Assalamu aleykum,

Looking at people of opposite gender is not a problem per se, although it should be avoided because it can cause lust thoughts, as you have described and experienced.

God commands the believing men and women to lower from their gazes in the Qur’an:

“Say to the believing men to lower their gazes and guard their chastity…”(An-Nur, 24: 30)

 “And tell the believing women to lower their eyes, and guard their modesty, ….”(An-Nur, 24: 31)

So we should always keep in mind that is it as requirement to avoid looking at opposite sex people and to try to lower our gaze as much as possible.

To understand more about the meaning of lower one’s gaze, please refer to 

1. Meaning of Casting Down

1. What “casting down the glances” mean?

It means that a person should not look at the member of the opposite sex except for those parts that may be uncovered.

So, for instance, a man is allowed to look at the face and hands of a non-mahram lady who is not related to him provided it is not done in with a lustful intention. (“Mahram” means person in whose presence hijab is not required. See the list at end of this section.)

https://www.al-islam.org/hijab-muslim-womens-dress-islamic-or-cultural-s...

Also, you may want to refer to the book A Code of Ethics for Muslim Men and Women in particular the rulings related to looking at others:

https://www.al-islam.org/code-ethics-muslim-men-and-women-sayyid-masud-m...