You should not invite them to your house as Islamic law strongly discourages or condemns the situation where a woman and a non-mahram male are alone together. (Whether you would be alone in the house or alone in a room.) There is no need for them to be at your house. This will prevent any sort of problems.
Also remember that your intentions are only your own, and you don't have any guarantee about what the other person is intending or experiencing.