You don't lose anything by studying and praying. Even if you do not get into the institute, you still retain the knowledge that you learned, as well as the discipline and willpower you strengthened by studying. You also still enjoy the benefits of prayer such as an increased closeness to Allah.

However, when something doesn't work out repeatedly, sometimes it is not meant to be. While there is no harm in continuing to try to get into the institute, it would be good to also look for other options or opportunities (and to pray to also have other options and opportunities). Maybe there is something better out there that you would not have seen before, or something related to your life path that is meant to be somewhere else.

Insofar as you have been studying very hard and are determined to succeed, I am sure you are qualified for many institutes and many things.

With duas for your success, wherever it may lie!