Many answers can be found in the book Kitab al-Mu'min by al-Husayn b. Sa'id al-Kufi al-Ahwazi, published Ansariyan. I think you find it also on this website.
Consider every hardship/challenge as a test, even if it’s a punishment, we are tested on how we act and react to hardship. Viewing it as a test may bring encouragement, viewing it as a punishment may bring anxiety.
Imam Ali (a.s.) said, ‘Overcome anxiety with patience, for anxiety erases [Allah’s] reward and augments the catastrophe.’[Ghurar al-Hikam, no. 2527]
Nevertheless, we must always strive to act patiently and repent our sins and faults. The Prophet (S) said, ‘Repentance erases whatever precedes it.’[Mustadrak al-Wasa’il, v. 12, p. 129, no. 13706] and Imam Ali (a.s.) said, ‘If you commit a misdeed, hasten to erase it with repentance.’[Ibid. v. 77, p. 208, no. 1].
Hardships are blessings. Either they remind us of our sins, when we still have time to seek forgiveness, or they erase our bad deeds. Either way they should bring about hope. Hope of the mercy and forbearance of Allāh (’aza wa jal). Hope that we have time to change our misdeeds, and hope that we will rise from our tests instead of failing them.
Imam al-Sadiq (a.s.) said, ‘Hope in Allah with such hope that it will not embolden you to disobey Him, and fear Allah with such fear that it will not cause you to despair of His mercy.’[Bihar al-Anwar, v. 70, p. 384, no. 39]
Allah, or God, is incomparable to any His creation. So it is impossible for people like us to even comprehend His immensity let alone become like Him.
So what then is the purpose of our life? There is a great lecture on this topic that you are sure to enjoy. Do go through it.
The answer to this question is simple, first Allah did not create you to test you, but so that you worship Him..as is clearly stated in the Quran. With that, the test you are refering to is being brought to this present life so that you act, and the results of your actions shall lead you to either Heaven or Hell. This test is not for Allah to know how you will act, but for you to act and come about with your results. Allah does not need to test you to know how will you act...I hope this is clear.
Second, There is a vast difference between knowing something and causing it. Someone who has AIDS-nastajer billah- knows that he is to die, or someone who is sentenced to death at a certain hour, knows that he is to die, does that make those people the doers of dying at those moments? NO, not at all.. The Fact that Allah SWTA knows what is happening to you now and tomorrow, is different than Allah causing you to do good or bad. Allah created the causes of every action, and left it to you to choose what you want. Now it is up to you to take whatever path.. Allah knows what path you will end up taking, but he will give you the choice for taking that path. Allah knows if you are bad, if you are going to repent, that is why there is bada..if you are eating poison as we speak, I know that you will kill yourself, but I know that if you go to the hospital in time, you will be saved..what responsiblity do I have in your actions, aside from knowing them?
I suggest that you read Ayatullah Subhani's book"Albada" in Farsi and Arabic.. Please let me know if you need further explaining. May Allah put the light of clarity in our hearts.