I swear to my own soul that there exists mystery and secret in non-acceptance of at least some parts of the prayers of the righteous and impeccable ones, in spite of the fact that Allah (the Glorious, the Exalted), has himself promised that He will grant such prayers. Because Allah (the Glorious, the Exalted), possesses unlimited mercy and compassion towards His righteous servants, and this very infinite love and affection compels Him to keep them immune from whatever constitutes as bad and evil, even though in their own imaginations they regard it as something good and a blessing. The non-acceptance of their supplications and wants is due to divine expediency beyond our comprehension like the expediency of getting a child killed through Prophet Khidr [a] - the killing which apparently from the point of view of the parents was evil and bad, but in reality was blessing and goodness for them.
Now let us suppose that this mother and father because of their being unaware of tragic and harmful future happenings would have considered their son's remaining alive as something good and a blessing for themselves, and with this intention would have beseeched Allah (the Glorious, the Exalted), for their son's life. Then in that case would the granting of their prayer (with the intention of good and blessing) be - that their son remains alive and instead of being good and blessed makes them losers in both the worlds, or he better be killed so that the parent could be saved from loosing both the words, and instead of him another righteous son be awarded to them? It is obvious that acceptance of their prayer could be materialized only through the killing of their son.
Because a supplicant prays with the intention of achieving goodness, prosperity, and salvation through this means; in case he did not mention anything particular in his prayer and, wished for its grant, then it would be appropriate that he should consider his prosperity and welfare in whatever is granted by Allah (the Glorious, the Exalted). Now if He sees and is knowledgeable that whatever is being asked by a supplicant (unknowingly) would bring nothing except loss and calamity for him; then in that case the real grant of supplicant's prayer would be not to accept his prayer, instead to bestow upon him whatever is considered good and useful for him.
Non-acceptance of such prayer by Allah (the Glorious, the Exalted), are commonly practised among human beings, and very often it happens that someone would turn down the request of a dear one for the sake of prudence. For example, if a sick person mistakenly considers a fatal harmful drug as health-giving and begs his father to give him that medicine, and the father being knowledgeable of its fatal harmful effects, would then the father grant the son's request? Never. And, if he knowingly grants his son's request by giving him that fatal drug, then in that case wouldn't he be criticised by the wise people, and wouldn't it be said that the son requested a health-giving medicine and instead his father gave him a fatal drug?
Similar is the story of Allah (the Glorious, the Exalted), and his servant, sometimes it may happen that a servant through the means of his prayer asks something from Allah say wealth and property, because, he considers it something good for his wellbeing, but since Allah's vast and infinite wisdom which covers all do's and don'ts relevant to a servant's life affairs knows that wealth and property do not suit this servant; knows that increase of wealth for him even though he is not aware of it would result in his taking distance from truth and reality. Here it is that Allah (the Glorious, the Exalted), because, of his profound love and compassion towards his servant does not grant the servant's prayer and instead makes him deprived from the wealth and property asked by him in his prayer.
Because, if a servant seeks wealth and property through prayer from Allah (the Glorious, the Exalted), it is because of this reason that he regards it as a means of good and prosperity for himself, without being aware of the fact that his prosperity and welfare lies in his being poor and destitute instead of being rich and wealthy. And since Allah, (the Glorious, the Exalted), is knowledgeable about the fact that wealth and property is not the real grant of his prayer, i.e., to bestow upon a servant the blessings which are indeed required for his prosperity and eternal salvation. The real grant of prayer does not mean that Allah should bestow upon a servant whatever is desired by him - even though being aware of the fact - that unknowingly the servant has asked for a thing which would be harmful and disastrous for him. Or, in other words the grant of a servant's prayer could be explained as follows:
In cases where a servant prays for something which would bring good and prosperity for the servant, the grant of his prayer (by Allah) would mean to bestow upon him the desired things.
But whereby a servant unknowingly or in his imagination considering it good for his prosperity, while in reality it is harmful and disastrous for him asks something from Allah the grant of his prayer would be to not to grant his prayer, thus, making him deprived from his desired wants.
The above matter that - occasionally Allah (the Glorious, the Exalted), grants a supplicants prayer by non-acceptance of his request has been described in many traditions. Even it has been narrated in some of the traditions, that Allah (the Glorious, the Exalted), because of his love towards a servant, sometimes not only did not accept his prayer, instead due to His love, gets the servant occasionally involved into some minor sins, in order to keep him immune from greater sins like arrogance and self-conceit which would bring greater calamities upon the servant.
Therefore, whatever Allah (the Glorious, the Exalted), decides about a servant, without least doubt it would bring for him salvation and prosperity; whatever He does with a servant, even though apparently and over all may not appear in his favour, but relative to that person's affairs and characteristics, certainly means goodness and prosperity, and is executed in accordance with wisdom and prudence. Apart from that, Allah (the Glorious, the Exalted), bestows all sorts of favours and blessings which are useful for His righteous believers and are permitted by His wisdom and discretion, even though if they do not seek them in their prayers.