It is narrated that if a person eats forbidden food, their prayers will not be answered for 40 days.

In Mustadrak Safinat al-Bihar, the focus of this narration is specified as food obtained via forbidden means, such as theft or usury, rather than things such as pork or non-zabiha meat, although that would be included in it. This would include any type of food obtained unlawfully, not just meat. The idea here seems to be that a person who lives unlawfully off of other people should not expect to be considered a pious person or close to God even if they are praying and fasting and performing the hajj.

To my understanding, non-zabiha meat was much less of an issue among Muslims in the past, when people tended to raise their own animals or get them from nearby farms, before the mass migration of Muslims to the West and the globalization of commerce. Embezzling things that rightfully belonged to others was much more of an issue, particularly when we look at the way the Umayyad caliphs lived. Of course, it is still an issue, but it is oftentimes easier to focus on the details of how meat is slaughtered rather than the ethical question of how one is earning a living - or, for that matter, the ethics of food production and factory farming.

Anyway, there are a couple factors.

One is if someone is thieving or eating forbidden food intentionally to thumb their nose at the divine, this would naturally distance someone from Allah. Furthermore, there is a sense that food is not just food; rather, one's evil acts involved in obtaining the food taints it.

The other is that even if someone does it accidentally, in practice, the food they are eating has a negative effect on their spiritual side. (That is, there is a cause-and-effect relationship) Many people argue that what one eats does affect people spiritually and thus it is good to be thoughtful about what you eat. 

Anyway, it is good to be reasonably careful about, while at the same time it is also important not to become too obsessive, as happens sometimes. If it happens by accident, the best thing to do is to admit one's error to Allah and move on rather than worry about it. Accidents happen!