Leather is a durable and flexible material created by tanning animal rawhides, mostly cattle hide. Leather is used to make various goods, including footwear, automobile seats, leather goods, clothing and in furniture.
Yes it is permissible to do that, even you can keep Najis leather or any other Najis items in your pocket during Salah as far as Najis items are not wet because wet Najis item will make your pocket Najis which will invalidate your Salah.
You must refer back to the specific view of your Marja' taqleed. According to A.U. Seyid Sistani, if you had forgotten, and it was due to your carelessness or negligence, then according to obligatory precaution you must repeat the prayers. If it was not due to carelessness or negligence, and you had simply just forgotten to take it off, then you do not need to repeat your prayers.
And Allah knows best.
The Maraji' have generally ruled that wearing things like leather belts in salat is not allowed if the leather is not slaughtered in an Islamic way. See Islamic Laws of Ayatullah Seestani's Tawdhihu'l Masa'il p. 160.