Thank you for your question. Not at all. Praying through the wasilah of the Infallibles (as) is a direct prayer to God. The form it takes is to ask Allah by the honor of that wasilah in His eyes. So one may take the Quran as a wasilah, or the Kaba or a prophet and so on. It is perfectly fine for someone to pray without asking by the right of a wasilah.
May you always be successful.