This statement is not correct.
The name of Twelvers came from the Prophetic statement that my successors will be twelve. This most authentic Hadeeth is narrated in main Sunni books including Bukhari, Muslim, Tirmithi, Ibn Maajah, Musnad Ibn Hanbal hundreds of Hadeeth books.
While all Muslim scholars believe that the Prophet said this, but only Shia Muslims follow the Prophetic teachings by following the Twelve successors.
In fact every Muslim must believe and follow the Twelve Imams If he really wants to abide to the Prophetic orders.
The Osooli and Akhbari and Shaikhi are not different sects, as all of them believe and follow the Twelve Imams from Ahlul Bayt (AS), but some of their scholars have their attitude to derive the Islamic rules from the Hadeeths.
The criteria to know the truth among all the different opinions and sects is the authentic evidence from Quran and authentic Hadeeth.