In his talks with Russian President Vladimir Putin over the phone, Obama said the vote in the Ukraine's autonomous republic violated the Ukrainian constitution, Xinhua quoted the White House as saying Sunday.
It was also held "under duress of Russian military intervention", Obama said.
The plebiscite, in which 95.5 percent of voters were in favor of rejoining Russia as shown by the preliminary results, "would never be recognized by the US and the international community", Obama said.
He told Putin that "Russia's actions were in violation of Ukraine's sovereignty and territorial integrity and that, in coordination with our European partners, we are prepared to impose additional costs on Russia for its actions".
US Secretary of State John Kerry Thursday threatened "a very serious series of steps" against Russia with the European Union as early as a week ago depending on Russia's next moves.
Washington has canceled trade talks and military exchanges with Russia, and Obama has authorized visa bans and assets freeze on some Russians and Ukrainians blamed for Moscow's military takeover of Crimea.
Putin told Obama, however, that Crimea's vote Sunday fully complied with the international law, according to the Kremlin's readout of the phone talks.
Putin told German Chancellor Angela Merkel earlier that Russia would "respect the choice" of the Crimeans.
Washington and its allies have been pushing for Moscow to withdraw its military personnel to bases in Crimea, home to Russia 's Black Sea Fleet, allow international monitors into Crimea and to open a dialogue with the Ukrainian government.
Crimea, historically part of the Russian Federation, was transferred to Ukraine in May 1954, then a republic of the Soviet Union.