"We reject a referendum patched together in a few weeks with Russian military personnel taking over Crimea," Xinhua quoted Obama as telling reporters after meeting with Ukrainian Prime Minister Arseniy Yatsenyuk at the White House.
Obama said he hoped diplomatic efforts over the next several days can cause a "rethinking" of plans for the referendum, which is slated for Sunday.
US Secretary of State John Kerry will travel to London to meet with Russian Foreign Minister Sergei Lavrov Friday. The top US diplomat told lawmakers Wednesday morning that the US will offer "certain choices" to Russia in hopes of finding a way forward.
Last Thursday, the Crimean parliament voted to join Russia and for a referendum March 16 on Crimea's future. On Tuesday, it adopted a declaration of independence, which specifies that Crimea will become independent if its residents vote in favour of splitting from Kiev to join Moscow in Sunday's referendum.
Obama said the idea that Ukraine is forced to choose between good relations with the West or economic ties with Russia is a " zero sum formulation" that doesn't make any sense in the 21st century with a highly integrated global economy.
"I actually think that in the end it's not in the interests of Russia either," he added.
Obama also warned Russia that if it continues on the current path, the US, together with the international community, will " be forced to apply a cost to Russia's violations of international law and its encroachments on Ukraine".