The opposition groups, led by cricketer-turned-politician Khan and Canada-based cleric Tahirul Qadri, will protest in the capital to press Sharif to call an early election little over a year after his landslide victory in the polls.
The protestors reached here from Lahore after travelling more than 300 kms in over 35 hours.
Addressing his thousands of supporters here amid rains early in the morning, Khan announced to stage a sit-in from 3 pm today till Sharif's ousting.
"I am not derailing democracy because there is no democracy in the country. We will not go away till Nawaz Sharif resigns," he told his supporters.
Senior leaders of his party and workers also echoed the same sentiments when they chanted "Go Nawaz Go" slogan.
The rain and journey took toll on Khan who had not slept for 40 hours, supporters said.
According to Pervez Khattak, his party's chief minister in Khyber-Pakhtunkhwa province, Khan was suffering from high fever. He asked the workers to pray for Khan's quick recovery.
Thousands of supporters of Qadri's "revolution march" also reached at a separate venue in Islamabad.
Qazi Faiz, spokesman of Qadri, said that the cleric was suffering from a sour throat but will address the workers.
Both Khan and Qadri had initially planned to take out a joint rally to oust Sharif but later differences cropped between the two.
Government expects the marchers disperse peacefully as backdoor efforts are already going on to meet some of the demands of Khan and Qadri, who are now separately protesting against alleged rigging of last year polls.
Khan initiated his 'Azadi March' from his Zaman Park residence in Lahore, while Qadri launched his 'Inqelab March' (revolution march) from the Model Town area of the city.
Khan has already underlined his demands, saying Sharif should resign and a caretaker government should be formed?for holding a fresh general election in the country.
Earlier yesterday, as the march led by Khan passed through the city of Gujranwala, clashes erupted between his convoy and supporters of Sharif's Pakistan Muslim League-N after shots were fired at Khan's vehicle.
Sharif, on last Tuesday, had announced appointment of a panel of three judges of the Supreme Court to probe the charges of fraud in the elections.
The government has deployed thousands of police and paramilitary soldiers to maintain law and order.