"I think he should be arrested. There should be zero tolerance for anybody who is harassing a woman and the time has come for that," said Gandhi.
Justice A K Ganguly is facing allegations of sexually assaulting a law intern at a hotel in Delhi last year.
A three-judge committee of the Supreme Court, which probed the complaint filed by the law graduate against Ganguly, said there was prima facie evidence supporting it.
But later Chief Justice P Sathasivam stopped all investigative proceedings on grounds that administratively Justice Ganguly could not be probed, which invited criticism and flak from all quarters.
Denying the allegations, Justice Ganguly, 66, has said that he is "shocked and shattered" by the allegations made against him by a 22-year-old lawyer who had interned with him.
Pressure is mounting on Justice Ganguly to resign from the post of Chairman of Human Rights Commission of West Bengal.
Meanwhile, senior advocate and Chairman of Bar Council of India, Manan Kumar Mishra said the girl should come forward for a proper trial to take place.
"The Bar Council of India is of the opinion that after the report of the sub-committee of the Supreme Court, now the matter should be fairly and thoroughly examined by the appropriate authorities, the truth should come out and it cannot come out, investigated or inquired unless the girl comes forward.
The statement, the examination of the girl, is a must now; only then the truth will come out. What offence is there, whether it was an offence or not, what type of misbehaviour, what was the extent of the misbehaviour, all this can be ascertained only after the girl comes out," said Mishra.
The law intern wrote in a blog post on November 6 that a top judge assaulted her in a hotel room last December
The intern's allegations created a media furore - with female lawyers and activists calling for the country's top court to investigate the incident and for the accused judge to be named publicly
Congress leader Jagdambika Pal said the former judge should resign from his post.
"Mr Ganguly has been found guilty by the Supreme Court sub-committee also, so he should at least resign from his post and whatever judicial proceedings have to take place, it will," said Pal
Lawyers and feminists are calling for a police complaint to be registered and for an inquiry
Janata Dal-United leader Sharad Yadav said: "Yes, there should be a case against Ganguly. The new law which has come into being of late will also get recognition and the women will also get courage.
The case is one of a small but growing number in which victims of alleged sexual harassment have come forward to complain about powerful male superiors
Activists say sexual harassment and abuse by powerful and privileged men is widespread in India, but few women have been willing to talk about it.