"I have withdrawn the statement and deputy chairman of Rajya Sabha (P.J. Kurien) expunged it. I have asked the home secretary to inquire into the matter," Shinde told reporters.
Shinde read out a statement on the gruesome incident in Maharashtra where three minor sisters were kidnapped, raped and murdered last month. Their bodies were taken out from a well.
He named all the rape victims, which according to the law is prohibited.
The matter was raised by Leader of Opposition Arun Jaitley, who said Shinde should not have named Bhandara rape victims.
"As per the Criminal Law Ordinance, 2013, the identity of rape victims is never to be revealed. However, the home minister in paragraph two has given names of all the three victims," the Bharatiya Janata Party leader (BJP) said.
Jaitley demanded the home minister withdraw the statement, have it corrected and table it again in the house.
Kurien ordered that the names be expunged.
"It is a very important matter and it (names) should be treated as expunged and media should also not use the names in any reports, else action will be taken," he said.
Reacting to the development, Communist Party of India-Marxist (CPI-M) leader Sitaram Yechury said: "It was shocking and unfortunate and should not have happened."
Defending Shinde, External Affairs Minister Salman Khurshid said the minister withdrew the remarks immediately after it was brought to his notice.
"I think what happened was clearly something that the home ministry doesn't want to insist upon and very graciously he withdrew. So I don't see there should be any issue on it at all," said Khurshid.
Women activists condemned the minister's slip, saying it was highly uncalled for.
"He is responsible for violating law of the land and should be held accountable. A cabinet minister's statement is a collective responsibility of the cabinet and we demand action from prime minister's office," said Ranjana Kumari, director, Centre for Social Research.