Japan catches about 1,000 whales each year in the region for what it calls scientific research, BBC reported.
Australia in May 2010 filed a case with the ICJ, arguing that Japan's programme - under which it kills whales - is actually commercial whaling in disguise.
The court's decision is considered legally binding, and Japan earlier did say it would abide by the court's ruling, the report said.
Presiding Judge Peter Tomka Monday ordered a temporary halt to the Japanese programme.
The court said it has decided, by 12 votes to four, "that Japan shall revoke any extant authorisation, permit or licence granted in relation to whaling programme in the Antarctic and refrain from granting any further permits in pursuance of that programme".
The court also in a statement said that "the programme is not for scientific purposes."