Asked about the comment, Bharatiya Janata Party spokesperson Nirmala Sitharaman said the party had "nothing to do with the statement".
Party leader and former Bihar deputy chief minister Sushil Kumar Modi called it an irresponsible statement.
"BJP doesn't approve of irresponsible statement given by Giriraj Singh," Sushil Kumar Modi said in a tweet.
BJP sources said party president Rajnath Singh has sent a message to Giriraj Singh and asked him to refrain from making such comments.
Giriraj Singh, addressing an election rally in Deogarh, Jharkhand, had said: "Jo log Narendra Modi ko rokna chahte hain, woh Pakistan dekh rahein hai. Aane wale dinon mein in logon ke liye jagah Hindustan mein nahi, Jharkhand mein nahi, balki Pakistan mein hoga (Those opposing Narendra Modi are looking at Pakistan, and such people will have no place in India but in Pakistan)."