Video footage emerged showing veteran protester Ian Bone telling one of the Tory Brexiteer’s sons: “Your daddy’s a totally horrible person. Lots of people don’t like your daddy. Do you know that?”
Police look on as a small group of protestors are seen confronting the MP, his wife and four of their six children on the pavement outside their London home.
Mr Bone is seen accusing them of failing to properly pay the family’s nanny, Veronica Cook.
He says: “Poor Nanny Crook who looks after you and wipes your bottom, she doesn’t get enough money every week. Daddy doesn’t care.”
Mr Bone then turns his attentions to Ms Crook, challenging her to say how much she is paid, to which she replies: “I don’t need to tell that. All I can tell you is I’m very happy.”
But speaking on LBC radio later on Wednesday, Jacob Rees-Mogg played down the barrage of abuse, and insisted his children were “absolutely fine”.
He said: “I wouldn’t get too excited about it. It was a few anarchists who turned up and it wasn’t very well organised. It wasn’t terribly serious.
“We are a free country. They weren’t violent. They aren’t admirers of mine.
“I am in public life and not everybody is going to like me. That is a reality of public life.
“I’d have preferred it if it hadn’t happened but I don’t want to get it out of perspective.
“I think much worse things happen to many other people.”
Theresa May’s official spokesman immediately condemned the protestors’ actions, labelling them “completely unacceptable”.
He said: “No elected member or their family should be subjected to intimidation or abuse in that way.”
Mr Rees-Mogg has since received widespread support from his peers following the protests outside his home.
Home Secretary Sajid Javid tweeted his backing to the Brexiteer and his family, branding the protest as “absolutely disgraceful.”
Tory MEP Daniel Hannan also tweeted: “A Leftist mob ambushes Mogg on his doorstep, shouting at his young children: ‘Your daddy’s a totally horrible person, lots of people hate your daddy’.
“What is happening to this country, for the love of God?”
But Mr Bone dismissed criticisms of the protest, claiming Mr Rees-Mogg had brought his family out of their house after they had confronted him on the street.
The veteran protestor warned that because the protest had attracted so much attention, the group planned to make it a regular event.
He told the Press Association: “They are hardly the most stinging examples of invective are they? He is a feudalist landlord scum of the worst kind.
“We are going to be back same time every week.”