The media star was speaking on the BBC’s Question Time programme, which has just recommenced after Parliament returned from its annual summer recess.
During the lively televised debate in Banbury, Oxfordshire, audience member George Stowe asked the panel whether the Tory party needed to “toss out” Theresa May to save Brexit.
Ms Hartley-Brewer emphatically replied: “Yes. The most generous explanation for the Chequers agreement I can think of is that Theresa May was taken hostage by the Europhile bureaucrats in Whitehall, and when we saw that video of her dancing in Africa that was a proof of life video – it is the only explanation I can think of.”
The journalist and broadcaster offered that Mrs May had got herself elected as Tory leader by saying she would follow through on Brexit, then stood for election in June 2017 on a Brexit manifesto.
She said: “The Euro-sceptics in her party are accused of being rebels.
“They are not rebelling. She is rebelling against her own manifesto.
“She is the only captain of a ship who is running her own mutiny against herself.”
The newspaper columnist also argued there was a good reason why MPs chose to put the EU question to the public because the House of Lords and the House of Commons does not represent the British public on the issue.
She said: “And to keep saying, ‘there isn’t a majority in the House of Commons,’ I don’t care!
“This is not a decision for the House of Commons; this is a decision for the British people.
“They asked the question, we gave the answer, and we want the Brexit that we were promised.
“And if Theresa May can’t deliver it, she should go.”
The BBC audience applauded Ms Hartley-Brewer’s comments as Prison Minister Rory Stewart shook his head in disagreement.
Defending the Government’s position, he said: “We have to be realistic. You have to understand that there are two sides in this.”
The Tory minister argued what the Prime Minister had put in place was clearly what the British people had voted for.
He said: “She has put in lines on free movement of people, lines on taking back control of our laws, lines on doing our own customs agreements.”
The MP for Penrith and The Border suggested the Conservative Party’s stance on Brexit was an indication it was moving into the political centre ground.
He said: “The fact that Julia is unhappy with this, and the fact that many Remain people in the audience are unhappy with this, probably indicates that we are moving into the centre ground.
“And the centre ground is where we need to be on this.
“That is not a weakness. It is a strength.”