MPs from the ERG, which is led by Jacob Rees-Mogg, believe they had blown their chances to control the outcome of Brexit, Buzzfeed News reports.
The group of up to 80 Tory Brexiteers this week launched what they claimed was a simple solution to the Northern Ireland border questions.
However group’s paper on the Irish border has been criticised for not having any clear ideas or new proposals.
One ERG MP told BuzzFeed News that he felt “despair” and his “heart sank” when he realised their plans for the Irish border were being ignored.
Another MP said: “The abiding image of our week is that photo of Boris, Jacob Rees-Mogg and Peter Bone with their heads in their hands.”
Some members of the group have said said a strong, confident Boris Johnson is needed as leader to generate momentum and media coverage.
But this week Mr Johnson had details of his marriage breakdown splashed over the front pages after it was revealed his marriage was ending in divorce.
Adding to this are reports of a scuppered bid to oust Theresa May as Prime Minister and push for a more eurosceptic leader.
The ERG are believed to have held a secret meeting this week where talks centred on Mrs May and a possible successor.
ITV’s reporter Robert Peston said: “Virtually the only topic of conversation for 40/50 mins was ‘how best do we get rid of her? What’s the best way to use our letters?’
“Comments included: ‘Everyone I know says she has to go’, ‘she’s a disaster’, ‘this can’t go on’.”
However Mrs May hit out at her rivals yesterday in a televised interview, telling them she was “irritated” the focus was on her leadership and not Brexit.
During a BBC interview to mark the six-month countdown to Brexit, Mrs May was asked if she would reassure the Conservative party she was not determined to continue her job.
She told BBC Panorama’s Inside No 10: Deal or No Deal?: “I get a little bit irritated but this debate is not about my future, this debate is about the future of the people of the UK and the future of the United Kingdom.
“That’s what I’m focused on and that’s what we should all be focused on.
“It’s ensuring that we get that good deal from the European Union which is good for people in the UK, wherever they live in the UK, that’s what’s important for us.”