He also highlighted two more recent polls, which he said showed 29 per cent have now switched to being undecided or now back Remain.
Speaking at the Edinburgh International Book Festival, Mr Brown said this adds up to a more than one million Labour leave voters who have shifted their view since the historic referendum in June 2016.
Mr Brown claimed both Leavers and Remainers now have one thing in common – “both are increasingly losing hope for a better future for Britain”.
He also insisted none of the proposed exit deals from Brussels can deliver what was promised and the UK was lurching from “one Brexit shambles to another.”
The former Labour leader said: “Ironically, Leavers and Remainers now do have one thing in common – both are increasingly losing hope for a better future for Britain.
“Remain voters are despondent, fearing that we have moved from a soft Brexit to a hard Brexit to a no-deal Brexit.
“But leave voters believe that none of the Brexit options- a Norway-style deal, a Swiss-style deal or the Canadian option – can deliver what they were promised and now feel betrayed.
“In truth we have now spent the past two years immobilised by divisions, failing to deal with the causes of Brexit.
“Our country is often accused of being stuck in the past.
“The problem this time is that we are stuck in the present – and, as each day passes, becoming less and less optimistic about the future.”
His comments mark his most outspoken Brexit intervention yet.
Last November Mr Brown urged Jeremy Corbyn to push for a second EU referendum if Brussels can be persuaded to make a “game-changer” offer on curbing free movement.
Mr Brown also yesterday claimed that freedom of movement could be retained while addressing the immigration concerns of Leave voters.
He said it could be tackled through measures such as requiring immigrants to register first at local job centres – as in Switzerland – and demand they leave if they do not work, as happens in Belgium.
Mr Brown also argued an “immigration pressures fund” could be established to support public services in areas of high demand.
Concerns over a loss of sovereignty to Brussels could be addressed by a constitutional statute that insists that European decisions have to respect and uphold the UK’s “national identity”, Mr Brown added.
He said it was necessary to build a “new social consensus for the age of the gig economy” and rebuild trust in institutions.