A new survey, published on the opening day of the party’s annual conference in Liverpool, reveals the scandal could affect how 900,000 people vote and has sparked calls for Jeremy Corbyn to stand down.
The YouGov poll, commissioned by the campaign group Labour Against Anti-Semitism, found 32 percent of potential Labour voters think the party has a problem with anti-Semitism.
Of those, 39 percent would be much less likely to vote Labour as a result.
Labour Against Anti-Semitism said this figure equates to 12 percent of potential Labour voters, or about 500,000 people.
Among those who say they would currently vote Labour but could change their mind, 28 percent believe Labour has a problem with anti-Semitism.
Within that group, 26 percent say if the problem is not addressed they would be much less likely to vote Labour. This equates to 7 percent of Labour waverers, or about 300,000 voters.
A further 100,000 votes from solid Labour supporters could be at risk with 11 percent saying the party has a problem with anti-semitism and 2 percent of those much less likely to back the party in an election.
Labour Against Anti-Semitism spokesman Euan Philipps said: “This poll delivers a damning judgment on a Labour leader who has lost the moral authority to lead.
“Jeremy Corbyn’s foolish and misguided decision to spend an entire summer baiting the British Jewish community appears to have backfired drastically, and turned thousands of vital swing voters away from the Labour Party.
“Labour Party members must now ask themselves how much they wish to remain loyal to Corbyn, how much they want to risk further tarnishing the party’s already sullied reputation for fighting racism, and how much they want a Labour government.
“Because while Mr Corbyn remains leader the prospect of a Labour government remains increasingly far away.”
A Labour spokesman said: “The party is fully committed to the support, defence and celebration of the Jewish community and its organisations.
“We are taking action against anti-Semitism, standing in solidarity with Jewish communities, and rebuilding trust.”
The survey also found just 17 percent of all voters say Mr Corbyn is performing “well” or “very well” compares to 19 percent who say “badly” and 31 percent “very badly”.
It found 20 percent of all voters say the leader has been “competent” in his handling of anti-Semitism accusations and 58 percent think he has been “incompetent”.
YouGov questioned 5,000 people between September 10 and 13.