The former UKIP leader was responding sharply to suggestions that Tommy Robinson, leader of the English Defence League, could be allowed to become a member of UKIP.
Mr Farage responded to the suggestion stating he was “really upset”.
Speaking on the eve of the UKIP party conference in Birmingham, he said: “I wrote very explicitly into the rules of the party when I first became leader. We did not want anybody in the party that had taints with organisations we deemed to be on the far right of British politics.
“I couldn’t have been clearer.
“It really upsets me to see the fact this debate is even taking place.”
Mr Farage has taken aim at current UKIP leader Gerard Batten, who has voiced support for Mr Robinson.
Mr Batten also drew criticism for attending an anti-Muslim rally in Sunderland, and a protest in support of Mr Robinson in London in July.
In a prepared speech delivered at the rally, Mr Batten said: “The great British media do not want to talk about the group identity of the rapists, or the ideology that inspires them.
“The rape gang members are predominantly followers of the cult of Muhammad.
“They can justify their crimes to themselves because their ideology tells them that non-cult followers are lesser human beings.”
He also took aim at the British media and hate laws for curbing their freedom of expression.
He said: “But we, the infidels and kaffirs, are not supposed to talk about it.
“And people who do face possibly criminal prosecution under our so-called ‘hate laws’.”
Mr Farage responded sharply to Mr Batten’s actions, and warned UKIP was in risk of undermining its many achievements over the past numerous years.
He said: “I’m less than impressed with the current direction in which we’re going.
“It devalues and undermines many of our incredible achievements.
“UKIP is in danger now unless it changes direction very, very quickly, of total and utter marginalisation.”
Mr Farage ruled out returning as leader of UKIP “right at the moment”, and noted he was focused on campaigning for Brexit.
The former UKIP leader has been highly critical of Theresa May’s handling of the Brexit negotiations, and encouraged her to “chuck Chequers”.
Speaking on his LBC show, he said: “She was talked into this Chequers deal by her own civil servants who played a superior role to David Davis which is why he simply had to walk away.
“Barnier played the Northern Ireland card to an extent and she allowed herself to be boxed in, that is what it was actually all about.
“It is a deal that will not free us from the European Court of Justice.
“It is a deal that will tie us into a European rulebook.
“It’s the wrong deal, Chequers is Brexit in name only. It simply isn’t good enough.”