The peer, who is among the most prominent critics of the Beeb, has ruled out even applying for the post after Boris Johnson favoured him. Following six weeks of talks about his proposed pay package, his appointment was “virtually a done deal” according to a report last week.
The Prime Minister had asked Lord Moore to fill the role as part of his radical shake-up of the BBC.
Sources suggested the corporation would have had to fork out at least £280,000 to the former editor of The Telegraph.
This would have risked infuriating BBC staff facing a raft of job cuts.
The BBC has long been criticised for overpaying its top stars.
Had he been paid £280,000 per year, it would mean he would have earned more than three times the salary of the current BBC chairman, David Clementi.
Sir David, who is due to step down in February, had said Lord Moore was an unsuitable replacement.
Those who remain in the running for the job are said to include former Tory chancellor George Osborne and ex-culture secretary Baroness Morgan.
The role of BBC chairman is the only post for which the broadcaster does not set the pay.
READ MORE: Sadiq Khan ‘running London into ground’ and risks collapsing Red Wall
He said the peer’s political views would have made him unsuitable to fill the space.
Speaking to MPs on the Digital, Culture, Media and Sport committee, Sir David said: “My point is impartiality starts at the top of the organisation.
“It doesn’t start halfway down.
“If the candidate comes from that sort of background, he or she will have to demonstrate to you that they have left those strong political views at the door.”
Committee chairman Julian Knight appeared to agree with Sir David’s assessment.
Citing Lord Moore’s conviction for refusing to pay his licence fee, he said handing him the role would be like placing “someone who committed fraud in charge of a bank.”
A job advertisement for the Beeb’s next chairman is expected to be issued shortly.
The Government is in the process of reviewing whether to decriminalise non-payment of the BBC licence fee.
Free TV licences for over 75s were scrapped in August.