The effect of the decision is to put the power into the hands of cabinet office civil servant Mr Robbins, the architect of the much derided Chequers agreement, who has long been criticised by Brexiteers as a Remainer in Downing Street.
The announcement came just minutes before Mr Raab and Mr Robbins were due to give evidence to the Commons Brexit committee and appeared to be a major snub to the new Brexit secretary who is only in his second week of the job.
Former cabinet minister John Whittingdale, a senior Leave supporter, led criticisms of the move saying he was “extraordinary” by the decision.
He asked if Mr Robbins and Mr Raab should change seats because the the civil servant had been made Secretary of State and the minister reduced to his “official”.
Addressing Mr Raab, Mr Whittingdale said: “Is it not extraordinary that you are the secretary of state for the department of exiting the EU and yet you do not have responsibility for the negotiations?”
He added: “The chain of command is from the Prime Minister to you via Mr Robbins because he has overall responsibility not you.”
Jacob Rees-Mogg, the leader of the power European Research Group (ERG), compared Mr Robbins to Sir Humphrey from Yes Prime Minister.
He pressed Mr Robbins four times over when his unit had started writing the Chequers paper which led to former Brexit Secretary David Davis resigning and accusations that ministers had been kept in the dark.
Mr Rees-Mogg told the committee it was “worrying” that Mrs May’s Chequers proposals had been drawn up by the Europe Unit apparently without the knowledge of DExEU at a time when Mr Davis’s former department was still in the lead suggesting it was “a failure in cabinet government.”
Another Tory Brexiteer Craig Mackinlay said the whole saga “failed the sniff test” and suggested Mr Robbins was pushing for “Brexit in name only”.
But Mr Raab dismissed suggestions that his department had been downgraded as a “caricature”, playing the development down as “shifting of the Whitehall deckchairs”.
Mr Raab said he and the Prime Minister discussed “what we do to make sure we have the optimum arrangements – one team, one negotiation, clear chain of command – that would put us in the very best professional position to get the best deal”.
Mr Robbins also denied he had kept the Brexit department and Mr Davis in the dark.
He said: “It’s certainly not a picture of the days and weeks leading up to Chequers that I, or more importantly the Prime Minister, would recognise.”