Brandon Lewis has asked Boris Johnson to apologise for the comments
The call followed a hail of criticism for Mr Johnson’s remarks, which were described as “offensive” by Foreign Office minister Alistair Burt and “bigotry” by former Tory chair Baroness Warsi.
Writing in the Daily Telegraph on Monday, the former foreign secretary described the burka as “ridiculous” and “weird” and said women wearing them looked like letter-boxes or bank robbers.
While agreeing with Mr Johnson that the burka should not be banned, Mr Burt told BBC Radio 4’s Today programme: “I would never have made such a comment, I think there is a degree of offence in that, absolutely right.”
And Lady Warsi accused Mr Johnson of adopting the “dog-whistle” tactics of former Donald Trump aide Steve Bannon in the hope of attracting support from right-wing Tories for an eventual leadership bid.
Repeating her call for an independent inquiry into Islamophobia in the Conservative Party, Lady Warsi told Channel 4 News: “Muslim women should not be a useful political battleground for Old Etonians.
“It is crass and it must stop, and it must be condemned by the leadership right from the Prime Minister down.”
In a message on Twitter, Mr Lewis said: “I agree with Alistair Burt. I have asked Boris Johnson to apologise.”
In his regular Telegraph column, Mr Johnson said he opposed a ban on face-covering veils, but said it was “absolutely ridiculous that people should choose to go around looking like letter-boxes”.
Friends insisted he was making a liberal case against the UK following countries like Denmark, France, Germany and Austria in imposing a ban on wearing the burka or niqab in public places.
But Lady Warsi suggested it was a calculated move by the former Cabinet minister to garner right-wing support.
“These were offensive comments but clever politics,” she said. “Boris knew the effect and the impact that this kind of dog-whistle politics would have.
“This is literally the kind of bigotry of Bannon and the tactics of Trump finding its way into mainstream British politics. It’s not the first time that Boris has tried to make a leadership bid.”
Mr Johnson’s father, Stanley, came to his defence, telling ITV1’s Good Morning Britain: “I think we are just picking holes here. I think people are whipping up a little mountain out of a molehill on this one, I really do.”
And Tory MP Nadine Dorries said the former foreign secretary “did not go far enough” and it was the Government which should be apologising.
“Any clothing a woman is forced to wear which hides both her beauty and her bruises should be banned and have no place in our liberal, progressive country,” said Ms Dorries.
Shadow chancellor John McDonnell called for a campaign of resistance against the rise of the far-right, along the lines of the Anti-Nazi League.
Mr McDonnell lumped Mr Johnson in with Tommy Robinson and a group of vandals who stormed a left-wing bookshop as proof of the emergence of far-right forces in British society.
Baroness Warsi accused Boris Johnson of bigotry
Ex-EDL leader Robinson became a cause celebre for elements of the political right around the world when he was jailed for 13 months for contempt of court for broadcasting footage on social media of people involved in a criminal trial.
He was released earlier this month after judges quashed the contempt finding, but warned he could still be returned to prison at a later hearing on the same allegations.
The Bookmarks bookshop in Bloomsbury, central London, said that “around a dozen mask-wearing fascists” attempted to intimidate staff and customers and to destroy books and other materials on Saturday.
Ukip suspended three party members understood to have been involved in the incident.
Mr McDonnell called for anti-racist campaigners to emulate the ANL, which was launched in response to the rise of the National Front in the 1970s, with the backing of the Socialist Workers Party.
The group was linked to Rock Against Racism, which staged high-profile festivals and concerts featuring acts such as The Clash, Elvis Costello and Sham 69.
The shadow chancellor said: “With the scale of Tommy Robinson demonstrations, the storming of Bookmarks bookshop, and now Boris Johnson’s Islamophobic comments, we can no longer ignore the rise of far-right politics in our society.
“Maybe it’s time for an Anti-Nazi League-type cultural and political campaign to resist.
“The Anti-Nazi League was an iconic movement over several decades that successfully combated the far-right through the mass mobilisation of trade unionists and anti-racist campaigners.”