Sir Lindsay Hoyle's decision to depart from convention during a debate on the Gaza conflict was "very concerning", Prime Minister Rishi Sunak has said.
The Commons Speaker is facing pressure to quit, after chaotic scenes in Parliament and acrimonious debate.
Sir Lindsay has apologised for giving MPs a vote on a Labour amendment during an SNP debate on Gaza on Wednesday, but said it was to ensure MPs' safety.
The SNP are calling for a vote on Sir Lindsay's future.
Mr Sunak called Sir Lindsay's decision "very concerning because the usual ways in which Parliament works were changed".
"We should never let extremists intimidate us in to changing the way in which parliament works," Mr Sunak said.
"That is a very slippery slope and not the right way to go."
But the prime minister fell short of backing calls to replace Sir Lindsay as Speaker.
Speaker accused of 'party politics' over Gaza vote
What does the Speaker of the House of Commons do?
Why are Israel and Hamas fighting in Gaza?
Election poll tracker: How do the parties compare?
There is no formal way for MPs to oust the Speaker, but the withdrawal of support from the SNP, the third largest party at Westminster, makes his position significantly more precarious.
OceanNewsUK chief political correspondent Henry Zeffman said a Commons Speaker "lives or dies by their reputation as a fair referee", and with one in 10 MPs - almost all Conservative and SNP - signing the motion of no confidence, "that's a very bad position for a Commons Speaker to find himself in".
It comes after Sir Lindsay gave MPs a vote on a Labour amendment calling for a ceasefire in the Israel-Hamas war, during a debate allocated to the SNP on Wednesday.
The move meant Labour MPs could call for a ceasefire without backing a differently-worded motion from the SNP, side-stepping a potential Labour rebellion but provoking fury among the SNP's MPs.
After the Conservatives then said they would not take part, the Labour motion was passed, meaning standalone votes on the SNP motion, as well as a government amendment, were dropped.
Sir Lindsay said he had allowed the vote to give MPs the chance to vote on a wider range of positions, and to protect them from threats to their safety.
It has plunged his tenure as Speaker into crisis, with accusations he played "party politics" with the vote.
Several Conservative MPs have also questioned his decision to allow Labour's motion on safety grounds, arguing he had allowed Parliament to be intimidated by threats of violence.
He is expected to meet senior party figures later, as he battles to shore up support for his position.
Sir Lindsay apologised to the SNP for making a "mistake" - and offered the party another chance at a Gaza vote through a future emergency debate.
In an emotional statement in the Commons, he added: "I have a duty of care, and I say that, and if my mistake is looking after [MPs] I am guilty."
He added that details of threats to MPs shared with him were "absolutely frightening".
Labour leader Sir Keir Starmer has rejected suggestions he had threatened to withdraw support from Sir Hoyle after the next election if he didn't allow a vote on Labour's motion.
Speaking to reporters, he said he did not "threaten the Speaker in any way," but did not directly answer a question on whether any of his MPs might have done so.
He accused the SNP of only being interested in "dividing the Labour Party", and the government of walking out on the debate because it didn't have support from Tory MPs for its own amendment.
'Best of reasons'
The acrimony continued in Parliament on Thursday, with Commons leader Penny Mordaunt accusing Labour of playing politics with the vote.
While she was critical of the situation and the actions taken by the Labour party in relation to the amendment to the SNP motion,
"I would never have done to [Sir Lindsay] what the Labour Party have done to him," she added, to jeers from Labour MPs.
Unlike some of her colleagues, it was noticeable she did not direct her criticism against Sir Lindsay.
One of the Tory MPs calling for Sir Lindsay to go, Danny Kruger, accused him of allowing "Labour to use the Islamist threat to change the way our democracy works".
Other Conservatives, however, gave him their backing, with former ministers Sir Edward Leigh and Mark Francois advising colleagues against forcing him out.
Another Tory MP, Sir Charles Walker, said Sir Lindsay had made a "genuine mistake" for the "best of reasons," adding the Speaker was "obsessed with the safety of MPs and their staff".
"The idea that we remove a Speaker for trying to do the right thing is just too ghastly to contemplate," he added.
SNP MP David Linden said he believed that Sir Lindsay he will be out of his position by "the weekend".
Speaking on Radio 4's PM programme, Mr Linden said: "The reality is, the Speaker of the House of Commons was bounced into this by the Labour leadership."
Mr Linden added that the Speaker should now "do the right thing and resign to allow us to get back to the issues that matter".
