A desperate Gordon Brown won his battle last night to let police hold terrorist suspects for 42 days without charge, but was forced to rely on the support of the Democratic Unionists and a string of extraordinary concessions to his backbenchers to save the vote, and possibly his premiership.
Brown was accused of trading civil liberties in a "grubby bazaar", accepting a shopping list of demands from the Northern Ireland unionists and offering a range of promises on backbenchers' pet projects, such as compensation for injured miners, increases in rail investment, the accountability of the intelligence services and lifting EU sanctions on Cuba.
Brown had staked his personal authority on winning the vote, but 36 of his backbenchers joined the Conservatives and Liberal Democrats in the No lobby, slashing his theoretical 66-seat majority. In the end, only the support of the nine DUP members, Ukip's Bob Spink and the Tory MP Ann Widdecombe saw the government home by nine votes.
The Tories claimed the vote had cost the government £1.2bn. No 10 denied any deals, insisting Brown had not known how the DUP would vote. But backbenchers claimed Brown's backstage bargaining handed peers an excuse to vote the measure down in the summer.
No 10 said Brown had shown himself in tune with the country and demonstrated political courage by standing firm on an issue of national security, even if it endangered his premiership.
This morning Brown will hold a press conference at which he will justify his tactics and map out his future economic strategy, including G8 and EU summits in the coming weeks. One Labour backbencher, Austin Mitchell, backed the government at the last minute and admitted he had voted "to save Gordon for the nation. If he had lost, he would have been on his way out".
One of the 36 Labour rebels, John Grogan, claimed the narrowness of the vote, and the dependence on the unionists, was "the worst possible result for the government" and it would have been "better to have lost on principle". Diane Abbott, who said Brown had spoken to her for the first time in 20 years, accused the government of "trading ancient civil liberties in a grubby bazaar".
Nick Clegg, the Lib Dem leader, said: "It is a victory for pork barrel politics, and nothing to do with principle."Mark Durkan, leader of the SDLP, said: "The result of all that has gone on here in the last few days is a humiliating victory."
Gordon Brown's reliance on the DUP last night revived memories of the dying days of John Major's government when the former Tory prime minister often had to rely on support from David Trimble's Ulster Unionists, then the biggest party in Northern Ireland. "This has the whiff of John Major," one veteran of the Major years said last night.
It was a day of drama, backstage deals, impassioned speeches and frayed nerves. Conservative whips were speculating on the consequences of a tied vote, and how the Speaker would be required to use his casting vote. Downing Street insisted with two hours to go that it was bracing itself for defeat, and Brown's closest allies despondently admitted they had lost.
But just before the vote, the DUP met the prime minister for talks in his study and changed their minds. As the stony-faced DUP group left its private meetings, they were accosted by Tory backbenchers urging them to hold to their opposition to an extension to 42 days.
The DUP had kept the Labour whips guessing by telling MPs during the debate that they were worried that extending detention without charge would give the terrorists the oxygen of publicity.
The government and the DUP claimed their discussions centred solely on the principles of pre-charge detention and Northern Ireland's history with internment.
The DUP's Rev William McCrae insisted "hand on heart, it was a principled decision based solely on security considerations".
But it was privately acknowledged the DUP had also pushed for a financial package to lift the burden of new water charges, speed up asset sales and exclude Northern Ireland from the Abortion Act
The DUP is hoping for a Anglo-American package to be announced when George Bush and Brown visit Northern Ireland on Monday.
Keith Vaz, chairman of the home affairs select committee, was even forced to deny claims that he had been offered a knighthood for delivering the votes of two Muslim MPs. The home secretary, Jacqui Smith, yesterday wrote to Mohammad Sarwar MP promising to introduce an ex-gratia payment to any suspect held more than 28 days and released without charge. The Home Office denied reports that the compensation would be as high as £3,000 a day.
At prime minister's questions Brown insisted his proposals were not only popular, but right. "We must take no risks with security," he said. The Tory leader, David Cameron, said Brown had managed to construct legislation both draconian and incompetent. "We have a symbolic assault on our liberty that is unnecessary; and we have a procedure that is unworkable. Is not the only way to describe what the prime minister is doing today as ineffective authoritarianism?"
· This article was amended on Thursday June 12 2008. It's Democratic, not Democractic, Unionists. This has been corrected.