Intelligence officials defend surveillance tactics in Congressional hearing - live

We're going to wrap up our live blog coverage for the day. Here's a summary of where things stand:

The leaders of America's intelligence services said the loss of surveillance capabilities could lead to a "catastrophic" 9/11-like attack on the United States. DNI James Clapper and NSA director Keith Alexander testified as new legislation was introduced in Congress with substantial bipartisan support to limit NSA surveillance capabilities.

The intelligence officials agreed in principle to the declassification of some Fisa court opinions, as a gesture of transparency. The length of time the NSA stores American's phone metadata might be reduced to three years from five. And the creation of a permanent advocate role on the court was deemed possible although the justice department witness opposed it.

Clapper did not deny that the US conducts surveillance on foreign leaders. A debate broke out on the committee over whether committee members had been informed of the practice. 

Alexander said reports that NSA collects tens of millions of phone calls in France are "completely false." He didn't talk about metadata collection performed by NSA. He said the reports have mistaken data "provided to NSA by foreign partners" for data NSA collects. 

The conversation was restricted to two intelligence programs that so far have garnered a lion's share of media coverage: phone metadata collection and the surveillance of foreign Internet users. Both programs are subject to Fisa court approval. Critics warn that both programs sweep up substantial intelligence about Americans in a way that relies on tendentious interpretations of the law.

White House spokesman Jay Carney said the president had already made some changes to US spy practices even as a review of those practices continues. Multiple anonymous White House officials said the president had ended spying on friendly leaders. Carney would not comment on those reports. 

There wasn't much talk of reform coming from the witness table, Guardian national security editor Spencer Ackerman notes: 

The proposal for a permanent advocate on the Fisa court was not rejected by the intelligence chiefs, Spencer notes, but deputy attorney general Cole opposed the idea.

The intelligence officials said Fisa court opinions could be declassified as a way of calming the public and meeting demands for greater transparency. 

Rep. Justin Amash, Republican of Michigan, co-sponsered legislation in July with Democratic Rep. John Conyers that would have ended the collection of phone metadata under section 215 of the Patriot Act. 

The legislation was voted down 205-217, a tantalizing near miss for privacy advocates. New legislation was introduced today in both Houses to place similar limits on NSA surveillance.

Amash wasn't impressed with what he saw this afternoon:

Rogers asks Cole, the deputy attorney general, about the proposal for Fisa court hearings to include a permanent privacy advocate. Do criminal grand juries include such an advocate? Rogers asks.

Nope, says Cole, and neither do courts that grant wiretap warrants. The court itself is meant to be the check – "That's the role of the judge," Cole says.

Rogers says for the Fisa court to include an advocate would be granting a "higher level of protection for terrorists overseas" than US citizens get before grand juries. 

That's assuming, of course, that the purview of Fisa approvals is exclusively "terrorists overseas," when a large part of the concern is the possible violation of Americans' privacy.

Rogers gavels the first panel to a close and brings in panel 2.

Clapper says NSA has spent thousands of man-hours cleaning up after the Snowden revelations, which he calls "a major distraction."

Rep. Lynn Westmoreland of Georgia asks about Edward Snowden and what he knows. "Is this stuff he's just making up, or did he really have knowledge?" Westmoreland asks.

Alexander replies that Snowden had access but not knowledge:

He had tremendous access as one of our web site administrators... he really didn't understand things about how these tools actually operated. ... He did get great access to what we'll call the core capabilities we have.. and he took a lot of that. He took a lot.

Alexander says the information is being "dribbled out" in a way that he believes is intended to inflict "maximum harm":

I believe it's being done in a way that would cause maximum harm. I don't know why they would want to harm our country. But that's what's being done. And our allies. 

Westmoreland asks what kind of information NSA collects on everyday citizens.

"Remember, we're a foreign intelligence agency," Alexander says.

Westmoreland concludes with a humdinger: "A lot of private companies in this country have a lot more information on our citizens than what the NSA does." 

The spy leaders and the members are joking about the "wire-brushing" the spy agencies have taken since this summer. "Wire brushing" has also been used during this hearing to describe extra care NSA takes to prepare Fisa requests. 

From inside the hearing room, Guardian national security editor Spencer Ackerman provides a snap summary of the Schiff-Rogers spat:

Two Democratic representatives, Adam Schiff of California and Jan Schakowsky of Illinois, suggested that the House intelligence committee was not informed about the foreign leader spying. Clapper, without confirming that the spying took place, seemed to concede that, saying only “we have by and large complied with the spirit and intent of the law.”

Schiff drew a heated and unexpected rebuke from Rogers, who called his suggestion “disingenuous.” The committee has access to “mounds of product” from the NSA, Rogers said. Schiff shot back a direct question about whether Rogers in fact knew about the foreign leader spying, which Rogers said he could not answer without confirming – but invited the committee member to view reams of intelligence in private.

Later, Schiff returned to the colloquy. The California Democrat said that yes, NSA reports the "selectors" it spies on, but "not all selectors are equal." The implication seemed to be that the NSA doesn't specify explicitly to the committee when a "leader of an allied nation" – "not an average selector," Schiff said – has been spied upon. 

Schiff said "we need a discussion here" about the standards by which the NSA contextualizes to the oversight committee when a major foreign figure has been spied upon. He suggested that the current standard was lower than the one the CIA must employ when reporting a covert action.

The NSA programs under discussion today are administered by the Fisa court under rules amended in 2008. The court has been criticized as a rubber stamp that automatically accedes to intelligence community requests. 

James Cole, the deputy attorney general on the panel next to Clapper and Alexander, points out that it's really more efficient that way:

An interesting exchange between Rep. Adam Schiff of California and Clapper, on whether the committee was informed about the program to tap the phones of world leaders friendly to the United States. Chairman Mike Rogers eventually jumps in and an intriguing debate breaks out. 

Schiff seems to take Rep. Schakowsky's side, that the intelligence committee did not know about the program.

"I have put a lot of emphasis on written notifications to Congress," Clapper says. ... The fact that we don't necessarily report each and every selector... that's something we could discuss, as to whether that level of detail is required in terms of congressional notification."

Schiff says he can help Clapper with this kind of decision. "If you're tapping the phone of a foreign leader and ally" then yes, tell Congress, Schiff says.

"There are many things we do in intelligence that, if revealed, would have the potential for all kinds of blowback," Clapper says. That's indisputable.

Schiff replies, this is just the activity Congress should know about, to evaluate the possible benefits of the programs versus the risks.

Clapper: "The conduct of intelligence is premised on the notion that we can do it secretly."

Schiff says if the program was visible to a "low-level systems analyst like Mr. Snowden" then it probably would have been OK for Congress to know. 

Rogers jumps in and says the committee is briefed on "activities that are sensitive to the national security of the United States" including "sources and methods."

"I would argue to make the case that somehow we are in the dark is mystifying to me," Rogers says. That seems true.

Schiff takes on Rogers: "Mr Chairman are you suggesting that you and committee were informed of the wiretapping of foreign leaders?"

Rogers replies: "We have access to all sources and methods, and there is lots of products to be reviewed... any implication that through those reviews that this committee has not been informed.. has not been correct." 

Schiff says he wants to know "If you're suggesting we have information if we don't have it." He asks if it's a case of document flooding, in which there's so much information there's no information, where "you have a warehouse full of documents."

Rogers lectures Schiff. "This is a very unique committee on Capitol Hill." He won't yield back. 

Rep. Michele Bachmann is an artist.

Is it fair to say that Mr. Snowden was a librarian who took books home for the night? she asks the director of NSA.

"And longer," Alexander says, pithily.

Rep Frank LoBiondo of New Jersey tells Alexander that he made an important point when he said the United States has not suffered a mass casualty terrorism incident since 2001.

"The fact that we haven't had an attack with mass casualties is not an accident," LoBiondo says.

Does the Boston marathon attack of last April not count?

Alexander says if section 215 collection of phone metadata ends, the United States will be newly vulnerable to a "catastrophic" attack on the scale of 9/11:

If we take away a program that's helping to defend this country... what we've got to do is try to help the American people understand it.

Clapper says that the loss of a surveillance program would incur "greater risks":

...when that happens, we are incurring greater risks, and I say that as a general comment.

Alexander says his paean to NSA patriotism "was meant as nothing against anyone else."

Schakowsky says no one is questioning the patriotism of NSA.

Alexander says "that's a discussion":

I don't read all the newspapers, but some of the stuff that comes out sure looks that way.

Rep. Jan Schakowsky of Illinois lays into Alexander a bit for his making such a big deal about how patriotic NSA employees are. 

Nobody's questioning their patriotism, Schakowsky says. The programs they're conducting are the issue":

I felt implied that someone was saying that the people who worked for the NSA were accused of not being patriots, of not working hard.

She says that Congress members who question surveillance programs, including Sensenbrenner and Leahy, who introduced new legislation today placing limits on the programs, are patriots too.

Then she asks:

Why did we not know that heads of state were being eavesdropped on, spied on? 

But earlier chairman Rogers said the committee did know about the program, Spencer points out:

Yup.

Alexander says there's rigorous internal training for analysts who operate Fisa surveillance programs: 

"There are at least six courses in 702 that they have to go through... to ensure that we handle the data properly."

NSA director Alexander says the nature of US surveillance on European phone records has been misreported. Guardian national security editor Spencer Ackerman is in the hearing room:

Alexander said "screen grabs" of NSA documents painted a misleading picture of the programs. Rogers jumped in and harangued the media for, he said, repeatedly mis-reporting the story.

Rogers emerges as a staunch defender of the intelligence services and opponent of the emergent White House-Feinstein-Reid axis promoting a "major review."

Rogers asks Clapper: Wouldn't the White House know pretty much everything you're doing? 

Clapper: "Speaking in totality, leadership intentions are an important dimension of the landscape out there..."

As I indicted earlier... they may not have information specifically related to a specific selector... what they would see though would be a specific output.

Rogers jumps in: "Well we're dancing around the bush." Huh? 

Then Rogers gives a speech about not making the intelligence community into the enemy. 

This is the time for leadership, it is not a time to apologize. We know that the 9/11 road was paved with a lot of very good intentions. We ought not to walk down that road again.

Rogers asks Clapper about a news report in which a French intelligence official says it's preposterous for politicians to assert they don't know what the intelligence services are up to. 

Rogers is trying to get at whether a certain other country's politicians know what that country's spies are up to. 

Clapper chuckles and makes an apropos cultural allusion, to Captain Renault:

"Some of this reminds me a lot of the classic movie Casablanca," Clapper says. "'My god, there's gambling going on here?!' – it's the same kind of thing."

Clapper is back before the House intelligence community, answering questions from chairman Mike Rogers.

Rogers nudges Clapper to comment on what the White House knows about specific intelligence activities. Clapper says is depends on the "level of detail."

They can and do [know], but I have to say that that does not necessarily extend down to the level of detail... so we do not necessarily review with the White House what the collection deck is... that is done at levels below the White House or national security staff.

The output of all this, in the form of intelligence analysis... that is what they use.

Senate majority leader Harry Reid is all for Senator Feinstein's proposed for a "major review into all intelligence collection programs," NBC News reports. 

Carney is asked whether the US spies on its friends.

I'm not going to get into a discussion of reported intelligence activities... I'm not going to talk about specific... operations.... I can tell you that the [intelligence] review has focused in part on this issue.

Reuters leans on an unnamed US official to report that the president has ordered NSA to "curtail eavesdropping on the United Nations headquarters in New York.

Is this what Carney was referring to when he said "some decisions have been made about our intelligence gathering"?

Reuters: 

The full extent of U.S. eavesdropping on the United Nations is not publicly known, nor is it clear whether the United States has stopped all monitoring of diplomats assigned to the U.N. in New York or elsewhere around the world.

"The United States is not conducting electronic surveillance targeting the United Nations headquarters in New York," said a senior Obama administration official, who spoke on condition of anonymity. [...]

The NSA declined to comment on the matter. Spokesmen for U.N. Secretary General Ban Ki-moon did not respond immediately to a request for comment. 

The House hearing has temporarily adjourned so the members can report to the floor for a vote. They'll be back in a bit.

Carney is asked about Feinstein's assertion that the White House has ordered a stop to the surveillance of friendly leaders.

We consult regularly with chairman Feinstein... and we appreciate her continued leadership on these issues. ... I would just refer you to chairman Feinstein on that.

Jay Carney, the White House spokesman, begins the daily briefing. "We need to look at and make sure that we are not just gathering intelligence because we can but because we need it," he says. 

It's a very different message from Alexander and Clapper's insistence that all surveillance targets are appropriate and all spying is legal. 

Carney says the administration has already changed surveillance behavior as a result of the ongoing review of intelligence activities:

Upon conclusion of the review we will endeavor to make public as much information as possible. ... In the course of the review some decisions have been made about our intelligence gathering even as we proceed with it. Some of those decisions have been made and ...implemented.

Alexander says there hasn't been a mass casualty terrorism incident in the United States since 2001. He's apparently not counting the Boston marathon bombings, in which five were killed and 280 wounded.

NSA director Keith Alexander begins with a sentimental tribute to the intelligence community. He said just this past weekend, his people provided support to US troops in Afghanistan.

In 8+ years, not one person has come up to me and said, I have to work tonight or the weekend. They always come in.

Then Alexander turns to the September 11 attacks:

First, how did we get here, how did we end up here? 9/11.. what I remember most is those firemen running up the stairs...

We see the threats that come into this nation... prior to 9/11, we had no way of connecting those dots... and so the question is, how can we connect these dots... and do it in the least intrusive manner? 

An uncommon degree of emotion in Alexander's voice. He's beseeching the members of the committee to recognize the value of the work the intelligence community does.

"Everything we do on this program is audited 100%, on the business records Fisa. 100%," Alexander says.

Clapper says "we've made mistakes... usually caused by human error," but he insists that the spying programs are valid:

We do not spy on anyone except for valid foreign intelligence purposes and we only work within the law.

It's unclear how to square that statement, for starters, with the tapping of the German chancellor's cell phone.

"We welcome this opportunity to make the case to the public," Clapper says. Then he says the disclosure of the intelligence programs has been "extremely damaging."

Clapper begins speaking but Rogers interrupts to kick out a protester wearing huge novelty glasses. 

Clapper had time to refer to the ongoing unauthorized disclosures of intelligence activities.

The House hearing begins. Chairman Mike Rogers forgoes reading his opening statement to allow more time for questions and make room for a House vote expected to interrupt the House hearing this afternoon.

Rep. Dutch Ruppersberger, the ranking Democrat, thanks intelligence community employees for their service, says the media has misreported the story, says Fisa surveillance only applies to foreign entities – not American citizens or residents – and says 9/11 shows we need surveillance.

Russia has denied reports it attempted to spy on foreign powers meeting at the G20 summit in St Petersburg earlier this year, denouncing the allegations as a "clear attempt to divert attention" from revelations concerning the United States' National Security Agency (NSA). 

The Guardian's Lizzy Davies reports from Rome:

Two Italian newspapers claimed on Tuesday that USB flash drives and cables to charge mobile phones which were given to delegates- including heads of state- at the September meeting were equipped with technology to retrieve data from computers and telephones. 

The St Petersburg summit on September 5th and 6th came at a particularly delicate point in relations between the Kremlin and the White House. Tensions were high over the possibility of military strikes on Syria, as well as over Russia's decision in August to grant asylum to NSA whistleblower Edward Snowden.

The revelations from his dossier of leaked material continue to shake Washington DC and inflame parts of Europe. 

According to La Stampa and Corriere della Sera, the first person to raise the alarm over the Russian devices was Herman von Rompuy, president of the European Council, who allegedly went to intelligence services in Brussels and Germany for advice on whether they were what they seemed. 

Spain's public prosecutor's office announced on Tuesday that it had launched a preliminary inquiry into the alleged widespread surveillance of Spanish citizens' private phone calls and emails by the US National Security Agency, to determine whether it could be prosecuted under Spanish law.

The Guardian's Paul Hamilos reports from Madrid:

It was reported on Monday that the NSA had monitored 60.5 million phone calls in the space of one month alone, in the latest revelations from the documents leaked by the US whistleblower Edward Snowden, causing outrage in Spain.

The attorney general, Eduardo Torres-Dulce, issued a statement saying that, having seen the media reports “about a possible interception and irregular access to telephonic and/or electronic metadata ... by foreign official services” it would determine the exact nature of any potential crime and whether this could be punished under Spanish law.

Julian Sanchez of the Cato Institute looks at the list of second-round witnesses at today's House hearing – about to begin – and sees a lot to yawn at:

Here are the two panels testifying:

Panel 1

James Clapper, Director of National Intelligence
James Cole, Deputy Attorney General, U.S. Department of Justice
General Keith Alexander, Director, National Security Agency/Chief, Central Security Service and Commander, U.S. Cyber Command
Chris Inglis, Deputy Director of the National Security Agency

Panel 2

Steven G. Bradbury, Partner, Dechert LLP
Stewart A. Baker, Partner, Steptoe & Johnson LLP
Stephen I. Vladeck, Professor of Law and Associate Dean for Scholarship, American University Washington College of Law 

Jay Carney's briefing at the White House has been pushed back to 1.15pm ET. 

Meanwhile we've spoken with Tim Weiner, author of the essential histories of the American intelligence services, Enemies: A History of the FBI and Legacy of Ashes: The History of the CIA. 

We wanted Weiner's insight on the brewing fight between the intelligence community and elected officials. "We've seen this movie before," Weiner said:

We've seen this movie before, and we saw it in the 1970s, when we learned through the post-Watergate hearings of the Senate into American intelligence that the NSA had been spying on Americans... this was political espionage.

Now, spying on a foreign leader like Angela Merkel is also political espionage, if you ask me. [...]

I think the president was on to something when he said, 'Just because you can do something doesn't mean you should do it.'

The full interview is embedded below. "Of all the messes that President Bush left for President Obama to clean up... the great expansion of the National Security Agency's powers that Bush ordered after the 9/11 attacks may take them longest to clean up," Weiner says.

Here's a summary of where things stand:

Two top American spymasters, the director of national intelligence and the director of NSA, are preparing to testify before Congress in a hearing scheduled to begin at 1.30pm ET.

For the first time since the Snowden revelations of last June, Senate intelligence committee chair Dianne Feinstein directly criticized the intelligence community and called for a "major review" of their activities. The president also put new distance between what "policy direction" he gives to spies and "what they're able to do."

Legislation was to be introduced in both houses of Congress to curtail the domestic collection of phone metadata by the NSA and otherwise to strengthen limits on spying. 

Officials are preparing a report on the fallout over US spying on its allies. The report is to be delivered to the president in the next two weeks, and it will provide the White House with a raft of possible reforms, reports Paul Lewis in Washington.

President Obama is meeting today with CEOs of major credit card, technology, banking and defense companies to talk about cybersecurity, according to a White House pool report. The Obama administration and the companies are drafting a new cybersecurity framework.

Here's the White House list of who's meeting with the president:

Ajay Banga, MasterCard

Steve Bennett, Symantec

Wes Bush, Northrop Grumman

Marillyn Hewson, Lockheed Martin

Renee James, Intel

Brian Moynihan, Bank of America

Joe Rigby, Pepco Holdings

With exceptions, congressional interrogation of intelligence officials in hearings since the Snowden revelations in June has been obsequious conspiratorial deeply collegial.

The question, in advance of today's House hearing with America's top spymasters, is whether that will change, now that Senate intelligence committee chair Dianne Feinstein has assumed a pose of shock and umbrage over what the spies are up to. Feinstein has long been criticized by privacy advocates for a perceived laxity of oversight of intelligence activities and deference to intelligence officials.

In the words of an anonymous NSA official that turned into a Foreign Policy headline Monday: 'We're Really Screwed Now': NSA's Best Friend Just Shivved The Spies. 

But does Feinstein's change in tack, and the president's distancing himself from intelligence gathering, really mean that elected officials are about to undertake a thorough review of spying activities and to start asking hard questions at hearings? We'll begin to find out in the House intelligence committee room at 1.30pm ET.

Guardian national security editor Spencer Ackerman (@attackerman), who will be attending the hearing, shares a widespread skepticism about the "major review into all intelligence collection programs" Feinstein announced in her statement Monday. Spencer points out (as did @emptywheel) that whatever the intended scope of the review, Feinstein isn't waiting for the results to go ahead with legislation that would leave NSA surveillance programs largely intact:

Spencer and Guardian Washington bureau chief Dan Roberts have published a preview of today's activity on the Hill and at the White House, where the daily briefing is scheduled to begin in about an hour:

US intelligence agencies were facing an increasingly hostile Congress on Tuesday as lawmakers prepared multiple efforts to rein in a series of surveillance programs from which even the White House was distancing itself.

The hearing comes just 24 hours after the intelligence chiefs found themselves abandoned by Washington's political establishment, with anonymous administration officials claiming President Obama was unaware of the extent of spying on foreign leaders – a practice unexpectedly rebuked by Senate intelligence chair Dianne Feinstein, who has been a significant NSA ally.

Meanwhile, legislative efforts to end bulk collection of Americans’ communications data gathered pace with two complementary bipartisan bills aimed at curbing “a trust deficit” sparked by the revelations by NSA whistleblower Edward Snowden.

Read the full piece here.

Did President Obama know about US spying on its friends, going back more than a decade apparently in the case of Angela Merkel? 

The answer is either "Of course he did, idiot" or "It's plausible he didn't, actually" – depending on whom you ask.

The White House message is clear: The president was in the dark.

The Wall Street Journal quoted an anonymous administration official on Sunday saying that the president didn't find out until this summer about spying on allies and he immediately ordered it stopped. The Washington Post published a corroborating report Monday.

Obama himself told ABC News on Monday that the White House merely gives the intelligence community "policy direction":

I'm the final user of all the intelligence that they gather. But they're involved in a whole wide range of issues.

We give them policy direction. But what we've seen over the last several years is their capacities continue to develop and expand, and that's why I'm initiating now a review to make sure that what they're able to do doesn't necessarily mean what they should be doing.

To members of the intelligence community, the president distancing himself in this way from data collection that fed his daily briefings is a betrayal. The intelligence officials involved aren't taking it sitting down, either. "Current and former U.S. intelligence officials" are talking to Ken Dilanian of the Los Angeles Times, among others. The White House "signed off on surveillance targeting phone conversations of friendly foreign leaders," he reports:

Obama may not have been specifically briefed on NSA operations targeting a foreign leader's cellphone or email communications, one of the officials said. "But certainly the National Security Council and senior people across the intelligence community knew exactly what was going on, and to suggest otherwise is ridiculous."

On the other side of the argument are those who point out that the president's intelligence briefings aren't sourced line by line and that there are layers of mastication and management between the president and the analysts. One of those layers is the National Intelligence Priorities Framework, which an NSA spokesperson pointed to on Monday in response to questions about its activities (h/t @emptywheel):

"NSA is not a free agent," said NSA spokesperson Vanee Vines. "The agency's activities stem from the National Intelligence Priorities Framework, which guides prioritization for the operation, planning, and programming of U.S. intelligence analysis and collection." The framework is approved by the top leaders of the government, but it leaves the question of how best to gather intelligence to the individual agencies.

With Senator Feinstein's startling about-face Monday on the strength of government oversight, it appears that a broader conflict between the intelligence community and elected officials may be brewing. John Schindler is a former NSA employee and a tireless explainer and advocate of the intelligence community perspective @20committee:

A site has launched to raise money to cover the "defence costs" of Edward Snowden. The site says it was commissioned by the Journalistic Source Protection Defence Fund (JSPDF), "dedicated to building support for Edward Snowden" and describes itself as "the only fund endorsed by Edward Snowden".

It also gives details of many of the revelations his leaks have resulted in.

A New York Times editorial calls the White House's response to the ongoing surveillance revelations "a pathetic mix of unsatisfying assurances about reviews under way, platitudes about the need for security in an insecure age, and the odd defense that the president didn’t know that American spies had tapped the German chancellor’s cellphone for 10 years".

It adds:

We are not reassured by the often-heard explanation that everyone spies on everyone else all the time. We are not advocating a return to 1929 when Secretary of State Henry Stimson banned the decryption of diplomatic cables because “gentlemen do not read each other’s mail.” But there has long been an understanding that international spying was done in pursuit of a concrete threat to national security.

That Chancellor Merkel’s cellphone conversations could fall under that umbrella is an outgrowth of the post-9/11 decision by President Bush and Vice President Dick Cheney that everyone is the enemy, and that anyone’s rights may be degraded in the name of national security. That led to Abu Ghraib, torture at the secret C.I.A. prisons, warrantless wiretapping of American citizens, grave harm to international relations, and the dragnet approach to surveillance revealed by the Snowden leaks.

“Spies will be spies” has been the response of some commentators to the NSA surveillance revelations. Yesterday Reuters’ Jack Shafer wrote that protests from world leaders “are largely contrived. As Max Boot and David Gewirtz wrote in Commentary's blog and ZDNet, respectively, nations have traditionally spied on allies both putative and stalwart. One excellent reason to spy on an ally, Gewirtz notes, is to confirm that the ally is really an ally. Allies sometimes become adversaries, so shifting signs must be monitored. Likewise, allies may be allies, but they always have their disagreements. What better way to prevent unpleasant surprises from an ally than by monitoring him?”

This response can also be heard in some quarters in Germany, as my colleague Philip Oltermann reports. "Everybody does it" has been one common reaction: the Germans shouldn't make such a fuss, as they were probably bugging other leaders' phones too.

So do German intelligence services spy on other national leaders? Even critics of the Bundesnachrichtendienst (BND - Germany’s foreign intelligence agency) aren't convinced. Erich Schmidt-Eenboom, an intelligence expert who was previously been himself the target of BND surveillance, told the Guardian it was "absurd" to assume that the German intelligence service spied on allied nations.

The BND is by no means staffed by saints, he argues. For one, it has been happy to share information gathered by US and UK agencies. But in its own activities it has traditionally concentrated on its core competence area of gathering information in countries where the German military is engaged, fighting international terrorism and tackling organised crime.

One simple reason for this focus was staffing issues. While the number of NSA employees is estimated at over 40,000, and the number of GCHQ employees at around 5,800, Germany is thought to employ only about 1,300 agents working in "technical surveillance".

If David Cameron was on the target list of the BND, said Schmidt-Eenboom, we would probably know about it: all the search terms for electronic surveillance by the German intelligence service had to be signed off by the German parliament.

According to an article in the Süddeutsche Zeitung, the US and EU member states were not systematically spied on by either Germany's secret service or intelligence agencies. This was a decision that has been backed by a succession of German governments. Andrea Nahles, general secretary of the Social Democrats, has in the last few days come on the record to say that her party had no plans to change this strategy.

Journalist Marcy Wheeler, aka Empty Wheel, notes Dianne Feinstein's comment that she wants “a total review of all intelligence programs … so that members of the Senate Intelligence Committee are fully informed as to what is actually being carried out by the intelligence community" and points out:

Today, her committee will nevertheless conduct a mark-up of her bill to not fix the spying targeted at Americans.

Umm, given that she just admitted she doesn’t know everything the NSA has been doing — and that she hasn’t been fully informed — don’t you think the comprehensive review should precede the new legislation?

A mark-up of a bill is the process by which a congressional committee debates and rewrites proposed legislation.

Reuters has also spoken to a "senior Obama administration official" who has suggested Obama may ban spying on friendly leaders. The news agency reports:

A senior administration official said the United States has made some individual changes in eavesdropping practices but as yet had not made across-the-broad policy changes such as ending intelligence collection that might be aimed at allies.

The official said the White House is considering a ban on intelligence collection aimed at allied leaders. A White House review that Obama ordered after NSA documents were made public by former contractor Edward Snowden is expected to be completed by year's end.

Mike Rogers, the Republican chair of the House intelligence committee, has spoken about the private talks between US legislators and a group of nine MEPs who travelled to Washington yesterday to discuss the spying scandal. Rogers said of the meeting:

It started to identify some of the differences that we have that we're going to have to bridge. That's a good thing. That's a good start and that's why we've pledged to take a delegation back to Brussels to follow up on this conversation.

It's important to understand that we're going to have to have a policy discussion that is bigger than any individual intelligence agency of either Europe or the United States.

Spain's prosecutor's office has said it has opened a preliminary inquiry into allegations that Spain was a target for surveillance by the NSA.

The office said the inquiry aimed to establish whether a crime was committed and if Spain should consider opening a formal investigation.

The inquiry may take weeks, AP reported.

The Associated Press predicts that Congress may get rid of the telephone surveillance programme but keep Prism, the internet surveillance programme.

Prism is meant to target foreigners and Americans who contact them, but the phone surveillance programme targets Americans, and US politicians are principally concerned with the idea the powers may have been used "improperly" to spy on Americans. AP writes:

Last month, documents released as part of a civil liberties lawsuit showed that NSA analysts for nearly three years accessed data on thousands of US phone numbers they shouldn't have, and then misrepresented their actions to a secret spy court to reauthorise the government's surveillance program. Separately, the NSA's inspector general reported 12 cases in which officers and analysts with access to the spying systems intentionally abused them, usually to monitor their lovers' phone calls.

AP also points out that NSA director General Keith Alexander, who will testify today, has said the phone surveillance programme has stopped "only one or two cases of terror activity out of about a dozen threats directed at the US".

This the news agency describes as "limited evidence showing why the telephone surveillance is important".

Ahead of today's House hearing, James Clapper, the US director of national intelligence declassified dozens of pages of top-secret documents in an apparent bid to show the NSA was acting legally when it gathered millions of Americans' phone records.

Clapper said he was following Obama's direction to make public as much information as possible about how US intelligence agencies spy under the Foreign Intelligence Surveillance Act.

The release of the documents yesterday focused on section 215 of the Patriot Act, which is used to allow the bulk collection of US phone records.

The Associated Press reports:

The documents support administration testimony that the NSA worked to operate within the law and fix errors when they or their systems overreached. One of the documents shows the NSA admitting to the House Intelligence Committee that one of its automated systems picked up too much telephone metadata. The February 2009 document indicates the problem was fixed.

Another set of documents shows the judges of the FISA court seemed satisfied with the NSA's cooperation. It says that in September 2009, the NSA advised the Senate Intelligence Committee about its continuing collection of Americans' phone records and described a series of demonstrations and briefings it conducted for three judges on the secretive U.S. spy court. The memorandum said the judges were "engaged throughout and asked questions, which were answered by the briefers and other subject matter experts," and said the judges appreciated the amount and quality of information the NSA provided.

It said that two days later, one of the judges, U.S. District Judge Reggie Walton, renewed the court's permission to resume collecting phone records.

The documents also included previously classified testimony from 2009 for the House Intelligence Committee by Michael Leiter, then head of the National Counterterrorism Center. He and other officials said collecting Americans' phone records helped indict Najibullah Zazi, who was accused in a previously disclosed 2009 terror plot to bomb the New York City subways.

Barack Obama will receive a classified dossier in the next two weeks that will lay out the consequences for US foreign relations of the National Security Agency's powerful surveillance apparatus and provide the White House with a raft of possible reforms, reports Paul Lewis in Washington.

The document is being drafted by a top-level group of experts appointed by the president to conduct an external review of US surveillance capabilities and the damage to public trust resulting from the Edward Snowden disclosures.

The review, parts of which will be declassified and released to the public, will be completed by mid-December. However, a senior administration official familiar with the process said a secret "interim report" will be shared with the president shortly.

The Associated Press has also been speaking to "a senior administration official" who claims the White House is considering ending its eavesdropping on friendly foreign leaders.

A final decision has not yet been made, the official told AP.

Barack Obama is “poised” to order the NSA to stop monitoring the communications of US allies, the New York Times writes.

Citing administration and congressional officials, the paper says the White House has informed congresswoman Dianne Feinstein – whose opinion of the NSA’s mass surveillance appears to have been swayed by the revelations of the monitoring of Angela Merkel’s phone – of its plans, which follow her statement that: “I do not believe the United States should be collecting phone calls or emails of friendly presidents and prime ministers.”

The White House said Monday evening that no final decision had been made on the monitoring of friendly foreign leaders. But the disclosure that it is moving to prohibit it signals a landmark shift for the NSA, which has had nearly unfettered powers to collect data on tens of millions of people around the world, from ordinary citizens to heads of state, including the leaders of Brazil and Mexico.

It is also likely to prompt a fierce debate on what constitutes an American ally. Prohibiting eavesdropping on Ms. Merkel’s phone is an easier judgment than, for example, collecting intelligence on the military-backed leaders in Egypt.

The NYT adds:

The administration will reserve the right to continue collecting intelligence in friendly countries that pertains to criminal activity, potential terrorist threats and the proliferation of unconventional weapons, according to several officials. It also appeared to be leaving itself room in the case of a foreign leader of an ally who turned hostile or whose actions posed a threat to the United States.

The paper also quotes Dennis C Blair, Obama’s first director of national intelligence, who had little sympathy for foreign leaders’ complaints:

If any foreign leader is talking on a cellphone or communicating on unclassified email, what the US might learn is the least of their problems.

Welcome to our hub for all Edward Snowden, NSA and GCHQ-related developments around the world, as controversy over revelations leaked by the whistleblower continue to make headlines. As arguments rage over how much of our day to day life should be monitored in the name of security, we'll be tracking the growing global debate about privacy in the digital age. We'd like to know what you think about the whole NSA story, what you're worried about – and any new areas you'd like to read more about.

Good morning. The focus turns to Washington today, where the two most senior intelligence officials in the US are to testify before the House of Representatives intelligence committee.

Both James Clapper, the director of national intelligence, and General Keith Alexander, the director of the NSA, are expected to be questioned on why they appears not to have informed either the White House or congressional oversight committees about the spying activities directed at foreign leaders revealed over the past weeks.

Clapper is also under fire for misleading Congress on bulk domestic collection. Alexander caused controversy last week when he mused that “we ought to come up with a way of stopping” reporters’ stories about the NSA.

Also today James Sensenbrenner, the Republican author of the 2001 Patriot Act, the act which authorised the bulk collection of phone records, will introduce a bill called the United and Strengthening America by Fulfilling Rights and Ending Eavesdropping, Dragnet Collection, and Online Monitoring Act or USA Freedom Act, which would ban warrantless bulk phone metadata collection and prevent the NSA from querying its foreign communications databases to identify information on Americans. Patrick Leahy, the Democratic chairman of the Senate judiciary committee, will simultaneously introduce a counterpart to the bill in the upper house.

The moves come as:

 The chair of the Senate intelligence committee, who has been a loyal defender of the National Security Agency, dramatically broke ranks, saying she was "totally opposed" to the US spying on allies and demanding a total review of all surveillance programmes.

California Democratic senator Dianne Feinstein strongly criticised the NSA's monitoring of the calls of friendly world leaders such as German chancellor Angela Merkel.

Feinstein, who has steadfastly defended the NSA's mass surveillance programs, added that both Barack Obama and members of her committee, which is supposed to received classified briefings, had been kept in the dark about operations to target foreign leaders.

"It is abundantly clear that a total review of all intelligence programs is necessary so that members of the Senate Intelligence Committee are fully informed as to what is actually being carried out by the intelligence community," Feinstein said in a statement to reporters.

"Unlike NSA's collection of phone records under a court order, it is clear to me that certain surveillance activities have been in effect for more than a decade and that the Senate Intelligence Committee was not satisfactorily informed.

"With respect to NSA collection of intelligence on leaders of US allies – including France, Spain, Mexico and Germany – let me state unequivocally: I am totally opposed," she said.

Obama told ABC News that he would not discuss classified information but accepted that security operations were being reassessed to ensure proper oversight of the NSA's technical abilities. He said:

The national security operations, generally, have one purpose and that is to make sure the American people are safe and that I'm making good decisions. I'm the final user of all the intelligence that they gather. But they're involved in a whole wide range of issues. We give them policy direction. But what we've seen over the last several years is their capacities continue to develop and expand, and that's why I'm initiating now a review to make sure that what they're able to do doesn't necessarily mean what they should be doing.

 The White House's chief spokesman, Jay Carney, said the administration "acknowledged the tensions" caused by Snowden's disclosures, saying:

The president clearly feels strongly about making sure we are not just collecting information because we can, but because we should. We recognise there needs to be additional constraints on how we gather and use intelligence.

 A new disclosure from the Electronic Frontier Foundation added to the NSA’s woes by suggesting that it began testing means to gather location data on cellphones inside the US before informing the secret surveillance court that oversees it.

The Spanish government warned of a potential breakdown of trust with the US following reports that the National Security Agency monitored more than 60m phone calls in Spain in the space of one month.

In Britain, David Cameron called on the Guardian and other newspapers to show "social responsibility" in the reporting of the leaked NSA files to avoid high court injunctions or the use of D notices to prevent the publication of information that could damage national security.

You can catch up with yesterday’s blog here.

We’ll have all this and more throughout the day.