Newly declassified documents released by the Obama Administration on Wednesday show that the National Security Agency (NSA) collected phone and Internet data on tens of thousands of Americans before it discovered and reported the issue to a secret court that oversees the program.
The documents were posted Wednesday by the Office of the Director of National Intelligence on a new Tumblr page called IC on the Record. The documents include two heavily redacted opinions by the secret Foreign Intelligence Surveillance Court (FISC), another file from 2011 describing the NSA's procedures for targeting and minimizing the data it collects, and several other documents.
In an accompanying statement, Director of National Intelligence James Clapper said the goal in releasing the documents is to give the U.S. public more insight into the "lawful foreign surveillance activities" carried out by the nation's intelligence community.
"In addition to comprehensive explanations of the authorities under which the Intelligence Community conducts foreign surveillance, the site will address methods of collection, use of collected data, and oversight and compliance," Clapper said.
President Barack Obama in June called on Clapper to release more details of the NSA's surveillance after widespread concerns were stoked by former NSA contract worker Edward Snowden's leaks to the media.
The most interesting document released today is a FISC opinion from two years ago that shows the NSA collected phone and Internet data on potentially hundreds of thousands of Americans with no link to terrorism before the court ended the practice in October 2011. The data collection apparently started before the court was established in 2008; FISC ruled the practice unconstitutional after being told about it by the NSA.
The declassified court opinion shows that the NSA's collection of wholly domestic communications happened because it was unable to properly filter the massive volumes of data it was collecting online. According to the court's description of the issue, the NSA's Internet collection devices at the time were incapable of accurately distinguishing between communications that originated outside the U.S with domestic communications.
An NSA analysis showed that the agency might have acquired between 46,000 and 56,000 domestic communications each year for several years, the court said.
The 85-page opinion written by the FSIC Chief Judge John Bates reflects frustration with the government's apparently inconsistent descriptions of the nature and scope of the NSA data collection practices.
The FISC was established in 2008 to oversee the data collection activities by U.S. intelligence activities carried out under the Foreign Intelligence Surveillance Act (FISA). Among other things, the secret court is responsible for reviewing and approving NSA requests for collecting data from Internet and phone service providers.
Details about the NSA's inadvertent domestic data collection emerged during one such request in 2011.
Bates said that the court had previously approved NSA requests based on the agency's description of its data collection practices, but noted with frustration that the description kept changing. The court initially understood that the NSA's technical measures would prevent the acquisition of domestic content, but later discovered that those measures were inadequate, Bates said in his opinion.
"The Court is troubled that the government's revelations regarding NSA's acquisition of Internet transactions marks the third instance in less than three years in which the government has disclosed a substantial misrepresentation regarding the scope of a major collection program," the opinion notes.
A footnote pointed to similar inconsistencies in the NSA's description of its phone metadata collection activities. "Contrary to the government's repeated assurances, NSA had been routinely running queries of the metadata using querying terms that did not meet the standard for querying." The querying standard had been "so frequently and systematically violated" that it never worked effectively, the court noted.
Such facts fundamentally alter the statutory and constitutional basis for the data collection, Bates said in the opinion.
In a statement referencing the October 2011 FISA opinion, the Office of the Director of National Intelligence said the problems stemmed from an incomplete understanding of data collection technology. "These incidents were due to a variety of factors, including gaps in technical understanding among various NSA components about how certain aspects of the complex architecture supporting the programs functioned."
Those gaps led to an unintended misrepresentation of the manner in which the NSA collected data, the statement said. It noted that after the issue was discovered, steps were take to minimize data collection and ensure that it was more targeted.
Jaikumar Vijayan covers data security and privacy issues, financial services security and e-voting for Computerworld. Follow Jaikumar on Twitter at @jaivijayan or subscribe to Jaikumar's RSS feed. His e-mail address is jvijayan@computerworld.com.
See more by Jaikumar Vijayan on Computerworld.com.
Read more about privacy in Computerworld's Privacy Topic Center.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.