In reality, it is hard to distinguish email metadata from email content. Distinctions that might make sense for telephone conversations and data about those conversations do not always hold for online communications.
"The calls you make can reveal a lot, but now that so much of our lives are mediated by the internet, your IP [internet protocol] logs are really a real-time map of your brain: what are you reading about, what are you curious about, what personal ad are you responding to (with a dedicated email linked to that specific ad), what online discussions are you participating in, and how often?" said Julian Sanchez of the Cato Institute.
"Seeing your IP logs -- and especially feeding them through sophisticated analytic tools -- is a way of getting inside your head that's in many ways on par with reading your diary," Sanchez added.
The purpose of this internet metadata collection program is detailed in the full classified March 2009 draft report prepared by the NSA's inspector general (IG).
One function of this internet record collection is what is commonly referred to as "data mining," and which the NSA calls "contact chaining." The agency "analyzed networks with two degrees of separation (two hops) from the target," the report says. In other words, the NSA studied the online records of people who communicated with people who communicated with targeted individuals.
Contact chaining was considered off-limits inside the NSA before 9/11. In the 1990s, according to the draft IG report, the idea was nixed when the Justice Department "told NSA that the proposal fell within one of the Fisa definitions of electronic surveillance and, therefore, was not permissible when applied to metadata associated with presumed US persons."
How the US government came to collect Americans' email recordsThe collection of email metadata on Americans began in late 2001, under a top-secret NSA program started shortly after 9/11, according to the documents. Known as Stellar Wind, the program initially did not rely on the authority of any court -- and initially restricted the NSA from analyzing records of emails between communicants wholly inside the US.
"NSA was authorized to acquire telephony and internet metadata for communications with at least one communicant outside the United States or for which no communicant was known to be a citizen of the United States," the draft report states.
George W Bush briefly "discontinued" that bulk internet metadata collection, involving Americans, after a dramatic rebellion in March 2004 by senior figures at the Justice Department and FBI, as the Washington Post first reported. One of the leaders of that rebellion was deputy attorney general James Comey, whom Barack Obama nominated last week to run the FBI.
But Comey's act of defiance did not end the IP metadata collection, the documents reveal. It simply brought it under a newly created legal framework.
As soon as the NSA lost the blessing under the president's directive for collecting bulk internet metadata, the NSA IG report reads, "DoJ [the Department of Justice] and NSA immediately began efforts to recreate this authority."
The DoJ quickly convinced the Fisa court to authorize ongoing bulk collection of email metadata records. On 14 July 2004, barely two months after Bush stopped the collection, Fisa court chief judge Collen Kollar-Kotelly legally blessed it under a new order -- the first time the surveillance court exercised its authority over a two-and-a-half-year-old surveillance program.
Kollar-Kotelly's order "essentially gave NSA the same authority to collect bulk internet metadata that it had under the PSP [Bush's program], except that it specified the data links from which NSA could collect, and it limited the number of people that could access the data."
How NSA gained more power to study Americans' online habitsThe Bush email metadata program had restrictions on the scope of the bulk email records the NSA could analyze. Those restrictions are detailed in a legal memorandum written on 27 November 2007, by assistant attorney general Kenneth Weinstein to his new boss, attorney general Michael Mukasey, who had taken office just a few weeks earlier.
The purpose of that memorandum was to advise Mukasey of the Pentagon's view that these restrictions were excessive, and to obtain permission for the NSA to expand its "contact chains" deeper into Americans' email records. The agency, the memo noted, already had "in its databases a large amount of communications metadata associated with persons in the United States."
But, Wainstein continued, "NSA's present practice is to 'stop' when a chain hits a telephone number or [internet] address believed to be used by a United States person."
(Note: You can view every article as one long page if you sign up as an Advocate Member, or higher).