SDI Blog from 2000 to 2019

In the first of a series of posts on the upcoming film A Good American, we look at the extraordinary surveillance program that caused NSA analyst-turned-whistleblower Bill Binney so many problems. What, exactly, was ThinThread?

Before Snowden, another man blew the whistle on bulk surveillance in the US. His name is William Binney, and until his abrupt resignation in 2001 he was a senior crypto-analyst for the US Government’s National Security Agency. He has been described as a mathematical genius and one of the agency’s best analysts ever.

The story of what happened to Binney and his colleagues when they spoke out is extraordinary. Told here in Friedrich Moser’s film A Good American, the story is about surveillance and government power, but not in the usual way. Because there are two narratives that dominate this issue: One is that existing government surveillance is terrifying, dystopian and out of control, while another is broadly summed up by the ‘nothing to hide, nothing to fear’ mantra. Neither are subtle arguments.

Protesting the government’s use of unwarranted tapping earned Bill Binney years of persecution by the administration. His home was raided by the FBI, at gunpoint. But Binney is not a radical or an activist; if anything, he comes across as a traditionalist, a believer in his country and government, and someone who greatly enjoyed his 30-year career at the NSA.

This is what makes the film a more ambiguous exploration of surveillance than you might find elsewhere. At the heart of the story is the software which Binney helped design, and which eventually landed him in so much trouble. This was a data collection system called ThinThread which, he maintains, proved that security need not come at the expense of privacy. The two can co-exist quite comfortably. Unfortunately, a potent mix of corruption, greed and bombast at the NSA meant that Binney’s was a message nobody wanted to hear, and the lengths to which the top brass went to cover their tracks are really what makes this case so incredible.

What was this programme that got Binney into so much trouble?

ThinThread was data collection software, but smart. Instead of gathering up and storing unwieldy amounts of information, ThinThread used the data to build a diagram of social relationships. Binney describes it as a "big-ass graph". A global map of nodes, intricately connected in a vast network, built from scraping digital communications and indexing it around metadata. Illustrated graphically in Moser’s film, it's a strangely beautiful sight, this complicated sphere of softly glowing data points. It’s pleasingly simple to understand, too: when one node is lifted up from the surface, those connected to it come along, as though you were plucking at a fabric surface.

We’re becoming increasingly familiar with graphs like this mapping our communications and relationships. Take, for instance, Newsdirect’s ‘Scottish Twitter’ graph. If you’re part of this particular scene, you can spend a fascinating few hours exploring the relationships, hierarchies, spheres of influence, and unexpected links mapped out here. The landscape of Scottish media and politics is laid bare in all its colour-coded, interactive glory (image credit: Lawrie Scott-McFarlane/Newsdirect).

The difference, of course, is that this relates to Twitter – something we signed up for, a profile we control. The graph shows our relationships to those with whom we discuss politics publicly and deliberately. Our ‘nodes’ on this graph do not lead to our secret lovers, or best friends, or parents.

But that’s what ThinThread can do. For every human, a node. Billions of nodes, their lives – personal and private – mapped out. The basic idea came from Binney’s early days in military intelligence, in the 1960s. Sent to Turkey to join a small band of Soviet-watchers, he managed to predict with extraordinary accuracy the movements of Soviet troops - including the invasion of Czechoslovakia in 1968, and Afghanistan in 1979. “I was off by an hour”, he says wryly. “I just couldn’t understand why a lot of people couldn’t see what I could see.” 

What Binney could see was the bigger picture – the patterns of behaviour that humans inevitably create, and which, analysed intelligently, give far more insight than a few tapped radio messages. When digitisation began, it was clear that the NSA was being left by the wayside. ThinThread was Binney’s solution: gathering a large amount of digital communications data, but dropping the vast majority of it quickly, and anonymizing any data belonging to US citizens in order to maintain privacy and keep within the law. It was, like much of Binney’s work at the NSA, meant to be directed abroad, to detect foreign military and terrorist threats.

Working on the fringes of the agency, in a little-known signals intelligence research centre, Binney worked with Ed Loomis to develop the program. Loomis, who would also become a whistleblower years later, created the metadata that would enable Binney’s traffic analysis. He explains: “You can extract an awful lot about a person’s life just by looking at metadata. Today when you take all the metadata being popped around the world – GPS, cellphones, transponders, a credit card inserted into an ATM, a cash register… all that data gets all brought together and it all traces back to you, the individual. It doesn’t matter what you said in an email, what website you’re going to, what you downloaded – what matters is what pattern your digital behaviour is taking, and what fingerprints that leaves of what you’ve been up to.”

The potential for ThinThread to be a dangerously invasive and powerful tool was recognised early on, and the team built in encryption to protect the identities of innocent US citizens. Anyone outwith the “zone of suspicion”, as they term it, would not be traceable in the data, and their sessions would quickly be deleted from the archive.  

ThinThread worked incredibly well on trial runs, detecting threats and suspects more efficiently and quickly than had previously been thought possible. It solved the problem of data overload: picking the needles from the haystack instead of creating more hay. By 2000 a handful of research sites were running the program, picking up data on Asia. Binney wanted to expand – by now there was a clear and growing terrorist threat and, he thought, ThinThread was by far the best method of identifying and preventing attacks. He proposed 18 potential targets in early 2001, eager to deploy this apparently successful program. But it was never used.

We’ll explore why not, and what the consequences were, in next week’s post.

A Good American in cinemas

Moving Docs is bringing A Good American to screens across Europe. Next screenings from 1 September in Spain and from 15 September in the UK, together with Take One Action and The Guardian Live.

This post was written by Jen Stout for Film & Campaign on behalf of Scottish Documentary Institute, a Moving Docs partner. Images and quotes are taken from A Good American unless attributed otherwise.