Reality check: intelligence agencies have been using social network analysis since the 1990s
There is a big hubbub today over a New York Times article N.S.A. Gathers Data on Social Connections of U.S. Citizens.
I fail to understand why this is big news, since US intelligence agencies have been using social network analysis (SNA) for domestic purposes since the 1990s, and likely even before that. The only issue here is that the NSA is tasked with non-domestic surveillance, so is not supposed to gather data on US citizens. However other US agencies that cover domestic intelligence have long been using SNA.
Certainly recent revelations suggest the NSA appears to have data surveillance capabilities that exceed those of US domestic intelligence agencies, but there is no good reason to imagine the CIA, among others, doesn’t have access to equally good data to seed its social network and other analysis.
I have been focused on networks since long before I wrote Living Networks in 2002. In the July 22, 1997 issue of The Bulletin (at the time Australia’s equivalent of Newsweek) included an article I wrote titled Beware! Netmap may be watching, which described how an Australian software package called Netmap was being used by police and intelligence around the world, taking examples of the identification of insider trading and a serial murderer. I wrote:
“For nearly 10 years, Netmap has been used primarily in high-level security and intelligence analysis. In Australia clients include the NSW Police… and the Australia Tax Office, while in the United States, several secretive government agencies use the software.”
I was never told which US agencies they were, but it is not hard to take a guess.
The 9/11 attacks brought attention to bear on new analysis tools, with Valdis Krebs showing that SNA could have helped identified the terrorists before the event. SNA has since used in a range of documented intelligence efforts including finding Saddam Hussein after the Iraq invasion.
CIA’s investment arm In-Q-Tel has invested in FMS to access tools that use SNA and has invested in social network monitoring company Visible Technologies, among other related companies.
As it happens, another Australian company, Distillery Software, has been operating since 1997 to “Capture and Analyse Social Networks in a Dedicated Intelligence Framework”, noting on its website that it “has a long history of providing intelligence management and intelligence-centric solutions to national security customers”.
The point here is that network analysis is a blindingly obvious approach to large data set management. If intelligence agencies have large data sets, it would be a massive failure on their part not to be using network analysis.
This means that governments have long been gathering and uncovering extraordinarily rich information about our relationships: personal, professional, and beyond. As I recently discussed in video conversations with Gerd Leonhard on The future of privacy in a world of government data gathering and The implications of Big Data, it is possible for them to know us in some ways better than we know ourselves.
As I wrote in my 1997 article:
“Unlike other “data mining” software used to extract information from large databases, Netmap does not require any set rules or questions from the user. Once the data has been entered, it highlights the relationships that stand out. “You wouldn’t know what to ask to find these,”” says [Netmap founder] Galloway. “You can approach your searches with a completely open mind rather than preconceptions.”
In other words, anomalous (or deviant) relationships are visible even without actively looking for them.
The uncovering of the social graph of the citizens of the US and many other countries has been long underway. While many are shocked to discover the power of these insights when applied to ordinary citizens, intelligence agencies have been using network analysis for domestic purposes since the 1990s. It’s a good thing that this is finally becoming well understood.
