William Binney: The Way Ahead in Big Data Exploitation — Lawful Electronic Surveillance

Advanced Cyber/IO
William Binney


When contemplating next steps in the discussion about the collection and monitoring of electronic data (both content and metadata), we are obligated to look to two areas of knowledge for guidance – the law and technology. How can these two areas come together to provide an improved, yet lawful ‘way ahead’?  The good news is that a roadmap to a solution already exists. It just needs to be implemented.

From law, we draw guidance from the Fourth Amendment to the United States Constitution, in particular the clause that requires that “probable cause” must be demonstrated before the personal information of a person can be lawfully searched. From technology we will draw from a robust, knowledge-centric capability to manage large amounts of information such that the identifying information of any innocent entity – innocent person, place, or thing – need not ever be revealed to law enforcement or national security authorities in the absence of fact-based suspicion of terrorism or other illegal activities meeting probable cause criteria approved by a duly constituted court of law.

In a word, it is eminently possible to protect the identities of the innocent while at the same time, ensuring the national security.  There is no balance – neither security, nor privacy must be sacrificed.


Whether data identifying an entity is captured or otherwise copied and stored by commercial entities or the government for surveillance purposes as part of a process to ensure national security, prevent crime, or to arrest those responsible for a crime, it must be immediately encrypted.  Such data must remain encrypted until such time probable cause criteria under the Fourth Amendment to the Constitution are met as determined by a judge in a court of law.

The encryption used for such purposes will be among the strongest available and may be commercial, or it may be developed by the government.  But in no circumstance will decryption algorithms be available to the Executive Branch of Government controlling intelligence or law enforcement agencies.  It will be managed and stored with small organizations representing either or both the Judicial and Legislative branches of government.  The intent is to ensure all three branches of government are responsible for the proper implementation and integrity of the process and software to ensure that the integrity of the innocent is protected to the maximum degree possible.  Should some argue that encryption of the identifying information associated with a particular entity or group of entities will interfere with the effective or timely analysis of data, be assured such is not the case.

Since all activities in data can be represented in a relational graph, with dots representing entities and lines between dots representing relationships between entities, software can represent entities – either those that are highly suspicious or those that are innocent as dots with or without the presence of true identifying information Those that are innocent will be assigned a randomly generated, but unique value for processing and reference purposes only, until evidence is discovered and submitted to the court demonstrating probable cause has been established and allowing the true identity of the entity to be revealed. In this way, both suspected entities and innocent entities can be shown in relationships across all sources of data without violating anyone’s Fourth Amendment privacy rights.

Such a graph of relationships might look as it does in Figure 1 of the Appendix located on the final page of this document (inserted below).


These analytic techniques are designed around demonstrated behavior that forms a basis of probable cause to examine individuals to determine if they are involved in criminal or terrorist activity.  This is not proof that they are involved in those activities; it is behavior that justifies considering them in order to rule them in or out of such activity. In this larger process, the analyst strives to move his approach ever closer to a deductive approach.

 Deductive Approach

  • Build social networks based on relationships in metadata such as phone numbers, email addresses, credit cards, money transfers, travel arrangements, and the like.
  • Isolate new members of these communities.
  • Extend the zone of suspicion to two degrees/hops from known criminal or terrorist entities, but
  • Exclude businesses and departments of governments to avoid including massive numbers of innocent individuals in the zone of suspicion.

For example, if you use Google, then you are two hops from all those using Google which eventually would include billions of entities around the world.

To add privacy to the social networks, all metadata identifiers can be uniquely encrypted to retain the network, but hide the identity of these individuals.

  • Alert when additional participants are added to these social networks.
  • Use latent semantic indexing to help in determining participation in these activities.
  • Calculate the probability of participation.

Inductive Approach

  • Monitor sites that advocate violence against the west, pedophile or other criminal activity

Those visiting these types of sites should be looked at to insure they are not becoming radicalized or active in criminal or terrorist activity.  Once resolved, they would be either excluded or targeted.

Abductive Approach

  • Look for communications links (no social network – three or more participants) and use GPS or other metadata to locate entities.

If in geographical areas of interest, they should be looked at to determine participation in criminal or terrorist activity.

Examine social networks that evidence a geographical distribution in or among countries associated with terror activity, drug smuggling, or other criminal activity.


Perhaps more than a few would argue that the court approval process under the Fourth Amendment, especially in matters of national security, could thwart the opportunity to interdict nefarious activities posing a threat to human life. However, this need not be the case, for it is eminently possible to capture many scenarios – criteria if you will – defined by courts but implemented in software as business rules that could automatically decide whether probable cause criteria had been met. If yes, then an automated response could be sent to the submitter’s information system, allowing the identifying information associated with the entity or entities involved to be revealed to the relevant intelligence analysts or investigators.  Such a transaction between an Executive Branch agency and the Judiciary could be accomplished in a matter of seconds.

More subtle, perhaps more complicated situations could be addressed through an “electronic court” made possible by the use of collaboration software (such as Go to Meeting, a popular Microsoft product) over a secure network connection between the court and analysts or investigators, allowing for the presentation of multimedia evidence in the form of relationship graphs, textual content, maps, whatever may be necessary to demonstrate to the court that available evidence meets probable cause criteria under the Fourth Amendment to the Constitution. This process could take just minutes in order to arrive at a verdict.

In summary, while FISA or other court judges serving under such conditions may need to be “on call”, the litigation process does not inherently or necessarily need to be long in terms of potential degradation in responding to some national security needs of the moment.


The relational graphic display depicting the relationships among entities engaged in an activity of interest is not new; indeed, its roots are found in the pre-World War I application of traffic analysis techniques. As shown in Figure 1 below, the elegance of such a display is found in the simplicity and clarity of the depicted relationships.  Because the graphic display leverages automation to ensure relationships are continuously updated for analysis, it allows the viewer to immediately grasp the span of an activity under investigation while allowing the user to focus on a single entity – or dot – as desired. In addition, such a depiction can be annotated with amplifying information that can define hierarchies of control or descriptions of roles and responsibilities for each of the member entities engaged in an Activity Of Interest (AOI).

The power of the display is found in the ability to focus analysis within two degrees (two hops) of any entity, while enabling the analyst to access in-depth virtually any information about the displayed activity as a whole, or the user can left-mouse or right-mouse on any entity to view metadata or content associated with it, such as a location, an address, an attachment to an email, and the like – any information stored in a knowledge base about the entity. In addition, the analyst may want to choose from a list of optional tools to use, such as a timeline to view changes in activities over a specified period of time. Finally, these capabilities make the relational graphic display the perfect medium for demonstrating probable cause to a court of law.

This description is but a brief summary of the usefulness of the relational graphic display. There are many other processes that can be brought to bear on entities and activities from the standpoint of advanced analytics used to build profiles of activities which then can be used as templates to ferret out similar activities of interest despite the fact than no names of entities are known.  This approach greatly increases the probability of discovering new, previously unknown threats in massive volumes of data, while maintaining the privacy of the innocent.

DOC (5 Pages): William Binney The Way Ahead March 2017

Published with the permission of the author.

Robert David STEELE Vivas

ROBERT STEELE: I treasure my friendship and professional relationship with William Binney, the most honest former National Security Agency (NSA) executive I know. He and I together represent “intelligence done right,” and I pray that we might eventually be able to restore ethics and integrity to the craft of intelligence. This is his manifesto, one that the European Union has agreed is appropriate to balancing privacy for the individual with the legitimate needs of state.

See Especially:

Robert Steele. “Healing the Self & Healing the World: The Open Source Way,” Defence and Intelligence Norway, September 13, 2017.

Steele, Robert with Lance Schuttler, “CIA Spy Speaks Out After 30 Years of Censorship – Is It Time to Listen?,The Mind Unleashed, January 31, 2017.

Steele, Robert, “Intelligence for the President–AND Everyone Else,” CounterPunch, March 1, 2009.

Steele, Robert. “SPECIAL FEATURE: Creating a Smart Nation–Strategy, Policy, Intelligence, and Information,”  Government Information Quarterly, pp. 159-173.

See Also:

Big Data @ Phi Beta Iota

Opt in for free daily update from this free blog. Separately The Steele Report ($11/mo) offers weekly text report and live webinar exclusive to paid subscribers, who can also ask questions of Robert. Or donate to ask questions directly of Robert.