In 1986-1989, building on the work of prior pioneers, a very small band of committed intelligence professionals defined the needs for the 21st century. Across many years, over 750 international speakers and 7,500 mid-career professionals came together to explore and share alternatives.
A quarter century later the core needs remain unmet. This is the story of what was, what is and what could be.
Updated 24 March 2014.
NOTE: Now that the best of the best from the conferences have been displayed, we will begin to weave in a selection of books and articles and other references that in our view represent the most transformative thinking (which is to say, the 3% no one in power listens to). Anyone wishing to nominate an entry (or multiple entries) is invited to email robert.david.steele.vivas [at] gmail [dot] com.
2013 Story Board: Public Intelligence in the Public Interest [OSE + M4IS2 + Panarchy = World Brain & Global Game = Prosperous World at Peace] – this is the short eight slide version of the “big picture.”
An analyst — however superior in education, training, and diligence, cannot overcome a system that is dysfunctional in its source acquisition and processing, its moral and intellectual models for studying system dynamics and the relationships among causes and effects, and its provision to the analyst of access to all relevant human sources irrespective of their provenance. An analyst is also made redundant — worthless — in the absence of a decision-maker to whom the analysis has value in developing strategy, deciding on acquisition or policy, or in pursuing operations that support strategy and policy with what has been acquired. The system the analyst serves has to want ethical evidence-based decision support. To emphasize this point, below we show the “shredded spear” [“broken” does not do justice to how hosed the system is] that leaves the U.S. Special Operations Command (USSOCOM) — perhaps our most important defense capability, it is also our least effective in relation to what it could do if the spear were whole.
We were naive — for a quarter century. Investments have not been made in the obvious — open sources, analytic tools, and mulitnational sharing — for the simple (but totally unethical) reason that they are not an expensive enough challenge to feed the 50% corruption network. Surprisingly, this has applied to the private sector as well — corporations have chosen to be unethical and rely on fraud (banks) or the externalizing of cost (everyone else), rather than embrace the craft of intelligence (decision-support) and get a grip on true cost economics in a whole systems context.
Below is an illustrated timeline of what we knew, when, and the prospects for transformation outside “the system.” Crisis Mappers is now worth more to the public than NSA. What happens next? This is one narrative.
Intelligence Transformation Timeline
General Al Gray, USMC, then Commandant of the Marine Corps (CMC) created the Marine Corps Intelligence Center (MCIC) at the same time that he created the Marine Corps University (MCU). He was guided by the reality that the larger serices–the Army, Navy, and Air Force–devoted all of their attention to training, equipping, and organizing for “Big War” against the Soviet Union and China. He also recognized that there were many strategic assumptions in the Planning, Programming, and Budgeting System (PPBS), including the assumption, by the large services, that they would have lead times for deploying and employing forces abroad.
1989 Webb (US) Computer Aided Tools for the Analysis of Science & Technology (CATALYST)
Neither CIA nor the other agencies were willing to pool requirements, funds, or outcomes.
1989 USMC to Joint National Intelligence Development Staff*
USMC JNIDS IV won, but the Admiral in charge said “we are a Navy shop, we will do a Navy problem.” This is emblematic of the corruption that afflicts every level of decision-making across every mission area.
1989 Brunn, USMC to the Defense Imagery and Mapping Agency annual meeting:
“I don’t care how much data you provide, if I cannot plot it on a map it is useless to me.
1989 Gray, USMC, Commandant of the Marine Corps, on Emerging Threats
He also called for being serious about open source intelligence, focusing on non-state revolutions, and using intelligence to make the case for Whole of Government “peaceful preventive measures.”
1990 Steele #2 USMC Intelligence Civilian, 6 Challenges for Intelligence in the 1990’s
USD1.2 Trillion later, the current US IC fails across all six needs. We now realize the US IC just moves money, it does not “do” decision-support. Open Source Intelligence (OSINT) as a movement failed for two reasons: it was not an expensive enough solution, and it threatened the endemic fraud, waste, and abuse that is expensive.
1991 Leavitt (US) The Analysit and Technology–2000 Survey of Needed Research & Development
There has been no lack of precise knowledge of what is needed — what has been lacking is integrity in leadership.
1991 Steele (US) Applying the “New Paradigm” to Avoid Strategic Intelligence Failures in the Future
We knew everything needed to be fixed to avoid 9/11 (a counterintelligence failure against domestic enemies) by 1991-1992.
1992 Dedijer (SE) OSS, Intelligence, Secrecy, and Open Sources [no link]
The father of international commercial intelligence. “Know who knows.”
1992 Shepard (US) Intelligence Analysis in the Year 2002 — A Concept of Operations
“We will have to remain near the leading edge of information manipulation, if we are to give the US Government a strategic advantage in the ‘Information Age.'”
In fact, when you look at the Department of Defense, you discover that we have inherited an information structure that reflects an organizational design of the past. The organizational design provided for information systems that wer built contract by contract. The Department of Defense has ten thousand integrated systems except none of them integrate with one another. … The information assets that we have in place are very labor intensive, and they do not have adequate protection for security. [In addition] the redundancy ratio is generally about 1 in 15; in other words, there are basically about 15 travel voucher systems [with] no fundamental difference; in the financial area alone we find that of 90 systems, only seven, actually, are necessary to get the job done. ….somewhere here, in the public sector, there lies anywhere from 50-70% operating unit cost productivity-gain opportunity.
The moral of the story is simple. If information access is controlled, development is slow and the economy suffers. If information access is open, development and the economy boom. In the interests of national security we need to use this knowledge to bring prosperity to the rest of the world before all its people become immigrants, refugees, or pensioners of the West.
1993 Beaumard (FR) Learned Nations and National Competitive Advantage Through Knowledge Strategies
At the age of 27 Dr. Professor Beaumard was the youngest leader for French strategic planning in modern history. His mantra: “Intelligent coordination (of information-sharing) plus intelligent (legal ethical collection) intrusiveness equals national competitive advantage.”
1993 Herring (US) Six Roles of Intelligence in Formulating Strategy
Intelligence demands integrity not only in the producer, but in the consumer. Of particular importance is the willingness of the consumer to have their assumptions challenged, their weaknesses identified, and the sustainability of their strategy questioned.
1993 Kumoon (JP), From Wealth (and Violence) to Wisdom (and Co-Evolution)
The next step in human evolution demands recognition of information rights as a major category of human rights.
To worry about war or anti-war in the future without rethinking intelligence and seeing how it fits into the concept of knowledge strategy is an exercise in futility. The restructuring and reconceptualization of intelligence–and military intelligence as part of it–is a step toward the formulation of knowledge strategies needed either to fight or forestall the wars of tomorrow.
1994 Fuchs (CH) Director General of the International Committee of the Red Cross on Handling Information in Humanitarian Operations within Armed Conflicts
Neither governments nor the media can be relied upon for complete coverage of any conflict situation, or for reliable coverage of all conflict situations — “the media has an attention span of one conflict at a time.”
1994 Steele (US) Private Enterprise Intelligence – It’s Potential Contribution to National Security
The hierarchical compartmented approach to sources is extremely inefficient, expensive, and least likely to connect a consumer with a question with the expert that can answer that question. The analyst is a mid-wife, a manager of sources, not the oracle.
1995 Bjore (SE) Six Years of Open Source Information Lessons Learned
Analysts need cyber-sleuths. The only persistent URL you can rely on is your own–copy it in full text.
1995 Markowitz (US) Community Open Source Program Office Briefing to OSS ’95
In 1995 both the immaturity of technology and the deep security concerns of the classified world prevented direct connections to the Internet. It was in this context that from 1992 to 1995 there were two competing schools of thought — the first, led by Robert Steele, advocated a separate Open Source Center with complete access to everything, will its finished products being air gapped (digitally scanned) to the high-side. The second, led by MITRE, mis-led to the IC and said that six open source products, among them LEXIS-NEXIS and Predicasts, would be sufficient to meet the IC needs through an internal Open Source Intelligence System (OSIS). This mis-step set US IC access to open sources in all languages and mediums back by at least a decade.
1995 Schnittker (CA) Use of Open Sources in the Criminal Intelligence Program of the Royal Canadian Mounted Police
Open source intelligence is an essential — an integral — part of human intelligence and the human intelligence cycle of spotting, assessing, developing, recruiting, handling, and terminating — and it is hugely important for counterintelligence against both domestic and foreign actors.
We are a smart people today, but a dumb nation Our national security and our national attractiveness as a site for international investment which permiu our citizens to prosper arc both at risk. We have no alternative but to completely redefine the role of government to emphasize its responsibility for the nurturing of our national information commons, and to redefine national intelligence so as to create a Virtual Intelligence Community in which every citizen is a collector, producer, and consumer of intelligence. To do this, we must have a National Information Strategy.
1996 Aspin-Brown Commission recommends open sources be top priority for funding and DCI attention
About this time the CIA, viscerally opposed to addressing this challenge, began speaking of “open sores.” The testimony would not have occurred without the integrity of Britt Snider, Loch Johnson, and Phyllis Provost-McNeil.
1996 Roger (AU) Open Source Strategies for Law Enforcement
The concept of global time travel as presented remains one of the most extraordinary and useful concepts in open source intelligence.
1997 Botbol (FR) The Open Source Revolution Early Failures and Future Prospects
Nations and organizations still need secrets, but they now must take a back seat to multinational openness and sharing.
1997 Davis (US) A Compendium of Analytic Tradecraft Notes
Jack Davis is the most able pioneer of “analytic tradecraft,” the best proponent for the value of human analysis over technical processing, and one of those very special individuals who helped define the end of 20th Century centralized analysis and the beginning of 21st Century distributed multinational multiagency analysis.
1997 Herman (UK) Has the (Secret Intelligence) Community a Future?
Michael Herman is perhaps the dean of the practitioner-scholar community in the English language, having begun his intelligence career in 1947 He has done much to emphasize that at the strategic level, most of what the nation-state needs to know is openly available.
1997 Mayer-Kress (DE) The World Brain
Complexity cannot be controlled — adaptive evolution is about distributed decision-making — the world brain is between inter-terrestrial and extra-terrestial orders of magnitude.
1997 Pinchot (US) The End of Bureaucracy and the Rise of the Intelligent Organization
This remains – perhaps equal to Alvin Toffler in 1993, but focused on doing — the single best presentation across 20 years of the conference, “National Security & National Competitveness: Open Source Solutions.
1997 Sutton (US) The Challenge of Global Coverage
This is the single most important briefing across 20 years in relation to the IC’s persistent failure to do Global Coverage — along with the most considered deep sketch of how to achieve a robust Global Coverage program using predominantly but not exclusively open sources and methods.
1998 Beaver (US) Operationalizing Information Operations (IO) in Bosnia-Herzegovina
To the best of our knowledge, this was the first proper integration of Open Source Intelligence as a foundation for Information Operations in support of full spectrum whole of government campaign.
1998 Clerc (FR) [National] Economic Intelligence
Unlike the Japanese and Chinese, who have emphasized massive collection with very poor processing, or the Israelis, who have emphasized secret technical espionage and political influence over economic intelligence, the French — and to a lesser extent the Germans and Spanish — have focused on an extraordinary mix of ethical economic intelligence, innovation in complex tenders, and a forcing function for Ambassadors — they are evaluated in relation to achieving economic goals.
1998 Dearth (US) Government and the Information Marketplace
Doug Dearth, then dean of the defense analytic corps, understood that open sources done properly would force all classified disciplines to justify their return on investment, and provide leaders authentically interested in producing decision-support with a powerful new foundation for transforming the craft of intelligence.
1998 Fry (UK) Open Sources and Law Enforcement–Learning Curves and Pain Barriers within Scotland Yard
Not only did open source cut their costs of criminal surveillance and monitoring by 80%, but it also increased the revenue from confiscated assets a 100 fold, finding, through legal and ethical means not exploited previously global financial holdings linked to those convicted.
1998 Mti (ZA), Coordinator for Intelligence, Open Source Intelligence, the African Renaissance, and Sustainable Development: The Emerging National Intelligence Model for South Africa
In 1997 South Africa created the first continent-wide early warning system among nations. In some ways South Africa is a counterpart to France, clearly understanding that open source intelligence and its support to cultural and economic priorities is the foundation for national security.
1998 Myers (US) US Atlantic Command (USACOM) Virtual Information Center (VIC) Concept Refinement
Both the US Atlantic and Pacific Commands developed innovative approaches to open source exploitation. Their initiatives were destroyed when CIA’s Open Source Center (OSC)lied to various military commanders, made promises it never kept, and essentially destroyed all of the seed crystals created from 1992 to 2000. This became a catastrophe instead of merely a disaster when the OSC classified all of its production, and was forbidden by the clandestine service from leveraging overt human subject matter experts. OSD and DIA are no better, treating OSINT as a technical and document challenge, rather than as a human times 183 language challenge.
1998 Politi (IT), Klaus Becher (DE) et al, Towards a European intelligence policy
The efficacy and legitimacy of governance depend on sound decision-making — ethical evidence-based decision-making. This is inherently both a public process (subject to public dialog and scrutiny) and a regional process (necessitating cross-boundary inquiry, dialog, and consensus).
1998 Prinslow (US) and Doug Bond (US) Information Sharing in Humanitarian Emergencies
Hard to believe, but fifteen years later, the core needs have not been addressed. This is still one of the best starting points around. Most importantly, it makes clear that within the Department of State and AID, it is the Bureau for Democracy, Conflict, and Humanitarian Assistance, NOT the Bureau for Intelligence and Research, that should be the diplomatic lead for open source everything and multinational information-sharing and sense-making — a logical place to put the Office of Information-Sharing Treaties and Agreements.
1999 Allen (US) Requirements, Collection, and Current Awareness: The Human Dimension (The Role of Open Sources as the Foundation for Successful All-Source Collection Strategies)
With the best of intentions, the DDCI/C tried for four years to make OSINT its own “program” (budget) at the national level. He was unable to prevail against the toxic combination of selfish interests across multiple agencies, led by a DCI unable to comprehend, much less adapt, to the 21st Century.
1999 Anon / BVD Chief (NL) Open Source Intelligence: The Lingua Franca or Common Language for Regional Intelligence Co-Ordination and Information Sharing
Alone among the intelligence communities of the world, the Dutch appear to have mastered the art and science of open sources as the source of first resort.
1999 Connors (US) US Pacific Command JV2010 Initiative Virtual Information Center (VIC) J-08
When USDI surveyed DoD OSINT, USPACOM was both the location of some of the best thinking, and also the location of some of the most vicious unprofessional back-stabbing by those in the classified world, against those seeking to support them from the open source world. Capt Carl Schuster, USN (Ret), today remains the last man standing from this initiative.
1999 (Ambassador) Marks (US) Joint Proposal for IRIN and ReliefWeb (Information) Activities and Funding
Ambassador Marks is one of the very few diplomats that has consistently championed multinational information sharing as a major foundation for early warning, rapid planning, and coherent multi-national, multi-agency response to disasters and conflicts.
1999 Rolington (UK) The Changing Messages in Western Information, Knowledge and Intelligence
Alfred Rolington is unique for having grown Jane’s Information Group to a $100 million a year enterprise, and then “retired” to manage Oxford Analytica. Today he is exploring new approaches to energy-related information and intelligence.
2000 George (BE) Open Source Intelligence Developments in Belgium
Intertia remains — it is easier to spend lots of money in old ways, than very little money in new ways.
2000 Mueller (AT) Some Thoughts on Open Source
Austrian military intelligence is concerned with central Asia as well as former elements of the USSR.
2000 Mulschlegel (EU) EUROPOL’s Open Sources & Documentation Unit
If the EU could embrace the spirit of EUROPOL and create a joint centre with NATO, some pretty spectacular outcomes could be assured.
2000 Reynold (J2A USTRANSCOM) Global Reach Without Global Intelligence
The global providers of lift and logistics are the red-headed step-children of the government world, the last to receive proper intellience support, and the most likely to benefit hugely from Open Source Intelligence (OSINT).
2000 Steele (US) ON INTELLIGENCE: Spies and Secrecy in an Open World (AFCEA), p. 205
Intelligence power is from the people, to the people — it cannot be reserved (or mastered) by any government alone.
2000 (General) Zinni (US) — 4% “at best” from classified sources, 96% from open sources
Despite the Aspin-Brown findings and recommendations, US IC leaders refuse to be responsible about open sources & methods.
2001 Dumaine (US) Global Futures Partnership “Are You Ready”
There are two truly extraordinary women in modern CIA history — the first was Diane Webb, who left when not heard, and the second is Carol Dumaine, who retired recently, also not heard. If and when there is a global intelligence network, it is women (smaller egos, higher intuition) who will run it, and Dumaine is easily among a tiny handful who should be considered for global responsibilities in this vein.
2001 (Ambassador) Oakley (US) The Use of Military and Civilian Power for Engagement and Intervention
Along with Ambassador Mark Palmer and Ambassador Ed Marks, one of the most ethical brilliant Foreign Services Officers ever to serve.
2001 Porter (US) Tools of the Trade — A Long Way to Go
The bottom line then and now: proprietary solutions are not affordable, do not integrate or even inter-operate, and cannot be scaled.
2001 Rotheray (UK) New Risks of Crisis: Fresh Perspectives from Open Source
Brian Rotheray is one of the “Dissident Majority” at BBC who have objected, among other things, to terrible management and a rather loose grasp of the truth, some would say because of its excessive dependence on US funding sources.
2002 Hardee (US) Growing an Open Source Program for the U.S. Special Operations Command and the Global War on Terrorism
SOCOM JIC J-23 answers 40% of all Global War on Terror (GWOT) requirements from all SOF elements worldwide, for under $10 million dollars. There is not another element of the U.S. Government that is remotely as capable, or as cost efficient.
2002 Henk (US) Respecting the Cultural Dimension: Intelligence and Africa
Col Dr. Dan Henk, USAF is a classical scholar-warrior, and one of the finest observers and interpreters of African realities and all of their nuances that we have encountered. We’ve found no finer briefing on regional, religious, ethnic, and tribal nuances than this officer.
2002 Hock (US) The World of Information: Content, Content, and Content
Dr. Ran Hock has done more than any single individual to educate both government and private sector parties with respect to the value of the deep web.
2002 Klavans (US) Tomorrow’s Hotspots: Identifying Commercial Opportunities from Science
Dick Klavens did for competitive intelligence what Jan Herring tried to do for government open source intelligence. A co-founder of the Society of Competitive Intelligence Progressionals (SCIP), he has done more than any other person to advance the science of citation analytics, building on the foundation established by Eugene Garfield.
2002 Manwaring (US) Asymmetry, Conflict, and the Need to Achieve Both Vertical and Horizonal Integration
Dr. Manwaring may well be America’s top authority on both “uncomfortable small wars” and on “gangs against governments.” He is the originator of the six generations of warfare (the US still fights 4th generation warfare at best) and inspired the definition by Robert Steele of the seventh generation, Information Peacekeeping at “total war” using information and intelligence as the sole munition.
2002 Moore (US) & Krizan (US) Core Competencies for Intelligence Analysts
Golden Candle Award: Mr. David Moore and Ms. Lisa Krizan, National Security Agency, USA OSS ’02: For their personal commitment to nurturing intelligence education and the craft of intelligence analysis, in part by studying, defining, and then promulgating “best practices” from both within the government and from the external private sector.
2002 Steele (US) New Rules for the New Craft of Intelligence (Chapter 15)
Need to go two-levels down, and strike a balance that does not exist now.
2003 Rodenberg (BE) Competitive Best Practices
There is no national intelligence community we know of that comes remotely close to this kind of integrated thinking.
2004 Arnold (US) Missing Information From Digital Shallow and Deep to Analog
As of 21 July 2013, Mr. Arnold observes that the digital web is worse than ever with more walled garden, fewer persistent URLs, and of course more “directed” search results (in the graphic below, Robert Steele added the red portion when he first adopted the slide).
2004 Defense Science Board (US) Transition to and from Hostilities
The study, drafted in part by Dr. Jospeh Markowitz, found that DoD cannot do transitions to and from hostilities without comprehensive information sharing among US Government inter-agency elements, coalition partner inter-agency elements, and non-governmental organizations; and that open sources of information are the cheapest, fastest, and best means of establishing common understandings of threats and solutions among interagency and coalition (multinational) stakeholders.