DOC (10 Pages): the-osint-story-2-1
25 June 2004, Version 2.1
Public Intelligence Must Supersede Secret Intelligence. 1
Trade-Off Decisions: What $100M Will Buy & Why Secrecy Hurts the Taxpayer. 1
Copyright & Patent Reform Needed to Fully Exploit New Knowledge. 2
Thomas Jefferson Had It Right: Educated Citizenry = Public Intelligence. 2
Open Source Intelligence (OSINT) Is the Wedge in Three Different Revolutions. 2
Learning Matters I: What the Marines Could Not Get From the Spies. 2
Learning Matters II: What the Public Cannot Get From Its Own Embassies. 3
Washington Is Operating on 2% of the Relevant Available Unclassified Information. 4
Spies Have Too Much Money, Diplomats Do Not Have Enough. 4
What is Open Source Intelligence or OSINT?. 4
Definition of OSINT.. 4
Proving the Point I: Testimony to the Aspin-Brown Commission on Future of Intelligence. 4
Proving the Point II: The Problem with Spies. 6
Seven Tribes, Seven Standards, Seven Issues: The Way Forward. 6
We Have Seven Intelligence Tribes, Not Just the One Secret National Tribe. 7
The Seven Tribes Need Seven Standards To Enable Sharing. 8
We Can Begin Building the World Brain By Focusing on Seven Issues. 8
Putting It All Together. 8
Eight Points of the World Brain Node on Anything of Public Interest 9
Citizen-Centered Intelligence, Networking, & Influence. 9
Small Obstacles. 9
Two Sources of Public Intelligence. 10
Signature & Short Biography of the Author. 10
Imagine an America in which public intelligence supersedes secret intelligence, and elitist corruption is displaced by an informed democracy in which consensus conferences at every level assure that “We the People” all serve the public interest.
Let us begin by defining what $100M will buy. $100M will buy a Navy warship, or an Army brigade, or 1000 diplomats, or 10,000 Peace Corps volunteers, or a water desalination plant doing 100M cubic meters of water a year, or a day of war over water in the Middle East.
Today these trade-off decisions are made by a policy process that is neither public, nor in the public interest. We need to create an atmosphere where secret intelligence processes and products—which cost the US taxpayers between $35 billion and $50 billion a year—cannot easily be manipulated or ignored in what amounts to a betrayal of the public trust. The issues of how and what we can know as a Nation about the real world also need to be protected and distinguished from illegal secret agreements among politicians and private sector campaign contributors. In my view, public intelligence in support of public policy is half the solution—the other half, as championed by Senators John McCain and Joe Lieberman, among others, is complete campaign finance reform.
One area where openness could contribute directly to national security and prosperity is in copyright and patent reform. These processes are used today to protect inefficient and socially-costly products and processes, keeping useful knowledge out of the public domain solely to protect profits that impose a heavy external diseconomy on the public—the technologies of portable, environmentally-friendly power alternatives are one example.
In essence, as things now stand, wars are fought, secret military bases established, subsidies given, deficits encroaching on the future amassed, all without adequate public review.
There is a simple democratic solution. It is called public intelligence. My vision is rooted in what our Founding Father, Thomas Jefferson, articulated: “A Nation’s best defense is an educated citizenry.” I am the evangelist for something called Open Source Intelligence or OSINT. Like open source software, and open spectrum, OSINT is, I believe, a foundation for our future….our bright future.
OSINT is the wedge in the door for three different revolutions:
1) Intelligence reform–if we get OSINT right, at very low cost, just 5% of what we spend now on secret intelligence, we can increase by a factor of 100, perhaps 1000, perhaps even 10,000, what America can know about itself, about the world, and about the possibilities for peace and prosperity.
2) Acquisition reform–if we get OSINT right, we will realize that we still need $500B a year for national security, but we must invest these taxpayer dollars in different ways—we can cut the heavy metal military to half this amount; substantially increase our special operations and gendarme “white hat” capabilities for delivering humanitarian assistance under combat conditions; substantially increase our homeland security through an emphasis on intelligence and prevention; and redirect $100B a year toward faith-based diplomacy, a dramatic increase in our foreign assistance including a digital Marshall Plan, and a focus on eliminating dictatorships that spawn terrorism, poverty, and genocide.
3) Governance reform–OSINT restores the morality of capitalism, including the proper calculation of the negative economic costs including ignored social costs, creates an informed citizenry, and makes true democracy possible around the world–the only way of actually containing and then eliminating terrorism.
—– SKIP TO LAST PAGE FOR SHORT VERSION —–
Now, given that the Director of Central Intelligence (DCI) has said that intelligence returns to America the greatest “bang for the buck”, something I happen to agree with, how is it that we are failing, at great expense, to get it right? Let me tell you two short stories.
The first story deals with my epiphany, a very expensive Republican word for the more Democratic “aha.” It was my great privilege to be the senior civilian responsible for designing and opening the Marine Corps Intelligence Command, our Nation’s newest all-source intelligence production facility. Being a former spy, and having no reason to question my faith in secrets, I spent $20 million of the US taxpayers’ money on ensuring that we had all the special equipment needed to get direct access to all of the secrets—the human spy secrets, the signals secrets, the imagery secrets, and the other secrets so secret that, like Pogo, I have had to forget them.
Imagine my shock, when I actually had to produce intelligence for the Marine Corps generals responsible for policy, for acquisition, and for operations, in discovering that 80% of what I needed in the way of raw information was not secret, not online, not in English, and not available from anyone in the national or defense intelligence communities.
It turns out there is a simple explanation. The Marine Corps must concern itself with the Third World, the under-developed countries where most genocide, poverty, terrorism, crime, and other natural and man-made disasters occur. The Center that I helped create was funded precisely because our Marine Corps leaders understood that we had spent over 50 years obsessing on the Soviet Union and the nuclear threat, and all of this other stuff—what is known as “Global Coverage”, had been neglected. Despite the fact that the Commandant of the Marine Corps called for a major shift in U.S, intelligence in 1988, the national intelligence bureaucracy and the national politicians that this bureaucracy supports, have refused to change their focus. As recently as July of 1997, the DCI himself made a decision that he was in the business of doing secrets about hard targets, not “all-source” intelligence about global issues and lower tier countries in Africa, Asia, and Latin America, where most global problems are spawned.
Now a second story, to put this in the context of your typical American collection environment overseas, i.e. the U.S. Embassy and the loose collection of U.S. government organizations that are represented overseas in that environment.
There are four things you need to know about the U.S. Embassies and other U.S. Government capabilities overseas that you pay for:
First, they are generally not yet “wired”, which is to say that most Embassy employees do not have access to the Internet and cannot use the Internet as a basic tool for obtaining and sharing unclassified knowledge essential to our national security and national prosperity.
Second, within any given Embassy, the Foreign Service Officers or FSOs, the elite from the Department of State, labor under numerous burdens. They are in the minority in their own Embassies, outnumbered by the mélange of officers from other departments and agencies, none of whom really care what the Ambassador wants from them. They have lost their secretaries to budget cuts, resulting in the junior officers becoming very expensive clerks. They are without any realistic budget for buying local knowledge legally and ethically. Finally, they are limited by the physics of the 24-hour day and the politics of who is willing to meet with them in the national capital area—their lack of funds precludes routine travel in to the provinces.
Third, within any given Embassy, there are two ways for information to be sent back to Washington: electronically, or via the diplomatic pouch in hard-copy. Electronic transmissions require staff coordination across all Embassy section, pouch hard-copy dispatches do not. The result: less than 10% of what we capture goes electronically, with the result that 90%, send via the diplomatic pouch, is not visible to the rest of the government, and more often than not is filed or discarded and thus not exploited as it should be.
This is why I came to the conclusion, in my second graduate thesis studying three Embassies with which I was familiar, that Washington is operating on 2% of the relevant unclassified multi-lingual information that can and should be known in support of our foreign relations decisions.
Fourth, and carefully separated from our Embassies, contrast this with the unlimited funding available to your clandestine officers that operate throughout the country, but have one rather interesting condition for listening to a local person: that person must be a traitor. They must agree to betray their country or their tribe or their organization or their family, either for money or because of a mutual interest that generally is not consistent with the interests of the host country’s leadership.
In summary, it is my view that we are wasting at least 10 billion dollars a year going after secrets out of context, and we are failing to spend a modest 1 or 2 billion going after open sources of information that are much more valuable to our national security and prosperity when taken in the aggregate, in near-real-time, and in the original languages where nuances are clear.
Now, what is OSINT, you may ask. Let me begin with the quasi-official definition that I have put forward over the past 15 years, and then explain that with two relatively short stories.
Open Source Intelligence (OSINT) combines the proven process of intelligence (requirements definition, collection management, source discovery and validation, multi-source fusion, and compelling actionable presentation) with a deep and broad understanding of what open sources of information (OSIF) are available in 29+ languages. While legally and ethically available, roughly 80% of those sources are not known to and not exploited by standard bureaucratic elements of the U.S. Government such as the Foreign Broadcast Information Service (FBIS) of the Central Intelligence Agency. The Department of State, which has the statutory responsibility for collecting, translating, and interpreting open sources of information relevant to U.S. foreign policy and national security, gave up its responsibility and its competency in this arena during the Cold War. Internationally, a number of nations, notably Australia, Norway, South Africa, and Sweden, have created specialist units to focus on OSIF/OSINT, with considerable success.
First, the Burundi Exercise. In August 1995, it was my privilege to testify the Aspin-Brown Commission on the future of intelligence, and more specifically, focused on what reforms were needed to make American intelligence effective. Had their recommendations been implemented, I do not believe 9-11 would have occurred. At the end of my testimony, as I was about to leave for the airport to speak to hackers in Las Vegas, General Les Aspin asked me if I would be willing to do a pro bono benchmark exercise—me and my Rolodex against the entire US Intelligence Community. I readily agreed. He turned to Britt Snider, the dynamic and very knowledgeable Staff Director and said, as I recall: “Burundi. By 1030 on Monday.” This was a Thursday, at the end of the day. I called my assistant from the car, told her who to alert, and had her send an additional fax to my hotel in Las Vegas, where I spent the day persuading my counterpart CEOs to donate roughly $100,000 in free services to make my point.
By 1030 Monday—the early delivery time for Federal Express—the Commission received the following:
- From LEXIS-NEXIS in Dayton (OH), one of the finest commercial online subscription services—there are others, such as Factiva and DIALOG, but they were the best for this specific task—a list of the top ten journalists reporting on the genocide happening in Burundi and Rwanda, all immediately available for debriefing.
- From the Institute of Scientific Information in Philadelphia (PA), publisher of the science and the social science citation directories—this is a very clever way of identifying the most influential authorities based on who else has quoted them or cited their work in other work published later—a list of the top 100 experts in the world, cutting across all cultures and countries, on the Burundi situation—all available for immediate debriefing, and also helpful in identifying the unpublished experts within governments and non-governmental organizations.
- From Oxford Analytica in Oxford (GB), which published superb Presidential and CEO-level strategic reports, a series of 22 two-page executive summaries of the situation in Burundi in relation to risk, UN operations there, and US foreign policy objectives or lack thereof.
- From Jane’s Information Group outside London (GB), two separate products were received. The first was a complete summary of all stories published in the various Jane’s publications, such as Jane’s Intelligence Review, on Burundi. This collection of one-paragraph summaries provided a superb overview of the historical and current situation unique to Jane’s. A second product, created overnight for this purpose, was a series of one-page tribal “orders of battle”, essentially describing tribal leadership, tribal manpower, and tribal capabilities including what are called “technicals” or normal small trucks with mounted machine-guns.
- From East View Cartographic in Minneapolis (MN), a list of all immediately-available Russian military combat charts (charts are geospatially accurate about contour lines and cultural features, where maps are simply generic guides to roads and other features) for the entire country, all at the 1:50,000 tactical level, essential for carrying out military and humanitarian assistance operations. This is very important because the United States of America does not have such maps for 90% of the world, having focused on the Soviet Union and aviation charts instead of ground truth in the Third World.
- Finally, from SPOT Image of Toulouse (FR), through its US affiliate in Reston (VA), and belatedly as I recall, since I met them a couple of weeks after the exercise, confirmation that all of Burundi was available in the form of commercial imagery, less than three years old, cloud-free, and inexpensive for being already in an archive.
This is the power of open sources of information. We did not produce intelligence for this test case—tailored answers to specific questions—but you can see how impressive the private sector can be when properly engaged. The US Intelligence Community had almost nothing—they are not structured to do overnight open source collection and their secret collection is focused on hard targets and also difficult to redirect. The provided a regional map of the area, the chapter from the World Fact Book, and a regional economic study that was very inaccurate because it applied American assumptions to African economics. There may have been one or two other small things also provided, but you get the idea.
As I have learned over the years, first as a spy—a clandestine case officer—then as a specialist seeking to help CIA exploit emerging Internet and other advanced information technologies—and finally as the senior civilian responsible for creating our Nation’s newest national intelligence facility, the Marine Corps Intelligence Command—our national intelligence service is neither truly national, nor all-knowing. We have two problems.
First, the biggest problem with spies is that they only know secrets. Think about that. They have access to—at best—less than 10% of what can and should be known in order to make informed decisions. This problem stems from a combination of excessive focus on secret technical collection, excessive focus on excessive use of official cover and acceptance of secret information from other governments, most of which cannot be trusted, and two deliberate decisions that will haunt us forever after 9-11: a refusal to invest in processing what we collect, and a refusal to take open sources of information seriously. This culture of secrecy, which extends to other governments, also imposed what is called the “third party rule,” meaning that even when information is known to be of value to multiple other countries, it cannot be shared directly—the originator must be asked to provide it directly and bilaterally—a very inefficient way of sharing, and a major contributor to the failure of Europe, Asia, and America to prevent the 9-11 debacle.
Second, however, the spies also have a mind-set that prevents sharing. For a spy, a secret shared is a secret lost forever. Both the FBI and the CIA as well as NSA—the National Security Agency that eavesdrops on everyone but processes fewer than a million out of every billion or so messages it collects—a processing rate of 1%, at best—have a culture of secrecy that is so deep as to have become an obstacle to achieving our goals. Even now, two years after 9-11, we have not achieved proper sharing despite direct orders to do so, in part in part because we are not yet serious about intelligence reform. As recently as March of 2004, I have learned that the most senior FBI leaders have not been able to get the CIA and NSA to include the names of American citizens known to be supporting terrorism in messages about terrorism. The FBI is forced, still, now, two years after 9-11, to go back to CIA and NSA, point out that the law permits reporting on US citizens when they are known to be supporting terrorists or are known agents of a foreign power.
Now, for the second story, the seven tribes, seven standards, and seven issues. In the course of spending $20 million of the taxpayer’s hard-earned funds on the Marine Corps, I discovered that 80-90% of what we need to know to keep America safe from terrorists and other unconventional threat, and to further our economic prosperity, public health and education, and ecological prosperity—is not secret, not online, not in English, and not available from anyone in Washington, D.C. or other major foreign capitals. It is all open source information, most of it is not digital, and most of it is in 29 languages, and in some instances a further 10-20 important dialects, none of which are understood by US intelligence officers, with a handful of exceptions.
As I cast about for solutions to the challenge of Global Coverage, to the need for finding ways to share the burden of 24/7 monitoring of the seven major issues that we must solve if we are to survive as a species on this planet, I realized that our concept of national intelligence was neither national nor intelligent. I already knew about spies and military intelligence as well as law enforcement, and of course academics and journalists. Beginning in 1988 I went on to explore business intelligence, then found religious intelligence and by 1994 began writing about creating smart nations with truly national and even global intelligence, through the harnessing of the distributed intelligence of the Whole Earth. My 1992 article in Whole Earth Review, entitled “E3i: Ethics, Ecology, Evolution, and Intelligence: An Alternative Paradigm for National Intelligence” was ridiculed by some as “assuring my place on the lunatic fringe.” My earlier articles in the American Intelligence Journal, from 1988 through 1992, and my later article in Government Information Quarterly as well as various chapters and speeches and additional testimony to Presidential and Congressional bodies, were simply ignored.
It was not until I attended a Dutch conference in 2002, sponsored by the Netherlands Intelligence Studies Association, that I found this final story. I call it, with help from many brilliant minds that have spoken at my annual conferences, “Seven Tribes, Seven Standards, Seven Issues.”
There are seven intelligence tribes in every nation—and we must have nations, for they are the only body that could and should protect the public interest instead of selfish private interests.
First, we have the national intelligence tribe—the spies and the signals and imagery and other technical intelligence specialists, and the all-source analysts that serve national government policy makers, usually just a handful of individuals. We still need this tribe, but we must make it smarter and connect it to the real world of open sources.
Second, we have the military intelligence tribe—the strategic, operational, tactical, and technical intelligence specialists that support military policy, military acquisition, and military operations.
Third, we have the law enforcement intelligence tribe—preventive policing, pro-active counterintelligence, a tribe very much in its infancy, but doing well in some areas such as London, where Scotland Yard has grown wisely.
Fourth, we have the business intelligence tribe—business intelligence is too often confused with data mining, middle managers think they know everything they need to know, and the competition rather than the customer is the focus of effort, but at least the tribe exists.
Fifth, we have the academic intelligence tribe—many brilliant scholars and diligent students, all eager to contribute the world brain, held back mostly by a lack of connectively and structure. This tribe is actually the most important tribe with respect to helping governments detect tax avoidance and import-export pricing fraud, or crop insurance and medical claim fraud. It is also the most important in terms of historical understanding and cultural intelligence research.
Sixth, we have the ground truth tribe, consisting of a handful of investigative journalists that actually leave their hotel rooms to go into the jungle, or Chechnya, or the mines of Papua New Guinea, and a vast multitude of non-governmental (NGO) and international governmental (IGO) specialists, among whom the World Bank and the European Centre for Conflict Prevention (ECCP) stand out. This tribe lacks processing power, and needs to be fully integrated into the world brain that is emerging.
Seventh, and finally, we have the citizen intelligence tribe, where I group religions, civil societies, and neighborhood associations as well as labor unions and individual “lone scouts.” Faith-based diplomacy, empowered labor unions, energized “cultural creative” associations, and citizens who share values and information for the common good comprise the foundation as well as the glue for bring all the tribes together, across national and cultural boundaries.
There are also seven standards and seven issues that I will name but not discuss today.
The seven standards areas where we must develop generic “good enough” means of sharing the burden of global intelligence are:
- Collection (Data Capture)
- Processing (Data Mining)
- Analytic Tool-Kits
- Analytic Trade-Craft
- Defensive Security & Counterintelligence
- Overt Action
- Mind-Sets—Leadership, Training, & Organizational Culture
The seven issue areas where we will begin organizing the world brain functions of weekly reports, expert forums, distance learning, virtual libraries, shared calendars, shared directories, virtual budgets (don’t move money, just deconflict spending), and a global “plot” that anyone can use to navigate the vastness of our newly-shared global digital intelligence community are:
- Co-Intelligence—Creating the World Brain
- Democracy through Faith-Based Diplomacy and Consensus Conferencing
- Digital Divide, Digital Marshall Plan
- Ecological Economics—Pricing to the Value of Life
- Faith-Based Education for Humanitas
- Public Health—Mental, Physical, Cultural, & Economic
- Rules of the Road—Moral Capitalism & the Public Interest
Now let us conclude by taking this story to its logical conclusion, ten years into the future.
In ten years’ time, it will be fashionable for each of you to see yourself as a member of one of the seven tribes, and to wear a pin such as I am wearing, that instantly connects you to strangers who share your vision, your values, your commitment to saving the future by husbanding the commonwealth through public intelligence applied to public policy.
In ten years’ time, with the Internet as the backbone for global connectivity, including the necessary tools for protecting privacy as well as confidences, these tribes will be well-established at the local, state or province, national, and regional levels. Governments will be important partners, but citizens and civil societies will do the leading.
In ten years’ time, every issue and every location of common concern to our “intelligence minutemen” will have an Internet “hub” that is well-structured and replete with validated information. Such a hub will have eight functional offerings, and be maintained by volunteers around the world who are proven and trusted must as the developers of LINUX are proven and trusted.
These eight offerings will be, in increasing order of importance:
- Weekly Report—what has changed, early warning, cost implications to the public
- Virtual Library—all that can be known, easily visualized and accessed, distributed
- Distance Learning—basic, intermediate, and advanced self-study on every aspect
- Calendar—of conferences and events, all monitored for the group by local minds
- Directory—of interested and qualified parties, self-validating
- Active Map—of the time, space, costs, benefits, and imminent dangers
- Expert Forum—certified, well-behaved, public, distilling wisdom on the fly
- Virtual Budget—the reality of what is being spent, easily redirected by consensus
My second book, entitled THE NEW CRAFT OF INTELLIGENCE: Personal, Public, & Political, begins with a preface that outlines my vision of how citizen or public intelligence will supersede secret intelligence and decision-making by the elites, for the elites. At every level, from neighborhoods to counties, to nations and regions, I see citizens being able to get the information they need, connect immediately with like-minded others regardless of race, religion, nationality, or occupation, establish consensus, and impose the will of the people on the elected officials and the hired bureaucrats.
There are several small obstacles. The first is an educational system that focuses on rote learning rather than learning how to learn and work with others. We have to change that. The second is the manner in which elected officials are compensated, where wealthy corporations have the influence to avoid taxes and violate the public interest. We have to change that. The third is the manner in which bureaucrat decisions—and especially homeland and national security decisions—are not subject to public review. We have to change that. Finally, the fourth—one you can take on if you wish—is the inattentiveness of the public. Between mediocre public education and the alternative reality portrayed by our really dumbed down media industry, we have a Nation of drones that are sleep-walking over the chasm created by elites who have betrayed our trust—some out of ignorance, others out of greed. This revolution is going to be a bottom-up revolution, and it starts right here with you. Get organized, focus on public intelligence as your tool for restoring the power of We the People, and begin the hard task of helping every waitress, every truck drive, every member of the working poor as well as the lower middle class, understand that it is their life, their brain, their attention to these details, that will determine whether or not we save this Nation and this world for our fulfillment and the fulfillment of our children.
I commend to you my web site, www.oss.net as a source of continuing education and networking among the seven tribes, and my 450+ reviews of national security and global issues non-fiction books at Amazon.com, where the people’s intelligence is visible for all to appreciate. I am sponsoring books by others about each of the seven intelligence tribes, and I hope you will share with me—and with those that visit our web site, any stories of your own as they emerge over the next few years.
—– RESUME FROM PAGE TWO FOR SHORT VERSION —–
We are all a family, a family of informed citizens of good heart, and we need to help one another into the future. The new craft of intelligence is personal, public, and political. It supports peace and prosperity. I ask for your help in communicating the vision to every citizen in every country. For myself, I dedicate the remainder of my life to telling this story. Thank you.
Roberto David de Steele y Vivas
 Robert David STEELE Vivas is the son of a Colombian mother, a naturalized US citizen since the early 1950’s, with roots in the Basque region of Spain in the 1700’s, and a USA father, with roots in Scotland (mother) and England (father), also in the 1700’s. Born in the USA on 16 July 1952, he is married to Kathy Lynette Steele of West Virginia, and has three sons to whom this effort is dedicated—if we fail they will not have children and the story becomes irrelevant. Robert has been a student of political science (multinationals issues with home and host countries), international affairs (prediction of revolution), public administration (strategic and tactical national security knowledge management), intelligence (policy), and defense (strategy, economics, and operations). He has served as a Marine Corps infantry officer, a Marine Corps reserve intelligence officer, a spy with three clandestine tours overseas and three Washington tours, in combination covering the full gamut of human, signals, imagery, counterintelligence, and covert action operations, and ended his government career as the senior civilian responsible for creating the Marine Corps Intelligence Command. Since 1 April 1993, when he chose to give up retirement as well as his savings, he has been committed to the evangelical mission of empowering people through Open Source Intelligence (OSINT). Robert is the author of two books, ON INTELLIGENCE: Spies and Secrecy in an Open World (2000) and THE NEW CRAFT OF INTELLIGENCE: Personal, Public, & Political (2002). He is also the publisher and contributing editor of PEACEKEEPING INTELLIGENCE: Emerging Concepts for the Future (2003), and the publisher of three books on other intelligence tribes, all forthcoming: Commercial Intelligence by Mats Bjore (2004), Academic Intelligence (edited work, for 2005), and Law Enforcement Intelligence: Global Gangs Against Local Governments (edited work, for 2006). He has a strong interest in the Ethics of Intelligence, and is a prolific reader and reviewer of books, generally standing as the #1 Amazon reviewer for national security and global issues non-fiction, #51 or so overall. Robert is now on “world tour” and will accept any invitation to speak that covers travel expenses and guarantees 250 minds across multiple audiences during a single visit. He can be reached at firstname.lastname@example.org, responds to all serious emails, and prefers to not receive uninvited telephone calls. Please do not add him to any email lists or digests without explicit permission. Robert wants to visit you, not be buried by your information. Pass it on….