Beacon and Warning: Sherman Kent, Scientific Hubris, and the CIA’s Office of National Estimates
J. Peter Scoblic, a doctoral candidate at Harvard Business School, is a fellow in the International Security Program at New America and the author of U.S. vs. Them: Conservatism in the Age of Nuclear Terror.
“From bias to probabilities, Kent anticipated the findings of modern scholars by many decades. However, in his conviction that intelligence analysts would soon be able to forecast geopolitical events with scientific accuracy, Kent overreached. Enthusiasm turned out to be hubris, and the predictive record of the Office of National Estimates was decidedly mixed. In light of this history, today's enthusiasm for the predictive potential of Big Data and artificial intelligence seems overzealous. Kent’s relentless pursuit of the truth makes him a model in today’s political climate, but his failures should serve as a warning to those who believe that technology can eliminate the uncertainty of the future.”
ROBERT STEELE: Unlike most articles in the camp follower literature, this one is worthy of being read with care. This is an important useful article. I was even reminded of the one other great piece to come out of Harvard Business School, Charles Hampden-Turner's RADICAL MAN: The Process of Psycho-Social Development (review here).
“When Kent joined the Research and Analysis Branch, U.S. intelligence analysis was a haphazard affair.”
It still is. CIA lacks a holistic analytic model, does not do true cost economics, has no clue what the open source everything engineering alternative means, and is generally flighty and dishonest — it provides “at best” 4% of what the president needs and nothing for everyone else.
“Kent anticipated — by decades — seminal findings in the field of judgment and decision-making, such as the prevalence of overconfidence and the dangers of confirmation bias, that have validated the role of social science in intelligence analysis.”
What Kent did not anticipate was the hijacking of the US Government (USG) by the Deep State and a Shadow Government (and the hijacking of Central Intelligence Agency (CIA) by the clandestine service and its covert operations mentality on behalf of the Deep State). No one in Washington does evidence-based decision-making for two reasons: 01) CIA is not up to the job and 02) the system is rigged for “pay to play” special interest accommodation and the FBI is never going to be a real counterintelligence organization absent Presidential intervention.
“The problem the United States faced as it approached a global showdown with the Axis powers was how to collect, synthesize, and present massive amounts of information about foreign countries and potential theaters of battle.”
The author provides a fine discussion of why historians and geographers were so logically called to action, and in passing documents the signal role the Library of Congress played in the formation of the Office of Strategic Services — although the Library has always had a special relationship at the highest levels with the secret world, it never rose to the opportunities of global multi-lingual open source harvesting and exploitation, a lasting tragedy.
Citing Kent: “‘From our visits and meetings with the various intelligence officers of the Armed Forces, we had some pretty solid evidence that any active intelligence work must have ended with the First World War. … There could be no speedier way to bring to light the shocking state of U.S. intelligence than the imminent outbreak of war.'”
The military blew it and they continue to blow it. The Defense Intelligence Agency (DIA) is a travesty, the National Security Agency (NSA) processes 1% of what it collects while violating the Constitution with mass surveillance, and more. The CIA no longer does warning intelligence or estimative intelligence, they got caught up with being a drone assassination and torture agency while relying on Zionist Israel, the Saudis, and other indigenous intelligence services for lies they republish with the pretense they came from unilateral sources that by and large simply do not exist. CIA has completely dismissed Open Source Intelligence (OSINT) outside a few very fat white women pretending to go through the motions.
“It would quickly become clear that the R&A scholars could produce most of what the military (and other elements of the war effort) needed simply by hitting the books.”
While Donovan got this, and embraced what could be done with open sources, he was sucked into the vortex of clandestine and covert operations and CIA has never recovered from that. CIA was created to be a coordinator of information, not a rogue element blundering about with torture, assassination, regime change, pedophilia blackmail operations, and such.
“R&A also forced its 900 scholars to work across disciplines. The bureau included historians, economists, political scientists, geographers, psychologists, and anthropologists.”
This is over-stated in part because CIA never mastered the art and science of multi-lingual analytics (or deep time and space analytics, for example, visualizing every statement on the Spratley Islands made by China, Viet-Nam, and the Philippines over the past 200 years). CIA created a new kind of ivory tower, one generally removed from the real world. Daniel Elsberg's comment to Henry Kissnger bears repeating:
much greater than yours.”
“Henry, there’s something I would like to tell you, for what it’s worth, something I wish I had been told years ago. You’ve been a consultant for a long time, and you’ve dealt a great deal with top secret information. But you’re about to receive a whole slew of special clearances, maybe fifteen or twenty of them, that are higher than top secret.
“I’ve had a number of these myself, and I’ve known other people who have just acquired them, and I have a pretty good sense of what the effects of receiving these clearances are on a person who didn’t previously know they even existed. And the effects of reading the information that they will make available to you.
“First, you’ll be exhilarated by some of this new information, and by having it all — so much! incredible! — suddenly available to you. But second, almost as fast, you will feel like a fool for having studied, written, talked about these subjects, criticized and analyzed decisions made by presidents for years without having known of the existence of all this information, which presidents and others had and you didn’t, and which must have influenced their decisions in ways you couldn’t even guess. In particular, you’ll feel foolish for having literally rubbed shoulders for over a decade with some officials and consultants who did have access to all this information you didn’t know about and didn’t know they had, and you’ll be stunned that they kept that secret from you so well.
“You will feel like a fool, and that will last for about two weeks. Then, after you’ve started reading all this daily intelligence input and become used to using what amounts to whole libraries of hidden information, which is much more closely held than mere top secret data, you will forget there ever was a time when you didn’t have it, and you’ll be aware only of the fact that you have it now and most others don’t….and that all those other people are fools.
“Over a longer period of time — not too long, but a matter of two or three years — you’ll eventually become aware of the limitations of this information. There is a great deal that it doesn’t tell you, it’s often inaccurate, and it can lead you astray just as much as the New York Times can. But that takes a while to learn.
“In the meantime it will have become very hard for you to learn from anybody who doesn’t have these clearances. Because you’ll be thinking as you listen to them: ‘What would this man be telling me if he knew what I know? Would he be giving me the same advice, or would it totally change his predictions and recommendations?’ And that mental exercise is so torturous that after a while you give it up and just stop listening. I’ve seen this with my superiors, my colleagues….and with myself.
“You will deal with a person who doesn’t have those clearances only from the point of view of what you want him to believe and what impression you want him to go away with, since you’ll have to lie carefully to him about what you know. In effect, you will have to manipulate him. You’ll give up trying to assess what he has to say. The danger is, you’ll become something like a moron. You’ll become incapable of learning from most people in the world, no matter how much experience they may have in their particular areas that may be much greater than yours.”
Kevin Drum, “Daniel Ellsberg on the Limiits of Knowledge,” Mother Jones, 27 February 2010.
CIA still does not get that. They have children rather than top scholars, and their analysts are investigated when they dare to talk directly to foreigners. I grade CIA at a C- (at best) on open source access — almost all digital, no real ground truth over human access — and a solid F in terms of useful decision support against all threats, all policies, all costs.
Almost immediately after Japan’s surrender in August 1945, Truman fired Donovan, whom he found arrogant, and shelved his plan to create a centralized intelligence service, telling Harold Smith, who directed the Bureau of the Budget, that he had “in mind a different kind of intelligence service from what this country has had in the past.”
Donovan betrayed the President's trust. While Kent and his Research & Analysis unit did good work, Donovan was enthralled with espionage and sabotage, and lost sight of his mission: decision-support. He failed to meet the President's expectation.
Harry S. Truman, “Limit CIA Role to Intelligence,” The Washington Post, 22 December 1963
“Kent explicitly categorized intelligence analysis by time — that is, whether it focused on the past, the present, or the future.”
This was an important contribution in its time, but in context must be categorized as elementary. Kent did not do what Tim Hendrickson at the National Ground Intelligence Center (NGIC) tried to do with GRAND VIEW: establish a holistic analytic model that would not only weave all of the strands together (political-military, socio-economic, ideo-cultural, techno-demographic, and natural-geographic), but also show with precision the financial affordability and feasibility of the “plans and intentions” of each actor. CIA production must be evaluated, still today, as stove-piped pablum.
Kent's confidence in prediction belies the larger counterintelligence and compare and contrast challenges that still today no one other than myself appears willing to address:
01 No amount of brilliance in decision-support can overcome the Deep State and Shadow Government if they choose to commit treason inclusive of domestic false flag attacks intended to justify wars based on lies.
02 The future is best predicted by creating it with transparency and truth.
“The use of mathematics similarly spiked, as did the tendency to explicitly connect empirical findings to theoretical literature. Effectively, American scholars thought they would soon be able to codify and quantify human behavior. The result was greater faith in social scientists’ ability to explain and predict.”
There is no finer antidote to academic bullshit — and the abuse of mathematics by academics — than the books by Nassim Nicholas Taleb, particularly Anti-Fragile and Skin in the Game. As a political scientist myself, intimately familiar with Arthur S. Banks and Robert B. Textor, A Cross-Polity Survey (MIT, 1971), I watched both political science (which used to be called “current history,” and international relations migrate from demanding competency in foreign languages and field work among the indigenous being studied, to absolute bullship called “comparative analytics” which was code for being able to complete a career without a foreign language, without field work, working with second and third hand data of dubious provenance.
The Big Data and Aritificial Intelligence (AI) fads today are more bullshit. Whether at NSA or among the major universities and corporations and governments, we process less than 1% of the Big Data and what we collect is generally far removed from what we need to know. AI is nothing more than artificial stupidity pimped up to make sales — the latest fraud is AI firms secretly using humans while pretending the insights come from machines. On this Jim Bamford's final line in Body of Secreets: Anatomy of the Ultra-Secret National Security Agency (Anchor, 2002) is instructive:
“Eventually NSA may secretly achieve the ultimate in quickness, compatibility, and efficiency-a computer with petaflop and higher speeds shrunk into a container about a liter in size, and powered by only about ten watts of power: the human brain.”
“He anticipated the dangers of confirmatory bias, the importance of allowing dissent, and the need for precision when estimating probabilities.”
There is no denying the importance of Sherman Kent as the father of the second era of national intelligence — the first was secret war the third is the era of collective intelligence, absolutely multinational and generally open source.
Steele, Robert. “The Evolving Craft of Intelligence,” in Robert Dover, Michael Goodman, and Claudia Hillebrand (eds.). Routledge Companion to Intelligence Studies, Oxford, UK: Routledge, July 31, 2013.
Kent is credited with creating an atmosphere of “psychological safety” that encouraged dissent and was open to all opinions, to the point of including unresolvable disagreements in the National Intelligence Estimates (NIE).
This is not the case today. Not only does the secret world demand “unanimity” but it also lies with malice aforethought — the lies told by Clapper, Hayden, Brennan and others with respect to the Russians “hacking” the election are in my view the stake in the heart of secret intelligence. When the President has time, this is a “fix big” matter. As Amy Zegart noted so ably in Flawed by Design: The Evolution of the CIA, JCS, and NSC (Stanford, 1999), the US secret intelligence community is so hosed that it is a “fix big or don't fix at all” matter — the political capital required to make changes on the margin are not worth it. I would add that given the President's objectives, nothing less than a rebooting of America the Beautiful, both the secret world including the deeply corrupt and dysfunctional Federal Bureau of Investigation (FBI) and the Department of Defense (DoD) are ripe for Presidential command, along with the abolition of the Department of Homeland Security and the Internal Revenue Service (IRS).
Despite Kent’s best efforts, ONE often erred in its predictions. One of its biggest mistakes was its insistence, in September 1962, that the Soviets would not place nuclear weapons in Cuba. . . . There were other errors as well. In 1973, Robert Gates, a young CIA analyst who would go on to become director of central intelligence and secretary of defense, wrote, “We failed to anticipate the construction of the Berlin Wall, the ouster of Khrushchev, the timing of the invasion of Czechoslovakia, and other events of importance.”
I am reminded of how the Soviet Union was on the right side of history with the wars of national liberation across Africa and to a lesser extent in Central Asia and South America, while the USA and CIA particularly was on the wrong side. You simply cannot do serious predictive intelligence if you leave your ethics at the door and do not do holistic analytics and true cost economics in a multi-lingual manner. In my second graduate degree I studied the “intakes” of three Embassies with which I was intimately familiar, and found that the diplomats are outnumbered and have no money with which to buy open sources of information and the spies, who have way too much money (I was spending $100,000 a month as a 33 year old — and racking up five times the regional average in recruitments and foreign intelligence reports) insist that you commit treason as a condition for paying you. My final conclusion: we collect at best 20% of what we can and should collect, the vast majority of that open sources in languages we do not read well, and we spill 80% of what we collect by sending it home in the pouch rather than processing it electronically (which requires inter-agencystaffing kin the field — pouch contents are not subject to staffing). 2% is what our total USG collection enterprise harvests, and we process 1% of that.
Citing George Tenet: Crucially, he continued, intelligence “has developed a recognized methodology; it has developed a vocabulary; it has developed a body of theory and doctrine; it has elaborate and refined techniques.”
Yes but it is the vocabulary of a child that only speaks English and has not progressed past the fifth grade, at best, in terms of local to global understanding. Furthermore, CIA analysts still do not have the tools that we defined by Diane Webb, Dennis McCormick, and Gordon Oehler as needed in 1989, nor does CIA have any mastery at all of all sources of information in all languages across space and time. CIA also is still incapable of doing what Ellen Seidman, then on the National Economic Council, needed: compare and contrast decision-support across each US industry. CIA never has understood true cost economics (or holistic analytics) and it is never going to be useful as a national analytic capability until it does so.
“The thing he felt intelligence needed most to help it become a discipline was a body of literature, which is why he lobbied for the creation of Studies in Intelligence. Its authors, he said, should grapple with “first principles,” but by “first principles” he meant, “What is our mission?” and “What is our method?” Despite repeated references to theory, in a quarter-century of intelligence work Kent never articulated an intelligence equivalent of, say, political science’s “realism.” He never even attempted such a contribution despite his insistence that intelligence needed the kind of brilliant thinkers — he cited Darwin, Freud, Keynes, and Pareto — who had defined or redefined their fields at the theoretical level.”
This is the meat of the work and well worth waiting for. The author is absolutely correct. Intelligence today lacks integrity and goes through the motions with impunity, knowing that its USG customers have given up all hope of every getting anything useful from the secret world that escshews open source intelligence (if the USG had a serious intelligence and counterintelligence capability Monsanto would have been closed down long ago and every director of the Center for Disease Control (CDC) would be in jail for fielding vaccines without ever testing them — EVER).
In a nutshell, it is possible to advance the craft of intelligence to a discpline, but not until you are rooted in ethics and committed to the public interest.
Steele, Robert, “Intelligence at a Cross Roads: To Be Or Not To Be (Review of Principled Spying by David Omand and Mark Phythian),” American Herald Tribune, 25 June 2018. PBI BackUp
The author concludes with useful observations on AI as a fad, and an exaggerated appreciation for the work of Dr. Philip Tetlock at the University of Wharton. “Delphic” techniques have been around for a long time, and it is certainly valuable to expand that concept, but until we get to where we can do multinational and multilingual overt human exploitation that incudes representatives from all eight tribes of information in the target, adjacent, and invested countries, at all levels from global to local, we will be infants.
Lots of work to do. No progress in likely in the near term. St.
