Category: 09 Justice

Culturomics 2.0: Forecasting large-scale human behavior using global news media tone in time and space

Kalev Leetaru

First Monday, Volume 16, Number 9 – 5 September 2011

Abstract

News is increasingly being produced and consumed online, supplanting print and broadcast to represent nearly half of the news monitored across the world today by Western intelligence agencies. Recent literature has suggested that computational analysis of large text archives can yield novel insights to the functioning of society, including predicting future economic events. Applying tone and geographic analysis to a 30–year worldwide news archive, global news tone is found to have forecasted the revolutions in Tunisia, Egypt, and Libya, including the removal of Egyptian President Mubarak, predicted the stability of Saudi Arabia (at least through May 2011), estimated Osama Bin Laden’s likely hiding place as a 200–kilometer radius in Northern Pakistan that includes Abbotabad, and offered a new look at the world’s cultural affiliations. Along the way, common assertions about the news, such as “news is becoming more negative” and “American news portrays a U.S.–centric view of the world” are found to have merit.

Contents

Introduction
Data sources
Method
Forecasting unrest: Conflict early warning
The spatial dimension of news
Mapping the “civilizations” of the world’s press
Conclusions

Phi Beta Iota:  Interesting, and no doubt to be presented to IARPA as a proposed project.  However, there are four major flaws in this approach:

1) it does not recognize the difference between preconditions of revolution and precipitants;

2)  is has no underlying analytic model for understanding true costs and severe imbalances between the few and the many;

3)  it relies on English-language second and third hand depictions of the indigenous press in a handful of languages (there are actually 183 that need to be studied as indigenous populations strive to overturn the Treaty of Westphalia and its artificial boundaries); and

4) it assumes that published media interpretations are a reliable representation of the public mood–in our experience, not only are all media generally biased as they are owned by the “establishment” in one form or another, but they also fail to capture the 80% that is “unpublished” or “unarticulated” but simmering and very reality-based.