It is fashionable now to talk about data as the new oil (or dirt), and to proclaim breathlessly that the ever-increasing masses of data allow for ever more wonderous things to be done including my personal favorite, situational awareness.
However, no one is yet serious about holistic analytics (which also implies a holistic collection management strategy and a clear definition of both what is to be collected and what is to be done with anomalous data encountered in passing). Neither is anyone serious about True Cost Economics, Man-Machine Translation, Global Near-Real-Time Crowd-Sourcing (for observations, translations, and culturally-grounded interpretations) or M4IS2 (Multinational, Multiagency, Multidisciplinary, Multidomain Information-Sharing and Sense-Making).
I cannot help but recall my briefing to the National Research Council in 1994, when I was asked to comment on the US Army's multi-billion dollar communications plan for the future. I pointed out the obvious: the US Army was assuming that all data would be generated from within the US Army or other US Government systems, and was making no provision for ingesting and digesting data from the 99% of the data sources outside the US Army. Of course they blew me off then, and they still do not get it today, 22 years later.
The World Bank, ostensibly one of the more impressive collections of expertise and data, has no strategic model for actually managing data to achieve the eradication of the ten high-level threats to humanity, by harmonizing understanding and then spending by many disparate stakeholders. The United Nations is even worse. NATO, in a most interesting turn of events, is suddenly realizing what I personally briefed to all the generals and colonels in charge of the 66 or so military intelligence organizations across NATO and PfP (this was in 2000): in the 21st Century, heavy metal military does not answer the mail — Stabilization & Reconstruction requires both a deep grasp of non-secret or open data, and the ability to share virtually all that you know — with the government and military generally being the beneficiaries of sharing from others, NOT the benefactors of others that are so much more in touch with ground truth than any Western government or military could ever be. We are being flooded with largely meaningless data burdened by multiple information pathologies, and refusing to get a grip on the most fundamental data — true cost data — for every supply chain.
Below are a number of articles on “Data Is the New Oil” followed by a number of Phi Beta Iota posts intended to illuminate all that is NOT being done today by major governments, international organizations, and corporations, among others.
What I see is a very unhealthy obession with social and personal data, and a general dismissal of environmental, cause and effect, and true cost data.
It remains my view that every nation needs national legislation and a national plan for creating what I called in 1995 a Smart Nation. This is both vastly more complex than any secret intelligence or presidential information mandarin might imagine, and also vastly easier and inexpensive than one might imagine, if you know what you need to know. Creating an Open Source Agency under diplomatic and/or commercial auspices, or as an organic part of the Office of Management and Budget, would be a good step to take now.
Data Is the New Oil
Holistic Analytics, True Cost Economics, Translation, M4IS2