Pierre Levy: The Limits of Big Data – The More You Have, The More Human Judgment Matters

0Shares

Michael Schrage

CNN, 10 October 2012

Two new books argue that algorithms can't fully replace human judgment. A review of Samuel Arbesman's The Half-Life of Facts and Nate Silver's The Signal and The Noise.

EXTRACT:

The ironic takeaway from both these fine books, however, is that the more data and facts one has, and the more predictions matter, the more important human judgment becomes. The co-evolution of human beings, datasets, and algorithms will ultimately determine whether Big Data creates new wealth or destroys old value.

Read full review.

Samuel Arbesman, The Half-Life of Facts: Why Everything We Know Has an Expiration Date (Current Hardcover, 2012)

FROM DESCRIPTION:

But it turns out there’s an order to the state of knowledge, an explanation for how we know what we know. Samuel Arbesman is an expert in the field of scientometrics—literally the science of science. Knowledge in most fields evolves systematically and predictably, and this evolution unfolds in a fascinating way that can have a powerful impact on our lives.

Doctors with a rough idea of when their knowledge is likely to expire can be better equipped to keep up with the latest research. Companies and governments that understand how long new discoveries take to develop can improve decisions about allocating resources. And by tracing how and when language changes, each of us can better bridge generational gaps in slang and dialect.

Nate Silver, The Signal and the Noise: Why So Many Predictions Fail-but Some Don't (Penguin Press, 2012)

FROM DESCRIPTION:

Drawing on his own groundbreaking work, Silver examines the world of prediction, investigating how we can distinguish a true signal from a universe of noisy data. Most predictions fail, often at great cost to society, because most of us have a poor understanding of probability and uncertainty. Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. This is the “prediction paradox”: The more humility we have about our ability to make predictions, the more successful we can be in planning for the future.