CNN, 10 October 2012
Two new books argue that algorithms can’t fully replace human judgment. A review of Samuel Arbesman’s The Half-Life of Facts and Nate Silver’s The Signal and The Noise.
The ironic takeaway from both these fine books, however, is that the more data and facts one has, and the more predictions matter, the more important human judgment becomes. The co-evolution of human beings, datasets, and algorithms will ultimately determine whether Big Data creates new wealth or destroys old value.
Samuel Arbesman, The Half-Life of Facts: Why Everything We Know Has an Expiration Date (Current Hardcover, 2012)
Doctors with a rough idea of when their knowledge is likely to expire can be better equipped to keep up with the latest research. Companies and governments that understand how long new discoveries take to develop can improve decisions about allocating resources. And by tracing how and when language changes, each of us can better bridge generational gaps in slang and dialect.
Nate Silver, The Signal and the Noise: Why So Many Predictions Fail-but Some Don’t (Penguin Press, 2012)
Drawing on his own groundbreaking work, Silver examines the world of prediction, investigating how we can distinguish a true signal from a universe of noisy data. Most predictions fail, often at great cost to society, because most of us have a poor understanding of probability and uncertainty. Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. This is the “prediction paradox”: The more humility we have about our ability to make predictions, the more successful we can be in planning for the future.