So, I pay attention whenever Tetlock writes something, even a book review. Thanks to my colleague, Steve Marrin (who knows of my interest in all things Tetlockian...), I recently had a chance to read Tetlock's review of three new books by people who claim to have an inside track when it comes to predicting the future.
I will let you read Tetlock's review of the books on your own. Suffice it to say that his critique of all three works is based on his research and, as a result, is skeptical (in varying degrees) of the claims of accuracy in the three books.
What really fascinated me about this review is buried at the end of it, though. Tetlock worries about how to improve forecasting, about how we can know which forecasters are worth listening to and which are modern day snake oil salesmen. Then, he makes a point that I agree with in whole:
- "There is one potential savior on the horizon: a big institutional purchaser of forecasting services that has the financial clout and technical-support staff ready to run forecasting tournaments that would shed light on the relative performance of competing approaches—a big player that also has powerful incentives to discover superior analytical strategies, for even small improvements in its prediction accuracy can translate into billions of dollars and millions of lives saved. And that player is the Office of the Director of National Intelligence."
- "Players high up in the political system—who really do want the best-possible forecasts—could decide that it is worth investing a nontrivial share of their intelligence agencies’ budgets into a series of long-term forecasting tournaments designed to distinguish the more from the less promising forecasting approaches across policy problems."
Still, it is a great idea. If we are ever to break out of the "two-thirds right" trap we are in, we need a robust research program aimed at validating intelligence methods and the DNI will have to be the one to sponsor it.