Experts aim to make forecasting future more accurate in wake of blunders
It’s officially known as the Forecasting World Events Project and is sponsored by the Intelligence Advanced Research Activity (IARPA), a little-known agency run by Lisa Porter, who is occasionally described as America’s answer to the fictional Agent Q who designs cutting-edge gadgets for James Bond. Much of IARPA’s work is classified, as is its budget. But the forecasting project is not classified. Invitations to participate are now on the Internet.
The idea is to raise five large competing teams of people of diverse backgrounds who will be asked to make predictions on fields that range from politics and global security to business and economics, public health, social and cultural change and science and technology.
One of the teams is being put together by University of Pennsylvania professor Philip Tetlock, whose ground-breaking 2005 book (Expert Political Judgment: How Good is It? How Can We Know?) analysed 27,450 predictions from a variety of experts and found they were no more accurate than random guesses or, as he put it — a dart-throwing chimpanzee.
“To test various hypotheses,” Tetlock said in an interview, “we want a large number on my team, 2,500 or so, which would make it almost ten times bigger than the number I analysed in my book.” Dan Gardner, the author of a book that also highlights the shortcomings of expert predictions, believes the IARPA-sponsored project will be the biggest of its kind.
The title of Gardner’s book, Future Babble. Why expert predictions are next to worthless and you can do better, leaves no doubts over his conclusion. The book is an entertaining, well researched guide to decades of totally wrong predictions from eminent figures. There was the British writer HN. Norman, for example, who, in the peaceful early days of 1914, predicted there would be no more wars between the big powers of the time. World War I started a few months later.
Not to forget the US Defense Intelligence Agency’s September 1978 prediction that the Shah of Iran “is expected to remain actively involved in power over the next ten years.” The Shah fled into exile three months later, forced out by increasingly violent demonstrations against his autocratic rule.
“We are not clairvoyant,” America’s intelligence czar, James Clapper, told a hearing of the House Intelligence Committee where criticism of the sprawling US intelligence community was aired.
Specific triggers for how and when instability would lead to the collapse of various regimes cannot always be known or predicted.”
True enough. Who could have predicted that the assassination of an archduke in Sarajevo in 1914 would lead to the deaths of 16 million people in World War I? Who could have predicted Japan’s recent earthquake, tsunami and nuclear reactor disaster? On the other hand, there were accurate predictions that US troops invading Iraq in 2003 would not be showered with flowers, as Washington officials had so confidently predicted.
IARPA’s aim, as explained in an online invitation to participate, is to “dramatically enhance the accuracy, precision and timeliness” of forecasts. Gardner, the forecast sceptic, thinks they will remind us that there are things that simply can’t be predicted.
