
Professor Philip E Tetlock
As lawyers, a lot of what we do is make predictions. At its most basic level, every bit of legal advice as to what the legal position is in any case is essentially a prediction: what would the courts make of it? In practice, the prediction is more subtle. What is the likely outcome of a legal engagement, bearing in mind that the vast majority of cases settle?
A good question is “How do we make these predictions?” And more importantly, “What can we do to make such predictions more accurate?”
These questions have hit a bit of additional topicality following the recent row in the UK about Andrew Sabisky. He is a special adviser, only recently appointed by Downing Street, who is what is known as a “superforecaster”. There is more science to this expression that you might think, not least because of the work of Philip Tetlock, who is a professor at the University of Pennsylvania, in the USA. He has done a good deal of work over the years on the subject of forecasting, or predictions, and has organised, through the Good Judgment Project, a series of forecasting tournaments over the last three decades. It turns out that the majority of people, including people ordinarily regarded as experts, are pretty rubbish at forecasts, barely better than random. In the longer the range of forecasts, the poorer their accuracy. But there are a few people – the so-called superforecasters – who are really rather good at it. It is not just luck. Time and again, these superforecasters (typically the best 2% of forecasters) prove themselves to be much better than run-of-the-mill experts are predicting what is going to happen. And Professor Tetlock and his team have done a good deal of work in identifying what makes these superforecasters better than others.
The answer turns out to be quite complicated; the superforecasters have a range of talents and techniques. One is the use of Continue reading