What does Superforecasting do? Not do?

Forecasting is a basic part of business planning in every organisation – forecasts of the external business environment, the market, the competition. Forecasts aim for accuracy. Professor Philip Tetlock is a social scientist now at the University of Pennsylvania who has designed and run major experiments on the characteristics of successful forecasters. A summary of his findings is captured in the acronym CHAMP below, and his latest book is “Superforecasting: the art and science of prediction”, written with Dan Gardner, Random House, ISBN 978-1-847-94714-7.

Tetlock’s CHAMP Comparisons are important:   use relevant comparisons as a starting point Historical trends can help: look at history unless you have a strong reason to expect change Average out opinions: experts disagree so find out what they think and pick a mid-point Mathematical models: when model based predictions are available you should take them into account Predictable biases exist and can be allowed for. Don’t confuse hopes with forecasts; don’t cling to old forecasts in the face of news.

The book is excellent and should be read by all who have responsibility for making decisions – whether personal or in organisations. It offers practical advice on how to use the methods used by successful forecasters.

We in SAMI have been scenario planning practitioners for all of our 25 years. Does this mean that we are going over to superforecasting?

We recognise and endorse the ingredients of CHAMP:

Absolutely, comparisons between industries and between countries are a staple ingredient of building scenarios

Absolutely, historical trends can help because then as Eisenhower famously said, Plans are worthless, but planning is everything, and planning needs to build on what we carry forward from the past.

Average out opinions – here is the crux of the difference – scenarios aim to explore different possible futures, often based on different current competing philosophies.

Mathematical models – we will explore in later blogs the role of mathematical models in exploring the future – essentially the implications that models are based on assumptions and can mislead unless these assumptions are transparent and explicit to all users of the model. And most models are not constructed to be able to handle fundamental shifts in underlying behaviour.

Absolutely, predictable biases exist – yes! This is so important, and we run a training course on this, details on www.samiconsulting.co.uk/training.

It is important to note the scope of the superforecasters in terms of both subject matter and timescale.

Subject matter: the experimental data on successful forecasters is based on global political forecasting (in the original book, Tetlock, P.E. Expert political judgment: How good is it? How can we know?. Princeton University Press. ISBN 9780691128719.) and in creating his subsequent panel of volunteers. Do the heuristics apply to forecasting of take up of technology and ensuing social change? Here, for instance, it can be argued that averaging out of opinions may mean that prudent planning is not done, as in the telecoms companies who were late in explore the changes in traffic patterns from mobiles and the internet-driven data flows. Or Steve Ballmer, then CEO of Microsoft, “There is no chance that the iPhone is going to get any significant market share. No chance”

Timescale: judging forecasts is difficult unless they are explicit about the timescale. And most forecasts don’t – like Steve Ballmer’s above. But we note that the experimental data about superforecasters quoted in the book is that superforcasters looking out 300 days were more accurate the forecasters looking out 100 days.

So, as our work is often about futures 5 years to 50 out, we learn what we can from Superforecasters and then need to add in the potential paradigm changers and explore these through scenarios – possible different futures.

Written by Gill Ringland, SAMI Fellow Emeritus.

The views expressed are those of the author and not necessarily of SAMI Consulting.

If you enjoyed this blog from SAMI Consulting, the home of scenario planning, please sign up for our monthly newsletter at eSAMIsignup@samiconsulting.co.uk and/or browse our website at http://www.samiconsulting.co.uk

 

Advertisement