When forecasting outside of Italy, we advise forecasters to use evidence-based principles of forecasting. For example, forecasters should provide evidence-based information on uncertainty. Failing to use evidence-based principles seems to be the only sound basis to challenge bad forecastsâ€”other than intent to deceive, which sometimes occurs in financial markets. Consider the field of medicine: Doctors are often successful sued when they fail to use evidence-based procedures. For forecasting, we refer readers to "Standards and Practices for Forecasting". As far as we are aware, it is the only published list of evidence-based principles of forecasting. It contains a short review of law cases involving faulty forecasts in the last section.
Scott Armstrong and Kesten Green
Compare your forecast for Chart A with the blue trend line in the following chart of temperatures. Monitor your Chart B forecast accuracy progress at The Climate Bet site. The Quiz is based on an idea from Warren Meyer's presentation of two charts cut and pasted from a larger temperature chart (see p. 48 of his presentation here).
Copy of "Mystery" temperature chart quiz result as pdf, here.
Grade your response out of 10*, as follows...
A squiggly line with a strong trend: 0
A squiggly line with a moderate trend: 1
A squiggly line with no obvious trend: 2
A straight line with a strong trend: 3
A straight line with a moderate trend**: 7
A straight line with no trend***: 10
*The grades are based on the principle that one should be conservative in forecasting when there is much uncertainty and complexity."
**Give yourself a 3 point bonus if your line matched the very modest downward trend exhibited by the actual temperature series over the 25 years you forecasted.
***Congratulations! Your forecast was consistent with the evidence-based forecasting principle of conservatism that applies on this situation.
Results for the Mystery Temperature Chart Quiz from an informal survey
# of responses
A squiggly line with a strong trend: 15
A squiggly line with a moderate trend: 8
A squiggly line with no obvious trend: 4
A straight line with a strong trend: 9
A straight line with a moderate trend: 5
A straight line with no trend: 9
A straight line that best matched the blue trend line above 1
Current version of working paper (please do not cite):
Green, KC, Soon, W, & Armstrong, JS. (2013) "Evidence-based Forecasting for Climate Change Policies".
Temperature Forecasting Quiz
Just for fun, test your climate forecasting skill by printing the image of the Quiz page showing two charts, below, and draw in your 25-year forecasts. (A pdf file copy of the Mystery Temperature Chart Quiz is available here.)
When you have made your forecasts, check them against the actuals and grade them, here.
Slides used in 25 June 2012 International Symposium on Forecasting (Boston) panel session on climate forecasting. Panelists providing commentary were Leonard Smith, Richard Lindzen, and Robert Fildes.
Previous evidence-based climate forecasting papers
Fildes, R. & Kourentzes, N. (2011). Validation and forecasting accuracy in models of climate change. International Journal of Forecasting, 27, 968â€“995.
Armstrong, J. S., Green, K. C., & Soon, W. (2011). Research on forecasting for the manmade global warming alarm. Energy and Environment, 22, 1091-1104. [Testimony to U.S. House Committee on Science, Space, and Technology]
Green, K. C., Armstrong, J. S., & Soon, W. (2009). Validity of climate change forecasting for public policy decision making. International Journal of Forecasting, 25, 826-832.
Armstrong, J. S., Green, K. C., & Soon, W. (2008). Polar bear population forecasts: A public-policy forecasting audit. Interfaces (with commentary), 38, 382-404.
Armstrong, J. S., Green, K. C., & Soon, W. (2008). What is the appropriate public-policy response to uncertainty? Interfaces, 38, 404-405.
Green, K. C. & Armstrong, J. S. (2007). Global warming: Forecasts by scientists versus scientific forecasts. Energy and Environment, 18, 997-1021.