Three recommendations for evaluating climate predictions

Thomas E. Fricker; Christopher A. T. Ferro; David B. Stephenson

doi:10.1002/met.1409

journal article Open Access Jun 01, 2013

Three recommendations for evaluating climate predictions

Thomas E. Fricker Christopher A. T. Ferro David B. Stephenson

Meteorological Applications Vol. 20 No. 2 pp. 246-255 · Wiley

View at Publisher Save 10.1002/met.1409

Abstract

ABSTRACTEvaluation is important for improving climate prediction systems and establishing the credibility of their predictions of the future. This paper shows how the choices that must be made about how to evaluate predictions affect the outcome and ultimately our view of the prediction system's quality. The aim of evaluation is to measure selected attributes of the predictions, but some attributes are susceptible to having their apparent performance artificially inflated by the presence of climate trends, thus rendering past performance an unreliable indicator of future performance. We describe a class of performance measures that are immune to such spurious skill. The way in which an ensemble prediction is interpreted also has strong implications for the apparent performance, so we give recommendations about how evaluation should be tailored to different interpretations. Finally, we explore the role of the timescale of the predictand in evaluation and suggest ways to describe the relationship between timescale and performance. The ideas in this paper are illustrated using decadal temperature hindcasts from the CMIP5 archive. © 2013 The Authors. Meteorological Applications published by John Wiley & Sons Ltd on behalf of the Royal Meteorological Society.

Topics

No keywords indexed for this article. Browse by subject →

References

49

[1]

10.1002/qj.456

[2]

10.1002/qj.1891

[3]

10.5194/npg-18-1-2011

[4]

10.1175/waf966.1

[5]

Uncertainty estimates in regional and global observed temperature changes: A new data set from 1850

P. Brohan, J. J. Kennedy, I. Harris et al.

Journal of Geophysical Research: Oceans 2006 10.1029/2005jd006548

[6]

10.1007/s00382-009-0545-4

[7]

10.1256/qj.04.71

[8]

10.1175/2010jcli3164.1

[9]

10.1175/waf1034.1

[10]

10.1002/qj.1924

[11]

10.1002/met.45

[12]

10.1029/2011gl049508

[13]

10.1007/s00382-009-0614-8

[14]

10.1198/jasa.2011.r10138

[15]

10.1111/j.1467-9868.2007.00587.x

[16]

10.1198/016214506000001437

[17]

10.1007/s11749-008-0114-x

[18]

10.1007/s00382-012-1481-2

[19]

10.1029/2012ms000173

[20]

10.1256/qj.06.25

[21]

10.1175/jcli‐d‐12‐00512.1

[22]

Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems

Hans Hersbach

Weather and Forecasting 10.1175/1520-0434(2000)015<0559:dotcrp>2.0.co;2

[23]

10.1175/2011bams3110.1

[24]

Hudson D "Intraseasonal forecasting of the 2009 summer and winter Australian heat waves using POAMA" Weather Forecast. (2011) 10.1175/waf-d-10-05041

[25]

Jolliffe IT (2012)

[26]

10.1175/jcli-d-11-00586.1

[27]

10.1038/nature06921

[28]

10.1175/1520-0442(2003)016<1684:ispf>2.0.co;2

[29]

10.1002/joc.2273

[30]

10.1175/1520-0469(1963)020<0130:dnf>2.0.co;2

[31]

10.1175/2008mwr2553.1

[32]

10.1175/2009bams2778.1

[33]

10.1073/pnas.0906531107

[34]

10.1175/1520-0450(1973)012<0595:anvpot>2.0.co;2

[35]

10.1016/j.shpsb.2010.07.006

[36]

10.1175/2009jcli2535.1

[37]

10.1175/bams-84-4-455

[38]

Randall DA (2007)

[39]

10.1175/mwr-d-11-00350.1

[40]

10.1034/j.1600-0870.2003.201378.x

[41]

10.1073/pnas.012580599

[42]

10.1126/science.1139540

[43]

10.1175/1520-0493(2004)132<1522:etloef>2.0.co;2

[44]

Stephenson DB "Forecast assimilation: a unified framework for the combination of multi‐model weather and climate predictions" Tellus (2005) 10.3402/tellusa.v57i3.14664

[45]

10.1002/env.2153

[46]

10.1034/j.1600-0870.2000.d01-5.x

[47]

Can multi‐model combination really enhance the prediction skill of probabilistic ensemble forecasts?

A. P. Weigel, M. A. Liniger, C. Appenzeller

Quarterly Journal of the Royal Meteorological Soci... 10.1002/qj.210

[48]

World Meteorological Organization (2010)

[49]

10.1175/2011jcli4085.1

Cited By

36

Seasonal Climate Prediction: A New Source of Information for the Management of Wind Energy Resources

Verónica Torralba, Francisco J. Doblas-Reyes · 2017

Journal of Applied Meteorology and...

Fair scores for ensemble forecasts

Christopher A. T. Ferro · 2013

Quarterly Journal of the Royal Mete...

Metrics

36

Citations

49

References

Details

Published: Jun 01, 2013
Vol/Issue: 20(2)
Pages: 246-255
License: View

Authors

T

Thomas E. Fricker

College of Engineering, Mathematics and Physical Sciences University of Exeter UK

C

Christopher A. T. Ferro

College of Engineering, Mathematics and Physical Sciences University of Exeter UK

D

David B. Stephenson

College of Engineering, Mathematics and Physical Sciences University of Exeter UK

Cite This Article

Thomas E. Fricker, Christopher A. T. Ferro, David B. Stephenson (2013). Three recommendations for evaluating climate predictions. Meteorological Applications, 20(2), 246-255. https://doi.org/10.1002/met.1409

Three recommendations for evaluating climate predictions

You May Also Like