A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

Maria J. Molina; David John Gagne; A. F. Prein

doi:10.1029/2020ea001490

journal article Open Access Aug 30, 2021

A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

Maria J. Molina

David John Gagne

A. F. Prein

Earth and Space Science Vol. 8 No. 9 · American Geophysical Union (AGU)

View at Publisher Save 10.1029/2020ea001490

Abstract

AbstractThis is a test case study assessing the ability of deep learning methods to generalize to a future climate (end of 21st century) when trained to classify thunderstorms in model output representative of the present‐day climate. A convolutional neural network (CNN) was trained to classify strongly rotating thunderstorms from a current climate created using the Weather Research and Forecasting model at high‐resolution, then evaluated against thunderstorms from a future climate and found to perform with skill and comparatively in both climates. Despite training with labels derived from a threshold value of a severe thunderstorm diagnostic (updraft helicity), which was not used as an input attribute, the CNN learned physical characteristics of organized convection and environments that are not captured by the diagnostic heuristic. Physical features were not prescribed but rather learned from the data, such as the importance of dry air at mid‐levels for intense thunderstorm development when low‐level moisture is present (i.e., convective available potential energy). Explanation techniques also revealed that thunderstorms classified as strongly rotating are associated with learned rotation signatures. Results show that the creation of synthetic data with ground truth is a viable alternative to human‐labeled data and that a CNN is able to generalize a target using learned features that would be difficult to encode due to spatial complexity. Most importantly, results from this study show that deep learning is capable of generalizing to future climate extremes and can exhibit out‐of‐sample robustness with hyperparameter tuning in certain applications.

Topics

No keywords indexed for this article. Browse by subject →

References

82

[1]

Abadi M. (2016)

[2]

10.1093/acrefore/9780190228620.013.62

[3]

10.1175/bams-d-15-00150.1

[4]

Barlage M. (2018)

[5]

10.1029/2019gl084944

[6]

10.5194/ascmo-5-147-2019

[7]

Random Forests

Leo Breiman

Machine Learning 10.1023/a:1010933404324

[8]

10.1016/j.atmosres.2012.04.002

[9]

10.1175/1520-0434(2003)018<0626:ceoldt>2.0.co;2

[10]

10.1175/waf949.1

[11]

SMOTE: Synthetic Minority Over-sampling Technique

N. V. Chawla, K. W. Bowyer, L. O. Hall et al.

Journal of Artificial Intelligence Research 10.1613/jair.953

[12]

Chollet F. (2015)

[13]

10.1175/waf-d-12-00038.1

[14]

The ERA‐Interim reanalysis: configuration and performance of the data assimilation system

D. P. Dee, S. M. Uppala, A. J. Simmons et al.

Quarterly Journal of the Royal Meteorological Soci... 10.1002/qj.828

[15]

10.1073/pnas.1307758110

[16]

10.1175/1520-0434(1996)011<0560:fffaib>2.0.co;2

[17]

10.1175/2009waf2222338.1

[18]

Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition

Kunihiko Fukushima, Sei Miyake

Lecture Notes in Biomathematics 10.1007/978-3-642-46466-9_18

[19]

10.1175/mwr-d-18-0316.1

[20]

10.1038/s41612-018-0048-2

[21]

10.1007/s10584-014-1320-z

[22]

Glorot X. (2010)

[23]

Goodfellow I. (2016)

[24]

10.1175/waf-d-17-0150.1

[25]

10.1038/s41586-019-1559-7

[26]

10.1175/mwr3199.1

[27]

10.1175/jcli-d-16-0885.1

[28]

10.1016/0169-2070(86)90048-8

[29]

10.1029/2008jd009944

[30]

Ioffe S. (2015)

[31]

10.1175/waf-d-19-0170.1

[32]

10.1175/waf2007106.1

[33]

Kingma D. P. (2014)

[34]

Krizhevsky A. (2012)

[35]

10.1175/waf-d-18-0183.1

[36]

10.1175/mwr-d-19-0372.1

[37]

10.1175/2008jtecha1153.1

[38]

10.1175/jtech-d-13-00205.1

[39]

Deep learning

Yann LeCun, Yoshua Bengio, Geoffrey Hinton

Nature 10.1038/nature14539

[40]

LeCun Y. (1990)

[41]

Gradient-based learning applied to document recognition

Y. Lecun, L. Bottou, Y. Bengio et al.

Proceedings of the IEEE 10.1109/5.726791

[42]

Continental-scale convection-permitting modeling of the current and future climate of North America

Changhai Liu, Kyoko Ikeda, Roy Rasmussen et al.

Climate Dynamics 10.1007/s00382-016-3327-9

[43]

Liu Y. (2016)

[44]

Mamalakis A. (2021)

[45]

Mason I. "A model for assessment of weather forecasts" Australian Meteorological Magazine (1982)

[46]

10.1175/bams-d-18-0195.1

[47]

10.1016/j.wace.2020.100244

[48]

10.1175/waf-d-19-0240.1

[49]

Molina M. J. (2020)

[50]

10.1038/nature08823

Showing 50 of 82 references

Cited By

31

Machine Learning of Key Variables Impacting Extreme Precipitation in Various Regions of the Contiguous United States

Xinming Lin, Jiwen Fan · 2023

Journal of Advances in Modeling Ear...

Using an Explainable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence

Sam J. Silva, Christoph A. Keller · 2022

Journal of Advances in Modeling Ear...

Metrics

31

Citations

82

References

Details

Published: Aug 30, 2021
Vol/Issue: 8(9)
License: View

Authors

M

Maria J. Molina

National Center for Atmospheric Research Boulder CO USA

D

David John Gagne

National Center for Atmospheric Research Boulder CO USA

A

A. F. Prein

National Center for Atmospheric Research Boulder CO USA

Funding

National Science Foundation Award: ICER‐2019758

U.S. Department of Energy Award: NSF IA 1947282

Cite This Article

Maria J. Molina, David John Gagne, A. F. Prein (2021). A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate. Earth and Space Science, 8(9). https://doi.org/10.1029/2020ea001490

A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

You May Also Like