A Robust Learning Methodology for Uncertainty-Aware Scientific Machine Learning Models

E.A. Costa; Carine de Menezes Rebello; Márcio Fontana; L. Schnitman; Idelfonso B. R. Nogueira

doi:10.3390/math11010074

journal article Open Access Dec 25, 2022

A Robust Learning Methodology for Uncertainty-Aware Scientific Machine Learning Models

E.A. Costa Carine de Menezes Rebello

Márcio Fontana

L. Schnitman

Idelfonso B. R. Nogueira

Mathematics Vol. 11 No. 1 pp. 74 · MDPI AG

View at Publisher Save 10.3390/math11010074

Abstract

Robust learning is an important issue in Scientific Machine Learning (SciML). There are several works in the literature addressing this topic. However, there is an increasing demand for methods that can simultaneously consider all the different uncertainty components involved in SciML model identification. Hence, this work proposes a comprehensive methodology for uncertainty evaluation of the SciML that also considers several possible sources of uncertainties involved in the identification process. The uncertainties considered in the proposed method are the absence of a theory, causal models, sensitivity to data corruption or imperfection, and computational effort. Therefore, it is possible to provide an overall strategy for uncertainty-aware models in the SciML field. The methodology is validated through a case study developing a soft sensor for a polymerization reactor. The first step is to build the nonlinear model parameter probability distribution (PDF) by Bayesian inference. The second step is to obtain the machine learning model uncertainty by Monte Carlo simulations. In the first step, a PDF with 30,000 samples is built. In the second step, the uncertainty of the machine learning model is evaluated by sampling 10,000 values through Monte Carlo simulation. The results demonstrate that the identified soft sensors are robust to uncertainties, corroborating the consistency of the proposed approach.

Topics

No keywords indexed for this article. Browse by subject →

References

34

[1]

Rackauckas, C., Ma, Y., Martensen, J., Warner, C., Zubov, K., Supekar, R., Skinner, D., Ramadhan, A., and Edelman, A. (2020). Universal Differential Equations for Scientific Machine Learning. arXiv. 10.21203/rs.3.rs-55125/v1

[2]

Chuang "Adversarial Controls for Scientific Machine Learning" ACS Chem. Biol. (2018) 10.1021/acschembio.8b00881

[3]

Gaikwad "Heterogeneous sensing and scientific machine learning for quality assurance in laser powder bed fusion—A single-track study" Addit. Manuf. (2020)

[4]

Using scientific machine learning to develop universal differential equation for multicomponent adsorption separation systems

Idelfonso B. R. Nogueira, Vinicius V. Santana, Ana M. Ribeiro et al.

The Canadian Journal of Chemical Engineering 2022 10.1002/cjce.24495

[5]

Meng "Learning functional priors and posteriors from data and physics" J. Comput. Phys. (2022) 10.1016/j.jcp.2022.111073

[6]

Das "A Robust Machine Learning Based Framework for the Automated Detection of ADHD Using Pupillometric Biomarkers and Time Series Analysis" Sci. Rep. (2021) 10.1038/s41598-021-95673-5

[7]

Psaros, A.F., Meng, X., Zou, Z., Guo, L., and Karniadakis, G.E. (2022). Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons. arXiv. 10.1016/j.jcp.2022.111902

[8]

Hariri "Uncertainty in big data analytics: Survey, opportunities, and challenges" J. Big Data (2019) 10.1186/s40537-019-0206-3

[9]

Li, J.Z. (2018). Principled Approaches to Robust Machine Learning and Beyond. [Ph.D. Thesis, Massachusetts Institute of Technology].

[10]

Gal, Y., and Ghahramani, Z. (2016, January 19–24). Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA.

[11]

Yu "Optimal performance of hybrid energy system in the presence of electrical and heat storage systems under uncertainties using stochastic p-robust optimization technique" Sustain. Cities Soc. (2022) 10.1016/j.scs.2022.103935

[12]

Nogueira "Chromatographic studies of n-Propyl Propionate: Adsorption equilibrium, modelling and uncertainties determination" Comput. Chem. Eng. (2018) 10.1016/j.compchemeng.2018.09.020

[13]

Gneiting "Probabilistic and sharpness forecasts, calibration" Jrssb (2013) 10.1111/j.1467-9868.2007.00587.x

[14]

Abdar "A review of uncertainty quantification in deep learning: Techniques, applications and challenges" Inf. Fusion (2021) 10.1016/j.inffus.2021.05.008

[15]

Siddique, T., Mahmud, M., Keesee, A., Ngwira, C., and Connor, H. (2022). A Survey of Uncertainty Quantification in Machine Learning for Space Weather Prediction. Geosciences, 12. 10.3390/geosciences12010027

[16]

Costa, E., Rebello, C., Santana, V., Rodrigues, A., Ribeiro, A., Schnitman, L., and Nogueira, I. (2022). Mapping Uncertainties of Soft-Sensors Based on Deep Feedforward Neural Networks through a Novel Monte Carlo Uncertainties Training Process. Processes, 10. 10.3390/pr10020409

[17]

Evaluating and Calibrating Uncertainty Prediction in Regression Tasks

Dan Levi, Liran Gispan, Niv Giladi et al.

Sensors 10.3390/s22155540

[18]

Migon, S.H., Gamerman, D., and Louzada, F. (2014). Statistical Inference: An Integrated Approach, CRC Press. [2nd ed.]. 10.1201/b17229

[19]

Bard, Y. (1974). Nonlinear Parameter Estimation, Academic Press.

[20]

Gamerman, D., and Lopes, H.F. (2006). Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, Chapman and Hall/CRC. [2nd ed.]. 10.1201/9781482296426

[21]

Haario "DRAM: Efficient adaptive MCMC" Stat. Comput. (2006) 10.1007/s11222-006-9438-0

[22]

BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP, and OIML (2008). Evaluation of Measurement Data—Guide to the Expression of Uncertainty in Measurement, BIPM.

[23]

BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP, and OIML (2008). Evaluation of Measurement Data—Supplement 1 to the “Guide to the Expression of Uncertainty in Measurement”—Propagation of Distributions Using a Monte Carlo Method, BIPM.

[24]

BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP, and OIML (2011). Evaluation of Measurement Data—Supplement 2 to the "Guide to the Expression of Uncertainty in Measurement"—Models with Any Number of Output Quantities, BIPM.

[25]

Haykin, S. (1999). Neural Networks and Learning Machines, Pearson Prentice Hall. [2nd ed.].

[26]

He, X., and Asada, H. (1993, January 2–4). New method for identifying orders of input-output models for nonlinear dynamic systems. Proceedings of the 1993 American Control Conference, San Francisco, CA, USA. 10.23919/acc.1993.4793346

[27]

Li "Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization" J. Mach. Learn. Res. (2016)

[28]

An Adaptive Metropolis Algorithm

Heikki Haario, Eero Saksman, Johanna Tamminen

Bernoulli 2001 10.2307/3318737

[29]

Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., and Rubin, D.B. (2013). Bayesian Data Analysis Third Edition (with Errors FiXed as of 13 February 2020), Routledge. 10.1201/b16018

[30]

Hidalgo "Nonlinear model predictive control of styrene polymerization at unstable operating points" Comput. Chem. Eng. (1990) 10.1016/0098-1354(90)87022-h

[31]

Alvarez "Optimization and control of a continuous polymerization reactor" Braz. J. Chem. Eng. (2012) 10.1590/s0104-66322012000400012

[32]

Brooks "Assessing Convergence of Markov Chain Monte Carlo Algorithms" Stat. Comput. (1998) 10.1023/a:1008820505350

[33]

Possolo "Copulas for uncertainty analysis" Metrologia (2010) 10.1088/0026-1394/47/3/017

[34]

Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv.

Metrics

9

Citations

34

References

Details

Published: Dec 25, 2022
Vol/Issue: 11(1)
Pages: 74
License: View

Authors

E

E.A. Costa

Programa de Pós-Graduação em Mecatrônica, Universidade Federal da Bahia, Rua Prof. Aristides Novis, 2, Federação, Salvador 40210-630, Brazil

C

Carine de Menezes Rebello

Programa de Pós-Graduação em Engenharia Industrial, Universidade Federal da Bahia, Rua Prof. Aristides Novis, 2, Federação, Salvador 40210-630, Brazil

M

Márcio Fontana

Programa de Pós-Graduação em Mecatrônica, Universidade Federal da Bahia, Rua Prof. Aristides Novis, 2, Federação, Salvador 40210-630, Brazil

L

L. Schnitman

Programa de Pós-Graduação em Mecatrônica, Universidade Federal da Bahia, Rua Prof. Aristides Novis, 2, Federação, Salvador 40210-630, Brazil

I

Idelfonso B. R. Nogueira

Department of Chemical Engineering, Norwegian University of Science and Technology (NTNU), 7491 Trondheim, Norway

Cite This Article

E.A. Costa, Carine de Menezes Rebello, Márcio Fontana, et al. (2022). A Robust Learning Methodology for Uncertainty-Aware Scientific Machine Learning Models. Mathematics, 11(1), 74. https://doi.org/10.3390/math11010074

A Robust Learning Methodology for Uncertainty-Aware Scientific Machine Learning Models

You May Also Like