SOLVING HIGH-DIMENSIONAL INVERSE PROBLEMS WITH AUXILIARY UNCERTAINTY VIA OPERATOR LEARNING WITH LIMITED DATA

Joseph Hart; Mamikon Gulian; I. Manickam; Laura P. Swiler

doi:10.1615/jmachlearnmodelcomput.2023048105

journal article Jan 01, 2023

SOLVING HIGH-DIMENSIONAL INVERSE PROBLEMS WITH AUXILIARY UNCERTAINTY VIA OPERATOR LEARNING WITH LIMITED DATA

Joseph Hart Mamikon Gulian I. Manickam Laura P. Swiler

Journal of Machine Learning for Modeling and Computing Vol. 4 No. 2 pp. 105-133 · Begell House

View at Publisher Save 10.1615/jmachlearnmodelcomput.2023048105

Abstract

In complex large-scale systems such as climate, important effects are caused by a combination of confounding processes that are not fully observable. The identification of sources from observations of the system state is vital for attribution and prediction, which inform critical policy decisions. The difficulty of these types of inverse problems lies in the inability to isolate sources and the cost of simulating computational models. Surrogate models may enable the many-query algorithms required for source identification, but data challenges arise from high dimensionality of the state and source, limited ensembles of costly model simulations to train a surrogate model, and few and potentially noisy state observations for inversion due to measurement limitations. The influence of auxiliary processes adds an additional layer of uncertainty that further confounds source identification. We introduce a framework based on (1) calibrating deep neural network surrogates to the flow maps provided by an ensemble of simulations obtained by varying sources, and (2) using these surrogates in a Bayesian framework to identify sources from observations via optimization. Focusing on an atmospheric dispersion exemplar, we find that the expressive and computationally efficient nature of the deep neural network operator surrogates in appropriately reduced dimension allows for source identification with uncertainty quantification using limited data. Introducing a variable wind field as an auxiliary process, we find that a Bayesian approximation error approach is essential for reliable source inversion when uncertainty due to wind stresses the algorithm.

Topics

No keywords indexed for this article. Browse by subject →

References

58

[1]

Arias, P., Bellouin, N., Coppola, E., Jones, R., Krinner, G., Marotzke, J., Naik, V., Palmer, M., Plattner, G.K., Rogelj, J., et al., Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change

[2]

Technical Summary, The Intergovernmental Panel on Climate Change AR6, V. Masson-Delmotte, P. Zhai, A. Pirani, S. Conners, C. Pean, S. Berger, N. Caud, Y. Chen, L. Goldfarb, M. Gomis, M. Huang, K. Leitzell, E. Lonnoy, J. Matthews, T. Maycock, T. Waterfield, O. Yeleki, R. Yu, and B. Zhou, Eds., accessed from https://elib.dlr.de/137584/ pp. 1-40, 2021.

[3]

Arridge, S.R., Kaipio, J.P., Kolehmainen, V., Schweiger, M., Somersalo, E., Tarvainen, T., and Vauhkonen, M., Approximation Errors and Model Reduction with an Application in Optical Diffusion Tomography, Inverse Probl., vol. 22, no. 1, pp. 175-195, 2006. 10.1088/0266-5611/22/1/010

[4]

Asch, M., A Toolbox for Digital Twins: From Model-Based to Data-Driven, Philadelphia, PA: SIAM, 2022. 10.1137/1.9781611976977

[5]

Barnett, J. and Farhat, C., Quadratic Approximation Manifold for Mitigating the Kolmogorov Barrier in Nonlinear Projection-Based Model Order Reduction, J. Comput. Phys., vol. 464, p. 111348, 2022. 10.1016/j.jcp.2022.111348

[6]

Bhattacharya, K., Hosseini, B., Kovachki, N.B., and Stuart, A.M., Model Reduction and Neural Networks for Parametric PDEs, J. Comput. Math., vol. 7, pp. 121-157, 2021. 10.5802/smai-jcm.74

[7]

Biegler, L., Biros, G., Ghattas, O., Heinkenschloss, M., Keyes, D., Mallick, B., Marzouk, Y., Tenorio, L., van Bloemen Waanders, B., and Willcox, K., Eds., Large-Scale Inverse Problems and Quantification of Uncertainty, Hoboken, NJ: John Wiley and Sons, 2011. 10.1002/9780470685853

[8]

Boncoraglio, G. and Farhat, C., Piecewise-Global Nonlinear Model Order Reduction for PDE-Constrained Optimization in High-Dimensional Parameter Spaces, SIAM J. Sci. Comput., vol. 44, no. 4, pp. A2176-A2203, 2022. 10.1137/21m1435343

[9]

Brunton, S.L. and Kutz, J.N., Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control, Cambridge, UK: Cambridge University Press, 2022. 10.1017/9781009089517

[10]

A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems Part I: The Linearized Case, with Application to Global Seismic Inversion

Tan Bui-Thanh, Omar Ghattas, James Martin et al.

SIAM Journal on Scientific Computing 10.1137/12089586x

[11]

Model Reduction for Large-Scale Systems with High-Dimensional Parametric Input Space

T. Bui-Thanh, K. Willcox, O. Ghattas

SIAM Journal on Scientific Computing 10.1137/070694855

[12]

Cai, S.,Wang, Z., Lu, L., Zaki, T.A., and Karniadakis, G.E., DeepM&Mnet: Inferring the Electroconvection Multiphysics Fields Based on Operator Approximation by Neural Networks, J. Comput. Phys., vol. 436, p. 110296, 2021. 10.1016/j.jcp.2021.110296

[13]

Cleary, E., Garbuno-Inigo, A., Lan, S., Schneider, T., and Stuart, A.M., Calibrate, Emulate, Sample, J. Comput. Phys., vol. 424, no. 1, 2021. 10.1016/j.jcp.2020.109716

[14]

Cui, T., Law, K.J., and Marzouk, Y.M., Dimension-Independent Likelihood-Informed MCMC, J. Comput. Phys., vol. 304, no. 1, pp. 109-137, 2016. 10.1016/j.jcp.2015.10.008

[15]

Likelihood-informed dimension reduction for nonlinear inverse problems

T Cui, J Martin, Y M Marzouk et al.

Inverse Problems 10.1088/0266-5611/30/11/114015

[16]

Department of Energy Office of Science, Explains...Earth System and Climate Models, accessed from https://www.energy.gov/science/doe-explainsearth-system-and-climate-models, 2023.

[17]

Enting, I.G., Inverse Problems in Atmospheric Constituent Transport, Cambridge, UK: Cambridge University Press, 2002. 10.1017/cbo9780511535741

[18]

Gao, X., Huang, A., Trask, N., and Reza, S., Physics-Informed Graph Neural Network for Circuit Compact Model Development, 2020 Int. Conf. on Simulation of Semiconductor Processes and Devices (SISPAD), IEEE, pp. 359-362, September 23-October 6, 2020. 10.23919/sispad49475.2020.9241634

[19]

Glorot, X. and Bengio, Y., Understanding the Difficulty of Training Deep Feedforward Neural Networks, Proc. of the Thirteenth Int. Conf. on Artificial Intelligence and Statistics, JMLR Workshop and Conf. Proc., pp. 249-256, Chia Laguna Resort, Sardinia, Italy, May 13-15, 2010.

[20]

Heavens, N.G., Ward, D., and Mahowald, N., Studying and Projecting Climate Change with Earth System Models, Nat. Ed. Knowl., vol. 4, no. 5, p. 4, 2013.

[21]

Non-intrusive reduced order modeling of nonlinear problems using neural networks

J.S. Hesthaven, S. Ubbiali

Journal of Computational Physics 10.1016/j.jcp.2018.02.037

[22]

Isaac, T., Petra, N., Stadler, G., and Ghattas, O., Scalable and Efficient Algorithms for the Propagation of Uncertainty from Data through Inference to Prediction for Large-Scale Problems, with Application to Flow of the Antarctic Ice Sheet, J. Comput. Phys., vol. 296, pp. 348-368, 2015. 10.1016/j.jcp.2015.04.047

[23]

Kaipio, J. and Kolehmainen, V., Approximate Marginalization over Modeling Errors and Uncertainties in Inverse Problems, Bayesian Theory and Applications, P. Damien, P. Dellaportas, N.G. Polson, and D.A. Stevens, Eds., pp. 644-672, Oxford, UK: Oxford University Press, 2013. 10.1093/acprof:oso/9780199695607.003.0032

[24]

Kissas, G., Seidman, J.H., Guilhoto, L.F., Preciado, V.M., Pappas, G.J., and Perdikaris, P., Learning Operators with Coupled Attention, J. Machine Learn. Res., vol. 23, no. 215, pp. 1-63, 2022.

[25]

Laird, C.D., Biegler, L.T., van Bloemen Waanders, B., and Bartlett, R.A., Contamination Source Determination for Water Networks, J. Water Res. Plan. Manag., vol. 131, no. 2, pp. 125-134, 2005. 10.1061/(asce)0733-9496(2005)131:2(125)

[26]

Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders

Kookjin Lee, Kevin T. Carlberg

Journal of Computational Physics 10.1016/j.jcp.2019.108973

[27]

Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A., Neural Operator: Grap. Kernel Network for Partial Differential Equations, Advances in Neural Information Processing Systems 33 (NeurIPS), pp. 1-21, December 6-12, 2020.

[28]

Long, Z., Lu, Y., Ma, X., and Dong, B., PDE-Net: Learning PDEs from Data, Proc. of the 35th Int. Conf. on Machine Learning, J. Dy and A. Krause, Eds., Vol. 80, Proc. Machine Learning Research, PMLR, pp. 3208-3216, 2018.

[29]

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

Lu Lu, Pengzhan Jin, Guofei Pang et al.

Nature Machine Intelligence 10.1038/s42256-021-00302-5

[30]

A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion

James Martin, Lucas C. Wilcox, Carsten Burstedde et al.

SIAM Journal on Scientific Computing 10.1137/110845598

[31]

Maupin, K.A., Swiler, L.P., and Porter, N.W., Validation Metrics for Deterministic and Probabilistic Data, J. Verificat., Validat. Uncertain. Quantif., vol. 3, no. 3, p. 031002, 2018. 10.1115/1.4042443

[32]

McQuarrie, S.A., Huang, C., and Willcox, K.E., Data-Driven Reduced-Order Models via Regularised Operator Inference for a Single-Injector Combustion Process, J. Roy. Soc. New Zealand, vol. 51, no. 2, pp. 194-211, 2021. 10.1080/03036758.2020.1863237

[33]

McQuarrie, S.A., Khodabakhshi, P., andWillcox, K.E., Non-Intrusive Reduced-Order Models for Parametric Partial Differential Equations via Data-Driven Operator Inference, arXiv: 2110.07653, 2023. 10.1137/21m1452810

[34]

Nicholson, R., Petra, N., and Kaipio, J.P., Estimation of the Robin Coefficient Field in a Poisson Problem with Uncertain Conductivity Field, Inverse Probl., vol. 34, p. 115005, 2018. 10.1088/1361-6420/aad91e

[35]

Nissinen, A., Heikkinen, L.M., Kolehmainen, V., and Kaipio, J.P., Compensation of Errors Due to Discretization, Domain Truncation and Unknown Contact Impedances in Electrical Impedance Tomography, Measure. Sci. Technol., vol. 20, no. 10, p. 105504, 2009. 10.1088/0957-0233/20/10/105504

[36]

Patel, R., Manickam, I., Lee, M., and Gulian, M., Error-in-Variables Modelling for Operator Learning, Proc. of 3rd Annual Conf. on Mathematical and Scientific Machine Learning, PMLR, vol. 145, pp. 142-157, 2022. 10.2172/2004345

[37]

Patel, R.G. and Desjardins, O., Nonlinear Integro-Differential Operator Regression with Neural Networks, arXiv: 1810.08552, 2018.

[38]

Patel, R.G., Trask, N.A.,Wood, M.A., and Cyr, E.C., A Physics-Informed Operator Regression Framework for Extracting Data-Driven Continuum Models, Comput. Methods Appl. Mech. Eng., vol. 373, p. 113500, 2021. 10.1016/j.cma.2020.113500

[39]

Data-driven operator inference for nonintrusive projection-based model reduction

Benjamin Peherstorfer, Karen Willcox

Computer Methods in Applied Mechanics and Engineer... 10.1016/j.cma.2016.03.025

[40]

Petra, N., Martin, J., Stadler, G., and Ghattas, O., A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems. Part II: Stochastic Newton MCMC with Application to Ice Sheet Flow Inverse Problems, SIAM J. Sci. Comput., vol. 36, no. 4, pp. A1525-A1555, 2014. 10.1137/130934805

[41]

Petra, N., Zhu, H., Stadler, G., Hughes, T.J., and Ghattas, O., An Inexact Gauss-Newton Method for Inversion of Basal Sliding and Rheology Parameters in a Nonlinear Stokes Ice Sheet Model, J. Glaciol., vol. 58, no. 211, pp. 889-903, 2012. 10.3189/2012jog11j182

[42]

Qin, T., Chen, Z., Jakeman, J.D., and Xiu, D., Data-Driven Learning of Nonautonomous Systems, SIAM J. Sci. Comput., vol. 43, no. 3, pp. A1607-A1624, 2021. 10.1137/20m1342859

[43]

Qin, T., Wu, K., and Xiu, D., Data Driven Governing Equations Approximation Using Deep Neural Networks, J. Comput. Phys., vol. 395, pp. 620-635, 2019. 10.1016/j.jcp.2019.06.042

[44]

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

M. Raissi, P. Perdikaris, G.E. Karniadakis

Journal of Computational Physics 10.1016/j.jcp.2018.10.045

[45]

Ratnaswamy, V., Stadler, G., and Gurnis, M., Adjoint-Based Estimation of Plate Coupling in a Non-Linear Mantle Flow Model: Theory and Examples, Geophys. J. Int., vol. 202, no. 2, pp. 768-786, 2015. 10.1093/gji/ggv166

[46]

Sargsyan, K., Surrogate Models for Uncertainty Propagation and Sensitivity Analysis, Handbook of Uncertainty Quantification, pp. 673-698, Berlin, Germany: Springer, 2017. 10.1007/978-3-319-12385-1_22

[47]

Shlens, J., A Tutorial on Principal Component Analysis, arXiv: 1404.1100, 2014.

[48]

Shukla, K., Xu, M., Trask, N., and Karniadakis, G.E., Scalable Algorithms for Physics-Informed Neural and Grap. Networks, Data-Centric Eng., vol. 3, p. e24, 2022. 10.1017/dce.2022.24

[49]

Smith, R.C., Uncertainty Quantification: Theory, Implementation, and Applications, Vol. 12, Philadelphia, PA: SIAM, 2013. 10.1137/1.9781611973228

[50]

Stuart, A.M., Inverse Problems: A Bayesian Perspective, Acta Numer., vol. 19, pp. 451-559, 2010. 10.1017/s0962492910000061

Showing 50 of 58 references

Metrics

10

Citations

58

References

Details

Published: Jan 01, 2023
Vol/Issue: 4(2)
Pages: 105-133

Authors

J

Joseph Hart

University of North Carolina, Chapel Hill, NC, USA; Investigation performed at the University of Wisconsin–Madison, Madison, Wisconsin, USA

Cite This Article

Joseph Hart, Mamikon Gulian, I. Manickam, et al. (2023). SOLVING HIGH-DIMENSIONAL INVERSE PROBLEMS WITH AUXILIARY UNCERTAINTY VIA OPERATOR LEARNING WITH LIMITED DATA. Journal of Machine Learning for Modeling and Computing, 4(2), 105-133. https://doi.org/10.1615/jmachlearnmodelcomput.2023048105

SOLVING HIGH-DIMENSIONAL INVERSE PROBLEMS WITH AUXILIARY UNCERTAINTY VIA OPERATOR LEARNING WITH LIMITED DATA

You May Also Like