Library Subscription: Guest
Begell Digital Portal Begell Digital Library eBooks Journals References & Proceedings Research Collections
International Journal for Uncertainty Quantification
IF: 3.259 5-Year IF: 2.547 SJR: 0.531 SNIP: 0.8 CiteScore™: 1.52

ISSN Print: 2152-5080
ISSN Online: 2152-5099

Open Access

International Journal for Uncertainty Quantification

DOI: 10.1615/Int.J.UncertaintyQuantification.2019028753
pages 103-121

HESSIAN-BASED SAMPLING FOR HIGH-DIMENSIONAL MODEL REDUCTION

Peng Chen
Institute for Computational Engineering & Sciences, The University of Texas at Austin, Austin, Texas 78712, USA
Omar Ghattas
Institute for Computational Engineering & Sciences, The University of Texas at Austin, Austin, Texas 78712, USA; Department of Mechanical Engineering, and Department of Geological Sciences, The University of Texas at Austin, Austin, Texas 78712, USA

ABSTRACT

In this work we develop a Hessian-based sampling method for the construction of goal-oriented reduced order models with high-dimensional parameter inputs. Model reduction is known to be very challenging for high-dimensional parametric problems whose solutions also live in high-dimensional manifolds. However, the manifold of some quantity of interest (QoI) depending on the parametric solutions may be low-dimensional. We use the Hessian of the QoI with respect to the parameter to detect this low-dimensionality, and draw training samples by projecting the high-dimensional parameter to a low-dimensional subspace spanned by the eigenvectors of the Hessian corresponding to its dominating eigenvalues. Instead of forming the full Hessian, which is computationally intractable for a high-dimensional parameter, we employ a randomized algorithm to efficiently compute the dominating eigenpairs of the Hessian whose cost does not depend on the nominal dimension of the parameter but only on the intrinsic dimension of the QoI.We demonstrate that the Hessian-based sampling leads to much smaller errors of the reduced basis approximation for the QoI compared to a random sampling for a diffusion equation with random input obeying either uniform or Gaussian distributions.

REFERENCES

  1. Giles, M.B., Multilevel Monte Carlo Methods, Acta Numer, 24:259-328, 2015.

  2. Dick, J., Kuo, F. Y., and Sloan, I.H., High-Dimensional Integration-The Quasi-Monte Carlo Way, Acta Numer., 22:133-288, 2013.

  3. Dick, J., Le Gia, Q.T., and Schwab, C., Higher Order Quasi-Monte Carlo Integration for Holomorphic, Parametric Operator Equations, SIAM/ASA J. Uncertainty Quantif., 4(1):48-79,2016.

  4. Cohen, A., Devore, R., and Schwab, C., Analytic Regularity and Polynomial Approximation of Parametric and Stochastic Elliptic PDE's, Anal. Appl., 9(1):11-47, 2011.

  5. Xiu, D. and Hesthaven, J.S., High-Order Collocation Methods for Differential Equations with Random Inputs, SIAM J. Sci. Comput., 27(3):1118-1139, 2005.

  6. Babuska, I., Nobile, F., and Tempone, R., A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data, SIAMRev., 52(3):317-355, 2010.

  7. Kressner, D. and Tobler, C., Low-Rank Tensor Krylov Subspace Methods for Parametrized Linear Systems, SIAM J. Matrix Anal. Appl., 32(4):1288-1316, 2011.

  8. Nouy, A., Low-Rank Methods for High-Dimensional Approximation and Model Order Reduction, arXiv preprint arXiv:1511.01554, 2015.

  9. Bonizzoni, F. and Nobile, F., Perturbation Analysis for the Darcy Problem with Log-Normal Permeability, SIAM/ASA J. Uncertainty Quantif, 2(1):223-244, 2014.

  10. Alexanderian, A., Petra, N., Stadler, G., and Ghattas, O., Mean-Variance Risk-Averse Optimal Control of Systems Governed by PDEs with Random Parameter Fields using Quadratic Approximations, SIAM/ASA J. Uncertainty Quantif., 5(1):1166- 1192,2017.

  11. Rozza, G., Huynh, D.B.P., and Patera, A.T., Reduced Basis Approximation and a Posteriori Error Estimation for Affinely Parametrized Elliptic Coercive Partial Differential Equations, Arch. Comput. Methods Eng., 15(3):229-275,2008.

  12. Bui-Thanh, T., Willcox, K., and Ghattas, O., Model Reduction for Large-Scale Systems with High-Dimensional Parametric Input Space, SIAMJ. Sci. Comput, 30(6):3270-3288,2008.

  13. Binev, P., Cohen, A., Dahmen, W., DeVore, R., Petrova, G., and Wojtaszczyk, P., Convergence Rates for Greedy Algorithms in Reduced Basis Methods, SIAMJ. Math. Anal., 43(3):1457-1472, 2011.

  14. Chen, P. and Schwab, C., Model Order Reduction Methods in Computational Uncertainty Quantification, in Handbook of Uncertainty Quantification, R. Ghanem, D. Higdon, and H. Owhadi, Eds., Berlin: Springer, pp. 1-53, 2016.

  15. Chen, P., Quarteroni, A., and Rozza, G., Reduced Basis Methods for Uncertainty Quantification, SIAM/ASA J. Uncertainty Quantif., 5(1):813-869, 2017.

  16. Benner, P., Gugercin, S., and Willcox, K., A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems, SIAMRev., 57(4):483-531, 2015.

  17. Hesthaven, J.S., Rozza, G., and Stamm, B., Certified Reduced Basis Methods for Parametrized Partial Differential Equations, SpringerBriefs in Mathematics, Cham, Switzerland: Springer, 2015.

  18. Quarteroni, A., Manzoni, A., and Negri, F., Reduced Basis Methods for Partial Differential Equations: An Introduction, vol. 92, Berlin: Springer, 2015.

  19. Chen, P., Quarteroni, A., and Rozza, G., Comparison of Reduced Basis and Stochastic Collocation Methods for Elliptic Problems, J. Sci. Comput., 59:187-216, 2014.

  20. Elman, H. and Liao, Q., Reduced Basis Collocation Methods for Partial Differential Equations with Random Coefficients, SIAM/ASA J. Uncertainty Quantif, 1(1):192-217, 2013.

  21. Chen, P. and Quarteroni, A., A New Algorithm for High-Dimensional Uncertainty Quantification based on Dimension-Adaptive Sparse Grid Approximation and Reduced Basis Methods, J. Comput. Phys., 298:176-193, 2015.

  22. Bashir, O., Willcox, K., Ghattas, O., van Bloemen Waanders, B., and Hill, J., Hessian-Based Model Reductionfor Large-Scale Systems with Initial Condition Inputs, Int. J. Numer. Methods Eng., 73:844-868, 2008.

  23. Lieberman, C., Willcox, K., and Ghattas, O., Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems, SIAMJ. Sci. Comp., 32(5):2523-2542, 2010.

  24. Cui, T., Marzouk, Y., and Willcox, K., Scalable Posterior Approximations for Large-Scale Bayesian Inverse Problems via Likelihood-Informed Parameter and State Reduction, J. Comput. Phys., 315:363-387, 2016.

  25. Barrault, M., Maday, Y., Nguyen, N.C., and Patera, A.T., An Empirical Interpolation Method: Application to Efficient Reduced-Basis Discretization of Partial Differential Equations, C. R. Math. Anal. Numer, 339(9):667-672, 2004.

  26. Maday, Y., Nguyen, N.C., Patera, A.T., and Pau, G.S.H., A General, Multipurpose Interpolation Procedure: The Magic Points, Commun. Pure Appl. Anal, 8(1):383-404,2009.

  27. Chaturantabut, S. and Sorensen, D.C., Nonlinear Model Reduction via Discrete Empirical Interpolation, SIAMJ. Sci. Comp., 32(5):2737-2764, 2010.

  28. Chen, P., Quarteroni, A., and Rozza, G., A Weighted Empirical Interpolation Method: A Priori Convergence Analysis and Applications, ESAIM: Mathemat. Modell. Numer. Anal., 48:943-953, 2014.

  29. Drohmann, M., Haasdonk, B., and Ohlberger, M., Reduced Basis Approximation for Nonlinear Parametrized Evolution Equations based on Empirical Operator Interpolation, SIAMJ. Sci. Comput, 34(2):A937-A969, 2012.

  30. Nguyen, N.C., Patera, A.T., and Peraire, J., A "Best Points" Interpolation Method for Efficient Approximation of Parametrized Functions, Int. J. Numer. Methods Eng., 73(4):521-543,2008.

  31. Everson, R. and Sirovich, L., Karhunen-Loeve Procedure for Gappy Data, J. Opt. Soc. Am., 12(8):1657-1664,1995.

  32. Bui-Thanh, T., Damodaran, M., and Willcox, K.E., Aerodynamic Data Reconstruction and Inverse Design using Proper Orthogonal Decomposition, AIAAJ, 42(8):1505-1516, 2004.

  33. Carlberg, K., Bou-Mosleh, C., and Farhat, C., Efficient Non-Linear Model Reduction via a Least-Squares Petrov-Galerkin Projection and Compressive Tensor Approximations, Int. J. Numer. Methods Eng., 86(2):155-181,2011.

  34. Niederreiter, H., Random Number Generation and Quasi-Monte Carlo Methods, Philadelphia: SIAM, 1992.

  35. Chen, P., Quarteroni, A., and Rozza, G., A Weighted Reduced Basis Method for Elliptic Partial Differential Equations with Random Input Data, SIAMJ. Numer. Anal., 51(6):3163-3185, 2013.

  36. Spannring, C., Ullmann, S., and Lang, J., A Weighted Reduced Basis Method for Parabolic PDEs with Random Data, arXiv preprint arXiv:1712.07393, 2017.

  37. Venturi, L., Torlo, D., Ballarin, F., and Rozza, G., Weighted Reduced Order Methods for Parametrized Partial Differential Equationswith Random Inputs, arXiv preprint arXiv:1805.00828, 2018.

  38. Eftang, J.L., Patera, A.T., and Renquist, E.M., An "hp" Certified Reduced Basis Method for Parametrized Elliptic Partial Differential Equations, SIAMJ. Sci. Comput, 32(6):3170-3200, 2010.

  39. Hesthaven, J., Stamm, B., and Zhang, S., Efficient Greedy Algorithms for High-Dimensional Parameter Spaces with Applications to Empirical Interpolation and Reduced Basis Methods, ESAIM: Math. Modell. Numer. Anal., 48(1):259- 283, 2011.

  40. Chen, P. and Quarteroni, A., Accurate and Efficient Evaluation of Failure Probability for Partial Differential Equations with Random Input Data, Comput. Methods Appl. Mech. Eng., 267:233-260, 2013.

  41. Haasdonk, B., Urban, K., and B. Wieland, B., Reduced Basis Methods for Parameterized Partial Differential Equations with Stochastic Influences using the Karhunen-Loeve Expansion, SIAM/ASA J. Uncertainty Quantif, 1(1):79-105, 2013.

  42. Carlberg, K. and Farhat, C., A Low-Cost, Goal-Oriented Compact Proper Orthogonal Decomposition Basis for Model Reduction of Static Systems, Int. J. Numer. Methods Eng., 86(3):381-402, 2011.

  43. Tezzele, M., Ballarin, F., and Rozza, G., Combined Parameter and Model Reduction of Cardiovascular Problems by Means of Active Subspaces and POD-Galerkin Methods, arXiv preprint arXiv:1711.10884, 2017.

  44. Flath, P.H., Wilcox, L.C., Akcelik, V., Hill, J., van Waanders, B., and Ghattas, O., Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems based on Low-Rank Partial Hessian Approximations, SIAM J. Sci. Comput., 33(1):407-432,2011.

  45. Bui-Thanh, T. and Ghattas, O., Analysis of the Hessian for Inverse Scattering Problems. Part I: Inverse Shape Scatteringof Acoustic Waves, Inverse Probl., 28(5):055001, 2012.

  46. Bui-Thanh, T. and Ghattas, O., Analysis of the Hessian for Inverse Scattering Problems. Part III: Inverse Medium Scattering of Electromagnetic Waves, Inverse Probl. Imaging, 7(4):1139-1155,2013.

  47. Bui-Thanh, T. and Ghattas, O., Analysis of the Hessian for Inverse Scattering Problems. Part II: Inverse Medium Scattering of Acoustic Waves, Inverse Probl., 28(5):055002,2012.

  48. Bui-Thanh, T., Burstedde, C., Ghattas, O., Martin, J., Stadler, G., and Wilcox, L.C., Extreme-Scale UQ for Bayesian Inverse Problems Governed by PDEs, in SC12: Proc. of the Int. Conf. for High Performance Computing, Networking, Storage and Analysis, 2012.

  49. Bui-Thanh, T., Ghattas, O., Martin, J., and Stadler, G., A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems. Part I: The Linearized Case, with Application to Global Seismic Inversion, SIAM J. Sci. Comput., 35(6):A2494- A2523, 2013.

  50. Chen, P., Villa, U., and Ghattas, O., Hessian-Based Adaptive Sparse Quadrature for Infinite-Dimensional Bayesian Inverse Problems, Comput. Methods Appl. Mech. Eng., 327:147-172, 2017.

  51. Alexanderian, A., Petra, N., Stadler, G., and Ghattas, O., A Fast and Scalable Method for A-Optimal Design of Experiments for Infinite-Dimensional Bayesian Nonlinear Inverse Problems, SIAMJ. Sci. Comput, 38(1):A243-A272, 2016.

  52. Alexanderian, A., Petra, N., Stadler, G., and Ghattas, O., A-Optimal Design of Experiments for Infinite-Dimensional Bayesian Linear Inverse Problems with Regularized l0-Sparsification, SIAMJ. Sci. Comput, 36(5):A2122-A2148, 2014.

  53. Crestel, B., Alexanderian, A., Stadler, G., and Ghattas, O., A-Optimal Encoding Weights for Nonlinear Inverse Problems with Application to the Helmholtz Inverse Problem, Inverse Probl., 33(7):074008, 2017.

  54. Petra, N., Martin, J., Stadler, G., and Ghattas, O., A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems: Part II. Stochastic Newton MCMC with Application to Ice Sheet Inverse Problems, SIAM J. Sci. Comput, 36(4):A1525-A1555, 2014.

  55. Isaac, T., Petra, N., Stadler, G., and Ghattas, O., Scalable and Efficient Algorithms for the Propagation of Uncertainty from Data through Inference to Prediction for Large-Scale Problems, with Application to Flow of the Antarctic Ice Sheet, J. Comput. Phys, 296:348-368,2015.

  56. Martin, J., Wilcox, L.C., Burstedde, C., and Ghattas, O., A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion, SIAMJ. Sci. Comput., 34(3):A1460-A1487, 2012.

  57. Bui-Thanh, T. and Ghattas, O., A Scalable MAP Solver for Bayesian Inverse Problems with Besov Priors, Inverse Probl. Imaging, 9(1):27-54, 2015.

  58. Nocedal, J. and Wright, S., Numerical Optimization, Berlin: Springer Science & Business Media, 2006.

  59. Biegler, L.T., Ghattas, O., Heinkenschloss, M., and Bloemen Waanders, B., Large-Scale PDE-Constrained Optimization: An Introduction, in Large-Scale PDE-Constrained Optimization, Berlin: Springer, pp. 3-13, 2003.

  60. Chen, P., Villa, U., and Ghattas, O., Taylor Approximation and Variance Reduction for PDE-Constrained Optimal Control Problemsunder Uncertainty, arXiv:1804.04301,2018.

  61. Biegler, L., Biros, G., Ghattas, O., Heinkenschloss, M., Keyes, D., Mallick, B., Tenorio, L., van Bloemen Waanders, B., Willcox, K., and Marzouk, Y., Large-Scale Inverse Problems and Quantification of Uncertainty, Hoboken, NJ: John Wiley & Sons, 2011.

  62. Law, K., Stuart, A., and Zygalakis, K., Data Assimilation, Berlin: Springer, 2015.

  63. Avron, H. and Toledo, S., Randomized Algorithms for Estimating the Trace of an Implicit Symmetric Positive Semi-Definite Matrix, J. ACM, 58(2):8, 2011.

  64. Saibaba, A.K., Lee, J., and Kitanidis, P.K., Randomized Algorithms for Generalized Hermitian Eigenvalue Problems with Application to Computing Karhunen-Loeve Expansion, Numer. Linear Algebra Appl, 23(2):314-339,2016.

  65. Chen, P. and Schwab, C., Adaptive Sparse Grid Model Order Reduction for Fast Bayesian Estimation and Inversion, in Sparse Grids and Applications-Stuttgart 2014. Lecture Notes in Computational Science and Engineering, vol. 109, J. Garcke and D. Pflger, Eds., Cham, Switzerland: Springer, 2016.


Articles with similar content:

A GRADIENT-BASED SAMPLING APPROACH FOR DIMENSION REDUCTION OF PARTIAL DIFFERENTIAL EQUATIONS WITH STOCHASTIC COEFFICIENTS
International Journal for Uncertainty Quantification, Vol.5, 2015, issue 1
Miroslav Stoyanov, Clayton G. Webster
OPTIMIZATION-BASED SAMPLING IN ENSEMBLE KALMAN FILTERING
International Journal for Uncertainty Quantification, Vol.4, 2014, issue 4
Alexander Bibov, Heikki Haario, Antti Solonen, Johnathan M. Bardsley
COMPARISON OF FINITE ELEMENT AND FINITE VOLUME ANALYSIS FOR ROUGH SURFACES USING MAPPED DOMAINS
First Thermal and Fluids Engineering Summer Conference, Vol.4, 2015, issue
R. J. Cochran, Ashley F. Emery
LOW-COST MULTI-DIMENSIONAL GAUSSIAN PROCESS WITH APPLICATION TO UNCERTAINTY QUANTIFICATION
International Journal for Uncertainty Quantification, Vol.5, 2015, issue 4
Guang Lin, Bledar A. Konomi
DIMENSIONALITY REDUCTION FOR COMPLEX MODELS VIA BAYESIAN COMPRESSIVE SENSING
International Journal for Uncertainty Quantification, Vol.4, 2014, issue 1
Bert J. Debusschere, Habib N. Najm, Peter Thornton, Cosmin Safta, Khachik Sargsyan, Daniel Ricciuto