Iterative importance sampling algorithms for parameter estimation

Article, Preprint English OPEN
Morzfeld, Matthias ; Day, Marcus S. ; Grout, Ray W. ; Pau, George Shu Heng ; Finsterle, Stefan A. ; Bell, John B. (2016)
  • Publisher: eScholarship, University of California
  • Subject: Statistics - Computation | Mathematics - Numerical Analysis

© 2018 Society for Industrial and Applied Mathematics. In parameter estimation problems one computes a posterior distribution over uncertain parameters defined jointly by a prior distribution, a model, and noisy data. Markov chain Monte Carlo (MCMC) is often used for the numerical solution of such problems. An alternative to MCMC is importance sampling, which can exhibit near perfect scaling with the number of cores on high performance computing systems because samples are drawn independently. However, finding a suitable proposal distribution is a challenging task. Several sampling algorithms have been proposed over the past years that take an iterative approach to constructing a proposal distribution. We investigate the applicability of such algorithms by applying them to two realistic and challenging test problems, one in subsurface flow, and one in combustion modeling. More specifically, we implement importance sampling algorithms that iterate over the mean and covariance matrix of Gaussian or multivariate t-proposal distributions. Our implementation leverages massively parallel computers, and we present strategies to initialize the iterations using “coarse” MCMC runs or Gaussian mixture models.
  • References (73)
    73 references, page 1 of 8

    [1] J.L. Anderson. Spatially and temporally varying adaptive covariance in ation for ensemble lters. Tellus, 61(A):72{83, 2009.

    [2] M.S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle lters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Transactions on Signal Processing, 50(2):174 {188, 2002.

    [3] E. Atkins, M. Morzfeld, and A.J. Chorin. Implicit particle methods and their connection with variational data assimilation. Monthly Weather Review, 141:1786{1803, 2013.

    [4] J.M. Bardsley, A. Solonen, H. Haario, and M. Laine. Randomize-then-optimize: A method for sampling from posterior distributions in nonlinear inverse problems. SIAM Journal on Scienti c Computing, 36(4):A1895{A1910, 2014.

    [5] T. Bengtsson, P. Bickel, and B. Li. Curse of dimensionality revisited: the collapse of importance sampling in very large scale systems. IMS Collections: Probability and Statistics: Essays in Honor of David A. Freedman, 2:316{334, 2008.

    [6] N. Bergman, editor. Recursive Bayesian estimation: Navigation and tracking applications. Ph.D Dissertation, Linkoping University, Linkoping, Sweden, 1999.

    [7] A. Beskos, A. Jasra, E. Muza er, and A.M. Stuart. Sequential Monte Carlo methods for Bayesian elliptic inverse problems. Stat. Comp., 25:727{737, 2015.

    [8] P. Bickel, T. Bengtsson, and J. Anderson. Sharp failure rates for the bootstrap particle lter in high dimensions. Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, 3:318{329, 2008.

    [9] C.J.F. Ter Braak. A Markov chain Monte Carlo version of the genetic algorithm Di erential Evolution: easy Bayesian computing for real parameter spaces. Statistics and Computing, 16(3):239{249, 2006.

    [10] C.J.F. Ter Braak and A.A. Vrugt. Di erential evolution markov chain with snooker updater andfewer chains. Statistics and Computing, 18(4):435{446, 2008.

  • Metrics
    No metrics available
Share - Bookmark