Optimally Adjusted Mixture Sampling and Locally Weighted Histogram Analysis

Research output: Contribution to journalArticlepeer-review

20 Scopus citations


Consider the two problems of simulating observations and estimating expectations and normalizing constants for multiple distributions. First, we present a self-adjusted mixture sampling method, which accommodates both adaptive serial tempering and a generalized Wang–Landau algorithm. The set of distributions are combined into a labeled mixture, with the mixture weights depending on the initial estimates of log normalizing constants (or free energies). Then, observations are generated by Markov transitions, and free energy estimates are adjusted online by stochastic approximation. We propose two stochastic approximation schemes by Rao–Blackwellization of the scheme commonly used, and derive the optimal choice of a gain matrix, resulting in the minimum asymptotic variance for free energy estimation, in a simple and feasible form. Second, we develop an offline method, locally weighted histogram analysis, for estimating free energies and expectations, using all the simulated data from multiple distributions by either self-adjusted mixture sampling or other sampling algorithms. This method can be computationally much faster, with little sacrifice of statistical efficiency, than a global method currently used, especially when a large number of distributions are involved. We provide both theoretical results and numerical studies to demonstrate the advantages of the proposed methods.

Original languageEnglish (US)
Pages (from-to)54-65
Number of pages12
JournalJournal of Computational and Graphical Statistics
Issue number1
StatePublished - Jan 2 2017

ASJC Scopus subject areas

  • Statistics and Probability
  • Discrete Mathematics and Combinatorics
  • Statistics, Probability and Uncertainty


  • Free energy
  • Markov chain Monte Carlo
  • Normalizing constant
  • Parallel tempering
  • Potts model
  • Serial tempering
  • Stochastic approximation
  • Wang–Landau algorithm
  • Weighted histogram analysis method


Dive into the research topics of 'Optimally Adjusted Mixture Sampling and Locally Weighted Histogram Analysis'. Together they form a unique fingerprint.

Cite this