Mean-Field Models in Statistics

Frequently in high dimensional Bayesian models, the posterior distribution is complicated and exhibits non-trivial dependence. Understanding the behavior of such models, both theoretically and empirically, is a challenge. In these situations, variational inference provides an efficient and scalable method of inference, when more traditional methods based on Markov Chain Monte Carlo are computationally prohibitive. One of the most common techniques for variational inference is the naive mean field approximation. However, despite its wide usage, not much is known about rigorous guarantees for the naive mean field method. This project aims to address this question, by studying the validity of naive mean field methods for several examples of interest.

This project defines a formal notion of correctness of the naive mean field approximation, which requires that the leading order of the log normalizing constant of the high dimensional distribution is predicted correctly by the mean field prediction formula. This definition does not require the high dimensional distribution of interest to arise as a posterior distribution. If the naive mean field approximation is indeed asymptotically correct, the high dimensional distribution of interest should be well approximated by a mixture of product distributions. If further, the optimization in the mean field prediction formula has a unique maximizer, then essentially the high dimensional distribution should be close to a product measure, and it is natural enquire about Law of Large Numbers, Concentration, Fluctuations, and Asymptotic Properties of Estimators. The PI plans to study these questions for three concrete examples: (a) (Bayesian) Linear Regression, (b) (Bayesian) Mixture of Gaussians, and (c) Exponential Random Graph Models.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.