Informative priors for Bayesian inference, regularization, and computation

The research problem is the development and use of informative prior distributions for Bayesian inference, regularization, and computation. The objective is to unify the choice of prior distribution by placing noninformative and weakly informative priors into a common framework of hierarchical modeling. The technical approach involves Stan, our open-source C++ program for Bayesian inference which uses the no-U-turn sampler and Hamiltonian Monte Carlo to efficiently explore high-dimensional posterior distributions (or, alternatively, uses an efficient optimization algorithm to find posterior modes or penalized maximum likelihood estimates). The effort involved in thisunification includes setting up families of prior distributions for several standard classes of models (hierarchical models, variance component models, mixture models, regression, and generalized linear models). We have fit our models in ongoing research efforts in survey sampling, social networks, and public health, among others. In addition to their inherent interest, these applications have allowed us to understand the practical gains arising from our methods. In this project, we propose to extend this work to more complicated classes of models, including variable selection for regressions with large numbers of predictions, and Gaussian processes and network models of multivariate dependence. At the algorithmic level, we propose to develop tools such as expectation propagation and cross-validation for scalable computing. Impact on DOD capabilities includes quantitative analysis that it is (a) flexible, (b) takes advantage of diverse data sources, and (c) allows us to understand how effects vary so as to map to targeted decisions. Informative priors are potentially valuable in allowing the integration of additional information into decision making, and by making inferences and predictions more stable (regularization), which facilitates the use of more sophisticated models with contextually varying effects.