Bayesian inference in high-dimensional models

Models with dimensions exceeding the available sample size are now commonly used across various applications. A sensible inference is possible using a lower-dimensional structure. In regression problems with many predictors, the model is often assumed to be sparse, with only a few predictors active. Interdependence among a large number of variables is succinctly described by a graphical model, in which variables are represented as nodes on a graph, and an edge between two nodes indicates their conditional dependence given other variables. Many procedures for making inferences in the high-dimensional setting, typically using penalty functions to induce sparsity in the solution obtained by minimizing a loss function, were developed. Bayesian methods have been proposed more recently for such problems, where the prior accounts for the sparsity structure. These methods naturally quantify the uncertainty of the inference through the posterior distribution. Theoretical studies of Bayesian procedures in high dimensions have recently been conducted. Questions that arise are whether the posterior distribution contracts at the minimax optimal rate near the true parameter value, whether the correct lower-dimensional structure is discovered with high posterior probability, and whether a credible region has adequate frequentist coverage. In this paper, we review the properties of Bayesian and related methods for several high-dimensional models, including the many normal means problem, linear regression, generalized linear models, and Gaussian and non-Gaussian graphical models. Practical computational approaches are also discussed.