Recent Advances in Bayesian Inference for Complex Systems

In many real-world problems, the interest is in learning about unknowns from observed data. One widely adopted approach to learning is to employ Bayesian inference. Over the last half-century, Monte Carlo-based methods have kept the main stage of Bayesian reasoning, and they will not relinquish it any time soon. With these methods, Bayesianism has enjoyed tremendous success as they made possible the computation of posterior distributions of unknowns from very challenging models. In the new age of signal processing, the main problems of concern are those with large numbers of unknowns (complex systems) and/or large amounts of data (big data). This raises the concern that Bayesian inference may become computationally incapable of handling them because of the sheer size and complexity of the studied systems. This chapter discusses the main challenges related to the addressed problem and recent advances in the theory and practice of a class of Monte Carlo methods, adaptive importance sampling (AIS), for dealing exactly with the types of problems where the numbers of unknowns and/or data are large. The focus of the chapter will be on new schemes for adaptive importance sampling with emphasis on strategies for adaptive learning and for stable weight computation, model selection, and application of the theory to case-studies including reconstruction of gene regulatory (life science problem) and inference of demographic rates of penguin populations (ecological problem).