Bayesian Approach to Collaborative Inference in Networks of Agents

Bayesian inference has become a standard tool in modern statistical signal processing theory and machine learning. It is based on a probabilistically consistent representation of available knowledge about variables of interest and the amount of uncertainty contained in this knowledge. Unlike in the "standard" theory, the underlying inferential principles of Bayes' theory are generally applicable to virtually any inference task, from linear models to nonlinear, mixture, or hierarchical models. Furthermore, the rapid development of modern devices with high computational performance finally eliminated the major drawback of Bayesian theory: the frequent analytical intractability of posterior distributions. This chapter studies the possible implementation of Bayesian inference in networks of collaborating agents. We focus on diffusion networks, where agents may share information (measurements and/or estimates) with their adjacent neighbors and can incorporate it into own knowledge about unknown variables of interest. There are several ways of performing this incorporation in an optimal way according to a convenient user-selected information criterion. Under certain conditions where the underlying model belongs to the exponential family of distributions and the prior distributions are conjugate, the results are analytically tractable. The widely used Kalman filter serves as an illustrative example for demonstrating the application of the abstractly described principles. It is reformulated for the collaborative estimation task in networks where both the neighbors' observations and their posterior distributions are available to each agent. The resulting equations of the filter have analytical solutions.