Optimizing value of information over an infinite time horizon

Decision-making based on probabilistic reasoning often involves selecting a subset of expensive observations that best predict the system state. In an earlier work, adopting the general notion of value of information (VoI) first introduced by Krause and Guestrin, Ghosh and Ramakrishnan considered the problem of determining optimal conditional observation plans in temporal graphical models, based on non-myopic (non-greedy) VoI, over a finite time horizon. They cast the problem as determining optimal policies in finite-horizon, non-discounted Markov Decision Processes (MDPs). However, there are many practical scenarios where a time horizon is undefinable. In this paper, we consider the VoI optimization problem over an infinite (or equivalently, undefined) time horizon. Adopting an approach similar to Ghosh and Ramakrishnan's, we cast this problem as determining optimal policies in infinite-horizon, finite-state, discounted MDPs. Although our MDP-based framework addresses Dynamic Bayesian Networks (DBNs) that are more restricted than those addressed by Ghosh and Ramakrishnan, we incorporate Krause and Guestrin's general idea of VoI even though it was fundamentally envisioned for finite-horizon settings. We establish the utility of our approach on two graphical models based on real-world datasets.