Sufficient Conditions for Solving Statistical Filtering Problems by Dynamic Programming

The paper studies discrete-time statistical filtering problems with the goal to minimize expected total costs. Such problems are usually defined by pairs of stochastic equations and by one-step cost functions. Stochastic equations describe the state and observation processes, and these equations are defined by transition and observation functions. This paper provides sufficient conditions on observation, transition, and one-step cost functions for convergence of value-iteration algorithms for problems with finite and infinite horizons. It is well-known that nonlinear and linear filtering problems can be presented as Partially Observable Markov Decision Processes (POMDPs). The paper applies contemporary results on convergence of value iterations for Markov Decision Processes (MDPs) and for POMDPs to filtering problems. It formulates conditions on observation and transition functions which imply weak continuity of the filter. Weak continuity of the filter means weak continuity of transition probabilities between belief states. The sufficient condition on one-step functions is their K-inf-compactness. The described conditions hold for broad classes of nonlinear filters and for Kalman filters.