A Class of Solvable Markov Decision Models with Incomplete Information

Eugene A. Feinberg; Pavlo O. Kasyanov; Michael Z. Zgurovsky

doi:10.1109/CDC45484.2021.9683160

A Class of Solvable Markov Decision Models with Incomplete Information

Eugene A. Feinberg
, Pavlo O. Kasyanov
, Michael Z. Zgurovsky

Applied Mathematics and Statistics

National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

This paper investigates natural conditions for the existence of optimal policies for a Markov decision process with incomplete information (MDPII) and with expected total costs. MDPII is the classic model of a controlled stochastic process with incomplete state observations which is more general than Partially Observable Markov Decision Processes (POMDPs). For MDPII we introduce the notion of a semi-uniform Feller transition probability, which is stronger than the notion of a weakly continuous transition probability. We show that an MDPII has a semi-uniform Feller transition probability if and only if the corresponding belief MDP also has a semi-uniform Feller transition probability. This fact has several corollaries. In particular, it provides new and implies all known sufficient conditions for the existence of optimal policies for POMDPs with expected total costs.

Original language	English
Title of host publication	60th IEEE Conference on Decision and Control, CDC 2021
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1615-1620
Number of pages	6
ISBN (Electronic)	9781665436595
DOIs	https://doi.org/10.1109/CDC45484.2021.9683160
State	Published - 2021
Event	60th IEEE Conference on Decision and Control, CDC 2021 - Austin, United States Duration: Dec 13 2021 → Dec 17 2021

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
Volume	2021-December
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Conference

Conference	60th IEEE Conference on Decision and Control, CDC 2021
Country/Territory	United States
City	Austin
Period	12/13/21 → 12/17/21

Access to Document

10.1109/CDC45484.2021.9683160

Cite this

Feinberg, E. A., Kasyanov, P. O., & Zgurovsky, M. Z. (2021). A Class of Solvable Markov Decision Models with Incomplete Information. In 60th IEEE Conference on Decision and Control, CDC 2021 (pp. 1615-1620). (Proceedings of the IEEE Conference on Decision and Control; Vol. 2021-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC45484.2021.9683160

@inproceedings{a7dd8369f4f74afa8e90c8d7a5577964,

title = "A Class of Solvable Markov Decision Models with Incomplete Information",

abstract = "This paper investigates natural conditions for the existence of optimal policies for a Markov decision process with incomplete information (MDPII) and with expected total costs. MDPII is the classic model of a controlled stochastic process with incomplete state observations which is more general than Partially Observable Markov Decision Processes (POMDPs). For MDPII we introduce the notion of a semi-uniform Feller transition probability, which is stronger than the notion of a weakly continuous transition probability. We show that an MDPII has a semi-uniform Feller transition probability if and only if the corresponding belief MDP also has a semi-uniform Feller transition probability. This fact has several corollaries. In particular, it provides new and implies all known sufficient conditions for the existence of optimal policies for POMDPs with expected total costs.",

author = "Feinberg, \{Eugene A.\} and Kasyanov, \{Pavlo O.\} and Zgurovsky, \{Michael Z.\}",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 60th IEEE Conference on Decision and Control, CDC 2021 ; Conference date: 13-12-2021 Through 17-12-2021",

year = "2021",

doi = "10.1109/CDC45484.2021.9683160",

language = "English",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1615--1620",

booktitle = "60th IEEE Conference on Decision and Control, CDC 2021",

}

Feinberg, EA, Kasyanov, PO & Zgurovsky, MZ 2021, A Class of Solvable Markov Decision Models with Incomplete Information. in 60th IEEE Conference on Decision and Control, CDC 2021. Proceedings of the IEEE Conference on Decision and Control, vol. 2021-December, Institute of Electrical and Electronics Engineers Inc., pp. 1615-1620, 60th IEEE Conference on Decision and Control, CDC 2021, Austin, United States, 12/13/21. https://doi.org/10.1109/CDC45484.2021.9683160

A Class of Solvable Markov Decision Models with Incomplete Information. / Feinberg, Eugene A.; Kasyanov, Pavlo O.; Zgurovsky, Michael Z.
60th IEEE Conference on Decision and Control, CDC 2021. Institute of Electrical and Electronics Engineers Inc., 2021. p. 1615-1620 (Proceedings of the IEEE Conference on Decision and Control; Vol. 2021-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Class of Solvable Markov Decision Models with Incomplete Information

AU - Feinberg, Eugene A.

AU - Kasyanov, Pavlo O.

AU - Zgurovsky, Michael Z.

PY - 2021

Y1 - 2021

N2 - This paper investigates natural conditions for the existence of optimal policies for a Markov decision process with incomplete information (MDPII) and with expected total costs. MDPII is the classic model of a controlled stochastic process with incomplete state observations which is more general than Partially Observable Markov Decision Processes (POMDPs). For MDPII we introduce the notion of a semi-uniform Feller transition probability, which is stronger than the notion of a weakly continuous transition probability. We show that an MDPII has a semi-uniform Feller transition probability if and only if the corresponding belief MDP also has a semi-uniform Feller transition probability. This fact has several corollaries. In particular, it provides new and implies all known sufficient conditions for the existence of optimal policies for POMDPs with expected total costs.

AB - This paper investigates natural conditions for the existence of optimal policies for a Markov decision process with incomplete information (MDPII) and with expected total costs. MDPII is the classic model of a controlled stochastic process with incomplete state observations which is more general than Partially Observable Markov Decision Processes (POMDPs). For MDPII we introduce the notion of a semi-uniform Feller transition probability, which is stronger than the notion of a weakly continuous transition probability. We show that an MDPII has a semi-uniform Feller transition probability if and only if the corresponding belief MDP also has a semi-uniform Feller transition probability. This fact has several corollaries. In particular, it provides new and implies all known sufficient conditions for the existence of optimal policies for POMDPs with expected total costs.

UR - https://www.scopus.com/pages/publications/85126046930

U2 - 10.1109/CDC45484.2021.9683160

DO - 10.1109/CDC45484.2021.9683160

M3 - Conference contribution

AN - SCOPUS:85126046930

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 1615

EP - 1620

BT - 60th IEEE Conference on Decision and Control, CDC 2021

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 60th IEEE Conference on Decision and Control, CDC 2021

Y2 - 13 December 2021 through 17 December 2021

ER -

A Class of Solvable Markov Decision Models with Incomplete Information

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this