MDPs with setwise continuous transition probabilities

Eugene A. Feinberg; Pavlo O. Kasyanov

doi:10.1016/j.orl.2021.07.011

MDPs with setwise continuous transition probabilities

Eugene A. Feinberg
, Pavlo O. Kasyanov

Applied Mathematics and Statistics

National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"

Research output: Contribution to journal › Article › peer-review

15 Scopus citations

Abstract

This paper describes the structure of optimal policies for infinite-state Markov Decision Processes with setwise continuous transition probabilities. The action sets may be noncompact. The objective criteria are either the expected total discounted and undiscounted costs or average costs per unit time. The analysis of optimality equations and inequalities is based on the optimal selection theorem for inf-compact functions introduced in this paper.

Original language	English
Pages (from-to)	734-740
Number of pages	7
Journal	Operations Research Letters
Volume	49
Issue number	5
DOIs	https://doi.org/10.1016/j.orl.2021.07.011
State	Published - Sep 2021

Keywords

Average cost per unit time
Markov decision process
Optimal selection theorem
Total discounted cost

Access to Document

10.1016/j.orl.2021.07.011

Cite this

@article{e4184088bcb54aec9857b9d0501139ff,

title = "MDPs with setwise continuous transition probabilities",

abstract = "This paper describes the structure of optimal policies for infinite-state Markov Decision Processes with setwise continuous transition probabilities. The action sets may be noncompact. The objective criteria are either the expected total discounted and undiscounted costs or average costs per unit time. The analysis of optimality equations and inequalities is based on the optimal selection theorem for inf-compact functions introduced in this paper.",

keywords = "Average cost per unit time, Markov decision process, Optimal selection theorem, Total discounted cost",

author = "Feinberg, \{Eugene A.\} and Kasyanov, \{Pavlo O.\}",

note = "Publisher Copyright: {\textcopyright} 2021 Elsevier B.V.",

year = "2021",

month = sep,

doi = "10.1016/j.orl.2021.07.011",

language = "English",

volume = "49",

pages = "734--740",

journal = "Operations Research Letters",

issn = "0167-6377",

number = "5",

}

TY - JOUR

T1 - MDPs with setwise continuous transition probabilities

AU - Feinberg, Eugene A.

AU - Kasyanov, Pavlo O.

PY - 2021/9

Y1 - 2021/9

N2 - This paper describes the structure of optimal policies for infinite-state Markov Decision Processes with setwise continuous transition probabilities. The action sets may be noncompact. The objective criteria are either the expected total discounted and undiscounted costs or average costs per unit time. The analysis of optimality equations and inequalities is based on the optimal selection theorem for inf-compact functions introduced in this paper.

AB - This paper describes the structure of optimal policies for infinite-state Markov Decision Processes with setwise continuous transition probabilities. The action sets may be noncompact. The objective criteria are either the expected total discounted and undiscounted costs or average costs per unit time. The analysis of optimality equations and inequalities is based on the optimal selection theorem for inf-compact functions introduced in this paper.

KW - Average cost per unit time

KW - Markov decision process

KW - Optimal selection theorem

KW - Total discounted cost

UR - https://www.scopus.com/pages/publications/85112827672

U2 - 10.1016/j.orl.2021.07.011

DO - 10.1016/j.orl.2021.07.011

M3 - Article

AN - SCOPUS:85112827672

SN - 0167-6377

VL - 49

SP - 734

EP - 740

JO - Operations Research Letters

JF - Operations Research Letters

IS - 5

ER -

MDPs with setwise continuous transition probabilities

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this