AVERAGE-COST MDPS WITH INFINITE STATE AND ACTION SETS: NEW SUFFICIENT CONDITIONS FOR OPTIMALITY INEQUALITIES AND EQUATIONS

Eugene A. Feinberg; Pavlo O. Kasyanov; Liliia S. Paliichuk

doi:10.23952/COT.2026.17

AVERAGE-COST MDPS WITH INFINITE STATE AND ACTION SETS: NEW SUFFICIENT CONDITIONS FOR OPTIMALITY INEQUALITIES AND EQUATIONS

Eugene A. Feinberg
, Pavlo O. Kasyanov
, Liliia S. Paliichuk

National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"

Research output: Contribution to journal › Article › peer-review

Abstract

This paper studies discrete-time average-cost infinite-horizon Markov decision processes (MDPs) with Borel state and action sets. It introduces new sufficient conditions for the validity of optimality inequalities and optimality equations for MDPs with weakly and setwise continuous transition probabilities. These inequalities and equations imply the existence of deterministic optimal policies.

Original language	English
Article number	17
Journal	Communications in Optimization Theory
Volume	2026
DOIs	https://doi.org/10.23952/COT.2026.17
State	Published - 2026

Keywords

Average cost per unit time
Markov decision process
Optimality equation
Optimality inequality

Access to Document

10.23952/COT.2026.17

Cite this

@article{212d155f9be34a40ab7721b8fcdf1765,

title = "AVERAGE-COST MDPS WITH INFINITE STATE AND ACTION SETS: NEW SUFFICIENT CONDITIONS FOR OPTIMALITY INEQUALITIES AND EQUATIONS",

abstract = "This paper studies discrete-time average-cost infinite-horizon Markov decision processes (MDPs) with Borel state and action sets. It introduces new sufficient conditions for the validity of optimality inequalities and optimality equations for MDPs with weakly and setwise continuous transition probabilities. These inequalities and equations imply the existence of deterministic optimal policies.",

keywords = "Average cost per unit time, Markov decision process, Optimality equation, Optimality inequality",

author = "Feinberg, \{Eugene A.\} and Kasyanov, \{Pavlo O.\} and Paliichuk, \{Liliia S.\}",

year = "2026",

doi = "10.23952/COT.2026.17",

language = "English",

volume = "2026",

journal = "Communications in Optimization Theory",

issn = "2051-2953",

}

TY - JOUR

T1 - AVERAGE-COST MDPS WITH INFINITE STATE AND ACTION SETS

T2 - NEW SUFFICIENT CONDITIONS FOR OPTIMALITY INEQUALITIES AND EQUATIONS

AU - Feinberg, Eugene A.

AU - Kasyanov, Pavlo O.

AU - Paliichuk, Liliia S.

PY - 2026

Y1 - 2026

N2 - This paper studies discrete-time average-cost infinite-horizon Markov decision processes (MDPs) with Borel state and action sets. It introduces new sufficient conditions for the validity of optimality inequalities and optimality equations for MDPs with weakly and setwise continuous transition probabilities. These inequalities and equations imply the existence of deterministic optimal policies.

AB - This paper studies discrete-time average-cost infinite-horizon Markov decision processes (MDPs) with Borel state and action sets. It introduces new sufficient conditions for the validity of optimality inequalities and optimality equations for MDPs with weakly and setwise continuous transition probabilities. These inequalities and equations imply the existence of deterministic optimal policies.

KW - Average cost per unit time

KW - Markov decision process

KW - Optimality equation

KW - Optimality inequality

UR - https://www.scopus.com/pages/publications/105030952430

U2 - 10.23952/COT.2026.17

DO - 10.23952/COT.2026.17

M3 - Article

AN - SCOPUS:105030952430

SN - 2051-2953

VL - 2026

JO - Communications in Optimization Theory

JF - Communications in Optimization Theory

M1 - 17

ER -

AVERAGE-COST MDPS WITH INFINITE STATE AND ACTION SETS: NEW SUFFICIENT CONDITIONS FOR OPTIMALITY INEQUALITIES AND EQUATIONS

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this