On occupation measures for total-reward MDPs

Eric V. Denardo; Eugene A. Feinberg; Uriel G. Rothblum

doi:10.1109/CDC.2008.4739426

On occupation measures for total-reward MDPs

Eric V. Denardo
, Eugene A. Feinberg
, Uriel G. Rothblum

Applied Mathematics and Statistics

Yale University
Technion-Israel Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

This paper is based on our recent contribution [3] that studies Markov Decision Processes (MDPs) with Borel state and action spaces and with the expected total rewards. The initial state distribution is fixed. According to [3], for a given randomized stationary policy, its occupation measure as a convex combination of occupation measures for simpler policies. If this is possible for a given policy, we say that the policy can be split. In particular, we are interested in splitting a randomized stationary policy into (nonrandomized) stationary policies or into a randomized stationary policies that are nonrandomized on a given subset of states. Though [3] studies Borel-state MDPs with expected total rewards, some of its results are new for finite state and action discounted MDPs. This paper focuses on these results.

Original language	English
Title of host publication	Proceedings of the 47th IEEE Conference on Decision and Control, CDC 2008
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	4460-4465
Number of pages	6
ISBN (Print)	9781424431243
DOIs	https://doi.org/10.1109/CDC.2008.4739426
State	Published - 2008
Event	47th IEEE Conference on Decision and Control, CDC 2008 - Cancun, Mexico Duration: Dec 9 2008 → Dec 11 2008

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Conference

Conference	47th IEEE Conference on Decision and Control, CDC 2008
Country/Territory	Mexico
City	Cancun
Period	12/9/08 → 12/11/08

Access to Document

10.1109/CDC.2008.4739426

Cite this

Denardo, E. V., Feinberg, E. A., & Rothblum, U. G. (2008). On occupation measures for total-reward MDPs. In Proceedings of the 47th IEEE Conference on Decision and Control, CDC 2008 (pp. 4460-4465). Article 4739426 (Proceedings of the IEEE Conference on Decision and Control). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC.2008.4739426

@inproceedings{27d926ed488c4dd88fe6b7d3634937bf,

title = "On occupation measures for total-reward MDPs",

abstract = "This paper is based on our recent contribution [3] that studies Markov Decision Processes (MDPs) with Borel state and action spaces and with the expected total rewards. The initial state distribution is fixed. According to [3], for a given randomized stationary policy, its occupation measure as a convex combination of occupation measures for simpler policies. If this is possible for a given policy, we say that the policy can be split. In particular, we are interested in splitting a randomized stationary policy into (nonrandomized) stationary policies or into a randomized stationary policies that are nonrandomized on a given subset of states. Though [3] studies Borel-state MDPs with expected total rewards, some of its results are new for finite state and action discounted MDPs. This paper focuses on these results.",

author = "Denardo, \{Eric V.\} and Feinberg, \{Eugene A.\} and Rothblum, \{Uriel G.\}",

year = "2008",

doi = "10.1109/CDC.2008.4739426",

language = "English",

isbn = "9781424431243",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "4460--4465",

booktitle = "Proceedings of the 47th IEEE Conference on Decision and Control, CDC 2008",

note = "47th IEEE Conference on Decision and Control, CDC 2008 ; Conference date: 09-12-2008 Through 11-12-2008",

}

Denardo, EV, Feinberg, EA & Rothblum, UG 2008, On occupation measures for total-reward MDPs. in Proceedings of the 47th IEEE Conference on Decision and Control, CDC 2008., 4739426, Proceedings of the IEEE Conference on Decision and Control, Institute of Electrical and Electronics Engineers Inc., pp. 4460-4465, 47th IEEE Conference on Decision and Control, CDC 2008, Cancun, Mexico, 12/9/08. https://doi.org/10.1109/CDC.2008.4739426

On occupation measures for total-reward MDPs. / Denardo, Eric V.; Feinberg, Eugene A.; Rothblum, Uriel G.
Proceedings of the 47th IEEE Conference on Decision and Control, CDC 2008. Institute of Electrical and Electronics Engineers Inc., 2008. p. 4460-4465 4739426 (Proceedings of the IEEE Conference on Decision and Control).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - On occupation measures for total-reward MDPs

AU - Denardo, Eric V.

AU - Feinberg, Eugene A.

AU - Rothblum, Uriel G.

PY - 2008

Y1 - 2008

N2 - This paper is based on our recent contribution [3] that studies Markov Decision Processes (MDPs) with Borel state and action spaces and with the expected total rewards. The initial state distribution is fixed. According to [3], for a given randomized stationary policy, its occupation measure as a convex combination of occupation measures for simpler policies. If this is possible for a given policy, we say that the policy can be split. In particular, we are interested in splitting a randomized stationary policy into (nonrandomized) stationary policies or into a randomized stationary policies that are nonrandomized on a given subset of states. Though [3] studies Borel-state MDPs with expected total rewards, some of its results are new for finite state and action discounted MDPs. This paper focuses on these results.

AB - This paper is based on our recent contribution [3] that studies Markov Decision Processes (MDPs) with Borel state and action spaces and with the expected total rewards. The initial state distribution is fixed. According to [3], for a given randomized stationary policy, its occupation measure as a convex combination of occupation measures for simpler policies. If this is possible for a given policy, we say that the policy can be split. In particular, we are interested in splitting a randomized stationary policy into (nonrandomized) stationary policies or into a randomized stationary policies that are nonrandomized on a given subset of states. Though [3] studies Borel-state MDPs with expected total rewards, some of its results are new for finite state and action discounted MDPs. This paper focuses on these results.

UR - https://www.scopus.com/pages/publications/62949182216

U2 - 10.1109/CDC.2008.4739426

DO - 10.1109/CDC.2008.4739426

M3 - Conference contribution

AN - SCOPUS:62949182216

SN - 9781424431243

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 4460

EP - 4465

BT - Proceedings of the 47th IEEE Conference on Decision and Control, CDC 2008

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 47th IEEE Conference on Decision and Control, CDC 2008

Y2 - 9 December 2008 through 11 December 2008

ER -

On occupation measures for total-reward MDPs

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this