Continuous-Time Markov Decision Processes
Theory and Applications
Authors: Guo, Xianping, Hernandez-Lerma, Onesimo
Free Preview- To the best of our knowledge, this is the first book completely devoted to continuous-time Markov Decision Processes
- Studies continuous-time MDPs allowing unbounded transition rates, which is the case in most applications
- It is thus distinguished from other books that contain chapters on the continuous-time case
Buy this book
- About this book
-
Continuous-time Markov decision processes (MDPs), also known as controlled Markov chains, are used for modeling decision-making problems that arise in operations research (for instance, inventory, manufacturing, and queueing systems), computer science, communications engineering, control of populations (such as fisheries and epidemics), and management science, among many other fields. This volume provides a unified, systematic, self-contained presentation of recent developments on the theory and applications of continuous-time MDPs. The MDPs in this volume include most of the cases that arise in applications, because they allow unbounded transition and reward/cost rates. Much of the material appears for the first time in book form.
- About the authors
-
Onésimo Hernández-Lerma received the Science and Arts National Award from the Government of MEXICO in 2001, an honorary doctorate from the University of Sonora in 2003, and the Scopus Prize from Elsevier in 2008. Xianping Guo received the He-Pan-Qing-Yi Best Paper Award from the 7th Word Congress on Intelligent Control and Automation in 2008.
- Reviews
-
From the reviews:
“The book consists of 12 chapters. … this is the first monograph on continuous-time Markov decision process. … This is an important book written by leading experts on a mathematically rich topic which has many applications to engineering, business, and biological problems. … scholars and students interested in developing the theory of continuous-time Markov decision Processes or working on their applications should have this book.” (E. A. Feinberg, Mathematical Reviews, Issue 2011 b)
- Table of contents (12 chapters)
-
-
Introduction and Summary
Pages 1-8
-
Continuous-Time Markov Decision Processes
Pages 9-18
-
Average Optimality for Finite Models
Pages 19-53
-
Discount Optimality for Nonnegative Costs
Pages 55-70
-
Average Optimality for Nonnegative Costs
Pages 71-86
-
Table of contents (12 chapters)
Recommended for you
Bibliographic Information
- Bibliographic Information
-
- Book Title
- Continuous-Time Markov Decision Processes
- Book Subtitle
- Theory and Applications
- Authors
-
- Xianping Guo
- Onesimo Hernandez-Lerma
- Series Title
- Stochastic Modelling and Applied Probability
- Series Volume
- 62
- Copyright
- 2009
- Publisher
- Springer-Verlag Berlin Heidelberg
- Copyright Holder
- Springer-Verlag Berlin Heidelberg
- eBook ISBN
- 978-3-642-02547-1
- DOI
- 10.1007/978-3-642-02547-1
- Hardcover ISBN
- 978-3-642-02546-4
- Softcover ISBN
- 978-3-642-26072-8
- Series ISSN
- 0172-4568
- Edition Number
- 1
- Number of Pages
- XVIII, 234
- Topics
*immediately available upon purchase as print book shipments may be delayed due to the COVID-19 crisis. ebook access is temporary and does not include ownership of the ebook. Only valid for books with an ebook version. Springer Reference Works are not included.
