Hundreds of Mathematics books at half price + FREE shipping! Get yours now >>

Stochastic Modelling and Applied Probability

Continuous-Time Markov Decision Processes

Theory and Applications

Authors: Guo, Xianping, Hernandez-Lerma, Onesimo

  • To the best of our knowledge, this is the first book completely devoted to continuous-time Markov Decision Processes
  • Studies continuous-time MDPs allowing unbounded transition rates, which is the case in most applications
  • It is thus distinguished from other books that contain chapters on the continuous-time case
see more benefits

Buy this book

eBook $69.99
price for USA (gross)
  • ISBN 978-3-642-02547-1
  • Digitally watermarked, DRM-free
  • Included format: PDF
  • ebooks can be used on all reading devices
  • Immediate eBook download after purchase
Hardcover $99.00
price for USA
  • ISBN 978-3-642-02546-4
  • Free shipping for individuals worldwide
  • Usually dispatched within 3 to 5 business days.
Softcover $99.00
price for USA
  • ISBN 978-3-642-26072-8
  • Free shipping for individuals worldwide
  • Usually dispatched within 3 to 5 business days.
About this book

Continuous-time Markov decision processes (MDPs), also known as controlled Markov chains, are used for modeling decision-making problems that arise in operations research (for instance, inventory, manufacturing, and queueing systems), computer science, communications engineering, control of populations (such as fisheries and epidemics), and management science, among many other fields. This volume provides a unified, systematic, self-contained presentation of recent developments on the theory and applications of continuous-time MDPs. The MDPs in this volume include most of the cases that arise in applications, because they allow unbounded transition and reward/cost rates. Much of the material appears for the first time in book form.

About the authors

Onésimo Hernández-Lerma received the Science and Arts National Award from the Government of MEXICO in 2001, an honorary doctorate from the University of Sonora in 2003, and the Scopus Prize from Elsevier in 2008. Xianping Guo received the He-Pan-Qing-Yi Best Paper Award from the 7th Word Congress on Intelligent Control and Automation in 2008.

Reviews

From the reviews:

“The book consists of 12 chapters. … this is the first monograph on continuous-time Markov decision process. … This is an important book written by leading experts on a mathematically rich topic which has many applications to engineering, business, and biological problems. … scholars and students interested in developing the theory of continuous-time Markov decision Processes or working on their applications should have this book.” (E. A. Feinberg, Mathematical Reviews, Issue 2011 b)

Table of contents (12 chapters)

  • Introduction and Summary

    Guo, Xianping (et al.)

    Pages 1-8

  • Continuous-Time Markov Decision Processes

    Guo, Xianping (et al.)

    Pages 9-18

  • Average Optimality for Finite Models

    Guo, Xianping (et al.)

    Pages 19-53

  • Discount Optimality for Nonnegative Costs

    Guo, Xianping (et al.)

    Pages 55-70

  • Average Optimality for Nonnegative Costs

    Guo, Xianping (et al.)

    Pages 71-86

Buy this book

eBook $69.99
price for USA (gross)
  • ISBN 978-3-642-02547-1
  • Digitally watermarked, DRM-free
  • Included format: PDF
  • ebooks can be used on all reading devices
  • Immediate eBook download after purchase
Hardcover $99.00
price for USA
  • ISBN 978-3-642-02546-4
  • Free shipping for individuals worldwide
  • Usually dispatched within 3 to 5 business days.
Softcover $99.00
price for USA
  • ISBN 978-3-642-26072-8
  • Free shipping for individuals worldwide
  • Usually dispatched within 3 to 5 business days.
Loading...

Recommended for you

Loading...

Bibliographic Information

Bibliographic Information
Book Title
Continuous-Time Markov Decision Processes
Book Subtitle
Theory and Applications
Authors
Series Title
Stochastic Modelling and Applied Probability
Series Volume
62
Copyright
2009
Publisher
Springer-Verlag Berlin Heidelberg
Copyright Holder
Springer-Verlag Berlin Heidelberg
eBook ISBN
978-3-642-02547-1
DOI
10.1007/978-3-642-02547-1
Hardcover ISBN
978-3-642-02546-4
Softcover ISBN
978-3-642-26072-8
Series ISSN
0172-4568
Edition Number
1
Number of Pages
XVIII, 234
Topics