Read While You Wait - Get immediate ebook access, if available*, when you order a print book

Stochastic Modelling and Applied Probability

Discrete-Time Markov Control Processes

Basic Optimality Criteria

Authors: Hernandez-Lerma, Onesimo, Lasserre, Jean B.

Free Preview

Buy this book

eBook $84.99
price for USA in USD
  • ISBN 978-1-4612-0729-0
  • Digitally watermarked, DRM-free
  • Included format: PDF
  • ebooks can be used on all reading devices
  • Immediate eBook download after purchase
Hardcover $159.99
price for USA in USD
  • ISBN 978-0-387-94579-8
  • Free shipping for individuals worldwide
  • Immediate ebook access, if available*, with your print order
  • Usually dispatched within 3 to 5 business days.
Softcover $109.99
price for USA in USD
  • ISBN 978-1-4612-6884-0
  • Free shipping for individuals worldwide
  • Immediate ebook access, if available*, with your print order
  • Usually dispatched within 3 to 5 business days.
About this book

This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes (MCPs). Interest is mainly confined to MCPs with Borel state and control (or action) spaces, and possibly unbounded costs and noncompact control constraint sets. MCPs are a class of stochastic control problems, also known as Markov decision processes, controlled Markov processes, or stochastic dynamic pro­ grams; sometimes, particularly when the state space is a countable set, they are also called Markov decision (or controlled Markov) chains. Regardless of the name used, MCPs appear in many fields, for example, engineering, economics, operations research, statistics, renewable and nonrenewable re­ source management, (control of) epidemics, etc. However, most of the lit­ erature (say, at least 90%) is concentrated on MCPs for which (a) the state space is a countable set, and/or (b) the costs-per-stage are bounded, and/or (c) the control constraint sets are compact. But curiously enough, the most widely used control model in engineering and economics--namely the LQ (Linear system/Quadratic cost) model-satisfies none of these conditions. Moreover, when dealing with "partially observable" systems) a standard approach is to transform them into equivalent "completely observable" sys­ tems in a larger state space (in fact, a space of probability measures), which is uncountable even if the original state process is finite-valued.

Table of contents (6 chapters)

Table of contents (6 chapters)

Buy this book

eBook $84.99
price for USA in USD
  • ISBN 978-1-4612-0729-0
  • Digitally watermarked, DRM-free
  • Included format: PDF
  • ebooks can be used on all reading devices
  • Immediate eBook download after purchase
Hardcover $159.99
price for USA in USD
  • ISBN 978-0-387-94579-8
  • Free shipping for individuals worldwide
  • Immediate ebook access, if available*, with your print order
  • Usually dispatched within 3 to 5 business days.
Softcover $109.99
price for USA in USD
  • ISBN 978-1-4612-6884-0
  • Free shipping for individuals worldwide
  • Immediate ebook access, if available*, with your print order
  • Usually dispatched within 3 to 5 business days.
Loading...

Recommended for you

Loading...

Bibliographic Information

Bibliographic Information
Book Title
Discrete-Time Markov Control Processes
Book Subtitle
Basic Optimality Criteria
Authors
Series Title
Stochastic Modelling and Applied Probability
Series Volume
30
Copyright
1996
Publisher
Springer-Verlag New York
Copyright Holder
Springer Science+Business Media New York
eBook ISBN
978-1-4612-0729-0
DOI
10.1007/978-1-4612-0729-0
Hardcover ISBN
978-0-387-94579-8
Softcover ISBN
978-1-4612-6884-0
Series ISSN
0172-4568
Edition Number
1
Number of Pages
XIV, 216
Topics

*immediately available upon purchase as print book shipments may be delayed due to the COVID-19 crisis. ebook access is temporary and does not include ownership of the ebook. Only valid for books with an ebook version. Springer Reference Works are not included.