Logo - springer
Slogan - springer

Engineering - Control Engineering | Simulation-Based Algorithms for Markov Decision Processes

Simulation-Based Algorithms for Markov Decision Processes

Chang, H.S., Hu, J., Fu, M.C., Marcus, S.I.

2nd ed. 2013, XVII, 229 p. 29 illus., 1 illus. in color.

Available Formats:
eBook
Information

Springer eBooks may be purchased by end-customers only and are sold without copy protection (DRM free). Instead, all eBooks include personalized watermarks. This means you can read the Springer eBooks across numerous devices such as Laptops, eReaders, and tablets.

You can pay for Springer eBooks with Visa, Mastercard, American Express or Paypal.

After the purchase you can directly download the eBook file or read it online in our Springer eBook Reader. Furthermore your eBook will be stored in your MySpringer account. So you can always re-download your eBooks.

 
$99.00

(net) price for USA

ISBN 978-1-4471-5022-0

digitally watermarked, no DRM

Included Format: PDF and EPUB

download immediately after purchase


learn more about Springer eBooks

add to marked items

Hardcover
Information

Hardcover version

You can pay for Springer Books with Visa, Mastercard, American Express or Paypal.

Standard shipping is free of charge for individual customers.

 
$129.00

(net) price for USA

ISBN 978-1-4471-5021-3

free shipping for individuals worldwide

usually dispatched within 3 to 5 business days


add to marked items

  • Rigorous theoretical derivation of sampling and population-based algorithms enables the reader to expand on the work presented in the certainty that new results will have a sound foundation
  • New chapter on game-theoretic methods for solving Markov decision processes gives the researcher up-to-date information
  • Presents recently developed on-line methods in constrained and uncertain model settings for the reader to use and adapt in their own research

Markov decision process (MDP) models are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences.  Many real-world problems modeled by MDPs have huge state and/or action spaces, giving an opening to the curse of dimensionality and so making practical solution of the resulting models intractable.  In other cases, the system of interest is too complex to allow explicit specification of some of the MDP model parameters, but simulation samples are readily available (e.g., for random transitions and costs). For these settings, various sampling and population-based algorithms have been developed to overcome the difficulties of computing an optimal solution in terms of a policy and/or value function.  Specific approaches include adaptive sampling, evolutionary policy iteration, evolutionary random policy search, and model reference adaptive search.
This substantially enlarged new edition reflects the latest developments in novel algorithms and their underpinning theories, and presents an updated account of the topics that have emerged since the publication of the first edition. Includes:
. innovative material on MDPs, both in constrained settings and with uncertain transition properties;
. game-theoretic method for solving MDPs;
. theories for developing roll-out based algorithms; and
. details of approximation stochastic annealing, a population-based on-line simulation-based algorithm.
The self-contained approach of this book will appeal not only to researchers in MDPs, stochastic modeling, and control, and simulation but will be a valuable source of tuition and reference for students of control and operations research.

The Communications and Control Engineering series reports major technological advances which have potential for great impact in the fields of communication and control. It reflects

research in industrial and academic institutions around the world so that the readership can exploit new possibilities as they become available.

Content Level » Research

Keywords » Controlled Markov Chains - Markov Decision Processes - Simulation-based Algorithms - Stochastic Dynamic Programming - Stochastic Modeling

Related subjects » Applications - Control Engineering - Operations Research & Decision Theory - Probability Theory and Stochastic Processes - Theoretical Computer Science

Table of contents / Preface / Sample pages 

Popular Content within this publication 

 

Articles

Read this Book on Springerlink

Services for this book

New Book Alert

Get alerted on new Springer publications in the subject area of Control.