Skip to main content
SpringerLink
Log in

Discrete-Time Markov Control Processes

Basic Optimality Criteria

  • Book
  • © 1996

Overview

Authors:
  1. Onésimo Hernández-Lerma

    1. Departamento de Matemáticas, CINVESTAV-IPN, México DF, México

    View author publications

    You can also search for this author in PubMed   Google Scholar

  2. Jean Bernard Lasserre

    1. LAAS-CNRS, Toulouse Cédex, France

    View author publications

    You can also search for this author in PubMed   Google Scholar

Part of the book series: Stochastic Modelling and Applied Probability (SMAP, volume 30)

This is a preview of subscription content, log in via an institution to check access.

Access this book

Log in via an institution
eBook USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

Table of contents (6 chapters)

  1. Front Matter

    Pages i-xiv
    Download chapter PDF

  2. Introduction and Summary

    • Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 1-12

  3. Markov Control Processes

    • Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 13-21

  4. Finite-Horizon Problems

    • Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 23-42

  5. Infinite-Horizon Discounted-Cost Problems

    • Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 43-73

  6. Long-Run Average-Cost Problems

    • Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 75-124

  7. The Linear Programming Formulation

    • Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 125-167

  8. Back Matter

    Pages 169-216
    Download chapter PDF
Back to top

Keywords

About this book

This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes (MCPs). Interest is mainly confined to MCPs with Borel state and control (or action) spaces, and possibly unbounded costs and noncompact control constraint sets. MCPs are a class of stochastic control problems, also known as Markov decision processes, controlled Markov processes, or stochastic dynamic pro­ grams; sometimes, particularly when the state space is a countable set, they are also called Markov decision (or controlled Markov) chains. Regardless of the name used, MCPs appear in many fields, for example, engineering, economics, operations research, statistics, renewable and nonrenewable re­ source management, (control of) epidemics, etc. However, most of the lit­ erature (say, at least 90%) is concentrated on MCPs for which (a) the state space is a countable set, and/or (b) the costs-per-stage are bounded, and/or (c) the control constraint sets are compact. But curiously enough, the most widely used control model in engineering and economics--namely the LQ (Linear system/Quadratic cost) model-satisfies none of these conditions. Moreover, when dealing with "partially observable" systems) a standard approach is to transform them into equivalent "completely observable" sys­ tems in a larger state space (in fact, a space of probability measures), which is uncountable even if the original state process is finite-valued.

Authors and Affiliations

  • Departamento de Matemáticas, CINVESTAV-IPN, México DF, México

    Onésimo Hernández-Lerma

  • LAAS-CNRS, Toulouse Cédex, France

    Jean Bernard Lasserre

Bibliographic Information

Publish with us

Policies and ethics

Back to top

Search

Navigation