Skip to main content
SpringerLink
Log in

Dynamic Optimization

Deterministic and Stochastic Models

  • Textbook
  • © 2016

Overview

Authors:
  1. Karl Hinderer

    1. Karlsruher Institut für Technologie (KIT), Karlsruhe, Germany

    View author publications

    You can also search for this author in PubMed   Google Scholar

  2. Ulrich Rieder

    1. University of Ulm, Ulm, Germany

    View author publications

    You can also search for this author in PubMed   Google Scholar

  3. Michael Stieglitz

    1. Karlsruher Institut für Technologie (KIT), Karlsruhe, Germany

    View author publications

    You can also search for this author in PubMed   Google Scholar

  • Provides a self-contained and easy-to-read introduction to dynamic programming
  • Provides a comprehensive treatment of discrete-time multistage optimization
  • Presents the theory of Markov decision processes without advanced measure theory
  • Includes various examples and exercises (without solutions)
  • Includes supplementary material: sn.pub/extras

Part of the book series: Universitext (UTX)

This is a preview of subscription content, log in via an institution to check access.

Access this book

Log in via an institution
eBook USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

Table of contents (26 chapters)

  1. Front Matter

    Pages i-xxii
    Download chapter PDF

  2. Introduction and Organization of the Book

    • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
    Pages 1-11

  3. Deterministic Models

    1. Front Matter

      Pages 13-13
      Download chapter PDF

    2. The Stationary Deterministic Model and the Basic Solution Procedure

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 15-33

    3. Additional General Issues

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 35-49

    4. Examples of Deterministic Dynamic Programs

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 51-68

    5. Absorbing Dynamic Programs and Acyclic Networks

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 69-86

    6. Monotonicity of the Value Functions

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 87-104

    7. Concavity and Convexity of the Value Functions

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 105-123

    8. Monotone and ILIP Maximizers

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 125-148

    9. Existence of Optimal Action Sequences

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 149-167

    10. Stationary Models with Large Horizon

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 169-186

  4. Markovian Decision Processes

    1. Front Matter

      Pages 187-187
      Download chapter PDF

    2. Control Models with Disturbances

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 189-198

    3. Markovian Decision Processes with Finite Transition Law

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 199-219

    4. Examples of Markovian Decision Processes with Finite Transition Law

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 221-257

    5. Markovian Decision Processes with Discrete Transition Law

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 259-275

    6. Examples with Discrete Disturbances and with Discrete Transition Law

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 277-289

    7. Models with Arbitrary Transition Law

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 291-318

    8. Existence of Optimal Policies

      • Karl Hinderer, Ulrich Rieder, Michael Stieglitz
      Pages 319-325
Back to top

Keywords

About this book

This book explores discrete-time dynamic optimization and provides a detailed introduction to both deterministic and stochastic models. Covering problems with finite and infinite horizon, as well as Markov renewal programs, Bayesian control models and partially observable processes, the book focuses on the precise modelling of applications in a variety of areas, including operations research, computer science, mathematics, statistics, engineering, economics and finance.


Dynamic Optimization is a carefully presented textbook which starts with discrete-time deterministic dynamic optimization problems, providing readers with the tools for sequential decision-making, before proceeding to the more complicated stochastic models. The authors present complete and simple proofs and illustrate the main results with numerous examples and exercises (without solutions). With relevant material covered in four appendices, this book is completely self-contained.



Reviews

“Part I deals with deterministic dynamic optimization models describing the control of discrete-time systems. Part II is devoted to discrete-time stochastic control models. Part III … is devoted to Markovian decision processes with disturbances… The book comprises a lot of examples, problems for readers, and supplements with additional comments for the advanced reader and with bibliographic notes.” (Svetlana A. Kravchenko, zbMATH 1365.90002)

Authors and Affiliations

  • Karlsruher Institut für Technologie (KIT), Karlsruhe, Germany

    Karl Hinderer, Michael Stieglitz

  • University of Ulm, Ulm, Germany

    Ulrich Rieder

About the authors

Karl Hinderer was Professor of Stochastics at the Karlsruhe Institute of Technology KIT. He wrote the seminal book Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter (1970) and the textbook Grundbegriffe der Wahrscheinlichkeitstheorie (1972). His main research areas were stochastic dynamic programming, probability and stochastic processes.


Ulrich Rieder is Professor emeritus at the University of Ulm. From 1990 to 2008, he was Editor-in-Chief of Mathematical Methods of Operations Research. His main research areas include stochastic dynamic programming and control, risk-sensitive Markov decision processes, stochastic games, and financial optimization.


Michael Stieglitz was Professor at the University of Karlsruhe until 2002. He contributes to summability, approximation theory, and probability.

Bibliographic Information

Publish with us

Policies and ethics

Back to top

Search

Navigation