Skip to main content
SpringerLink
Log in
Book cover

Continuous-Time Markov Chains and Applications

A Two-Time-Scale Approach

  • Book
  • © 2013

Overview

Authors:
  1. G. George Yin

    1. Department of Mathematics, Wayne State University, Detroit, USA

    View author publications

    You can also search for this author in PubMed   Google Scholar

  2. Qing Zhang

    1. Dept. Mathematics, University of Georgia, Athens, USA

    View author publications

    You can also search for this author in PubMed   Google Scholar

  • New chapters added on backward equations and LQG control problems
  • Bridges the gap between theory and applications
  • Presents results on asymptotic expansions of the corresponding probability distributions

Part of the book series: Stochastic Modelling and Applied Probability (SMAP, volume 37)

This is a preview of subscription content, log in via an institution to check access.

Access this book

Log in via an institution
eBook USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

Table of contents (10 chapters)

  1. Front Matter

    Pages i-xxi
    Download chapter PDF

  2. Prologue and Preliminaries

    1. Front Matter

      Pages 1-1
      Download chapter PDF

    2. Introduction and Overview

      • G. George Yin, Qing Zhang
      Pages 3-16

    3. Mathematical Preliminaries

      • G. George Yin, Qing Zhang
      Pages 17-29

    4. Markovian Models

      • G. George Yin, Qing Zhang
      Pages 31-56

  3. Two-Time-Scale Markov Chains

    1. Front Matter

      Pages 57-57
      Download chapter PDF

    2. Asymptotic Expansions of Solutions for Forward Equations

      • G. George Yin, Qing Zhang
      Pages 59-140

    3. Occupation Measures: Asymptotic Properties and Ramification

      • G. George Yin, Qing Zhang
      Pages 141-234

    4. Asymptotic Expansions of Solutions for Backward Equations

      • G. George Yin, Qing Zhang
      Pages 235-257

  4. Applications: MDPs, Near-optimal Controls, Numerical Methods, and LQG with Switching

    1. Front Matter

      Pages 259-259
      Download chapter PDF

    2. Markov Decision Problems

      • G. George Yin, Qing Zhang
      Pages 261-283

    3. Stochastic Control of Dynamical Systems

      • G. George Yin, Qing Zhang
      Pages 285-318

    4. Numerical Methods for Control and Optimization

      • G. George Yin, Qing Zhang
      Pages 319-339

    5. Hybrid LQG Problems

      • G. George Yin, Qing Zhang
      Pages 341-372

  5. Back Matter

    Pages 373-427
    Download chapter PDF
Back to top

Keywords

About this book

This book gives a systematic treatment of singularly perturbed systems that naturally arise in control and optimization, queueing networks, manufacturing systems, and financial engineering. It presents results on asymptotic expansions of solutions of Komogorov forward and backward equations, properties of functional occupation measures, exponential upper bounds, and functional limit results for Markov chains with weak and strong interactions. To bridge the gap between theory and applications, a large portion of the book is devoted to applications in controlled dynamic systems, production planning, and numerical methods for controlled Markovian systems with large-scale and complex structures in the real-world problems. This second edition has been updated throughout and includes two new chapters on asymptotic expansions of solutions for backward equations and hybrid LQG problems. The chapters on analytic and probabilistic properties of two-time-scale Markov chains have been almost completely rewritten and the notation has been streamlined and simplified. This book is written for applied mathematicians, engineers, operations researchers, and applied scientists. Selected material from the book can also be used for a one semester advanced graduate-level course in applied probability and stochastic processes.

Reviews

From the reviews of the second edition:

“This book is the expanded second edition of ‘Continuous-time Markov chains and applications. A singular perturbation approach.’ which appeared 1998. … The book remains clearly of interest to researchers in stochastic control, operation research, manufacturing system, engineering, economics and applied mathematics.” (Michael Högele, zbMATH, Vol. 1277, 2014)

“The book is devoted to a study of continuous time singularly perturbed (SP) Markov chains and their applications in problems of control and optimization. … There is no doubt that this second, revised and updated edition will be also well received by the intended audience (researchers and graduate students in applied mathematics and control engineering interested in modeling and optimization of complex stochastic systems) and will excite further interest to the topic.” (Vladimir Gaitsgory, SIAM Review, Vol. 55 (4), 2013)

Authors and Affiliations

  • Department of Mathematics, Wayne State University, Detroit, USA

    G. George Yin

  • Dept. Mathematics, University of Georgia, Athens, USA

    Qing Zhang

Bibliographic Information

Publish with us

Policies and ethics

Back to top

Search

Navigation