Skip to main content
  • Textbook
  • © 2018

Markov Chains

  • Includes many results which are published for the first time in a textbook
  • Many results are illustrated with simple examples
  • Provides an accessible presentation of important ergodicity results of general state Markov chains with many new proof ideas

Buy it now

Buying options

eBook USD 39.99 USD 79.99
50% discount Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book USD 49.99 USD 99.99
50% discount Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

This is a preview of subscription content, log in via an institution to check for access.

Table of contents (23 chapters)

  1. Front Matter

    Pages i-xviii
  2. Foundations

    1. Front Matter

      Pages 1-1
  3. Foundations

    1. Markov Chains: Basic Definitions

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 3-25
    2. Examples of Markov Chains

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 27-52
    3. Stopping Times and the Strong Markov Property

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 53-74
    4. Martingales, Harmonic Functions and Poisson–Dirichlet Problems

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 75-96
    5. Ergodic Theory for Markov Chains

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 97-115
  4. Irreducible Chains: Basics

    1. Front Matter

      Pages 117-117
  5. Irreducible Chains: Basics

    1. Atomic Chains

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 119-144
    2. Markov Chains on a Discrete State Space

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 145-164
    3. Convergence of Atomic Markov Chains

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 165-189
    4. Small Sets, Irreducibility, and Aperiodicity

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 191-220
    5. Transience, Recurrence, and Harris Recurrence

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 221-239
    6. Splitting Construction and Invariant Measures

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 241-264
    7. Feller and T-Kernels

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 265-286
  6. Irreducible Chains: Advanced Topics

    1. Front Matter

      Pages 287-287
  7. Irreducible Chains: Advanced Topics

    1. Rates of Convergence for Atomic Markov Chains

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 289-312
    2. Geometric Recurrence and Regularity

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 313-337
    3. Geometric Rates of Convergence

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 339-359
    4. (fr)-Recurrence and Regularity

      • Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier
      Pages 361-383

About this book

This book covers the classical theory of Markov chains on general state-spaces as well as many recent developments. The theoretical results are illustrated by simple examples, many of which are taken from Markov Chain Monte Carlo methods. The book is self-contained, while all the results are carefully and concisely proven. Bibliographical notes are added at the end of each chapter to provide an overview of the literature.

Part I lays the foundations of the theory of Markov chain on general states-space. Part II covers the basic theory of irreducible Markov chains on general states-space, relying heavily on regeneration techniques. These two parts can serve as a text on general state-space applied Markov chain theory. Although the choice of topics is quite different from what is usually covered, where most of the emphasis is put on countable state space, a graduate student should be able to read almost all these developments without any mathematical background deeperthan that needed to study countable state space (very little measure theory is required).

Part III covers advanced topics on the theory of irreducible Markov chains. The emphasis is on geometric and subgeometric convergence rates and also on computable bounds. Some results appeared for a first time in a book and others are original. Part IV are selected topics on Markov chains, covering mostly hot recent developments.

Authors and Affiliations

  • Département CITI, Telecom SudParis, Évry, France

    Randal Douc

  • Centre de Mathématiques Appliquées, Ecole Polytechnique, Palaiseau, France

    Eric Moulines

  • Université Pierre et Marie Curie, Paris, France

    Pierre Priouret

  • Université Paris Nanterre, Nanterre, France

    Philippe Soulier

About the authors

Randal Douc is a Professor in the CITI Department at Telecom SudParis. His research interests include parameter estimation in general Hidden Markov models and Markov Chain Monte Carlo (MCMC) and sequential Monte Carlo methods.

Eric Moulines is a Professor at Ecole Polytechnique's Applied Mathematics Center (CMAP, UMR Ecole Polytechnique/CNRS).  


Pierre Priouret is a Professor at Université Pierre et Marie Curie


Philippe Soulier is a professor at Université de Paris-Nanterre

Bibliographic Information

Buy it now

Buying options

eBook USD 39.99 USD 79.99
50% discount Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book USD 49.99 USD 99.99
50% discount Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access