Markov Chains

1. Front Matter

   Pages i-xviii

   PDF

2. Probability Review

   • Pierre Brémaud

   Pages 1-52

3. Discrete-Time Markov Models

   • Pierre Brémaud

   Pages 53-93

4. Recurrence and Ergodicity

   • Pierre Brémaud

   Pages 95-124

5. Long Run Behavior

   • Pierre Brémaud

   Pages 125-165

6. Lyapunov Functions and Martingales

   • Pierre Brémaud

   Pages 167-193

7. Eigenvalues and Nonhomogeneous Markov Chains

   • Pierre Brémaud

   Pages 195-251

8. Gibbs Fields and Monte Carlo Simulation

   • Pierre Brémaud

   Pages 253-322

9. Continuous-Time Markov Models

   • Pierre Brémaud

   Pages 323-368

10. Poisson Calculus and Queues

    • Pierre Brémaud

    Pages 369-416

11. Back Matter

    Pages 417-445

    PDF

In this book, the author begins with the elementary theory of Markov chains and very progressively brings the reader to the more advanced topics. He gives a useful review of probability that makes the book self-contained, and provides an appendix with detailed proofs of all the prerequisites from calculus, algebra, and number theory. A number of carefully chosen problems of varying difficulty are proposed at the close of each chapter, and the mathematics are slowly and carefully developed, in order to make self-study easier. The author treats the classic topics of Markov chain theory, both in discrete time and continuous time, as well as the connected topics such as finite Gibbs fields, nonhomogeneous Markov chains, discrete- time regenerative processes, Monte Carlo simulation, simulated annealing, and queuing theory. The result is an up-to-date textbook on stochastic processes. Students and researchers in operations research and electrical engineering, as well as in physics and biology, will find it very accessible and relevant.

• Markov
• Markov Chains
• Markov model
• Martingale
• Monte Carlo Simulation
• Stochastic model
• Stochastic modelling
• electrical engineering
• ergodicity
• linear optimization
• modeling
• operations research
• regenerative process
• simulation
• stochastic process

• Laboratoire des Signaux et Systèmes, CNRS-ESE, Gif-sur-Yvette, France

  Pierre Brémaud

Policies and ethics