Stochastic Approximation and Recursive Algorithms and Applications

1. Front Matter

   Pages i-xxii

   PDF

2. Introduction: Applications and Issues

   Pages 1-27

3. Applications to Learning, Repeated Games, State Dependent Noise, and Queue Optimization

   Pages 29-62

4. Applications in Signal Processing, Communications, and Adaptive Control

   Pages 63-93

5. Mathematical Background

   Pages 95-115

6. Convergence with Probability One: Martingale Difference Noise

   Pages 117-159

7. Convergence with Probability One: Correlated Noise

   Pages 161-212

8. Weak Convergence: Introduction

   Pages 213-240

9. Weak Convergence Methods for General Algorithms

   Pages 241-290

10. Applications: Proofs of Convergence

    Pages 291-314

11. Rate of Convergence

    Pages 315-371

12. Averaging of the Iterates

    Pages 373-393

13. Distributed/Decentralized and Asynchronous Algorithms

    Pages 395-441

14. Back Matter

    Pages 443-474

    PDF

The basic stochastic approximation algorithms introduced by Robbins and MonroandbyKieferandWolfowitzintheearly1950shavebeenthesubject of an enormous literature, both theoretical and applied. This is due to the large number of applications and the interesting theoretical issues in the analysis of “dynamically de?ned” stochastic processes. The basic paradigm is a stochastic di?erence equation such as ? = ? + Y , where ? takes n+1 n n n n its values in some Euclidean space, Y is a random variable, and the “step n size” > 0 is small and might go to zero as n??. In its simplest form, n ? is a parameter of a system, and the random vector Y is a function of n “noise-corrupted” observations taken on the system when the parameter is set to ? . One recursively adjusts the parameter so that some goal is met n asymptotically. Thisbookisconcernedwiththequalitativeandasymptotic properties of such recursive algorithms in the diverse forms in which they arise in applications. There are analogous continuous time algorithms, but the conditions and proofs are generally very close to those for the discrete time case. The original work was motivated by the problem of ?nding a root of a continuous function g ¯(?), where the function is not known but the - perimenter is able to take “noisy” measurements at any desired value of ?. Recursive methods for root ?nding are common in classical numerical analysis, and it is reasonable to expect that appropriate stochastic analogs would also perform well.

• Averaging
• Markov chain
• Martingale
• algorithms
• diffusion process
• jump diffusion
• linear optimization
• numerical methods
• optimization
• programming
• random walk
• stability

From the reviews of the second edition:

"This is the second edition of an excellent book on stochastic approximation, recursive algorithms and applications … . Although the structure of the book has not been changed, the authors have thoroughly revised it and added additional material … ." (Evelyn Buckwar, Zentralblatt MATH, Vol. 1026, 2004)

"The book attempts to convince that … algorithms naturally arise in many application areas … . I do not hesitate to conclude that this book is exceptionally well written. The literature citation is extensive, and pertinent to the topics at hand, throughout. This book could be well suited to those at the level of the graduate researcher and upwards." (A. C. Brooms, Journal of the Royal Statistical Society Series A: Statistics in Society, Vol. 169 (3), 2006)

• Division of Applied Mathematics, Brown University, Providence, USA

  Harold J. Kushner

• Department of Mathematics, Wayne State University, Detroit, USA

  G. George Yin

Policies and ethics