The Nature of Statistical Learning Theory

1. Front Matter

   Pages i-xix

   PDF

2. Introduction: Four Periods in the Research of the Learning Problem

   • Vladimir N. Vapnik

   Pages 1-15

3. Setting of the Learning Problem

   • Vladimir N. Vapnik

   Pages 17-34

4. Consistency of Learning Processes

   • Vladimir N. Vapnik

   Pages 35-68

5. Bounds on the Rate of Convergence of Learning Processes

   • Vladimir N. Vapnik

   Pages 69-91

6. Controlling the Generalization Ability of Learning Processes

   • Vladimir N. Vapnik

   Pages 93-122

7. Methods of Pattern Recognition

   • Vladimir N. Vapnik

   Pages 123-180

8. Methods of Function Estimation

   • Vladimir N. Vapnik

   Pages 181-224

9. Direct Methods in Statistical Learning Theory

   • Vladimir N. Vapnik

   Pages 225-265

10. The Vicinal Risk Minimization Principle and the SVMs

    • Vladimir N. Vapnik

    Pages 267-290

11. Conclusion: What Is Important in Learning Theory?

    • Vladimir N. Vapnik

    Pages 291-299

12. Back Matter

    Pages 301-314

    PDF

The aim of this book is to discuss the fundamental ideas which lie behind the statistical theory of learning and generalization. It considers learning as a general problem of function estimation based on empirical data. Omitting proofs and technical details, the author concentrates on discussing the main results of learning theory and their connections to fundamental problems in statistics. These include: * the setting of learning problems based on the model of minimizing the risk functional from empirical data * a comprehensive analysis of the empirical risk minimization principle including necessary and sufficient conditions for its consistency * non-asymptotic bounds for the risk achieved using the empirical risk minimization principle * principles for controlling the generalization ability of learning machines using small sample sizes based on these bounds * the Support Vector methods that control the generalization ability when estimating function using small sample size. The second edition of the book contains three new chapters devoted to further development of the learning theory and SVM techniques. These include: * the theory of direct method of learning based on solving multidimensional integral equations for density, conditional probability, and conditional density estimation * a new inductive principle of learning. Written in a readable and concise style, the book is intended for statisticians, mathematicians, physicists, and computer scientists. Vladimir N. Vapnik is Technology Leader AT&T Labs-Research and Professor of London University. He is one of the founders of

• Conditional probability
• Statistical Learning
• Statistical Theory
• cognition
• control
• learning
• pattern recognition
• statistics

From the reviews of the second edition:

ZENTRALBLATT MATH

"...written in a concise style. It must be recommended to scientists of statistics, mathematics, physics, and computer science."

SHORT BOOK REVIEWS

"This interesting book helps a reader to understand the interconnections between various streams in the empirical modeling realm and may be recommended to any reader who feels lost in modern terminology, such as artificial intelligence, neural networks, machine learning etcetera."

"The book by Vapnik focuses on how to estimate a function of parameters from empirical data … . The book is concisely written and is intended to be useful to statisticians, computer scientists, mathematicians, and physicists. … This book is very well written at a very high level of abstract thinking and comprehension. The references are up-to-date." (Ramalingam Shanmugam, Journal of Statistical Computation and Simulation, Vol. 75 (2), February, 2005)

"The aim of the book is to introduce a wide range of readers to the fundamental ideas of statistical learning theory. … Each chapter is supplemented by ‘Reasoning and Comments’ which describe the relations between classical research in mathematical statistics and research in learning theory. … The book is well suited to promote the ideas of statistical learning theory and can be warmly recommended to all who are interested in computer learning problems." (S. Vogel, Metrika, June, 2002)

• Room 3-130, AT&T Labs-Research, Red Bank, USA

  Vladimir N. Vapnik

Policies and ethics