Support Vector Machines

1. Front Matter

   Pages 1-14

   PDF

2. Introduction

   Pages 1-20

3. Loss Functions and Their Risks

   Pages 21-47

4. Surrogate Loss Functions (*)

   Pages 48-109

5. Kernels and Reproducing Kernel Hilbert Spaces

   Pages 110-163

6. Infinite-Sample Versions of Support VectorMachines

   Pages 164-201

7. Basic Statistical Analysis of SVMs

   Pages 202-237

8. Advanced Statistical Analysis of SVMs (*)

   Pages 238-284

9. Support Vector Machines for Classification

   Pages 285-329

10. Support Vector Machines for Regression.

    Pages 330-351

11. Robustness

    Pages 352-407

12. Computational Aspects

    Pages 408-451

13. Data Mining

    Pages 452-466

14. Back Matter

    Pages 1-130

    PDF

Every mathematical discipline goes through three periods of development: the naive, the formal, and the critical. David Hilbert The goal of this book is to explain the principles that made support vector machines (SVMs) a successful modeling and prediction tool for a variety of applications. We try to achieve this by presenting the basic ideas of SVMs together with the latest developments and current research questions in a uni?ed style. In a nutshell, we identify at least three reasons for the success of SVMs: their ability to learn well with only a very small number of free parameters, their robustness against several types of model violations and outliers, and last but not least their computational e?ciency compared with several other methods. Although there are several roots and precursors of SVMs, these methods gained particular momentum during the last 15 years since Vapnik (1995, 1998) published his well-known textbooks on statistical learning theory with aspecialemphasisonsupportvectormachines. Sincethen,the?eldofmachine learninghaswitnessedintenseactivityinthestudyofSVMs,whichhasspread moreandmoretootherdisciplinessuchasstatisticsandmathematics. Thusit seems fair to say that several communities are currently working on support vector machines and on related kernel-based methods. Although there are many interactions between these communities, we think that there is still roomforadditionalfruitfulinteractionandwouldbegladifthistextbookwere found helpful in stimulating further research. Many of the results presented in this book have previously been scattered in the journal literature or are still under review. As a consequence, these results have been accessible only to a relativelysmallnumberofspecialists,sometimesprobablyonlytopeoplefrom one community but not the others.

• machine learning theory
• support vector machines
• consistency
• learning rates
• statistical robustness
• kernels
• reproducing kernel Hilbert spaces
• numerical aspects
• data mining
• bootstrap

From the reviews:

“This book has many remarkable qualities which make it commendable to a large mathematical audience. …It is probably the first book on this topic…which is genuinely aimed at a mathematician reader. No technical issue is avoided, and fine points like measurability, integrability, existence and regularity of solutions, etc., are addressed with due rigor and precision. …The authors take special care to make the book self-contained and accessible to non-specialists…always including very detailed proofs for all results. A substantial appendix acts as a handy reference of fundamental results of analysis and probability needed throughout the book, even including a full proof of Talagrand’s concentration inequality. Many well-thought –out exercises very nicely complete each chapter. Finally, the book as a whole, though voluminous and presenting for the most part some very recent results, always stays very coherent to its choices and goals, and obviously a lot of effort has gone into a clear organization of the material. This work is bound to be recognized as a classic reference on this topic.” (MathSciNet)

“This book presents an extensive account of … Support Vector Machines (SVMs). … The book has many remarkable qualities which make it commendable to a large mathematical audience. First of all it is probably the first book on this topic … which is genuinely aimed at a mathematician reader. … Secondly, the authors take special care to make the book self contained and accessible to non-specialists … . Many well thought-out exercises very nicely complete each chapter. … a classic reference on this topic.” (Gilles Blanchard, Mathematical Reviews, Issue 2010 f)

“A mathematically elaborated topic of support vector machines in a book full with definitions and lemmas. It presents support vector machines (SVMs) as a successful modeling and prediction tool with different examples. This book has 12 chapters and 9 appendices that introduce also marginal applications of SVMs. … This book is … suitable as a textbook on SVMs for graduate courses … .” (Adriana Horníková, Technometrics, Vol. 53 (2), May, 2011)

Ingo Steinwart is a researcher in the machine learning group at the Los Alamos National Laboratory. He works on support vector machines and related methods.

Andreas Christmann is Professor of Stochastics in the Department of Mathematics at the University of Bayreuth. He works in particular on support vector machines and robust statistics.

Policies and ethics