Model Selection and Error Estimation in a Nutshell

1. Front Matter

   Pages i-xiii

   PDF

2. Introduction

   • Luca Oneto

   Pages 1-3

3. The “Five W” of MS and EE

   • Luca Oneto

   Pages 5-11

4. Preliminaries

   • Luca Oneto

   Pages 13-23

5. Resampling Methods

   • Luca Oneto

   Pages 25-31

6. Complexity-Based Methods

   • Luca Oneto

   Pages 33-57

7. Compression Bound

   • Luca Oneto

   Pages 59-63

8. Algorithmic Stability Theory

   • Luca Oneto

   Pages 65-74

9. PAC-Bayes Theory

   • Luca Oneto

   Pages 75-86

10. Differential Privacy Theory

    • Luca Oneto

    Pages 87-97

11. Conclusions and Further Readings

    • Luca Oneto

    Pages 99-100

12. Back Matter

    Pages 101-132

    PDF

How can we select the best performing data-driven model? How can we rigorously estimate its generalization error? Statistical learning theory answers these questions by deriving non-asymptotic bounds on the generalization error of a model or, in other words, by upper bounding the true error of the learned model based just on quantities computed on the available data. However, for a long time, Statistical learning theory has been considered only an abstract theoretical framework, useful for inspiring new learning approaches, but with limited applicability to practical problems. The purpose of this book is to give an intelligible overview of the problems of model selection and error estimation, by focusing on the ideas behind the different statistical learning theory approaches and simplifying most of the technical aspects with the purpose of making them more accessible and usable in practice. The book starts by presenting the seminal works of the 80’s and includes the most recent results. It discusses open problems and outlines future directions for research.

• Statistical Learning Theory
• Empirical Data
• Model Selection
• Error Estimation
• Resampling Methods
• Complexity-Based Methods
• Union and Shell Bounds
• Vapnik-Chernovenkis Theory
• Rademacher Complexity Theory
• Compression Bound
• Algorithmic Stability Theory
• PAC-Bayes Theory
• Differential Privacy Theory
• Data-driven models

• DIBRIS, Università degli Studi di Genova, Genoa, Italy

  Luca Oneto

Luca Oneto was born in Rapallo, Italy in 1986. He received his BSc and MSc in Electronic Engineering at the University of Genoa, Italy respectively in 2008 and 2010. In 2014 he received his PhD from the same university in the School of Sciences and Technologies for Knowledge and Information Retrieval with the thesis ``Learning Based On Empirical Data''. In 2017 he obtained the Italian National Scientific Qualification for the role of Associate Professor in Computer Engineering and in 2018 he obtained the one in Computer Science. He worked as Assistant Professor in Computer Engineering at University of Genoa from 2016 to 2019. In 2018 he was co-founder of the spin-off ZenaByte s.r.l. He is currently Associate Professor in Computer Science at University of Pisa with particular interests in Statistical Learning Theory and Data Science. Besides being an editorial board member of the book series Modeling and Optimization in Science and Technologies he is also co-author of the textbook Introduction to Digital Systems Design (Donzellini et al., Springer, 2019). 

Policies and ethics