Statistical Learning from a Regression Perspective

Statistical Learning from a Regression Perspective considers statistical learning applications when interest centers on the conditional distribution of the response variable, given a set of predictors, and when it is important to characterize how the predictors are related to the response. As a first approximation, this is can be seen as an extension of nonparametric regression. Among the statistical learning procedures examined are bagging, random forests, boosting, and support vector machines. Response variables may be quantitative or categorical.

Real applications are emphasized, especially those with practical implications. One important theme is the need to explicitly take into account asymmetric costs in the fitting process. For example, in some situations false positives may be far less costly than false negatives. Another important theme is to not automatically cede modeling decisions to a fitting algorithm. In many settings, subject-matter knowledge should trump formal fitting criteria. Yet another important theme is to appreciate the limitation of one’s data and not apply statistical learning procedures that require more than the data can provide.

The material is written for graduate students in the social and life sciences and for researchers who want to apply statistical learning procedures to scientific and policy problems. Intuitive explanations and visual representations are prominent. All of the analyses included are done in R.

• Fitting
• bagging
• boosting
• classification
• random forests
• statistical learning
• support vector machines

From the reviews:

"I believe that the practical utility of statistical learning over more traditional non- and parametric regression approaches has yet to be truly demonstrate but the procedures presented in this text do show considerable potential…. The mathematical prerequisites for using this book are minimal…. Some familiarity with using a computer is necessary in order to gain the most benefit for the text, and some previous experience of using a statistical software package would be advantageous." (C.M. O’Brien, International Statistical Review, 2009, 77, 1)

"The readers of this book will obtain the knowledge of the dialectic of the regression modeling problems arising in the study of predictor –response. A large percent of the contents is devoted to discuss how to understand phenomena through regression equation fitting. … I recommend it for practitioners and professors have the responsibility of teaching on the subject, the book gives an interesting perspective for dealing with regression." (Sovandep.H. Kumar, Revista Investigación Operacional, Vol. 30 (2), 2009)

"On the positive side, SLRP is a nice addition to the data mining literature, more accessible than ESL. It gives good references and provides statistical detail. In general, I enjoyed the philosophical discussions about how statistical learning fits in with statistical inference. I may not hand it over to my colleagues in Biology and Sociology, but I will seriously consider recommending it to the undergraduates in my data mining seminar." (Richard D. DE VEAUX, The American Statistician, Novemeber 2009, Volume 63, Number 4, pp. 297-411)

“…The strength of this book is its extensive discussion of practical issues. Algorithmic details are a starting point for discussing why and how methods work, comparison with other methodologies, limitations and strengths, and so on. Throughout the book, examples are worked through in detail. Each chapter except the first and the last end with a section headed ‘Software Considerations’, followed by ‘Summary and Conclusions’ and data analysis exercises. …Regression methods, both the theory and the practice, remain a work in progress… .Berk has made a good start in pulling together commentary on issues of major importance.” (Journal of Statistical Software, Vol. 29, Book Review 12, February 2009)

“This book is unique in that statistical learning is discussed by a sociology–PhD scientist, Professor Richard Berk, who has extensive research  accomplishments in the intersection of social science and statistics. …The key strength of this book is in its emphasis on practical applications and hands-on learning of the statistical learning methods. Each chapter has real data examples … and  goes through their analyses using statistical software R (2009). This design effectively illustrates the use of the methods in practice. ‘Software consideration’  given at the end of each chapter provides discussions oncurrently available computational tools, both functions/packages of  R and other software, and is useful in practice. Emphasis on using R that is freely available worldwide is a major advantage in terms of readers’ accessibility to the methods. Furthermore, each chapter contains exercises for practicing different aspects of the methods in the chapter. The solutions and R codes of these exercises are provided at the author’s website…: this is another useful feature enhancing the hands-on learning. …A notable difference…is that this book is written with little mathematics. …Consequently, emphasis is not to understand the statistical-learning methods mathematically. Rather, the methods are explained mostly algorithmically in English, providing readers story-like descriptions of them. This would appeal to readers who are users of the statistical-learning methods but are not mathematically oriented. …” (Biometrics 65, 1309–1310, December 2009)

“The author covers a remarkable terrainin a relatively short book. Up-to-date methods are presented, and their main features are explained with a minimum of mathematical notation. … The problems at the end of each chapter are a real jewel: they lead the reader to a clear understanding of the issues treated in the chapter … . The book will no doubt be useful for the intended readership. Even the mathematically trained reader … may find useful ideas in it.” (Ricardo Maronna, Statistical Papers, Vol. 52, 2011)

Policies and ethics