Applied Machine Learning

1. Front Matter

   Pages i-xxi

   PDF

2. Classification

   1. Front Matter

      Pages 1-1

      PDF

   2. Learning to Classify

      • David Forsyth

      Pages 3-19

   3. SVMs and Random Forests

      • David Forsyth

      Pages 21-48

   4. A Little Learning Theory

      • David Forsyth

      Pages 49-65

3. High Dimensional Data

   1. Front Matter

      Pages 67-67

      PDF

   2. High Dimensional Data

      • David Forsyth

      Pages 69-91

   3. Principal Component Analysis

      • David Forsyth

      Pages 93-115

   4. Low Rank Approximations

      • David Forsyth

      Pages 117-138

   5. Canonical Correlation Analysis

      • David Forsyth

      Pages 139-151

4. Clustering

   1. Front Matter

      Pages 153-153

      PDF

   2. Clustering

      • David Forsyth

      Pages 155-182

   3. Clustering Using Probability Models

      • David Forsyth

      Pages 183-202

5. Regression

   1. Front Matter

      Pages 203-203

      PDF

   2. Regression

      • David Forsyth

      Pages 205-244

   3. Regression: Choosing and Managing Models

      • David Forsyth

      Pages 245-274

   4. Boosting

      • David Forsyth

      Pages 275-302

6. Graphical Models

   1. Front Matter

      Pages 303-303

      PDF

   2. Hidden Markov Models

      • David Forsyth

      Pages 305-332

   3. Learning Sequence Models Discriminatively

      • David Forsyth

      Pages 333-350

Machine learning methods are now an important tool for scientists, researchers, engineers and students in a wide range of areas.  This book is written for people who want to adopt and use the main tools of machine learning, but aren’t necessarily going to want to be machine learning researchers. Intended for students in final year undergraduate or first year graduate computer science programs in machine learning, this textbook is a machine learning toolkit. Applied Machine Learning covers many topics for people who want to use machine learning processes to get things done, with a strong emphasis on using existing tools and packages, rather than writing one’s own code.

A companion to the author's Probability and Statistics for Computer Science, this book picks up where the earlier book left off (but also supplies a summary of probability that the reader can use).


Emphasizing the usefulness ofstandard machinery from applied statistics, this textbook gives an overview of the major applied areas in learning, including coverage of:
• classification using standard machinery (naive bayes; nearest neighbor; SVM)
• clustering and vector quantization (largely as in PSCS)
• PCA (largely as in PSCS)
• variants of PCA (NIPALS; latent semantic analysis; canonical correlation analysis)
• linear regression (largely as in PSCS)
• generalized linear models including logistic regression
• model selection with Lasso, elasticnet
• robustness and m-estimators
• Markov chains and HMM’s (largely as in PSCS)
• EM in fairly gory detail; long experience teaching this suggests one detailed example is required, which students hate; but once they’ve been through that, the next one is easy
• simple graphical models (in the variational inference section)
• classification with neural networks, with a particular emphasis on
image classification
• autoencoding with neural networks
• structure learning

• machine learning
• naive bayes
• nearest neighbor
• SVM
• OCA
• PSCS
• linear regression
• Markov chains
• generalized linear models
• model selection
• EM
• structure learning

• Computer Science Department, University of Illinois Urbana Champaign, Urbana, USA

  David Forsyth

David Forsyth grew up in Cape Town. He received a B.Sc. (Elec. Eng.) from the University of the Witwatersrand, Johannesburg in 1984, an M.Sc. (Elec. Eng.) from that university in 1986, and a D.Phil. from Balliol College, Oxford in 1989. He spent three years on the faculty at the University of Iowa, ten years on the faculty at the University of California at Berkeley, and then moved to the University of Illinois. He served as program co-chair for IEEE Computer Vision and Pattern Recognition in 2000, 2011, 2018 and 2021; general co-chair for CVPR 2006 and ICCV 2019, and program co-chair for the European Conference on Computer Vision 2008, and is a regular member of the program committee of all major international conferences on computer vision. He has served six terms on the SIGGRAPH program committee. In 2006, he received an IEEE technical achievement award, in 2009 he was named an IEEE Fellow, and in 2014 he was named an ACM Fellow. He served as Editor-in-Chief of IEEE TPAMI from 2014-2017. He is lead co-author of Computer Vision: A Modern Approach, a textbook of computer vision that ran to two editions and four languages. He is sole author of Probability and Statistics for Computer Science, which provides the background for this book. Among a variety of odd hobbies, he is a compulsive diver, certified up to normoxic trimix level.

Policies and ethics