Skip to main content
SpringerLink
Log in
Book cover

Neural Networks: Tricks of the Trade

  • Book
  • © 2012

Overview

Editors:
  1. Grégoire Montavon

    1. Dept. of Computer Science, Technische Universität Berlin, Berlin, Germany

    View editor publications

    You can also search for this editor in PubMed   Google Scholar

  2. Geneviève B. Orr

    1. Dept. of computer Science, Willamette University, Salem, USA

    View editor publications

    You can also search for this editor in PubMed   Google Scholar

  3. Klaus-Robert Müller

    1. Dept. of Computer Science, Technische Universität Berlin, Berlin, Germany

    View editor publications

    You can also search for this editor in PubMed   Google Scholar

  • The second edition of the book "reloads" the first edition with more tricks
  • Provides a timely snapshot of tricks, theory and algorithms that are of use

Part of the book series: Lecture Notes in Computer Science (LNCS, volume 7700)

Part of the book sub series: Theoretical Computer Science and General Issues (LNTCS)

This is a preview of subscription content, log in via an institution to check access.

Access this book

Log in via an institution
eBook USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

Table of contents (39 chapters)

  1. Front Matter

    Download chapter PDF

  2. Introduction

    1. Introduction

      • Klaus-Robert Müller
      Pages 1-5

  3. Speeding Learning

    1. Speeding Learning

      • Klaus-Robert Müller
      Pages 7-8

    2. Efficient BackProp

      • Yann A. LeCun, Léon Bottou, Genevieve B. Orr, Klaus-Robert Müller
      Pages 9-48

  4. Regularization Techniques to Improve Generalization

    1. Regularization Techniques to Improve Generalization

      • Klaus-Robert Müller
      Pages 49-51

    2. Early Stopping — But When?

      • Lutz Prechelt
      Pages 53-67

    3. A Simple Trick for Estimating the Weight Decay Parameter

      • Thorsteinn S. Rögnvaldsson
      Pages 69-89

    4. Controlling the Hyperparameter Search in MacKay’s Bayesian Neural Network Framework

      • Tony Plate
      Pages 91-110

    5. Adaptive Regularization in Neural Network Modeling

      • Jan Larsen, Claus Svarer, Lars Nonboe Andersen, Lars Kai Hansen
      Pages 111-130

    6. Large Ensemble Averaging

      • David Horn, Ury Naftaly, Nathan Intrator
      Pages 131-137

  5. Improving Network Models and Algorithmic Tricks

    1. Improving Network Models and Algorithmic Tricks

      • Klaus-Robert Müller
      Pages 139-141

    2. Square Unit Augmented, Radially Extended, Multilayer Perceptrons

      • Gary William Flake
      Pages 143-161

    3. A Dozen Tricks with Multitask Learning

      • Rich Caruana
      Pages 163-189

    4. Solving the Ill-Conditioning in Neural Network Learning

      • Patrick van der Smagt, Gerd Hirzinger
      Pages 191-203

    5. Centering Neural Network Gradient Factors

      • Nicol N. Schraudolph
      Pages 205-223

    6. Avoiding Roundoff Error in Backpropagating Derivatives

      • Tony Plate
      Pages 225-230

  6. Representing and Incorporating Prior Knowledge in Neural Network Training

    1. Representing and Incorporating Prior Knowledge in Neural Network Training

      • Klaus-Robert Müller
      Pages 231-233

    2. Transformation Invariance in Pattern Recognition – Tangent Distance and Tangent Propagation

      • Patrice Y. Simard, Yann A. LeCun, John S. Denker, Bernard Victorri
      Pages 235-269

    3. Combining Neural Networks and Context-Driven Search for On-line, Printed Handwriting Recognition in the Newton

      • Larry S. Yaeger, Brandyn J. Webb, Richard F. Lyon
      Pages 271-293

    4. Neural Network Classification and Prior Class Probabilities

      • Steve Lawrence, Ian Burns, Andrew Back, Ah Chung Tsoi, C. Lee Giles
      Pages 295-309
Back to top

Keywords

About this book

The twenty last years have been marked by an increase in available data and computing power. In parallel to this trend, the focus of neural network research and the practice of training neural networks has undergone a number of important changes, for example, use of deep learning machines.

The second edition of the book augments the first edition with more tricks, which have resulted from 14 years of theory and experimentation by some of the world's most prominent neural network researchers. These tricks can make a substantial difference (in terms of speed, ease of implementation, and accuracy) when it comes to putting algorithms to work on real problems.

Editors and Affiliations

  • Dept. of Computer Science, Technische Universität Berlin, Berlin, Germany

    Grégoire Montavon, Klaus-Robert Müller

  • Dept. of computer Science, Willamette University, Salem, USA

    Geneviève B. Orr

Bibliographic Information

Publish with us

Policies and ethics

Back to top

Search

Navigation