Optimization

1. Front Matter

   Pages i-xvii

   PDF

2. Elementary Optimization

   • Kenneth Lange

   Pages 1-21

3. The Seven C’s of Analysis

   • Kenneth Lange

   Pages 23-52

4. The Gauge Integral

   • Kenneth Lange

   Pages 53-74

5. Differentiation

   • Kenneth Lange

   Pages 75-105

6. Karush-Kuhn-Tucker Theory

   • Kenneth Lange

   Pages 107-135

7. Convexity

   • Kenneth Lange

   Pages 137-170

8. Block Relaxation

   • Kenneth Lange

   Pages 171-183

9. The MM Algorithm

   • Kenneth Lange

   Pages 185-219

10. The EM Algorithm

    • Kenneth Lange

    Pages 221-244

11. Newton’s Method and Scoring

    • Kenneth Lange

    Pages 245-272

12. Conjugate Gradient and Quasi-Newton

    • Kenneth Lange

    Pages 273-290

13. Analysis of Convergence

    • Kenneth Lange

    Pages 291-312

14. Penalty and Barrier Methods

    • Kenneth Lange

    Pages 313-339

15. Convex Calculus

    • Kenneth Lange

    Pages 341-381

16. Feasibility and Duality

    • Kenneth Lange

    Pages 383-414

17. Convex Minimization Algorithms

    • Kenneth Lange

    Pages 415-444

18. The Calculus of Variations

    • Kenneth Lange

    Pages 445-472

19. Back Matter

    Pages 473-529

    PDF

Finite-dimensional optimization problems occur throughout the mathematical sciences. The majority of these problems cannot be solved analytically. This introduction to optimization attempts to strike a balance between presentation of mathematical theory and development of numerical algorithms. Building on students’ skills in calculus and linear algebra, the text provides a rigorous exposition without undue abstraction. Its stress on statistical applications will be especially appealing to graduate students of statistics and biostatistics. The intended audience also includes students in applied mathematics, computational biology, computer science, economics, and physics who want to see rigorous mathematics combined with real applications.

In this second edition the emphasis remains on finite-dimensional optimization. New material has been added on the MM algorithm, block descent and ascent, and the calculus of variations. Convex calculus is now treated in much greater depth.  Advanced topics such as the Fenchel conjugate, subdifferentials, duality, feasibility, alternating projections, projected gradient methods, exact penalty methods, and Bregman iteration will equip students with the essentials for understanding modern data mining techniques in high dimensions.

 

• Convexity
• Differentiation
• Gauge Integral
• Integration
• Optimization
• Statistical variance

• Biomathematics, Human Genetics, Statistics, University of California, Los Angeles, USA

  Kenneth Lange

Kenneth Lange is the Rosenfeld Professor of Computational Genetics at UCLA. He is also Chair of the Department of Human Genetics and Professor of Biomathematics and Statistics. At various times during his career, he has held appointments at the University of New Hampshire, MIT, Harvard, the University of Michigan, the University of Helsinki, and Stanford. He is a fellow of the American Statistical Association, the Institute of Mathematical Statistics, and the American Institute for Medical and Biomedical Engineering. His research interests include human genetics, population modeling, biomedical imaging, computational statistics, and applied stochastic processes. Springer previously published his books Mathematical and Statistical Methods for Genetic Analysis, Numerical Analysis for Statisticians, and Applied Probability, all in second editions.

Policies and ethics