Accelerated Optimization in Machine Learning: First-Order Algorithms
by Zhouchen Lin, Professor of the Key Laboratory of Machine Perception (Ministry of Education), School of EECS, Peking University
Machine learning relies heavily on optimization to solve problems with its learning models, and the acceleration of optimization algorithms is crucial for the efficiency of machine learning. While some existing books have introduced some accelerated algorithms, they are nevertheless incomplete, unsystematic, and not up-to-date. Thus, in early 2018 I decided to write a monograph on the state-of-the-arts of accelerated algorithms.
This book aims to deliver a systematic and self-contained reference, with sufficient preliminary materials and detailed proofs, so that the readers need not consult scattered literatures, be plagued by inconsistent notations, or be carried away from the central ideas by non-essential contents.
Since first-order methods are the mainstream approaches used in machine learning, this book focuses on accelerated first-order optimization algorithms. The algorithms are organized by their nature: deterministic algorithms for unconstrained convex problems (Chap. 2), constrained convex problems (Chap. 3), and (unconstrained) nonconvex problems (Chap. 4), as well as stochastic algorithms for centralized optimization (Chap. 5) and distributed optimization (Chap. 6).
To make our book self-contained, for each introduced algorithm we give the details of its proof. This book serves as a reference to part of the recent advances in optimization. It is appropriate for graduate students and researchers who are interested in machine learning and optimization. Nonetheless, the proofs for achieving critical points (Sect. 4.2), escaping saddle points (Sect. 4.3), and decentralized topology (Sect. 6.2.2) are highly non-trivial. So uninterested readers may skip them.
Finally, I am truly honored to have the forewords from Prof. Michael I. Jordan, Prof. Zongben Xu, and Prof. Zhi-Quan Luo.
Accelerated Optimization for Machine Learning: First-Order Algorithms
Authors: Lin, Zhouchen, Li, Huan, Fang, Cong
- The first monograph on accelerated first-order optimization algorithms used in machine learning
- Includes forewords by Michael I. Jordan, Zongben Xu, and Zhi-Quan Luo, and written by experts on machine learning and optimization
- Is comprehensive, up-to-date, and self-contained, making it easy for beginners to grasp the frontiers of optimization in machine learning
The book
About the author
Zhouchen Lin is a leading expert in the fields of machine learning and computer vision. He is currently a Professor at the Key Laboratory of Machine Perception (Ministry of Education), School of EECS, Peking University. He served as an area chair for several prestigious conferences, including CVPR, ICCV, ICML, NIPS/NeurIPS, AAAI and IJCAI. He is an associate editor of the IEEE Transactions on Pattern Analysis and Machine Intelligence and the International Journal of Computer Vision. He is a Fellow of IAPR and IEEE.
点击观看直播回放
在线直播：机器学习中优化算法前沿简介
Line