Overview
- The book will be valuable for researchers in the areas of global optimization, machine learning, evolutionary theory, and control theory
- Optimization is a fundamental problem that recurs across scientific disciplines and is pervasive in informatics research, from statistical machine learning to probabilistic models to reinforcement learning
- In the final main chapter of the book the author realizes that the basic mathematical objects developed to account for stochastic optimization have applications far beyond optimization, he thinks about them as stimulus-response systems, the key intuition coming from the Optimization Game
Part of the book series: Natural Computing Series (NCS)
Access this book
Tax calculation will be finalised at checkout
Other ways to access
Table of contents (18 chapters)
-
Front Matter
-
Back Matter
Keywords
About this book
This book examines the mismatch between discrete programs, which lie at the center of modern applied mathematics, and the continuous space phenomena they simulate. The author considers whether we can imagine continuous spaces of programs, and asks what the structure of such spaces would be and how they would be constituted. He proposes a functional analysis of program spaces focused through the lens of iterative optimization.
The author begins with the observation that optimization methods such as Genetic Algorithms, Evolution Strategies, and Particle Swarm Optimization can be analyzed as Estimation of Distributions Algorithms (EDAs) in that they can be formulated as conditional probability distributions. The probabilities themselves are mathematical objects that can be compared and operated on, and thus many methods in Evolutionary Computation can be placed in a shared vector space and analyzed using techniques of functionalanalysis. The core ideas of this book expand from that concept, eventually incorporating all iterative stochastic search methods, including gradient-based methods. Inspired by work on Randomized Search Heuristics, the author covers all iterative optimization methods and not just evolutionary methods. The No Free Lunch Theorem is viewed as a useful introduction to the broader field of analysis that comes from developing a shared mathematical space for optimization algorithms. The author brings in intuitions from several branches of mathematics such as topology, probability theory, and stochastic processes and provides substantial background material to make the work as self-contained as possible.
The book will be valuable for researchers in the areas of global optimization, machine learning, evolutionary theory, and control theory.
Authors and Affiliations
About the author
Alan J. Lockett received his PhD in 2012 at the University of Texas at Austin under the supervision of Risto Miikkulainen, where his research topics included estimation of temporal probabilistic models, evolutionary computation theory, and learning neural network controllers for robotics. After a postdoc in IDSIA (Lugano) with Jürgen Schmidhuber he now works for CS Disco in Houston.
Bibliographic Information
Book Title: General-Purpose Optimization Through Information Maximization
Authors: Alan J. Lockett
Series Title: Natural Computing Series
DOI: https://doi.org/10.1007/978-3-662-62007-6
Publisher: Springer Berlin, Heidelberg
eBook Packages: Computer Science, Computer Science (R0)
Copyright Information: Springer-Verlag GmbH Germany, part of Springer Nature 2020
Hardcover ISBN: 978-3-662-62006-9Published: 17 August 2020
Softcover ISBN: 978-3-662-62009-0Published: 18 August 2021
eBook ISBN: 978-3-662-62007-6Published: 16 August 2020
Series ISSN: 1619-7127
Series E-ISSN: 2627-6461
Edition Number: 1
Number of Pages: XVIII, 561
Topics: Theory of Computation, Artificial Intelligence, Optimization, Mathematics of Computing