Overview
- Solves existing problems without requiring deep knowledge of partial differential equations
- Presents a new framework for optimization of stochastic systems, promoting new research pathways
- Shows the reader how to link optimisation of continuous systems with reinforcement-learning algorithms and artificial intelligence
Part of the book series: Communications and Control Engineering (CCE)
Access this book
Tax calculation will be finalised at checkout
Other ways to access
Table of contents (6 chapters)
Keywords
About this book
This monograph applies the relative optimization approach to time nonhomogeneous continuous-time and continuous-state dynamic systems. The approach is intuitively clear and does not require deep knowledge of the mathematics of partial differential equations. The topics covered have the following distinguishing features: long-run average with no under-selectivity, non-smooth value functions with no viscosity solutions, diffusion processes with degenerate points, multi-class optimization with state classification, and optimization with no dynamic programming.
The book begins with an introduction to relative optimization, including a comparison with the traditional approach of dynamic programming. The text then studies the Markov process, focusing on infinite-horizon optimization problems, and moves on to discuss optimal control of diffusion processes with semi-smooth value functions and degenerate points, and optimization of multi-dimensional diffusion processes. The book concludes with a brief overview of performance derivative-based optimization.
Among the more important novel considerations presented are:
- the extension of the Hamilton–Jacobi–Bellman optimality condition from smooth to semi-smooth value functions by derivation of explicit optimality conditions at semi-smooth points and application of this result to degenerate and reflected processes;
- proof of semi-smoothness of the value function at degenerate points;
- attention to the under-selectivity issue for the long-run average and bias optimality;
- discussion of state classification for time nonhomogeneous continuous processes and multi-class optimization; and
- development of the multi-dimensional Tanaka formula for semi-smooth functions and application of this formula to stochastic control of multi-dimensional systems with degenerate points.
The book will be of interest to researchers and students in the field of stochastic control andperformance optimization alike.
Reviews
Authors and Affiliations
About the author
Professor Xi-Ren Cao gained his Masters degree in engineering and PhD in applied mathematics from Harvard University in 1982 and 1984 respectively. He has worked in an academic position at numerous institutions, including as a visiting professor at both the University of Massachusetts and the University of Maryland, a chair professor at the Hong Kong University of Science and Technology, and his current position of Chair Professor at Shanghai Jiao Tong University. He has published 125 peer-reviewed journal papers, 12 invited book chapters, and three books in areas related to stochastic and discrete control. He was Editor-in-Chief for Discrete Event Dynamic Systems: Theory and Applications from 2005 to 2014. He has extensive industrial experiences with Digital Equipment Corporation, Massachusetts, and AT&T Labs. He is Fellow of IEEE and IFAC.
Bibliographic Information
Book Title: Relative Optimization of Continuous-Time and Continuous-State Stochastic Systems
Authors: Xi-Ren Cao
Series Title: Communications and Control Engineering
DOI: https://doi.org/10.1007/978-3-030-41846-5
Publisher: Springer Cham
eBook Packages: Intelligent Technologies and Robotics, Intelligent Technologies and Robotics (R0)
Copyright Information: Springer Nature Switzerland AG 2020
Hardcover ISBN: 978-3-030-41845-8Published: 14 May 2020
Softcover ISBN: 978-3-030-41848-9Published: 14 May 2021
eBook ISBN: 978-3-030-41846-5Published: 13 May 2020
Series ISSN: 0178-5354
Series E-ISSN: 2197-7119
Edition Number: 1
Number of Pages: XIX, 365
Number of Illustrations: 9 b/w illustrations, 12 illustrations in colour
Topics: Control and Systems Theory, Calculus of Variations and Optimal Control; Optimization, Markov model