Cybernetics 2.0

This book takes the notions of adaptivity and learning from the realm of engineering into the realm of biology and natural processes. It introduces a Hebbian-LMS algorithm, an integration of unsupervised Hebbian learning and supervised LMS learning in neural networks, as a mathematical representation of a general theory for synaptic learning in the brain, and adaptation and functional control of homeostasis in living systems. Written in a language that is able to address students and scientists with different backgrounds, this book accompanies readers on a unique journey through various homeostatic processes in living organisms, such as body temperature control and synaptic plasticity, explaining how the Hebbian-LMS algorithm can help understand them, and suggesting some open questions for future research. It also analyses cell signalling pathways from an unusual perspective, where hormones and hormone receptors are shown to be regulated via the principles of the Hebbian-LMS algorithm. It further discusses addiction and pain, and various kinds of mood disorders alike, showing how they can be modelled with the Hebbian-LMS algorithm.  For the first time, the Hebbian-LMS algorithm, which has been derived from a combination of Hebbian theory from the neuroscience field and the LMS algorithm from the engineering field of adaptive signal processing, becomes a potent model for understanding how biological regulation works. Thus, this book is breaking new ground in neuroscience by providing scientists with a general theory for how nature does control synaptic learning. It then goes beyond that, showing that the same principles apply to hormone-mediated regulation of physiological processes. In turn, the book tackles in more depth the concept of learning. It covers computer simulations and strategies for training neural networks with the Hebbian-LMS algorithm, demonstrating that the resulting algorithms are able to identify relationships between unknown input patterns. It shows how this can translate in useful ideas to understand human memory and design cognitive structures. All in all, this book offers an absolutely, unique, inspiring reading for biologists, physiologists, and engineers, paving the way for future studies on what we could call the nature’s secret learning algorithm.

Bernard Widrow is Professor Emeritus in the Electrical Engineering Department at Stanford University. His research focuses on adaptive signal processing, adaptive control systems, adaptive neural networks, human memory, cybernetics, and human-like memory for computers. Applications include signal processing, prediction, noise cancelling, adaptive arrays, control systems, and pattern recognition.  He received the Doctor of Science Degree from MIT in 1956, and was appointed Professor from the same University. He has been active in the field of artificial neural networks since 1957, when there were only a half-dozen researchers working on this all over the world. In 1959, he moved to Stanford University. In the same year, together with his student Ted Hoff, he invented the Least Mean Square (LMS) algorithm, which has been the world’s most widely used learning algorithm to date. Since 2010, he has expanded his interest to living neural networks and biological adaptivity. A Life fellow of the Institute of Electrical and Electronic Engineering (IEEE), he was awarded with the IEEE Alexander Graham Bell Medal in 1986 and with the Benjamin Franklin Medal for Electrical Engineering in 2001. He has been inducted into both the US National Academy of Engineering and the Silicon Valley Engineering Hall of Fame, in 1995 and 1999, respectively.