Editors:
- Offers a computational framework for modeling active exploratory listening that assigns meaning to auditory scenes
- Presents a model that incorporates both signal-driven (bottom-up), and hypothesis-driven (top-down) processing
- Helps to understand auditory perception and cognitive processes involved with our interaction with the world
Part of the book series: Modern Acoustics and Signal Processing (MASP)
Buy it now
Buying options
Tax calculation will be finalised at checkout
Other ways to access
This is a preview of subscription content, log in via an institution to check for access.
Table of contents (25 chapters)
-
Front Matter
-
Forming and Interpreting Aural Objects: Effects and Models
-
Front Matter
-
-
Configuring and Understanding Aural-Space
-
Front Matter
-
-
Processing Cross-Modal Inference
-
Front Matter
-
-
Evaluating Aural-Scene Quality and Speech Understanding
-
Front Matter
-
About this book
Sound, devoid of meaning, would not matter to us. It is the information sound conveys that helps the brain to understand its environment. Sound and its underlying meaning are always associated with time and space. There is no sound without spatial properties, and the brain always organizes this information within a temporal–spatial framework. This book is devoted to understanding the importance of meaning for spatial and related further aspects of hearing, including cross-modal inference.
People, when exposed to acoustic stimuli, do not react directly to what they hear but rather to what they hear means to them.
This semiotic maxim may not always apply, for instance, when the reactions are reflexive. But, where it does apply, it poses a major challenge to the builders of models of the auditory system. Take, for example, an auditory model that is meant to be implemented on a robotic agent for autonomous search-&-rescue actions. Or think of a system that can performjudgments on the sound quality of multimedia-reproduction systems. It becomes immediately clear that such a system needs
• Cognitive capabilities, including substantial inherent knowledge
• The ability to integrate information across different sensory modalities
To realize these functions, the auditory system provides a pair of sensory organs, the two ears, and the means to perform adequate preprocessing of the signals provided by the ears. This is realized in the subcortical parts of the auditory system. In the title of a prior book, the term Binaural Listening is used to indicate a focus on sub-cortical functions. Psychoacoustics and auditory signal processing contribute substantially to this area.
The preprocessed signals are then forwarded to the cortical parts of the auditory system where, among other things, recognition, classification, localization, scene analysis, assignment of meaning, quality assessment, and action planning take place. Also, information from different sensory modalities is integrated at this level. Between sub-cortical and cortical regions of the auditory system, numerous feedback loops exist that ultimately support the high complexity and plasticity of the auditory system.
The current book concentrates on these cognitive functions. Instead of processing signals, processing symbols is now the predominant modeling task. Substantial contributions to the field draw upon the knowledge acquired by cognitive psychology. The keyword Binaural Understanding in the book title characterizes this shift.
Both books, The Technology of Binaural Listening and the current one, have been stimulated and supported by AABBA, an open research group devoted to the development and application of models of binaural hearing. The current book is dedicated to technologies that help explain, facilitate, apply, and support various aspects of binaural understanding. It is organized into five parts, each containing three to six chapters in order to provide a comprehensive overview of this emerging area. Each chapter was thoroughly reviewed by at least two anonymous, external experts.
The first part deals with the psychophysical and physiological effects of Forming and Interpreting Aural Objects as well as the underlying models. The fundamental concepts of reflexive and reflective auditory feedback are introduced. Mechanisms of binaural attention and attention switching are covered—as well as how auditory Gestalt rules facilitate binaural understanding. A general blackboard architecture is introduced as an example of how machines can learn to form and interpret aural objects to simulate human cognitive listening.
The second part, Configuring and Understanding Aural Space, focuses on the human understanding of complex three-dimensional environments—covering the psychological and biological fundamentals of auditory space formation. This part further addresses the human mechanisms used to process information and interact in complex reverberant environments, such as concert halls and forests, and additionally examines how the auditory system can learn to understand and adapt to these environments.
The third part is dedicated to Processing Cross-Modal Inference and highlights the fundamental human mechanisms used to integrate auditory cues with cues from other modalities to localize and form perceptual objects. This part also provides a general framework for understanding how complex multimodal scenes can be simulated and rendered.
The fourth part, Evaluating Aural-scene Quality and Speech Understanding, focuses on the object-forming aspects of binaural listening and understanding. It addresses cognitive mechanisms involved in both the understanding of speech and the processing of nonverbal information such as Sound Quality and Quality-of- Experience. The aesthetic judgment of rooms is also discussed in this context. Models that simulate underlying human processes and performance are covered in addition to techniques for rendering virtual environments that can then be used to test these models.
The fifth part deals with the Application of Cognitive Mechanisms to Audio Technology. It highlights how cognitive mechanisms can be utilized to create spatial auditory illusions using binaural and other 3D-audio technologies. Further, it covers how cognitive binaural technologies can be applied to improve human performance in auditory displays and to develop new auditory technologies for interactive robots. The book concludes with the application of cognitive binaural technologies to the next generation of hearing aids.
Reviews
“Thought-provoking, encyclopedic in its literature review, and deeply enjoyable. … The Technology of Binaural Understanding is a great physical representation of blackboard architecture: numerous experts coming together to contribute their collective experience. … The Technology of Binaural Understanding will certainly inspire current and future generations of acousticians.” (Brandon Cudequest, Journal of the Audio Engineering Society, Vol. 69 (5), 2021)
Editors and Affiliations
-
Institut für Kommunikationsakustik, Ruhr-Universität Bochum, Bochum, Germany
Jens Blauert
-
School of Architecture, Rensselaer Polytechnic Institute, Troy, USA
Jonas Braasch
About the editors
Jonas Braasch is Associate Professor at the School of Architecture at Rensselaer Polytechnic Institute, where he teaches in the Graduate Program in Architectural Acoustics. His research interests span binaural hearing, auditory modeling, multimodal integration, sensory substitution devices, aural architecture and creative processes in music improvisation. At RPI, he directs the Communication Acoustics and Aural Architecture Research Laboratory (CA^3 RL). He obtained a master’s degree from Dortmund University (Germany, 1998) in Physics and two PhD degrees from Ruhr-University Bochum, Germany (2001, 2004) in Electrical Engineering/Information Science and Musicology. His research interests include Binaural Hearing, Multi-channel Audio Technology, Telematic Music Systems, Perceptual Audio/Visual Integration, Intelligent Systems, and Musical Acoustics. For his work, he has received funding from NSF, NSERC, DFG (German Science Foundation), and NYSCA. As a soprano saxophonist and sound artist, he has worked with Curtis Bahn, Chris Chafe, Michael Century, Mark Dresser, Francisco Lopez, Pauline Oliveros, and Doug van Nort – among others. Jonas Braasch is an affiliated faculty member of RPI’s Experimental Media and Performing Arts Center (EMPAC) and Board Member of the Deep Listening Institute (Kingston, NY).
Bibliographic Information
Book Title: The Technology of Binaural Understanding
Editors: Jens Blauert, Jonas Braasch
Series Title: Modern Acoustics and Signal Processing
DOI: https://doi.org/10.1007/978-3-030-00386-9
Publisher: Springer Cham
eBook Packages: Engineering, Engineering (R0)
Copyright Information: Springer Nature Switzerland AG 2020
Hardcover ISBN: 978-3-030-00385-2Published: 13 August 2020
eBook ISBN: 978-3-030-00386-9Published: 12 August 2020
Series ISSN: 2364-4915
Series E-ISSN: 2364-4923
Edition Number: 1
Number of Pages: XI, 815
Number of Illustrations: 61 b/w illustrations, 152 illustrations in colour
Topics: Acoustics, Engineering Acoustics, Signal, Image and Speech Processing