Multimodal User Interfaces

1. Front Matter

   Pages i-x

   PDF

2. Introduction

   Pages 1-4

3. Multimodality Theory

   • Niels Ole Bernsen

   Pages 5-29

4. Information-Theoretic Framework for Multimodal Signal Processing

   • Jean-Philippe Thiran, Torsten Butz, A. Murat Tekalp

   Pages 31-59

5. Multimodality for Plastic User Interfaces: Models, Methods, and Principles

   • Jean Vanderdonckt, Gaëlle Calvary, Joëlle Coutaz, Adrian Stanciulescu

   Pages 61-84

6. Face and Speech Interaction

   • Mihai Gurban, Veronica Vilaplana, Jean-Philippe Thiran, Ferran Marques

   Pages 85-117

7. Recognition of Emotional States in Natural Human-Computer Interaction

   • R. Cowie, E. Douglas-Cowie, K. Karpouzis, G. Caridakis, M. Wallace, S. Kollias

   Pages 119-153

8. Two SIMILAR Different Speech and Gestures Multimodal Interfaces

   • Alexey Karpov, Sebastien Carbini, Andrey Ronzhin, Jean Emmanuel Viallet

   Pages 155-184

9. Multimodal User Interfaces in Ubiquitous Environments

   • Fabio Paternò, Carmen Santoro

   Pages 185-199

10. Software Engineering for Multimodal Interactive Systems

    • Laurence Nigay, Jullien Bouchet, David Juras, Benoit Mansoux, Michael Ortega, Marcos Serrano et al.

    Pages 201-218

11. Gestural Interfaces for Hearing-Impaired Communication

    • Oya Aran, Thomas Burger, Lale Akarun, Alice Caplier

    Pages 219-250

12. Modality Replacement Framework for Applications for the Disabled

    • Savvas Argyropoulos, Konstantinos Moustakas

    Pages 251-270

13. A medical Component-based Framework for Image Guided Surgery

    • Daniela G. Trevisan, Vincent Nicolas, Benoit Macq, Luciana P. Nedel

    Pages 271-280

14. Multimodal Interfaces for Laparoscopic Training

    • Pablo Lamata, Carlos Alberola, Francisco Sánchez-Margallo, Miguel Ángel Florido, Enrique J. Gómez

    Pages 281-290

15. Back Matter

    Pages 291-315

    PDF

tionship indicates how multimodal medical image processing can be unified to a large extent, e. g. multi-channel segmentation and image registration, and extend information theoretic registration to other features than image intensities. The framework is not at all restricted to medical images though and this is illustrated by applying it to multimedia sequences as well. In Chapter 4, the main results from the developments in plastic UIs and mul- modal UIs are brought together using a theoretic and conceptual perspective as a unifying approach. It is aimed at defining models useful to support UI plasticity by relying on multimodality, at introducing and discussing basic principles that can drive the development of such UIs, and at describing some techniques as proof-of-concept of the aforementioned models and principles. In Chapter 4, the authors introduce running examples that serve as illustration throughout the d- cussion of the use of multimodality to support plasticity.

• Haptik
• Interaction
• Interface
• Segment
• User Interfaces
• digital signal processing
• haptics
• human-computer interaction (HCI)
• image processing
• image registration
• information
• multimedia
• segmentation
• signal processing
• user interface

• Centre for Research and Technology Hellas, Informatics and Telematics Institute , Thermi-Thessaloniki, Greece

  Dimitrios Tzovaras

Policies and ethics