Skip to main content
Springer Nature Link
Log in

Multimodal Corpora

From Models of Natural Interaction to Systems and Applications

  • Book
  • © 2009

Overview

Editors:
  1. Michael Kipp

    1. Deutsches Forschungszentrum für künstliche Intelligenz (DFKI), Saarbrücken, Germany

    View editor publications

    You can also search for this editor in PubMed   Google Scholar

  2. Jean-Claude Martin

    1. Laboratoire d’Informatique pour la Mécanique et les Sciences de l’Ingénieur (LIMSI-CNRS), Orsay Cedex, France

    View editor publications

    You can also search for this editor in PubMed   Google Scholar

  3. Patrizia Paggio

    1. Faculty of Humanities, Centre for Language Technology, University of Copenhagen, Copenhagen, Denmark

    View editor publications

    You can also search for this editor in PubMed   Google Scholar

  4. Dirk Heylen

    1. Computer Science, Human Media Interaction, University of Twente, Enschede, The Netherlands

    View editor publications

    You can also search for this editor in PubMed   Google Scholar

Part of the book series: Lecture Notes in Computer Science (LNCS, volume 5509)

Part of the book sub series: Lecture Notes in Artificial Intelligence (LNAI)

This is a preview of subscription content, log in via an institution to check access.

Access this book

Log in via an institution
Softcover Book USD 18.99 USD 54.99
Discount applied Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

About this book

Empirically based research in multimodality starts with the collection of video data relevant for a specific research question. This data must then be enriched by human coders, who annotate the corpus and who also explore the potential interconnections between modalities (e.g., face, gesture, speech, posture, etc.). The data must then be validated for consistency using quantitative measures -- systematic analysis of the observed multimodal behaviors is the central characteristic of multimodal corpora. For analysis and modeling tasks, exploratory qualitative and quantitative analysis tools are needed for browsing, viewing, extraction, and modeling. At the same time, the intercultural dimension must be taken into account, since multimodal interaction is always embedded in a social setting.

This state-of-the-art survey documents the scientific outcome of the International Workshop on "Multimodal Corpora: From Models of Natural Interaction to Systems and Applications", held in conjunction with the 6th International Conference for Language Resources and Evaluation (LREC), on May 27, 2008, in Marrakech, Morocco. It presents current research from fields as diverse as psychology, artificial intelligence, robotics, signal processing, computational linguistics, and human-computer interaction.

The 6 selected workshop contributions are complemented by 7 invited research articles, including contributions from major international multimodal corpus projects like AMI and SmartWeb. All papers underwent a special review process for this volume, resulting in significant revisions and extensions based on the experts' advice.

Similar content being viewed by others

Keywords

Table of contents (13 chapters)

  1. Front Matter

    Download chapter PDF

  2. Collecting and Distributing Multimodal Corpora

    1. The SmartWeb Corpora: Multimodal Access to the Web in Natural Environments

      • Florian Schiel
      Pages 1-17

    2. Promoting free Dialog Video Corpora: The IFADV Corpus Example

      • R. J. J. H. van Son, Wieneke Wesseling, Eric Sanders, Henk van den Heuvel
      Pages 18-37

  3. Coding and Analyzing Multimodal Corpora

    1. Creating and Exploiting Multimodal Annotated Corpora: The ToMA Project

      • Philippe Blache, Roxane Bertrand, Gaëlle Ferré
      Pages 38-53

    2. Multi-track Annotation of Child Language and Gestures

      • Jean-Marc Colletta, Ramona N. Kunene, Aurélie Venouil, Virginie Kaufmann, Jean-Pascal Simon
      Pages 54-72

    3. Gesture, Gaze and Persuasive Strategies in Political Discourse

      • Isabella Poggi, Laura Vincze
      Pages 73-92

    4. Unsupervised Clustering in Multimodal Multiparty Meeting Analysis

      • Yosuke Matsusaka, Yasuhiro Katagiri, Masato Ishizaki, Mika Enomoto
      Pages 93-108

  4. Validating Multimodal Corpora

    1. Multimodal Corpora Annotation: Validation Methods to Assess Coding Scheme Reliability

      • Federica Cavicchio, Massimo Poesio
      Pages 109-121

    2. On the Contextual Analysis of Agreement Scores

      • Dennis Reidsma, Dirk Heylen, Rieks op den Akker
      Pages 122-137

  5. The Intercultural Dimension of Multimodal Corpora

    1. Creating Standardized Video Recordings of Multimodal Interactions across Cultures

      • Matthias Rehm, Elisabeth André, Nikolaus Bee, Birgit Endrass, Michael Wissner, Yukiko Nakano et al.
      Pages 138-159

    2. Multimodal Intercultural Information and Communication Technology – A Framework for Designing and Evaluating Multimodal Intercultural Communicators

      • Jens Allwood, Elisabeth Ahlsén
      Pages 160-175

  6. Tools for Browsing, Coding and Exchanging Data in Multimodal Corpora

    1. Tools and Resources for Visualising Conversational-Speech Interaction

      • Nick Campbell
      Pages 176-188

    2. Accessing a Large Multimodal Corpus Using an Automatic Content Linking Device

      • Andrei Popescu-Belis, Jean Carletta, Jonathan Kilgour, Peter Poller
      Pages 189-206

    3. An Exchange Format for Multimodal Annotations

      • Thomas Schmidt, Susan Duncan, Oliver Ehmer, Jeffrey Hoyt, Michael Kipp, Dan Loehr et al.
      Pages 207-221

  7. Back Matter

    Download chapter PDF
Back to top

Editors and Affiliations

  • Deutsches Forschungszentrum für künstliche Intelligenz (DFKI), Saarbrücken, Germany

    Michael Kipp

  • Laboratoire d’Informatique pour la Mécanique et les Sciences de l’Ingénieur (LIMSI-CNRS), Orsay Cedex, France

    Jean-Claude Martin

  • Faculty of Humanities, Centre for Language Technology, University of Copenhagen, Copenhagen, Denmark

    Patrizia Paggio

  • Computer Science, Human Media Interaction, University of Twente, Enschede, The Netherlands

    Dirk Heylen

Bibliographic Information

Publish with us

Policies and ethics

Back to top