Comparative Evaluation of Multilingual Information Access Systems

1. Front Matter

   PDF

2. Introduction

   1. Introduction

      • Julio Gonzalo, Carol Peters

      Pages 1-6

   2. CLEF 2003 Methodology and Metrics

      • Martin Braschler, Carol Peters

      Pages 7-20

   3. Analysis of the Reliability of the Multilingual Topic Set for the Cross Language Evaluation Forum

      • Thomas Mandl, Christa Womser-Hacker

      Pages 21-28

   4. Evaluation of Information Access Technologies at the NTCIR Workshop

      • Noriko Kando

      Pages 29-43

3. Part I. Ad-hoc Text Retrieval Tracks

   1. CLEF 2003 – Overview of Results

      • Martin Braschler

      Pages 44-63

4. Mainly Cross-Language Experiments

   1. Report on CLEF-2003 Multilingual Tracks

      • Jacques Savoy

      Pages 64-73

   2. The Impact of Word Normalization Methods and Merging Strategies on Multilingual IR

      • Eija Airio, Heikki Keskustalo, Turid Hedlund, Ari Pirkola

      Pages 74-84

   3. JHU/APL Experiments in Tokenization and Non-word Translation

      • Paul McNamee, James Mayfield

      Pages 85-97

   4. Report on CLEF-2003 Experiments: Two Ways of Extracting Multilingual Resources from Corpora

      • Nicola Cancedda, Hervé Déjean, Éric Gaussier, Jean-Michel Renders, Alexei Vinokourov

      Pages 98-107

   5. Combining Query Translation and Document Translation in Cross-Language Retrieval

      • Aitao Chen, Fredric C. Gey

      Pages 108-121

   6. Cross-Language Experiments with the IR-n System

      • Fernando Llopis, Rafael Muñoz

      Pages 122-132

   7. Multilingual Information Retrieval Using Open, Transparent Resources in CLEF 2003

      • Monica Rogati, Yiming Yang

      Pages 133-139

   8. ITC-irst at CLEF 2003: Monolingual, Bilingual, and Multilingual Information Retrieval

      • Nicola Bertoldi, Marcello Federico

      Pages 140-151

   9. Language-Dependent and Language-Independent Approaches to Cross-Lingual Text Retrieval

      • Jaap Kamps, Christof Monz, Maarten de Rijke, Börkur Sigurbjörnsson

      Pages 152-165

   10. Multilingual Retrieval Experiments with MIMOR at the University of Hildesheim

       • René Hackl, Ralph Kölle, Thomas Mandl, Alexandra Ploedt, Jan-Hendrik Scheufen, Christa Womser-Hacker

       Pages 166-173

   11. Concept-Based Searching and Merging for Multilingual Information Retrieval: First Experiments at CLEF 2003

       • Romaric Besançon, Gaël de Chalendar, Olivier Ferret, Christian Fluhr, Olivier Mesnard, Hubert Naets

       Pages 174-184

   12. CLEF 2003 Experiments at UB: Automatically Generated Phrases and Relevance Feedback for Improving CLIR

       • Miguel E. Ruiz

       Pages 185-191

   13. SINAI at CLEF 2003: Decompounding and Merging

       • Fernando Martínez-Santiago, Arturo Montejo-Ráez, Luis Alfonso Ureña-López, M. Carlos Díaz-Galiano

       Pages 192-201

   14. Merging Results by Predicted Retrieval Effectiveness

       • Wen-Cheng Lin, Hsin-Hsi Chen

       Pages 202-209

The fourth campaign of the Cross-language Evaluation Forum (CLEF) for European languages was held from January to August 2003. Participation in this campaign showed a slight rise in the number of participants from the previous year, with 42 groups submitting results for one or more of the different tracks (compared with 37 in 2002), but a steep rise in the number of experiments attempted. A distinctive feature of CLEF 2003 was the number of new tracks and tasks that were offered as pilot experiments. The aim was to try out new ideas and to encourage the development of new evaluation methodologies, suited to the emerging requirements of both system developers and users with respect to today’s digital collections and to encourage work on many European languages rather than just those most widely used. CLEF is thus gradually pushing its participants towards the ultimate goal: the development of truly multilingual systems capable of processing collections in diverse media. The campaign culminated in a two-day workshop held in Trondheim, Norway, 21–22 August, immediately following the 7th European Conference on Digital Libraries (ECDL 2003), and attended by more than 70 researchers and system developers. The objective of the workshop was to bring together the groups that had participated in the CLEF 2003 campaign so that they could report on the results of their experiments.

• DOM
• algorithms
• bilingual information retrieval
• cross-language queries
• cross-lingual text retrieval
• image retrieval
• information system
• interactive text retrieval systems
• machine translation
• multilingual information retrieval
• natural language processing
• ontology
• proving
• spoken document retrieval

• ISTI-CNR, Area di Ricerca, Pisa, Italy

  Carol Peters

• No Affiliations,  

  Julio Gonzalo, Martin Braschler

• German Institute for International and Security Affairs, Stiftung Wissenschaft und Politik (SWP), Berlin, Germany

  Michael Kluck

Policies and ethics