Logo - springer
Slogan - springer

Computer Science - Security and Cryptology | Language Modeling for Information Retrieval

Language Modeling for Information Retrieval

Croft, Bruce, Lafferty, John (Eds.)

2003, XIV, 246 p.

Available Formats:
eBook
Information

Springer eBooks may be purchased by end-customers only and are sold without copy protection (DRM free). Instead, all eBooks include personalized watermarks. This means you can read the Springer eBooks across numerous devices such as Laptops, eReaders, and tablets.

You can pay for Springer eBooks with Visa, Mastercard, American Express or Paypal.

After the purchase you can directly download the eBook file or read it online in our Springer eBook Reader. Furthermore your eBook will be stored in your MySpringer account. So you can always re-download your eBooks.

 
$149.00

(net) price for USA

ISBN 978-94-017-0171-6

digitally watermarked, no DRM

Included Format: PDF

download immediately after purchase


learn more about Springer eBooks

add to marked items

Hardcover
Information

Hardcover version

You can pay for Springer Books with Visa, Mastercard, American Express or Paypal.

Standard shipping is free of charge for individual customers.

 
$189.00

(net) price for USA

ISBN 978-1-4020-1216-7

free shipping for individuals worldwide

usually dispatched within 3 to 5 business days


add to marked items

Softcover
Information

Softcover (also known as softback) version.

You can pay for Springer Books with Visa, Mastercard, American Express or Paypal.

Standard shipping is free of charge for individual customers.

 
$189.00

(net) price for USA

ISBN 978-90-481-6263-5

free shipping for individuals worldwide

usually dispatched within 3 to 5 business days


add to marked items

  • About this book

A statisticallanguage model, or more simply a language model, is a prob­ abilistic mechanism for generating text. Such adefinition is general enough to include an endless variety of schemes. However, a distinction should be made between generative models, which can in principle be used to synthesize artificial text, and discriminative techniques to classify text into predefined cat­ egories. The first statisticallanguage modeler was Claude Shannon. In exploring the application of his newly founded theory of information to human language, Shannon considered language as a statistical source, and measured how weH simple n-gram models predicted or, equivalently, compressed natural text. To do this, he estimated the entropy of English through experiments with human subjects, and also estimated the cross-entropy of the n-gram models on natural 1 text. The ability of language models to be quantitatively evaluated in tbis way is one of their important virtues. Of course, estimating the true entropy of language is an elusive goal, aiming at many moving targets, since language is so varied and evolves so quickly. Yet fifty years after Shannon's study, language models remain, by all measures, far from the Shannon entropy liInit in terms of their predictive power. However, tbis has not kept them from being useful for a variety of text processing tasks, and moreover can be viewed as encouragement that there is still great room for improvement in statisticallanguage modeling.

Content Level » Research

Keywords » DOM - Performance - Text - cognition - database - filtering - machine translation - speech recognition

Related subjects » Artificial Intelligence - Computer Science - Database Management & Information Retrieval - Security and Cryptology

Table of contents 

Preface. Contributing Authors. 1: Probabilistic Relevance Models Based on Document and Query Generation; J. Lafferty, ChengXiang Zhai. 1. Introduction. 2. Generative Relevance Models. 3. Discussion. 4. Historical Notes. 2: Relevance Models in Information Retrieval; V. Lavrenko, W.B. Croft. 1. Introduction. 2. Relevance Models. 3. Estimating a Relevance Model. 4. Experimental Results. 5. Conclusions. 3: Language Modeling and Relevance; K. Sparck Jones, S. Robertson, D. Hiemstra, H. Zaragoza. 1. Introduction. 2. Relevance in LM. 3. A Possible LM Approach: Parsimonious Models. 4. Concluding Comment. 4: Contributions of Language Modeling to the Theory and Practice of IR; W.R. Greiff, W.T. Morgan. 1. Introduction. 2. What is Language Modeling in IR. 3. Simulation Studies of Variance Reduction. 4. Continued Exploration. 5: Language Models for Topic Tracking; W. Kraai, M. Spitters. 1. Introduction. 2. Language Models for IR Tasks. 3. Experiments. 4. Discussion. 5. Conclusions. 6: A Probabilistic Approach to Term Translation for Cross-Lingual Retrieval; Jinxi Xu, R. Weischedel. 1. Introduction. 2. A Probabilistic Model for CLIR. 3. Estimating Term Translation Probabilities. 4. Related Work. 5. Test Collections. 6. Comparing CLIR with Monolingual Baseline. 7. Comparing Probabilistic and Structural Translations. 8. Comparing Probabilistic Translation and MT. 9. Measuring CLIR Performance as a Function of Resource Sizes. 10. Reducing the Translation Cost of Creating a Parallel Corpus. 11. Conclusions. 7: Using Compression-Based Language Models for Text Categorization; W.J. Teahan, D.J. Harper. 1. Background. 2. Compression Models. 3. Bayes Classifiers. 4. PPM-Based Language Models. 5. Experimental Results. 6. Discussion. 8: Applications of Score Distributions in Information Retrieval; R. Manmatha. 1. Introduction. 2. Related Work. 3. Modeling Score Distributions of Search Engines. 4. Combining Search Engines Indexing the Same Database. 5. Applications to Filtering and Topic Detection and Tracking. 6. Combining Engines Indexing Different Databases or Languages. 7. Conclusion. 9: An Unbiased Generative Model for Setting Dissemination Thresholds; Yi Zhang, J. Callan. 1. Introduction. 2. Generative Models of Dissemination Thresholds. 3. The Non-Random Sampling Problem & Solution. 4. Experimental Methodology. 5. Experimental Results. 6. Conclusion. 10: Language Modeling Experiments in Non-Extractive Summarization; V.O. Mittal, M.J. Witbrock. 1. Introduction. 2. Related Work. 3. Statistical Models of Gisting. 4. Training the Models. 5. Output and Evaluation. 6.

Popular Content within this publication 

 

Articles

Read this Book on Springerlink

Services for this book

New Book Alert

Get alerted on new Springer publications in the subject area of Data Structures, Cryptology and Information Theory.