The Information Retrieval Series

Laboratory Experiments in Information Retrieval

Sample Sizes, Effect Sizes, and Statistical Power

Authors: Sakai, Tetsuya

Free Preview
  • Discusses the principles and limitations of statistical significance tests
  • Provides hands-on examples of t-tests, ANOVA, and multiple comparison procedures with Excel and R
  • Introduces tools for designing effective experiments by leveraging topic set size design and for power analysis
おすすめポイントをすべて見る

書籍の購入

イーブック ¥5,720
価格の適用国: Japan (日本円価格は個人のお客様のみ有効) (小計)
  • ISBN 978-981-13-1199-4
  • ウォーターマーク付、 DRMフリー
  • ファイル形式: EPUB, PDF
  • ebooks can be used on all reading devices
  • Immediate eBook download after purchase
ハードカバー ¥7,150
価格の適用国: Japan (日本円価格は個人のお客様のみ有効) (小計)
  • ISBN 978-981-13-1198-7
  • Free shipping for individuals worldwide
  • Institutional customers should get in touch with their account manager
  • Covid-19 shipping restrictions
  • Usually ready to be dispatched within 3 to 5 business days, if in stock
ソフトカバー ¥7,150
価格の適用国: Japan (日本円価格は個人のお客様のみ有効) (小計)
  • ISBN 978-981-13-4581-4
  • Free shipping for individuals worldwide
  • Institutional customers should get in touch with their account manager
  • Covid-19 shipping restrictions
  • Usually ready to be dispatched within 3 to 5 business days, if in stock
この教本について

Covering aspects from principles and limitations of statistical significance tests to topic set size design and power analysis, this book guides readers to statistically well-designed experiments. Although classical statistical significance tests are to some extent useful in information retrieval (IR) evaluation, they can harm research unless they are used appropriately with the right sample sizes and statistical power and unless the test results are reported properly. The first half of the book is mainly targeted at undergraduate students, and the second half is suitable for graduate students and researchers who regularly conduct laboratory experiments in IR, natural language processing, recommendations, and related fields.

Chapters 1–5 review parametric significance tests for comparing system means, namely, t-tests and ANOVAs, and show how easily they can be conducted using Microsoft Excel or R. These chapters also discuss a few multiple comparison procedures for researchers who are interested in comparing every system pair, including a randomised version of Tukey's Honestly Significant Difference test. The chapters then deal with known limitations of classical significance testing and provide practical guidelines for reporting research results regarding comparison of means.

Chapters 6 and 7 discuss statistical power. Chapter 6 introduces topic set size design to enable test collection builders to determine an appropriate number of topics to create. Readers can easily use the author’s Excel tools for topic set size design based on the paired and two-sample t-tests, one-way ANOVA, and confidence intervals. Chapter 7 describes power-analysis-based methods for determining an appropriate sample size for a new experiment based on a similar experiment done in the past, detailing how to utilize the author’s R tools for power analysis and how to interpret the results. Case studies from IR for both Excel-based topic set size design and R-based power analysis are also provided.

著者について

Tetsuya Sakai is a professor and the head of the Department of Computer Science and Engineering, Waseda University, Japan. He is also a visiting professor at the National Institute of Informatics. He joined Toshiba in 1993 and obtained a Ph.D. from Waseda in 2000. From 2000 to 2001, he was supervised by the late Karen Sparck Jones at the Computer Laboratory, University of Cambridge, as a visiting researcher. In 2007, he joined NewsWatch, Inc. as the director of the Natural Language Processing Lab. In 2009, he joined Microsoft Research Asia. He joined the Waseda faculty in 2013. He is an editor-in-chief of the Information Retrieval Journal (Springer) and an associate editor of ACM TOIS. He received a Waseda University Teaching Award in 2014 and a Waseda University Presidential Teaching Award in 2016.

Table of contents (8 chapters)

Table of contents (8 chapters)

書籍の購入

イーブック ¥5,720
価格の適用国: Japan (日本円価格は個人のお客様のみ有効) (小計)
  • ISBN 978-981-13-1199-4
  • ウォーターマーク付、 DRMフリー
  • ファイル形式: EPUB, PDF
  • ebooks can be used on all reading devices
  • Immediate eBook download after purchase
ハードカバー ¥7,150
価格の適用国: Japan (日本円価格は個人のお客様のみ有効) (小計)
  • ISBN 978-981-13-1198-7
  • Free shipping for individuals worldwide
  • Institutional customers should get in touch with their account manager
  • Covid-19 shipping restrictions
  • Usually ready to be dispatched within 3 to 5 business days, if in stock
ソフトカバー ¥7,150
価格の適用国: Japan (日本円価格は個人のお客様のみ有効) (小計)
  • ISBN 978-981-13-4581-4
  • Free shipping for individuals worldwide
  • Institutional customers should get in touch with their account manager
  • Covid-19 shipping restrictions
  • Usually ready to be dispatched within 3 to 5 business days, if in stock
Loading...

書誌情報

Bibliographic Information
Book Title
Laboratory Experiments in Information Retrieval
Book Subtitle
Sample Sizes, Effect Sizes, and Statistical Power
Authors
Series Title
The Information Retrieval Series
Series Volume
40
Copyright
2018
Publisher
Springer Singapore
Copyright Holder
Springer Nature Singapore Pte Ltd.
イーブック ISBN
978-981-13-1199-4
DOI
10.1007/978-981-13-1199-4
ハードカバー ISBN
978-981-13-1198-7
ソフトカバー ISBN
978-981-13-4581-4
Series ISSN
1871-7500
Edition Number
1
Number of Pages
IX, 150
Number of Illustrations
10 b/w illustrations, 43 illustrations in colour
Topics