Text Mining

1. Front Matter

   Pages i-xii

   PDF

2. Overview of Text Mining

   • Sholom M. Weiss, Nitin Indurkhya, Tong Zhang, Fred J. Damerau

   Pages 1-13

3. From Textual Information to Numerical Vectors

   • Sholom M. Weiss, Nitin Indurkhya, Tong Zhang, Fred J. Damerau

   Pages 15-46

4. Using Text for Prediction

   • Sholom M. Weiss, Nitin Indurkhya, Tong Zhang, Fred J. Damerau

   Pages 47-84

5. Information Retrieval and Text Mining

   • Sholom M. Weiss, Nitin Indurkhya, Tong Zhang, Fred J. Damerau

   Pages 85-102

6. Finding Structure in a Document Collection

   • Sholom M. Weiss, Nitin Indurkhya, Tong Zhang, Fred J. Damerau

   Pages 103-128

7. Looking for Information in Documents

   • Sholom M. Weiss, Nitin Indurkhya, Tong Zhang, Fred J. Damerau

   Pages 129-156

8. Case Studies

   • Sholom M. Weiss, Nitin Indurkhya, Tong Zhang, Fred J. Damerau

   Pages 157-195

9. Emerging Directions

   • Sholom M. Weiss, Nitin Indurkhya, Tong Zhang, Fred J. Damerau

   Pages 197-211

10. Back Matter

    Pages 213-237

    PDF

Data mining is a mature technology. The prediction problem, looking for predictive patterns in data, has been widely studied. Strong me- ods are available to the practitioner. These methods process structured numerical information, where uniform measurements are taken over a sample of data. Text is often described as unstructured information. So, it would seem, text and numerical data are different, requiring different methods. Or are they? In our view, a prediction problem can be solved by the same methods, whether the data are structured - merical measurements or unstructured text. Text and documents can be transformed into measured values, such as the presence or absence of words, and the same methods that have proven successful for pred- tive data mining can be applied to text. Yet, there are key differences. Evaluation techniques must be adapted to the chronological order of publication and to alternative measures of error. Because the data are documents, more specialized analytical methods may be preferred for text. Moreover, the methods must be modi?ed to accommodate very high dimensions: tens of thousands of words and documents. Still, the central themes are similar.

• Active learning
• Clustering and matching
• Document classification and correction
• Extraction
• Retrieval
• Summarization
• classification
• clustering
• data mining
• information retrieval
• text mining

• TJ Watson Labs, IBM Research, Yorktown Heights, USA

  Sholom M. Weiss, Tong Zhang, Fred J. Damerau

• School of Computer Science and Engineering, University of New South Wales, Sydney, Australia

  Nitin Indurkhya

Policies and ethics