An Introduction to Kolmogorov Complexity and Its Applications

1. Front Matter

   Pages i-xx

   PDF

2. Preliminaries

   • Ming Li, Paul Vitányi

   Pages 1-92

3. Algorithmic Complexity

   • Ming Li, Paul Vitányi

   Pages 93-188

4. Algorithmic Prefix Complexity

   • Ming Li, Paul Vitányi

   Pages 189-238

5. Algorithmic Probability

   • Ming Li, Paul Vitányi

   Pages 239-314

6. Inductive Reasoning

   • Ming Li, Paul Vitányi

   Pages 315-377

7. The Incompressibility Method

   • Ming Li, Paul Vitányi

   Pages 379-457

8. Resource-Bounded Complexity

   • Ming Li, Paul Vitányi

   Pages 459-520

9. Physics, Information, and Computation

   • Ming Li, Paul Vitányi

   Pages 521-589

10. Back Matter

    Pages 591-637

    PDF

Briefly, we review the basic elements of computability theory and prob­ ability theory that are required. Finally, in order to place the subject in the appropriate historical and conceptual context we trace the main roots of Kolmogorov complexity. This way the stage is set for Chapters 2 and 3, where we introduce the notion of optimal effective descriptions of objects. The length of such a description (or the number of bits of information in it) is its Kolmogorov complexity. We treat all aspects of the elementary mathematical theory of Kolmogorov complexity. This body of knowledge may be called algo­ rithmic complexity theory. The theory of Martin-Lof tests for random­ ness of finite objects and infinite sequences is inextricably intertwined with the theory of Kolmogorov complexity and is completely treated. We also investigate the statistical properties of finite strings with high Kolmogorov complexity. Both of these topics are eminently useful in the applications part of the book. We also investigate the recursion­ theoretic properties of Kolmogorov complexity (relations with Godel's incompleteness result), and the Kolmogorov complexity version of infor­ mation theory, which we may call "algorithmic information theory" or "absolute information theory. " The treatment of algorithmic probability theory in Chapter 4 presup­ poses Sections 1. 6, 1. 11. 2, and Chapter 3 (at least Sections 3. 1 through 3. 4).

• algorithms
• complexity
• statistics
• algorithm analysis and problem complexity

• Department of Computer Science, University of Waterloo, Waterloo, Canada

  Ming Li

• Centrum voor Wiskunde en Informatica, SJ Amsterdam, The Netherlands

  Paul Vitányi

Policies and ethics