Overview
- The author is a distinguished scientist in information theory and statistical modeling
Part of the book series: Information Science and Statistics (ISS)
Access this book
Tax calculation will be finalised at checkout
Other ways to access
Table of contents (9 chapters)
-
Front Matter
-
Introduction
-
Information and Coding
-
Front Matter
-
-
Statistical Modeling
-
Front Matter
-
-
Back Matter
Keywords
About this book
No statistical model is "true" or "false," "right" or "wrong"; the models just have varying performance, which can be assessed. The main theme in this book is to teach modeling based on the principle that the objective is to extract the information from data that can be learned with suggested classes of probability models. The intuitive and fundamental concepts of complexity, learnable information, and noise are formalized, which provides a firm information theoretic foundation for statistical modeling. Inspired by Kolmogorov's structure function in the algorithmic theory of complexity, this is accomplished by finding the shortest code length, called the stochastic complexity, with which the data can be encoded when advantage is taken of the models in a suggested class, which amounts to the MDL (Minimum Description Length) principle. The complexity, in turn, breaks up into the shortest code length for the optimal model in a set of models that can be optimally distinguished from the given data and the rest, which defines "noise" as the incompressible part in the data without useful information.
Such a view of the modeling problem permits a unified treatment of any type of parameters, their number, and even their structure. Since only optimally distinguished models are worthy of testing, we get a logically sound and straightforward treatment of hypothesis testing, in which for the first time the confidence in the test result can be assessed. Although the prerequisites include only basic probability calculus and statistics, a moderate level of mathematical proficiency would be beneficial. The different and logically unassailable view of statistical modelling should provide excellent grounds for further research and suggest topics for graduate students in all fields of modern engineering, including and not restricted to signal and image processing, bioinformatics, pattern recognition, and machine learning to mention just a few.
Reviews
From the reviews:
"Readership: Graduate students and researchers in statistics, computer science and engineering, anyone interested in statistical modelling. This book presents a personal introduction to statistical modelling based on the principle that the objective of modelling is to extract learnable information from data with suggested classes of probability models. It grew from lectures to doctoral students … and retains much of the economical style of a lecture series. … Therefore, this fascinating volume offers an excellent source of important statistical research problems calling for solution." (Erkki P. Liski, International Statistical Review, Vol. 75 (2), 2007)
"This book covers the minimum description length (MDL) principle … . For statistics beginners, this book is self-contained. The writing style is concise … . Overall, this is an authoritative source on MDL and a good reference book. Most statisticians would be fortunate to have a copy in their bookshelves." (Thomas C. M. Lee, Journal of the American Statistical Association, Vol. 103 (483), September, 2008)
"This book describes the latest developments of the MDL principle. … The book … is intended to serve as a readable introduction to the mathematical aspects of the MDL principle when applied to statistical modeling for graduate students in statistics and information sciences. … Overall, this interesting book will make an important contribution to the field of statistical modeling through the MDL principle." (Prasanna Sahoo, Zentralblatt Math, Vol. 1156, 2009)
Authors and Affiliations
Bibliographic Information
Book Title: Information and Complexity in Statistical Modeling
Authors: Jorma Rissanen
Series Title: Information Science and Statistics
DOI: https://doi.org/10.1007/978-0-387-68812-1
Publisher: Springer New York, NY
eBook Packages: Computer Science, Computer Science (R0)
Copyright Information: Springer-Verlag New York 2007
Hardcover ISBN: 978-0-387-36610-4Published: 25 January 2007
Softcover ISBN: 978-1-4419-2267-0Published: 23 November 2010
eBook ISBN: 978-0-387-68812-1Published: 15 December 2007
Series ISSN: 1613-9011
Series E-ISSN: 2197-4128
Edition Number: 1
Number of Pages: VIII, 142
Topics: Probability Theory and Stochastic Processes, Coding and Information Theory, Statistics for Engineering, Physics, Computer Science, Chemistry and Earth Sciences