# Information Theory

Information theory is a branch of applied mathematics, electrical engineering, and computer science involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and communicating data. Since its inception it has broadened to find applications in many other areas, including statistical inference, natural language processing, cryptography, neurobiology, the evolution and function of molecular codes, model selection in ecology, thermal physics, quantum computing, plagiarism detection and other forms of data analysis.

A key measure of information is known as entropy, which is usually expressed by the average number of bits needed to store or communicate one symbol in a message. Entropy quantifies the uncertainty involved in predicting the value of a random variable. For example, specifying the outcome of a fair coin flip (two equally likely outcomes) provides less information (lower entropy) than specifying the outcome from a roll of a die (six equally likely outcomes).

Applications of fundamental topics of information theory include lossless data compression (e.g. ZIP files), lossy data compression (e.g. MP3s and JPGs), and channel coding (e.g. for Digital Subscriber Line (DSL)). The field is at the intersection of mathematics, statistics, computer science, physics, neurobiology, and electrical engineering. Its impact has been crucial to the success of the Voyager missions to deep space, the invention of the compact disc, the feasibility of mobile phones, the development of the Internet, the study of linguistics and of human perception, the understanding of black holes, and numerous other fields. Important sub-fields of information theory are source coding, channel coding, algorithmic complexity theory, algorithmic information theory, information-theoretic security, and measures of information.

Read more about Information Theory:  Overview, Historical Background, Quantities of Information, Coding Theory

### Other articles related to "information theory, theory, information":

List Of Jewish American Computer Scientists
... Bricklin, creator of the original spreadsheet Sergey Brin, co-founder of Google Peter Elias, information theory Robert Fano, information theory Edward Feigenbaum ... Seymour Ginsburg, formal language theory Adele Goldberg, Smalltalk design team Herman Adele Goldstine, developers of ENIAC Shafi Goldwasser ... error correction Ray Solomonoff, algorithmic information theory Richard Stallman, GNU, FSF Gerald Jay Sussman, Scheme Eric Tobis, optimized Viterbi error correction algorithms Peter J ...
List Of Weight-of-evidence Articles - Mathematics, Statistics, and Information Theory
... as an alternative to classical hypothesis testing Ban (information), a logarithmic unit of information Quasi-empirical method, scientific methods that are "almost" or "socially ...
Mathematical Beauty - Beauty and Mathematical Information Theory
... In the 1970s, Abraham Moles and Frieder Nake analyzed links between beauty, information processing, and information theory ... In the 1990s, Jürgen Schmidhuber formulated a mathematical theory of observer-dependent subjective beauty based on algorithmic information theory the most beautiful objects among ...
Information Theory - Applications To Other Fields - Miscellaneous Applications
... Information theory also has applications in gambling and investing, black holes, bioinformatics, and music ...

### Famous quotes containing the words theory and/or information:

There could be no fairer destiny for any physical theory than that it should point the way to a more comprehensive theory in which it lives on as a limiting case.
Albert Einstein (1879–1955)

I have all my life been on my guard against the information conveyed by the sense of hearing—it being one of my earliest observations, the universal inclination of humankind is to be led by the ears, and I am sometimes apt to imagine that they are given to men as they are to pitchers, purposely that they may be carried about by them.