Science Fair Project Encyclopedia

Science Fair Project Encyclopedia Contents Page

Asymptotic equipartition property

The asymptotic equipartition property (AEP), or Shannon-McMillan theorem, is a direct consequence of the weak law of large numbers and is used extensively in information theory. It can be summed up by the phrase 'Almost everything is almost equally probable.'

Given independent identically-distributed random variables X₁, ..., X_n with entropy H(X) and p(X₁, ..., X_k) the probability of observing the sequence X₁, ..., X_k, the AEP states that in probability,

${|-\frac{1}{n}} \log p(X_1, X_2, ..., X_n) - H(X)| < \epsilon \qquad \forall \epsilon>0.$

The AEP is used to define the typical set, which is used extensively in theories of compression.

Categories: Information theory

Last updated: 10-16-2005 14:38:20

Science Fair Project Encyclopedia Contents Page

10-26-2009 08:16:03
The contents of this article is licensed from www.wikipedia.org under the GNU Free Documentation License. Click here to see the transparent copy and copyright details

All Science Fair Projects

Science Fair Project Encyclopedia for Schools!

Science Fair Project Encyclopedia

Asymptotic equipartition property