Science Fair Project Encyclopedia

Science Fair Project Encyclopedia Contents Page

Rényi entropy

In information theory, Rényi entropy, an extension of Shannon entropy, is a means of quantifying the entropy of a system. Entropy characterizes the disorder of the system, how much information can be compressed and consequently how much information is present in a signal or system (see Information entropy). Rényi entropy is defined as

$H_\alpha(p_1,p_2,\ldots,p_n) = \frac{1}{1-\alpha}\ln\Bigg(\sum_{i=1}^n p_i^\alpha\Bigg)$

where $p i$ are probabilities and $\alpha > 0, \alpha \ne 1$ . As $α$ approaches 1, $H α$ converges to Shannon entropy. For some $α$ and $α'$ where $\alpha \le \alpha'$ , Rényi entropy guarantees that

$H_\alpha \le H_{\alpha'}$ .

Categories: Information theory

Last updated: 05-27-2005 12:59:41

Science Fair Project Encyclopedia Contents Page

10-26-2009 08:16:03
The contents of this article is licensed from www.wikipedia.org under the GNU Free Documentation License. Click here to see the transparent copy and copyright details

All Science Fair Projects

Science Fair Project Encyclopedia for Schools!

Science Fair Project Encyclopedia

Rényi entropy