Information Theory b-log

Historically in the pre-word-processing era, this wasn’t done because inserting/deleting an equation would mean retyping or retouching a large portion of the manuscript.
The main reason for doing it is that the writer cannot foresee which equations a reader will want to reference. For example, as a reviewer, it annoys me to have to say “the third displayed equation after the beginning of Section VII”. It also avoids bad habits like “as we saw in the equation above” and “in the second to last inequality”.

Incidentally, Cover-Thomas abides by this rule and then some.When referring to equations in a chain, they also label some of the = or \leq with letters and then they refer to the letters. This is redundant because labels label equations, not their right sides.

Here are some properties of the binary entropy function (bits) courtesy of Sergio Verdú. Please contribute any others you have found useful.

${0 \leq h(p) \leq 1\phantom{\frac{p}{p}}}$
${h(0.5) = 1 \mbox{~bit} \phantom{\frac{p}{p}}}$
${ h(p) = h(1-p) \phantom{\frac{p}{p}}}$
${h(0.11003) = 0.5 \mbox{~bits} \phantom{\frac{p}{p}}}$
${h \left( \frac{1}{k} \right) = \log_2 k - \left(1 - \frac{1}{k} \right) \log_2 (k-1), ~~k> 1}$
${\frac{d h(p)}{dp}|_{p=0} = \infty \phantom{\frac{p}{p}}}$
${\frac{d h(p)}{dp} = \log_2 \frac{1-p}{p}, ~~ 0 < p < 1}$
${\frac{d}{dp} \frac{h(p)}{1-p} = \frac{1}{(1-p)^2 }\log_2 \frac{1}{p} }$
${ \frac{h(p)}{1-p} ~\mbox{is monotonically increasing on} ~(0,1)}$
${p \log \alpha - h(p) ~\mbox{is monotonically decreasing/increasing on} }$ ${~\left(0, \frac{1}{1 + \alpha}\right) ~\mbox{and}~\left(\frac{1}{1 + \alpha}, 1 \right) ~ \mbox{respectively, for any}~\alpha > 0}$
${h(p) ~\mbox{is concave on} ~(0,1) \phantom{\frac{p}{p}}}$
${h(p) = p \log_2 \frac{e}{p} - \frac{\log_2 e}{2} p^2 + o(p^2) \phantom{\frac{p}{p}}}$
${\lim_{\alpha \downarrow 0} \frac{1}{\alpha} h \left( \frac{1}{1+\alpha} \right) - \log_2 \left( 1 + \frac{1}{\alpha} \right) = \log_2 e}$
${h \left( \frac{1+x}{2} \right) = 1 - \sum_{k=1}^\infty \frac{x^{2k}}{(2k-1) 2k} \log_2 e , ~~| x | \leq 1}$
${\lim_{n \rightarrow \infty} \frac{1}{n} \log_2 \binom{n}{k_n} = h (p ) ~\Longleftarrow ~\lim_{n \rightarrow \infty} \frac{k_n}{n} = p}$
${(1 - p q ) h \left( \frac{p - p q}{1 - p q } \right) = h(p) + p h(q) - h(pq) ~~0\leq p \leq 1, 0\leq q \leq 1, pq < 1 }$
${h(p) \leq 2 \sqrt{p (1-p)}\phantom{\frac{p}{p}}}$
${q h (p) \leq h ( q p ) , ~(p,q) \in [0,1]^2 \phantom{\frac{p}{p}}}$
${\int_0^{1} h ( x ) \, d x = \frac{1}{2} \log_2 {e}}$
${\int_0^1 \frac{h \left(x^\alpha\right)}{x} \,dx = \frac{\pi^2}{6 \alpha}\log_2 {e} }$
${\int_0^{\frac{\pi}{2}} h ( \sin^2 \alpha ) \, d\alpha = \frac{\pi}{2} \log_2 \frac{4}{e}}$
${\int_0^{1} u^{2k} h \left( \frac{1-u}{2}\right) \, du = \frac{\log_2 {e}}{(2k+1)(2k+2)} \sum_{i=0}^k \frac{1}{2i+1}~~~k=0,1,2, \ldots}$
${\max_{0 \leq \alpha \leq 1} h(\alpha ) + (1- \alpha) \beta_0 + \alpha \beta_1 = \log_2 ( 2^{\beta_0} + 2^{\beta_1} )}$
${h ( h^{-1} ( x ) * p ) ~\mbox{is convex on}~ 0<x<1 ~\mbox{for any}~ 0 \leq p \leq 1, \mbox{~where}}$ ${ ~ a * p = a (1-p) + p (1-a) \mbox{~and}~ h^{-1} ~\mbox{is the inverse of}~ h( x ) ~\mbox{on}~ \left[0, 1/2 \right] }$