Log energy and Shannon entropy?

1. Why is there a log in Shannon entropy?
2. Why log base 2 in entropy?
3. What is Shannon entropy formula?
4. Is Shannon entropy always positive?

Why is there a log in Shannon entropy?

The purpose of log(pi) appearing in Shannon's Entropy is that log(pi) is the only function satisfying the basic set of properties that the entropy function, H(p1,…,pN), is held to embody.

Why log base 2 in entropy?

Two bits of entropy: In the case of two fair coin tosses, the information entropy in bits is the base-2 logarithm of the number of possible outcomes; with two coins there are four possible outcomes, and two bits of entropy.

What is Shannon entropy formula?

The Shannon entropy is defined as HSh = −Σipilog pi, while the von Neumann entropy is defined as HvN = −Tr ρ log ρ.

Is Shannon entropy always positive?

The Shannon entropy of the exponential distribution can be negative. 1+log(μ), and it can take negative values for many values of the mean μ!.