How to prove relationship between mutual information and differential entropy?

1. What is the relation between entropy and mutual information?
2. What is the difference between correlation and mutual information?
3. What is role of entropy and mutual information in digital communication?
4. How do you interpret mutual information score?

What is the relation between entropy and mutual information?

The concept of mutual information is intimately linked to that of entropy of a random variable, a fundamental notion in information theory that quantifies the expected "amount of information" held in a random variable.

What is the difference between correlation and mutual information?

Correlation analysis provides a quantitative means of measuring the strength of a linear relationship between two vectors of data. Mutual information is essentially the measure of how much “knowledge” one can gain of a certain variable by knowing the value of another variable.

What is role of entropy and mutual information in digital communication?

Mutual information of a channel is symmetric. Mutual information is non-negative. Mutual information can be expressed in terms of entropy of the channel output. Mutual information of a channel is related to the joint entropy of the channel input and the channel output.

How do you interpret mutual information score?

High mutual information indicates a large reduction in uncertainty; low mutual information indicates a small reduction; and zero mutual information between two random variables means the variables are independent.