Information Theory 2

Question 1

If noise equals zero, what is the mutual information between X to-> Y equal to?

I(X,Y) =

Answer 1

if noise=0, then X=Y
thus
I(X,Y)=H(X)=H(Y)
mutual information =entropy of X = entropy of Y

Question 2

If there is no noise between the information transfer between X to Y, what is the entropy at X and Y?

Answer 2

entropy of X is the same as the entropy at Y
H(X)=H(Y)

Question 3

What is the equation for mutual information between points X to-> Y?

Answer 3

mutual info of X and Y = entropy of Y - noise entropy

I(X,Y) = H(Y) – H(Y|X)

Question 4

Is the message identical at both points (X and Y) when there is noise?

Answer 4

no!
no noise = identical

Question 5

Why is the message at points X and Y not identical when there is noise?

Answer 5

because information is lost when sending message from X to Y due to noise

Question 6

How do you calculate the noise entropy of X->Y?

Answer 6

noise entropy = H(Y|X)
entropy of Y given X

Question 7

With a message sent from X to-> Y, why is the noise H(Y|X)?

Answer 7

H(Y|X) means the amount of information received when X is held constant ( when no information sent in channel). thus this means the amount of information in the NOISE transmitted in the channel