Problem $3$

As in problem $3$ in HW set $3,$ assume a neuron is tuned by a stimulus $s$ which

has a Gaussian distribution with mean $$ and variance ${}^2$ and it produces

the response $r=as+n,$ where noise $n$ is also assumed to be Gaussian with

zero mean and variance ${}^2,$ and $a$ is some constant. Recall that the entropy

of $r$ and conditional entropy of $r$ give $s$ are given as

What is the mutual information $r$ and $s?$

Describe how $a,s$ and the variance of noise influence the mutual information.

Calculate the average noise entropy, that is$,$

$E[H(r|s)]={\displaystyle \underset{\phantom{}}{\overset{\phantom{}}{}}}[-{\displaystyle \underset{\phantom{}}{\overset{\phantom{}}{}}}p(r|s)log(p(r|s)dr)]p(s)ds$