Question: 25. Show that the sigmoidal transfer function acts to transmit the maximum infor- mation if its inputs are distributed normally. Recall that the entropy (a


25. Show that the sigmoidal transfer function acts to transmit the maximum infor- mation if its inputs are distributed normally. Recall that the entropy (a measure of information) is defined as H = [ p(y)In[p(y)]dy. (a) Consider a continuous input variable r drawn from the density p(x) ~ N(0, o2). What is entropy for this distribution? (b) Suppose samples x are passed through an antisymmetric sigmoidal function to give y = f(x), where the zero crossing of the sigmoid occurs at the peak of the Gaussian input, and the effective width of the linear region equal to the range -0
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
