a

(a) Suppose we consider the function estimation problem (rather than the classification problem), but insist on using the probability of error as our success criterion (rather than squared error). That is, we “make an error”

whenever our estimate f (x) is not equal to y. What is the smallest probability of error we could hope for in the general case with densities?

(b) Now suppose we consider squared error, but the distributions are such that the y value is always equal to either 0 or 1. If P (y = 1|x) = p and P (y = 0|x) = 1 − p, what estimate f (x) minimizes the squared error?