For X ∈ RN×(p+1) and y ∈ RN , show each of the following:

(a) If the variance σ2 > 0 is known, the β ∈ Rp+1 that maximizes l := 

coincides with the least squares solution. Hint:

(b) If both β ∈ Rp+1 and σ2 > 0 are unknown, the maximum likelihood estimate of σ2 is given by

Hint: If we partially differentiate l with respect to σ2, we have

(c) For probabilistic density functions f and g over R, the Kullback–Leibler divergence is nonnegative, i.e.,

Transcribed Image Text:

log f(vi\xi, ) log f(vi\xi, )