Question: MSE Minimizer 3. (10 points) Recall from calculus that given some function g(x), the x you get from solving g(2) = 0 is called a

 MSE "Minimizer" 3. (10 points) Recall from calculus that given some

MSE "Minimizer" 3. (10 points) Recall from calculus that given some function g(x), the x you get from solving "g(2) = 0 is called a critical point of g - this means it could be a minimizer or a maximizer for g. In this question, we will explore some basic properties and build some intuition on why, for certain loss functions such as squared L2 loss, the critical point of the empirical risk function (defined as an average loss on the observed data) will always be the minimizer. Given some linear model f(x) = Ox for some real scalar 0, we can write the empirical risk of the model f given the observed data {r, yi}, for ie {1, ..., n} as the average L2 loss, also known as Mean Squared Error (MSE): (yi - ex.)? = _ (yi - Ox;)2 n i=1 i=1 n (a) (3 points) Let's investigate one of the n functions in the summation in the MSE. Define g:(0) = =(y; - Or;)? for i e {1,..., n}. In this case, note that the MSE can be written as _1_1 9:(0). Recall from calculus that we can use the 2nd derivative of a function to describe its curvature about a certain point (if it is facing concave up, down, or possibly a point of inflection). You can take the following as a fact: A function is convex if and only if the function's 2nd derivative is non-negative on its domain. Based on this property, verify that g:(0) is a convex function

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Mathematics Questions!