2. In this problem, we will investigate a simple case of bias due to underfitting. Suppose...

 

Transcribed Image Text:

2. In this problem, we will investigate a simple case of bias due to underfitting. Suppose that training data {(x, yi)} is modeled using a simple linear model of the form y = f (x; B) = Bo+B12. But suppose that the true relationship between x and y is noiseless and quadratic, i.e., y = f(x), f(x) = Boo + Bo1x + Bo2x, for some true parameters Bo (Boo, B01, B02). = (a) Describe how to compute the least-squares fit of the model parameters 3 = (Bo, 3) from the training data (xi, yi), i = 1,..., n. The procedure will involve multiple steps, and you do not need to simplify the equations. Just clearly state how one would compute Bis from the training data. (b) Using the fact that y = f(xi) for the training data {(xi, yi)}, write an expression for is in terms of the training features {x;} and the true parameter values o. Again, you do not need to simplify the equations. Just clearly state how is relates to 3o and {xi}^\_1. (c) Suppose that the true parameters are 3o = (1, 2, -1) and the model is trained using 10 values that are linearly spaced in the closed interval [0, 1]. Write a short Python pro- gram to compute the least-squares parameters 3. Plot the estimated function f(x; 3ls) and the true function f(x) for x = [0,3]. (d) At which of value x [0,3] does bias(x) = ( y) = ((x; ) (x)) grow largest? 2. In this problem, we will investigate a simple case of bias due to underfitting. Suppose that training data {(x, yi)} is modeled using a simple linear model of the form y = f (x; B) = Bo+B12. But suppose that the true relationship between x and y is noiseless and quadratic, i.e., y = f(x), f(x) = Boo + Bo1x + Bo2x, for some true parameters Bo (Boo, B01, B02). = (a) Describe how to compute the least-squares fit of the model parameters 3 = (Bo, 3) from the training data (xi, yi), i = 1,..., n. The procedure will involve multiple steps, and you do not need to simplify the equations. Just clearly state how one would compute Bis from the training data. (b) Using the fact that y = f(xi) for the training data {(xi, yi)}, write an expression for is in terms of the training features {x;} and the true parameter values o. Again, you do not need to simplify the equations. Just clearly state how is relates to 3o and {xi}^\_1. (c) Suppose that the true parameters are 3o = (1, 2, -1) and the model is trained using 10 values that are linearly spaced in the closed interval [0, 1]. Write a short Python pro- gram to compute the least-squares parameters 3. Plot the estimated function f(x; 3ls) and the true function f(x) for x = [0,3]. (d) At which of value x [0,3] does bias(x) = ( y) = ((x; ) (x)) grow largest?