Question: Please make sure your math is correct: Consider the following training dataset: We would like to fit a linear regression model y^=w0+w1x to this dataset

Please make sure your math is correct:

Please make sure your math is correct: Consider the following training dataset:

Consider the following training dataset: We would like to fit a linear regression model y^=w0+w1x to this dataset using gradient lescent and 2 regularization, with =0.1 and =0.1. We use mean squared error for our oss function and initialize w=(w0,w1)=0 Part(a): Perform the first two gradient updates to w (i.e. what are the weights after one gradient update? after two?). Part(b) : Perform the first two gradient updates to w with momentum using =0.9 (assume z(0)=0 for the first gradient update). Part(c) : What would happen if we set to be much smaller, e.g. =10100 ? What about much larger, e.g. =100 ? (You do not need to perform the computations manually; a written description is sufficient.)

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock

Question You are answering ... View full answer

blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!