Question: 3. Regularization (4 pts.): We have mentioned how when we are looking for parameters that optimize a loss/error function, sometimes we want to ensure these

 3. Regularization (4 pts.): We have mentioned how when we are

3. Regularization (4 pts.): We have mentioned how when we are looking for parameters that optimize a loss/error function, sometimes we want to ensure these parameters are not too big or too complex in order to avoid overfitting. This is called regularization. We will explore optimizing a regularized loss function here. Let {Z1,it ~ N(-1,.5), {Z2, ~ N(1, .5), and {} ~ N(0, .1). Generate N = 300 values of y, defined as: Yi = Biznit Bizzi + ci, where BY = 1 and ; = .1. In the following, you will only use {Yi, Z1,i, Z2{} and will assume no knowledge of Bi and By. Define your loss function as a regularized version of sum of squares: N 1 L(B) = N (Yi - BIZ1,i - B2Z2,i)2 + AlBil + AlBel. In this case we are not just trying to minimize the sum of squares but, in doing so, we are trying to keep the absolute value of the As small

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Mathematics Questions!