Below, we have already initialized the hyper$-$parameters needed for this experiment. We have also provided several functions which you should fill out. At the bare minimum, you must implement the LMSE loss function $($with no regularization, with l$1$ regularization, with l$2$ regularization$)$ and the train function. The loss function must return a real$-$valued loss given data, labels, and parameters, and the training function should use batch SGD to train the model for a given set of epochs, and return the loss accumulated over each epoch.

Please refer to the following formula when implementing the loss function with no regularization, l$1$ regularization, and l$2$ regularization respectively.

LMSE with no regularizationLMSE with l$1$ regularizationLMSE with l$2$ regularization$=12$i$=1$m$($yi$$w$$xi$)2=12$i$=1$m$($yi$$w$$xi$)2+\backslash$lambda $$j$=1$n$|$wj$|=12$i$=1$m$($yi$$w$$xi$)2+\backslash$lambda $$j$=1$nw$2$j

You are welcome to add additional functions as needed, but please make sure to thoroughly comment everything you add. Also do not import any additional functions from other libraries.

$[]$

DEFAULT$\_$R$=1$e$-3$ # fixed

DEFAULT$\_$EPOCHS$=10$

DEFAULT$\_$LAMBDA$=0\mathrm{.}1$

DEFAULT$\_$BATCH$\_$SIZE$=32$

NO$\_$REGULARIZATION$=\text{'}$no$\_$reg'

L$1\_$LOSS $=\text{'}$l$1\text{'}$

L$2\_$LOSS $=\text{'}$l$2\text{'}$

# Loss function $-$ MODIFY THIS

def loss$($X$,$ y$,$ w$,$ regularization$\_$type$=$NO$\_$REGULARIZATION, lam$=$DEFAULT$\_$LAMBDA$)$: