Question: In this problem, we would like to use the gradient descent to calculate the parameters $W$ for the parabola. The loss function $ mathcal

In this problem, we would like to use the gradient descent to calculate the parameters $W$ for the parabola.
The loss function $\mathcal{L}(W)$ now contains two parts: A squared $L_2$ norm and a $L_1$ norm.
A coefficient $\alpha$ is used to control the ratio of these two norms:
$$
\begin{aligned}
\mathcal{L}(W)
& =\sum_{i=1}^{n}
\Big(\alpha\big(\mathbf{x}_i^T W - y_i\big)^2+(1-\alpha)|\mathbf{x}_i^T W - y_i|\Big)\\
& =\alpha\left\lVert X W - Y \right\rVert_2^2+(1-\alpha)\left\lVert X W - Y \right\rVert_1\\
onumber
\end{aligned}
$$
Complete the following code to use the **gradient descent** to find $W^*$ when $\alpha=0,\alpha=0.03,\alpha=0.05,\alpha=0.1$, and $\alpha=1$, espectively.
Write your code in the `...` part.
**Hint**: You may refer to Q3.1 for the gradient of $L_2$ norm.
**Note:** It may take 2~3 mins to run the algorithm.

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Programming Questions!