Problem #$5$:

$(1$ point$)$ Stochastic Gradient Descent $($SGD$)$ is an iterative method for optimizing

an objective function, particularly useful for large$-$scale machine learning problems.

Consider a linear regression problem where we aim to minimize the mean squared

error $($MSE$)$ loss function:

L$($w$)=1$

$2$n

n

$$

i$=1$

$($w

$$xi $$ yi$)$

$2$

$,$

where w is the weight vector, xi

is the feature vector for the i$-$th training example, and

yi

is the corresponding target value.

Answer the following questions:

$$ Explain the update rule for the weight vector w using stochastic gradient descent.

$$ Describe the difference between stochastic gradient descent and traditional gradient

descent.Problem #$5$:

$(1$ point$)$ Stochastic Gradient Descent $($SGD$)$ is an iterative method for optimizing

an objective function, particularly useful for large$-$scale machine learning problems.

Consider a linear regression problem where we aim to minimize the mean squared

error $($MSE$)$ loss function:

$L(w)=\frac{1}{2n}{\displaystyle \underset{i=1}{\overset{n}{}}}(w^{TT}{x}_i-y_i{)}^2,$

where $w$ is the weight vector, $x_i$ is the feature vector for the $i-$th training example, and

$y_i$ is the corresponding target value.

Answer the following questions:

Explain the update rule for the weight vector $w$ using stochastic gradient descent.

Describe the difference between stochastic gradient descent and traditional gradient

descent.