Question $1$

$1$ pts

Calculate the centroid $($in$)$ of the section below from the top

Problem $2.[75$ points$]$ Consider a linear model $y=$vec$(w)*vec(x).$ The input vec$(x)in{R}^d$ is $d-$dimensional, the weight vec$(w)in{R}^d$ is of same dimension, and the output yinR is a scalar. We will optimize the weights vec$(w)$ in a linear model such that it can classify two types of datasets: the first data type has low frequency and the second data type has high frequency. Specifically, they are sinusoidal waves with frequencies $1$ Hz $($low$)$ and $5$ Hz $($high$).$ Random numbers are added to make them noisy. Given the low frequency data as the input $(vec(x)),$ the output of the linear model $(y)$ should be $1.$ On the other hand, given the high frequency data as the input $(vec(x)),$ the output $(y)$ should be $-1.$

Here is the Python code that generates two types of datasets.Answer the following questions.

$(10$ pts$)$ Plot several example data from each data type.

$(25$ pts $)$ Set up a loss function, measuring the difference between predicted and true data types. Set up a gradient descent algorithm that reduces the loss function. Use the reverse mode of automatic differentiation when computing the gradient of the loss with respect to vec$(w).$

$(15$ pts$)$ Show that the loss decreases as the training progresses.

$(25$ pts $)$ Once the loss becomes small, use the optimized vec$(w)$ to predict the data type. Compare the predicted and the true data types and show that the prediction is accurate.