Q$4.(10$ Marks$)$

Single layer perceptron with pure linear activation function can be used to fit a linear

model to a set of input$-$output pairs. Suppose that we are given the following pairs:

$\{(0\mathrm{.}5,8\mathrm{.}0),(1\mathrm{.}5,6\mathrm{.}0),(3,5),(4\mathrm{.}0,2),(5\mathrm{.}0,0\mathrm{.}5)\}$

and a single linear neuron as shown in the following figure.

a$).$ Find the solution of $w$ and $b$ using the standard linear least$-$squares $($LLS$)$ method.

Plot out the fitting result.

$(3$ Marks$)$

b$).$ Suppose that initial weight is chosen randomly and learning rate $$ is $0\mathrm{.}02.$ Find the

solution of $w$ and $b$ using the least$-$mean$-$square $($LMS$)$ algorithm for $100$ epochs. Plot

out the fitting result and the trajectories of the weights versus learning steps. Will the

weights converge?

$(3$ Marks$)$

c$).$ Compare the results obtained by LLS and the LMS methods.

$(2$ Marks$)$

d$)$ Repeat the simulation study in b$)$ with different learning rates $,$ and explain your

findings.