$1$ Gradient Descent $(45$ marks$)$

In this question, we will discuss properties of gradient descent. In the following, we will

consider the family of loss functions $`$ a$($w$)=$ aw$2$ for a $>0.$

$(1)$ Let a $=4$ and consider the two step sizes $1=1/4$ and $2=2.$ What do you observe

when starting gradient descent at w$0=1?(15$ marks$)$

$(2)$ Define the set of all step sizes $>0$ for which gradient descent will converge to the

minimum w $$ of $`$ a$($w$)$ when starting at w$0=0.(15$ marks$)$

$(3)$ Let $>0,$ and suppose is chosen such that gradient descent coverges to the minimum

w $$ of $`$ a$($w$)$ starting at w$0.$ After how many steps i do we have $|$wi $$ w $|<?(15$ marks$)$