Problem $3.$ In the gradient descent algorithm, $>0$ is the learning rate. If $$ is small enough,

then the function value guarantees to decrease. In practice, we may anneal $,$ meaning that

we start from a relatively large $,$ but decrease it gradually.

Show that $$ cannot be decreased too fast. If $$ is decreased too fast, even if it is strictly

positive, the gradient descent algorithm may not converge to the optimum of a convex

function.

Hint: Show a specific loss and an annealing scheduler such that the gradient descent

algorithm fails to converge to the optimum.