Question $3(50$ points$)$

Your task is to minimise the following two multivariate functions using the convex

optimisation techniques that you learnt in the module.

$f_1(x,y)=(1-x{)}^2+5**(y-2**x^2{)}^2$

$f_2(x,y)=x^2+y^2+x**y+5$

For the first function, use the starting point $(-1,2)$ and for the second function, use the

starting point $(-3,1\mathrm{.}5).$ Then, compare the performance of Gradient Descent, Momentum,

Nesterov Momentum and Adam optimiser for these tasks. For the first function, you can

use $100$ steps and for the second function, use $50$ steps of all these techniques. For each

Page $2$ of $3$

of these techniques, use the best from two different learning rates: $0\mathrm{.}01$ and $0\mathrm{.}001$ for the

first function and from $0\mathrm{.}1$ and $0\mathrm{.}01$ for the second function. You can find good values of

other parameters empirically. Plot the contour lines and show the iterative progress made

by the different optimisation techniques $($just as you did in your tutorial$).$ How do these

techniques compare empirically? Do you observe any benefit of Adam over other

techniques for these functions?