Exercise $1($up to Lecture $5)$ Unconstrained Optimization Methods.

a$)$ Code a routine in Matlab that can execute the Golden Section Search $($GSS$)$ method. The function should take as inputs the initial triplet of points $(a,b,c),$ the function $f,$ and the tolerance $lon.$ It should return the solution in terms of $x$ and $y$ value. Clearly name this function.

b$)$ Solve for the local maximum of $f(x)=e^{-x^2}$ using your routine from a$).$ Set $(a,b,c)=(-10,2,8)$ and $lon={10}^{-8}.$ Report the $x$ and the $f(x)$ value at the local maximum.

Hint: it should converge to the true solution!

c$)$ Code a routine in Matlab that can execute Newton's method for optimization. The function should take as inputs the initial point $x_0,$ the function $f,$ it's gradient gradf, the Hessian $H_f,$ two tolerances $,lon,$ and the maximum number of iterations I. It should return the solution as well as an exit flag. Clearly name this function.

d$)$ Solve for the local maximum of $f(x,y)=e^{-x^2-y^2}$ using your routine from c$).$ Set $x_0=[0\mathrm{.}3,0\mathrm{.}3{]}^T,lon={10}^{-8},={10}^{-8},I=100.$ What happens with $x_0=[0\mathrm{.}4,0\mathrm{.}4{]}^T?$

Hint: it should converge to the true solution!

e$)$ Code a routine in Matlab that can execute the BFGS method. The function should take as inputs the initial point $x_0,$ the function $f,$ it's gradient gradf, the initial guess for the Hessian tilde$(H{)}_f,$ two tolerances $,lon,$ and the maximum number of iterations I. It should return the solution as well as an exit flag. Clearly name this function.

f$)$ Solve for the local maximum of $f(x,y)=e^{-x^2-y^2}$ using your routine from e$).$ Set $x_0=[0\mathrm{.}3,0\mathrm{.}3{]}^T,$ tilde$(H{)}_f=\left[\begin{array}{cc}-1\mathrm{.}3& 0\mathrm{.}3\\ 0\mathrm{.}3& -1\mathrm{.}3\end{array}\right],lon={10}^{-8},={10}^{-8},I=100.$

Hint: it should converge to the true solution!

g$*)$ Repeat f$)$ but now use tilde$(H{)}_f=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right].$ For which initial guess of the Hessian does the algorithm converge $($in fewer iterations$)?$ Argue why.

h$)$ Using any of the three methods above, find all local minima of $f(x)=x^4-12x^3+15x^2+56x-55.$ What is the global minimum? Why? Also argue why there cannot be any other local minimum.