Non linear Program

(5) Let f : Rentm -> R be defined as f(x, y) = |Ax - (by)y|2, where A is an m x n matrix, bE RM, r E R" and y E Rm. (a) Find Vf(x, y) and V2f(x, y). Hint: you may want to expand the expression for f (x, y) and compute the partial derivatives with respect to x and y. (b) Find the points (x0, yo ) satisfying the necessary first and second order conditions for a minimizer of f(x, y) on ? = Rntm