answer with step by step explanation

4. Consider the non-linear regression problem of fitting the function class y = f (x) = 0(x)'0 = Zio big; (x) where o (x)" = [do (x) , ..., PM-1 (x)] are non-linear and 0' = [Go, . ..,0M-1] are parameters to be fitted. Given the data set D = {(21, y1) , (x2, 42) , . .., (XN, UN) } with inputs X = {21, 22, ..., N) and observation ) = {y1, y2, ..., yN} we use the following Loss function to minimise in order to choose the best parameters 0: N L(0) = ):= 202 lly - 2012 + 262 [1101?]. i= 1 where p > 1, o, b are fixed parameters and & (x) is the fixed N x (M - 1) matrix do (21) $2 (21 ) ... PM-1 (21) Po (12) $2 (12) ... QM-1 (22) Q = : Do (IN) $2 (N ) ... OM-1 (IN) (a) Assuming p = 1, use a formal, by hand, calculus argument, show that the system of linear equations that are required to be solved to obtain the optimal 0* parameters are: 1 7 + + ( 2 ) ' 1 0 - y's . [Hint: Recall that VuljuQu + bu + d] = Qu + b. ] (b) Now suppose only that p > 1 (and not necessarily 2) so we must solve this problem (mine L (0)) using a stochastic gradient descent. Assuming we take a single random sample from the set {VL; (0*) }; of N possible gradients at each iteration & what would be the form of the update rule ok+1 = 0k - nkidk ? Hint: You need to specify n' and d' in terms of the data of the problem.]