Question: Please answer questions a to e according to the following formula, and give detailed steps and explanations Recall that for a function f : RN

Please answer questions a to e according to the following formula, and give detailed steps and explanations

Please answer questions a to e according to thePlease answer questions a to e according to the
Recall that for a function f : RN - R, the gradient is simply the N x 1 vector formed by taking the derivative of f with respect to each entry in a: of of Vf(ac) = and the Hessian is the N x N symmetric matrix of second derivatives a2 f a2 f a2 f a2 x1 a2 f as f V2 f (ac) = Or20x1 82 x 2 . . . a2 f a2 f . . If f : RN -> RM is multivariate and multi-valued, f1(ac) f2 (20) f (ac) = LfM(ac) its Jacobian matrix is the M x N matrix a first derivatives: of1 of1 of1 Or1 ax2 Of2 of2 of2 ax2 ON F'(x) = . . . OfM OfM OfM Ox1 ON - Note that for M = 1, we of course have F'(x) = Vf(x)T. Also note that for M = 1, the Hessian is the Jacobian of the gradient map, i.e. if we let g : RN - RN be defined as g(x) = Vf(x), then G'(x) = V2 f(x). Recall also the standard chain rule: For differentiable g : R - R and h : R - R, let f (t) = g(h(t)). We have df dg dh at dh dt' or f' ( t ) = g' ( h (t) ) . h' (t ) . For (possibly multi-valued) functions of multiple variables, the chain rule is general- ized as follows. Let g : RK - RM and h : RN -> RK be differentiable functions, and let f : RN - RM be the composition f (x) = g(h(x)). Then F'(x) = G'(h(x)) . H'(2),where the . above is a matrix product. Note that the dimensions work out, since G is always a M x K matrix, and H' is always a K x N matrix. In the case where g (and hence f) is scalar valued (M = 1), we can rewrite the above as Vf(x) = (Vg(h(a) )TH'(x) )T = H'(x) TVg(h(x)). In the case where both g and h are scalar valued (M = K = 1), we have . (20 ) 4A ( (20 ) 4) , 6 = (20 ) f Using the above, (a) Compute the gradient of f(x) = ? ||Ax - |13. (b) Compute the Hessian of f(x) = 21| Ax - yl13. (c) Compute the gradient of f(x) = Em_1 -log(cmx - bm). (d) Compute the Hessian of f(a) = _M_ - log(cTa - bm). (e) Let hi, ..., hz be differentiable functions he : RNe-1 -> R' (we will take No = N) and let g : R L - R. Find an expression for the gradient of f : RN -> R, where f (a) = g (hL(hL-1(. . . h2(hi(ac)) . . . )))

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Mathematics Questions!