18. (10) We seek to predict multivariate responses y; = (yu1, . . ., yik)T E RK for some K 2 2 based on predictor variables x;. Our training data consists of {y, x} , and we suppose that yij are all interval-scaled. For a multivariate prediction function F(x.) = (Fi(xi), . .., FK(xi))T e RK , we consider the score criterion N K L(y, F) = _ _lyn - F,(x)12. (2) 1= 1 i=1 (a) One approach of learning a F(.) that provides multivariate predictions is to ignore the multivariate nature of the responses and instead learn K separate prediction models, one for cach response. That is, for cach j c {1, ..., K}, one could learn F,( ) by applying standard CART with training data {y, x }-1. Is such an approach justified by loss function (2)? (b) Describe one potential shortcoming of (2) and suggest another score criterion. (c) A second approach is to consider models of the form: M F(xi) = (Fi(x;), ..., FK(x;))T where F;(x) = >cmj l(x E Rm), (3) m=1 for regions Rm C R" and Cm,j E R. In words, the regions (tree structure) are shared across all K responses, but the leaf predictions cm ; are different across responses. Describe an advantage and a disadvantage of (3) compared to using a separate tree for each of the K responses. (d) Explain how CART can be used to learn a decision tree model of the form (3) that performs well in terms of the loss (2). What modification is needed compared to CART with a 1-dimensional interval-scaled response? Is there any aspect of CART that does not generalize from K = 1 to K 2 2? 19. (5) Suppose one uses gini impurity to find the best split while constructing the classifi- cation tree. Give an example of a scenario where the first best split does not improve the gini impurity compared to a random classification (with gini impurity ; ), however one may get a "significantly better" gini impurity when another split is made. The goal of this problem is to show stopping criterion based on improvement in gini impurity might be shortsighted. It may be a better idea to grow the tree until the height of the tree is large enough and do pruning afterwards