In this problem, we consider splitting when building a regression tree in the CART algorithm. We...

Transcribed Image Text:

In this problem, we consider splitting when building a regression tree in the CART algorithm. We assume that there is a feature vector X RP and dependent variable Y R. We have collected a training dataset (x, y),..., (In, Yn), where x R and y; E R for all i = 1,..., n. We also assume, for simplicity, that we are considering the initial split at the top (root node) of the tree. An arbitrary split simply divides the training dataset into a partition of size two. By appropriately reshuffling the data, we can represent this partition (again for simplicity) via two sub-datasets (x1, y),..., (TN, YN) and (TN+1, YN+1),..., (En, Yn) where N is the index of the last observation included in the first set. Assume throughout that our impurity function is the RSS error the standard choice for a regression tree. As mentioned, the above reasoning applies to an arbitrary split of the training dataset into a parti- tion of size 2. The CART algorithm only considers splits of a particular type - those corresponding to two regions R(j, s) = {X : Xj < s} and R(j, s) = {X : X; s} where j is the index of a chosen feature and s is a cutoff value. d) (10 points) Consider a modification of the regression tree algorithm such that, in addition to considering splits of the form described in the preceding paragraph, we also consider splits of the form R(j, t) = {X : eXi < t} and R(j,t) = {X : eX; t} where j is the index of a chosen feature and t is a cutoff value for the exponential function ei. Is it possible for these new splits to improve the regression tree? Explain. In this problem, we consider splitting when building a regression tree in the CART algorithm. We assume that there is a feature vector X RP and dependent variable Y R. We have collected a training dataset (x, y),..., (In, Yn), where x R and y; E R for all i = 1,..., n. We also assume, for simplicity, that we are considering the initial split at the top (root node) of the tree. An arbitrary split simply divides the training dataset into a partition of size two. By appropriately reshuffling the data, we can represent this partition (again for simplicity) via two sub-datasets (x1, y),..., (TN, YN) and (TN+1, YN+1),..., (En, Yn) where N is the index of the last observation included in the first set. Assume throughout that our impurity function is the RSS error the standard choice for a regression tree. As mentioned, the above reasoning applies to an arbitrary split of the training dataset into a parti- tion of size 2. The CART algorithm only considers splits of a particular type - those corresponding to two regions R(j, s) = {X : Xj < s} and R(j, s) = {X : X; s} where j is the index of a chosen feature and s is a cutoff value. d) (10 points) Consider a modification of the regression tree algorithm such that, in addition to considering splits of the form described in the preceding paragraph, we also consider splits of the form R(j, t) = {X : eXi < t} and R(j,t) = {X : eX; t} where j is the index of a chosen feature and t is a cutoff value for the exponential function ei. Is it possible for these new splits to improve the regression tree? Explain.

Answer rating: 100% (QA)

To build a regression tree using the CART Classification and Regression Trees algorithm we aim to mi... View the full answer