Use cross$-$validation to choose a hyperparameter value

Suppose you want to only learn decision trees that are limited in depth. This can be achieved using a $$maximum depth$$ hyperparameter in the decision tree learning algorithm implemented in sklearn. But how

should you choose this hyperparameter value? Using hold$-$out cross$-$validation is one possibility.

To implement hold$-$out cross$-$validation, you will first need to randomly partition the training data into two parts. The following code does exactly this.

from sklearn.model$\_$selection import train$\_$test$\_$split

X$\_$train, X$\_$val, y$\_$train, y$\_$val $=$ train$\_$test$\_$split$($covtype$[\text{'}$train$\_$data'$],$

covtype$[\text{'}$train$\_$labels'$],$ test$\_$size$=0\mathrm{.}2,$ random$\_$state$=0)$

You will also need to decide on which values to consider for the $$maximum depth$$ hyperparameter. Let$$s

fixed this to the following ten values: $5,10,15,20,25,30,35,40,45,50.$

$$ Problem $8.$ Use hold$-$out cross$-$validation to pick a value for the $$maximum depth$$ hyperparameter. $($You may want to use sklearn.metrics.accuracy$\_$score, and also save some of the intermediate

results for Problem $9,$ below.$)$ What is the chosen hyperparameter value?

$$ Problem $9.$ Use matplotlib to plot the validation accuracy $($or validation error rate, if you prefer$)$ as a function of the $$maximum depth$$ hyperparameter value. Label the axes of the plot and give the plot an appropriate title.

$$ Problem $10.$ Now use the value for the $$maximum depth$$ hyperparameter chosen in Problem $8$ to learn another decision tree, using the entirety of the training data $($covtype$[\text{'}$train$\_$data'$]$ and covtype$[\text{'}$train$\_$labels'$],$ not just X$\_$train and y$\_$train$).$ What is the test error rate of this new learned decision tree?

For Problem $9,$ the matplotlib Simple Plot example$3$ may be helpful if you are new to matplotlib.

all python code and calculations are need