I am preparing a machine Learning final

the Bold questions are that I have to know for the exam, and I wrote the answer as much as I know for the decision tree based on my powerpoint material.

is there anything I need to add for decision tree or do I have wrong information?

Machine Learning Test #3

- Decision Tree

1. is used for classification, regression, both, neither

* regression : by taking the mean of the target values in the node

* classification : by taking the mode of the target values in the node

2. is a parametric algorithm or not

3. tendencies towards bias, variance or overfitting, underfitting

* overfiting : tree grows too large it may overfit the data

* trees can be pruned which could have several consequences :

- the tree may perform better (no guarantees though)

- the tree may be easier to interpret

4. how it divides the space : flexible or linear boundaries

* the decision tree partitioning makes axis-parallel splits which is on of its limitations

- cannot make a diagonal split with decision trees

* Box-shaped regions

5. conceptually, how the algorithm works

* a decision tree segments the training data into simple regions by repeatedly selecting and

an attribute/value that divides the data well

6. conceptually, how the metrics work ex: entropy, Information Gain

7. Advantage, disadvantage (easy to interpret, data need scaling, etc)

* advantages : simple and interpretable

- easy to explain

- my mirror human decision making

- handles qualitative predictors without the need for dummy variables

* disadvantages : performance is not competitive with other supervised algorithms

- predictive accuracy not as high as other algorithms

- highly variable with changes in input data

* improvement : bagging boosting, random forests

* produce multiple trees that are combined in a single prediction

* these are more accurate but lose interpretability

Additional information

* I need to know how to solve entropy

* and information gain