Subject: Machine Learning

Question Number 1:

A dataset showing the decisions about presence of Heart Disease (HD) based on four attributes (Age, Blood pressure, Cholesterol and Heart rate) is listed in the following table:

Age	BP	CH	HR	HD
Old	Low	Low	Medium	No
Very old	Low	Medium	Low	Yes
Old	Medium	Medium	Medium	No
Old	Low	Low	Low	Yes
Mild	Low	Low	Medium	Yes
Very old	High	Medium	Low	Yes
Very old	Medium	Medium	Low	Yes
Mild	Medium	Very high	Low	Yes
Old	Low	Medium	Medium	No
Very old	Medium	Very high	Medium	No
Very old	Medium	Low	Medium	No
Mild	Low	Medium	Medium	No
Mild	Low	Medium	Medium	No

1. Compute the entropy for this data set.
2. What is the information gain for the Heart Rate (HR) feature?
3. Given a choice among Age, Blood pressure, Cholesterol and Heart rate, which feature would the ID3 algorithm choose as the root node for a decision tree.
4. What is the role of information gain and entropy in ID3 algorithm?