1. Introduction to Decision Trees

<aside> 💡 The basic intuition behind a decision tree is to map out all possible decision paths in the form of a tree.

</aside>

How to build a decision tree?

1. Choose an attribute from your dataset.
2. Calculate the significance of attribute in splitting of data
3. split data based on the value of the best attribute
4. Go to step 1

2. Building Decission Trees

Entropy

the entropy in a node is the amount of information disorder calculated in each node

• Measure of randomness or uncertainty
• The lower the Entropy, the less uniform the distribution, the purer the node

$$ Entropy = -p(A)log(p(A))-p(B)log(p(B)) $$

Choose the tree with the higher information gain after splitting

Information Gain is the information that can incease the lefel of certain after spliting

information gain = (entropy before split) - (weighted entropy after split)