Jesus A. Gonzalez
August 2, 2016
\[Entropy([9+, 5-]) = -(9/14) \log_2 (9/14) -(5/14) \log_2 (5/14)\] \[= 0.940\]
\[Gain(S,A) = Entropy(S) - \sum\limits_{v \in Values(A)} \frac{|S_v|}{|S|}Entropy(S_v)\] - \(Values(A)\): Set of values that attribute \(A\) may take - \(S_v\): Subset of \(S\) for which attribute \(A\) has value \(v\) - i.e. \(S_v={s \in S | A(S) = v}\)
\(Gain(S, Wind) = Entropy(S) - \sum\limits_{v \in \{Weak, Strong\}} \frac{|S_v|}{|S|} Entropy(S_v)\)
\(=Entropy(S) - (8/14)Entropy(S_{Weak}) - (6/14)Entropy(S_{Strong})\)
\(=Entropy(S) - (8/14)(-6/8 \log_2 6/8 - 2/8 \log_2 2/8)\) \(-(6/14)(-3/6 \log_2 3/6 - 3/6 \log_2 3/6)\)
\(= 0.940 - (8/14) 0.811 - (6/14) 1.0\)
\(= 0.048\)