Jesus A. Gonzalez
January 26, 2020
\[Entropy([9+, 5-]) = -(9/14) \log_2 (9/14) -(5/14) \log_2 (5/14)\] \[= 0.940\]
\(Gain(S, Wind) = Entropy(S) - \sum\limits_{v \in \{Weak, Strong\}} \frac{|S_v|}{|S|} Entropy(S_v)\)
\(=Entropy(S) - (8/14)Entropy(S_{Weak}) - (6/14)Entropy(S_{Strong})\)
\(=Entropy(S) - (8/14)(-6/8 \log_2 6/8 - 2/8 \log_2 2/8)\) \(-(6/14)(-3/6 \log_2 3/6 - 3/6 \log_2 3/6)\)
\(= 0.940 - (8/14) 0.811 - (6/14) 1.0\)
\(= 0.048\)