On uncertainty measures used for decision tree induction

This paper provides a further look at uncertainty or information criteria used in the context of deci- sion tree induction, and more generally of learn- ing conditional class probability models. We show the high degree of similarity among two main families of criteria based respectively on the logarithmic SHANNON entropy function and the quadratic GINI index. We start by introduc- ing a general family of entropy functions and then discuss the latter particular cases, and end up with a short review of the Kolmogorov-Smirnov dis- tance,anotherrelatedmeasure.