Consistency and Robustness Properties of Support Vector Machines for Heavy-Tailed Distributions

Recently results on consistency and robustness of support vector machines (SVMs) were derived for non-negative convex losses L of Nemitski type given some weak moment condition for the joint distribution P on X × Y (Christmann and Steinwart, 2007). However, this condition excludes heavy-tailed distributions such as the Cauchy distribution or several extreme value distributions.
The condition on P can be weakened to only a condition on the marginal distribution PX by shifting the loss L downwards, a trick that will enlarge the applicability of SVMs to heavy-tailed conditional distributions. More precisely, we define the shifted loss L⋆(x,y,f(x)) := L(x,y,f(x))− L(x,y,0). Obviously, this new “loss” L⋆(x,y,f(x)) can be negative. We define the decision function of the shifted SVM as fL⋆,P,λ.
We will give some properties of L⋆ and fL⋆,P,λ and we will state a representer theorem and results on both risk-consistency as well as consistency of the solution. Finally we will show that, given some conditions, fL⋆,P,λ is robust in the sense of both Hampel’s influence function as well as the Bouligand influence function introduced by Christmann and Van Messem (2008).