Abstract :
[en] Some results on consistency and robustness of support vector machines for both nonparametric classification and nonparametric regression were recently derived for non-negative convex loss func- tions L of Nemitski type, see Christmann and Steinwart (2007) and Steinwart and Christmann (2008), if a weak moment condition for the joint distribution P on X × Y is valid. However, this condition excludes heavy-tailed distributions such as the Cauchy distribution or several extreme value distributions which can occur in financial or actuarial problems.
In this talk we will weaken this condition on P to only a condition on the marginal distribution PX , such that the applicability of SVMs can be extended to heavy-tailed conditional distributions. As was already used by Huber (1967) in a different setting, we will shift the loss function L : X × Y × R → [0, ∞) downwards by some function which is independent of the estimator. More precisely, we define the shifted loss function L⋆(x,y,f(x)) := L(x,y,f(x))−L(x,y,0). It is obvious that this new “loss” L⋆(x,y,f(x)) can be negative. Let fL,P,λ be the decision function of the original support vector machine. We define
fL⋆,P,λ =arg inf EPL⋆(X,Y,f(X))+λ∥f∥2H, f∈H
where H is the reproducing kernel Hilbert space of a measurable kernel k : X × X → R and λ > 0 is some regularization parameter.
We will first discuss the used “L⋆-trick”, give some properties of the new loss L⋆, and show that fL⋆,P,λ exists and is unique. Furthermore, this SVM solution will coincide with fL,P,λ if the latter exists. We then give a representer theorem and results on both risk-consistency as well as consistency of the solution. And finally we will show that fL⋆,P,λ is robust in the sense of influence functions if the kernel is bounded and if the loss function L is Lipschitz continuous. This result holds true for both Hampel’s influence function as well as the Bouligand influence function proposed by Christmann and Van Messem (2008).
