work by Poggio and Girosi [1990b], and later extended to handle outliers and
negative examples, Girosi et al. [1991]. Poggio and Girosi also showed the equiv-
alence of nonlinear regularization algorithms with multilayer networks, Poggio
and Girosi [1990a].
the linear model of weight decay is called jitter. Jitter is equivalent to ridge
regression.
fier (PSVMC), Fung and Mangasarian [2001b]. They later introduced algorithms
for multicategory PSVMC (Fung and Mangasarian [2001a]) and incremental and
decremental PSVMC (Fung and Mangasarian [2002]). The PSVMC was devel-
oped by relaxing the constraints of the ordinary SVMC quadratic optimization
problem.
and that Fung and Mangasarians two main contributions were 1) very fast ways
of computing the linear RLSC and 2) empirical evidence that RLSC have approx-
imately the same classification accuracy as SVMC on benchmark datasets. It was
also proved that the SVMC and RLSC have the same generalization bounds, i.e.
theoretically supporting the prior empirical evidence, Rifkin [2002]; Rifkin et al.
[2003]. Agarwal showed that PSVMC can be transformed into classification using
ridge regression, Agarwal [2002] (This is also supported by the above-mentioned
relation between ridge regression and regularization).
promising performance and efficient memory utilization; results making it suit-
able in web intelligence applications, but could it be further improved to
web sites)
cremental PSVMC (i.e. RLSC) algorithms proposed by Fung and Mangasarian
[2002]. In paper F (Tveit and Hetland [2003]) we proposed memoization in order
to add efficient support for incremental multicategory classification with PSVMC.