Narrow Results

We confirm by the multi-Gaussian support vector machine (SVM) classification that the information of the intrinsic dimension of Riemannian manifolds can be used to illustrate the efficiency (learning rates) of learning algorithms. We study an approximation scheme realized by convolution operators involving the Gaussian kernels with flexible variances. The essential analysis lies in the study of its approximation order in L^p (1 ≤ p < ∞) norm as the variance of the Gaussian tends to zero. It is different from the analysis for approximation in C(X) since pointwise estimations do not work any more. The L^p approximation arises from the SVM case where the approximated function is the Bayes rule and is not continuous, in general. The approximation error is estimated by imposing a regularity condition that the approximated function lies in some interpolation spaces. Then, the learning rates for multi-Gaussian regularized classifiers with general classification loss functions are derived, and the rates depend on the intrinsic dimension of the Riemannian manifold instead of the dimension of the underlying Euclidean space. Here, the input space is assumed to be a connected compact C^∞ Riemannian submanifold of ℝⁿ. The uniform normal neighborhoods of the Riemannian manifold and the radial basis form of Gaussian kernels play an important role.

Correntropy-based learning has achieved great success in practice during the last decades. It is originated from information-theoretic learning and provides an alternative to classical least squares method in the presence of non-Gaussian noise. In this paper, we investigate the theoretical properties of learning algorithms generated by Tikhonov regularization schemes associated with Gaussian kernels and correntropy loss. By choosing an appropriate scale parameter of Gaussian kernel, we show the polynomial decay of approximation error under a Sobolev smoothness condition. In addition, we employ a tight upper bound for the uniform covering number of Gaussian RKHS in order to improve the estimate of sample error. Based on these two results, we show that the proposed algorithm using varying Gaussian kernel achieves the minimax rate of convergence (up to a logarithmic factor) without knowing the smoothness level of the regression function.

This paper studies the binary classification problem associated with a family of Lipschitz convex loss functions called large-margin unified machines (LUMs), which offers a natural bridge between distribution-based likelihood approaches and margin-based approaches. LUMs can overcome the so-called data piling issue of support vector machine in the high-dimension and low-sample size setting, while their theoretical analysis from the perspective of learning theory is still lacking. In this paper, we establish some new comparison theorems for all LUM loss functions which play a key role in the error analysis of large-margin learning algorithms. Based on the obtained comparison theorems, we further derive learning rates for regularized LUMs schemes associated with varying Gaussian kernels, which maybe of independent interest.