Narrow Results

The Self-Organizing Map (SOM) is an efficient tool for visualizing high-dimensional data. In this paper, an intuitive and effective SOM projection method is proposed for mapping high-dimensional data onto the two-dimensional grid structure with a growing self-organizing mechanism. In the learning phase, a growing SOM is trained and the growing cell structure is used as the baseline framework. In the ordination phase, the new projection method is used to map the input vector so that the input data is mapped to the structure of the SOM without having to plot the weight values, resulting in easy visualization of the data. The projection method is demonstrated on four different data sets, including a 118 patent data set and a 399 checical abstract data set related to polymer cements, with promising results and a significantly reduced network size.

The main motivation of this paper is to propose a method to extract the output structure and find the input data manifold that best represents that output structure in a multivariate regression problem. A graph similarity viewpoint is used to develop an algorithm based on LDA, and to find out different output models which are learned as an input subspace. The main novelty of the algorithm is related with finding different structured groups and apply different models to fit better those structures. Finally, the proposed method is applied to a real remote sensing retrieval problem where we want to recover the physical parameters from a spectrum of energy.

This paper presents a new framework for manifold learning based on a sequence of principal polynomials that capture the possibly nonlinear nature of the data. The proposed Principal Polynomial Analysis (PPA) generalizes PCA by modeling the directions of maximal variance by means of curves, instead of straight lines. Contrarily to previous approaches, PPA reduces to performing simple univariate regressions, which makes it computationally feasible and robust. Moreover, PPA shows a number of interesting analytical properties. First, PPA is a volume-preserving map, which in turn guarantees the existence of the inverse. Second, such an inverse can be obtained in closed form. Invertibility is an important advantage over other learning methods, because it permits to understand the identified features in the input domain where the data has physical meaning. Moreover, it allows to evaluate the performance of dimensionality reduction in sensible (input-domain) units. Volume preservation also allows an easy computation of information theoretic quantities, such as the reduction in multi-information after the transform. Third, the analytical nature of PPA leads to a clear geometrical interpretation of the manifold: it allows the computation of Frenet–Serret frames (local features) and of generalized curvatures at any point of the space. And fourth, the analytical Jacobian allows the computation of the metric induced by the data, thus generalizing the Mahalanobis distance. These properties are demonstrated theoretically and illustrated experimentally. The performance of PPA is evaluated in dimensionality and redundancy reduction, in both synthetic and real datasets from the UCI repository.

The detection of damages in engineering structures by means of the changes in their vibration response is called structural health monitoring (SHM). It is a promising field but presents fundamental challenges. Accurate theoretical models of the structure are generally unfeasible, so data-based approaches are required. Indeed, only data from the undamaged condition are usually available, so the approach needs to be framed as novelty detection. Data are acquired from a network of sensors to measure local changes in the operating condition of the structures. In order to distinguish changes produced by damages from those caused by the environmental conditions, several physically meaningful features have been proposed, most of them in the frequency domain. Nevertheless, multiple measurement locations and the absence of a principled criterion to select among the potentially damage-sensitive features contribute to increase data dimensionality. Since high dimensionality affects the effectiveness of damage detection, we evaluate the effect of a dimensionality reduction approach in the diagnostic accuracy of damage detection.

A challenge for statistical learning is to deal with large data sets, e.g. in data mining. The training time of ordinary Support Vector Machines is at least quadratic, which raises a serious research challenge if we want to deal with data sets of millions of examples. We propose a "hard parallelizable mixture" methodology which yields significantly reduced training time through modularization and parallelization: the training data is iteratively partitioned by a "gater" model in such a way that it becomes easy to learn an "expert" model separately in each region of the partition. A probabilistic extension and the use of a set of generative models allows representing the gater so that all pieces of the model are locally trained. For SVMs, time complexity appears empirically to local growth linearly with the number of examples, while generalization performance can be enhanced. For the probabilistic version of the algorithm, the iterative algorithm probably goes down in a cost function that is an upper bound on the negative log-likelihood.

Manifold learning has been demonstrated as an effective way to represent intrinsic geometrical structure of samples. In this paper, a new manifold learning approach, named Local Coordinates Alignment (LCA), is developed based on the alignment technique. LCA first obtains local coordinates as representations of local neighborhood by preserving proximity relations on a patch, which is Euclidean. Then, these extracted local coordinates are aligned to yield the global embeddings. To solve the out of sample problem, linearization of LCA (LLCA) is proposed. In addition, in order to solve the non-Euclidean problem in real world data when building the locality, kernel techniques are utilized to represent similarity of the pairwise points on a local patch. Empirical studies on both synthetic data and face image sets show effectiveness of the developed approaches.

The paper describes an integrated recognition-by-parts architecture for reliable and robust face recognition. Reliability and robustness are characteristic of the ability to deploy full-fledged and operational biometric engines, and handling adverse image conditions that include among others uncooperative subjects, occlusion, and temporal variability, respectively. The architecture proposed is model-free and non-parametric. The conceptual framework draws support from discriminative methods using likelihood ratios. At the conceptual level it links forensics and biometrics, while at the implementation level it links the Bayesian framework and statistical learning theory (SLT). Layered categorization starts with face detection using implicit rather than explicit segmentation. It proceeds with face authentication that involves feature selection of local patch instances including dimensionality reduction, exemplar-based clustering of patches into parts, and data fusion for matching using boosting driven by parts that play the role of weak-learners. Face authentication shares the same implementation with face detection. The implementation, driven by transduction, employs proximity and typicality (ranking) realized using strangeness and p-values, respectively. The feasibility and reliability of the proposed architecture are illustrated using FRGC data. The paper concludes with suggestions for augmenting and enhancing the scope and utility of the proposed architecture.

Using local data information, the recently proposed local Fisher Discriminant Analysis (LFDA) algorithm¹⁸ provides a new way of handling the multimodal issues within classes where the conventional Fisher Discriminant Analysis (FDA) algorithm fails. Like the FDA algorithm (global counterpart), the LFDA suffers when it is applied to the higher dimensional data sets. In this paper, we propose a new formulation by which a robust algorithm can be formed. The new algorithm offers more robust results for higher dimensional data sets when compared with the LFDA in most cases. By extensive simulation studies, we have demonstrated the practical usefulness and robustness of our new algorithm in data visualization.

The paper presents mathematical underpinnings of the locally linear embedding technique for data dimensionality reduction. It is shown that a cogent framework for describing the method is that of optimization on a Grassmann manifold. The solution delivered by the algorithm is characterized as a constrained minimizer for a problem in which the cost function and all the constraints are defined on such a manifold. The role of the internal gauge symmetry in solving the underlying optimization problem is illuminated.

This paper proposes an Enhanced Lipschitz Embedding based Classifier (ELEC) for the classification of multi-emotions from speech signals. ELEC adopts geodesic distance to preserve the intrinsic geometry at all scales of speech corpus, instead of Euclidean distance. Based on the minimal geodesic distance to vectors of different emotions, ELEC maps the high dimensional feature vectors into a lower space. Through analyzing the class labels of the neighbor training vectors in the compressed low space, ELEC classifies the test data into six archetypal emotional states, i.e. neutral, anger, fear, happiness, sadness and surprise. Experimental results on clear and noisy data set demonstrate that compared with the traditional methods of dimensionality reduction and classification, ELEC achieves 15% improvement on average for speaker-independent emotion recognition and 11% for speaker-dependent.

The locally linear embedding (LLE) algorithm is hypothetically able to find a lower dimensional space than a linear method for preserving a data manifold originally embedded in a high dimensional space. However, uneven sampling over the manifold in real-world data ultimately causes LLE to suffer from the disconnected-neighborhood problem. Consequently, the final dimensionality required for the data manifold is multiplied by the number of disjoint groups in the complete data representation. In addition, LLE as an unsupervised method is unable to suppress between-class connections. This means that samples from different classes are mixed during reconstruction. This study presents CLLE, a classification-oriented LLE method that uses class label information from training samples to guide unsupervised LLE. The criterion for neighbor selection is redesigned using class-conditional likelihood as well as Euclidean distance. This algorithm largely eliminates fractured classes and lowers the incidence of connections between classes. Also, a reconnection technique is proposed as a supporting method for ensuring a fully connected neighborhood graph, so that CLLE is able to extract the fewest features. Experiments with simulated and real data show that CLLE exceeds the performance of linear methods. Comparable classification performance can be achieved by CLLE using fewer features. In comparison with LLE, CLLE demonstrates a higher aptitude for and flexibility towards classification.

Many locality-based unsupervised dimensionality reduction (DR) algorithms have recently been proposed and demonstrated to be effective to a certain degree in some classification tasks. In this paper, we aim to show that: (1) a discriminant disposal is intentionally or unintentionally induced from the construction of locality in these unsupervised algorithms, however, such a discrimination is often inconsistent with the actual class information, so here called disguised discrimination; (2) sensitivities of these algorithms to local neighbor parameters stem from the inconsistency between the disguised discrimination and the actual class information; (3) how such inconsistency impacts the classification performance of these algorithms. The experiments on the benchmark face datasets testify our statements that are expected to provide some insight into the unsupervised leaning based on locality.

Many problems in pattern classification and feature extraction involve dimensionality reduction as a necessary processing. Traditional manifold learning algorithms, such as ISOMAP, LLE, and Laplacian Eigenmap, seek the low-dimensional manifold in an unsupervised way, while the local discriminant analysis methods identify the underlying supervised submanifold structures. In addition, it has been well-known that the intraclass null subspace contains the most discriminative information if the original data exist in a high-dimensional space. In this paper, we seek for the local null space in accordance with the null space LDA (NLDA) approach and reveal that its computational expense mainly depends on the quantity of connected edges in graphs, which may be still unacceptable if a great deal of samples are involved. To address this limitation, an improved local null space algorithm is proposed to employ the penalty subspace to approximate the local discriminant subspace. Compared with the traditional approach, the proposed method can achieve more efficiency so that the overload problem is avoided, while slight discriminant power is lost theoretically. A comparative study on classification shows that the performance of the approximative algorithm is quite close to the genuine one.

An improved manifold learning method, called Uncorrelated Local Fisher Discriminant Analysis (ULFDA), for face recognition is proposed. Motivated by the fact that statistically uncorrelated features are desirable for dimension reduction, we propose a new difference-based optimization objective function to seek a feature submanifold such that the within-manifold scatter is minimized, and between-manifold scatter is maximized simultaneously in the embedding space. We impose an appropriate constraint to make the extracted features statistically uncorrelated. The uncorrelated discriminant method has an analytic global optimal solution, and it can be computed based on eigen decomposition. As a result, the proposed algorithm not only derives the optimal and lossless discriminative information, but also guarantees that all extracted features are statistically uncorrelated. Experiments on synthetic data and AT&T, extended YaleB and CMU PIE face databases are performed to test and evaluate the proposed algorithm. The results demonstrate the effectiveness of the proposed method.

Pseudoinverse linear discriminant analysis (PLDA) is a classical method for solving small sample size problem. However, its performance is limited. In this paper, we propose an improved PLDA method which is faster and produces better classification accuracy when experimented on several datasets.

Automatic fingerprint identification systems (AFIS) have been studied extensively and are widely used for biometric identification. Given its importance, many well-engineered methods have been developed for the different stages that encompass those systems. The first stage of any such system is the segmentation of the actual fingerprint region from the background. This is typically achieved by classifying pixels, or blocks of pixels, based on a set of features. In this paper, we describe novel features for fingerprint segmentation that express the underlying manifold topology associated with image patches in a local neighborhood. It is shown that fingerprint patches seen in a high-dimensional space form a simple and highly regular circular manifold. The characterization of the manifold topology suggests a set of optimal features that characterize the local properties of the fingerprint. Thus, fingerprint segmentation can be formulated as a classification problem based on the deviation from the expected topology. This leads to features that are more robust to changes in contrast than mean, variance and coherence. The superior performance of the proposed features for fingerprint segmentation is shown in the eight datasets from the 2002 and 2004 Fingerprint Verification Competitions.

Recently, many dimensionality reduction (DR) algorithms have been developed, which are successfully applied to feature extraction and representation in pattern classification. However, many applications need to re-project the features to the original space. Unfortunately, most DR algorithms cannot perform reconstruction. Based on the manifold assumption, this paper proposes a General Manifold Reconstruction Framework (GMRF) to perform the reconstruction of the original data from the low dimensional DR results. Comparing with the existing reconstruction algorithms, the framework has two significant advantages. First, the proposed framework is independent of DR algorithm. That is to say, no matter what DR algorithm is used, the framework can recover the structure of the original data from the DR results. Second, the framework is space saving, which means it does not need to store any training sample after training. The storage space GMRF needed for reconstruction is far less than that of the training samples. Experiments on different dataset demonstrate that the framework performs well in the reconstruction.

We propose a novel unsupervised subspace learning method to optimize graph construction for face recognition called Datum Adaptive Weighted Collaborative Representation (DAWCR). Different from sparsity preserving projection (SPP), a recently proposed linear dimensionality reduction method inspired by sparse representation, where graph is constructed using sparse reconstructive relationship by minimizing a l₁-regularization-based objective function, DAWCR aims to optimize the graph construction by incorporating the locality structure and features variance among data elements into a unified framework using regularized linear representation i.e. weighted regularized least square using l₂-minimization approach. The neighborhood selection method in DAWCR is datum dependent, moreover neighborhood size for each datum is chosen automatically by considering data distribution probability. Hence the resulting graph is sparse, models the nonlinear geometry of data set, and conveys more discriminate information. The DAWCR problem formulation has the algebraic solution without involving any parameter tuning for optimal neighbors selection, which makes it computationally efficient than SPP. Extensive experiments on several publicly available real-world face and some UCI data sets, are conducted to verify the feasibility and effectiveness of the proposed method. Experimental results show that the proposed method achieves competitive performance with encouraging results.

Fisher discriminant analysis (FDA) is a classic supervised dimensionality reduction method in statistical pattern recognition. FDA can maximize the scatter between different classes, while minimizing the scatter within each class. As it only utilizes the labeled data and ignores the unlabeled data in the analysis process of FDA, it cannot be used to solve the unsupervised learning problems. Its performance is also very poor in dealing with semi-supervised learning problems in some cases. Recently, several semi-supervised learning methods as an extension of FDA have proposed. Most of these methods solve the semi-supervised problem by using a tradeoff parameter that evaluates the ratio of the supervised and unsupervised methods. In this paper, we propose a general semi-supervised dimensionality learning idea for the partially labeled data, namely the reconstruction probability class of labeled and unlabeled data. Based on the probability class optimizes Fisher criterion function, we propose a novel Semi-Supervised Local Fisher Discriminant Analysis (S2LFDA) method. Experimental results on real-world datasets demonstrate its effectiveness compared to the existing similar correlation methods.

Multidimensional scaling techniques are unsupervised Dimension Reduction (DR) techniques which use multidimensional data pairwise similarities to represent data into a plane enabling their visual exploratory analysis. Considering labeled data, the DR techniques face two objectives with potentially different priorities: one is to account for the data points' similarities, the other for the data classes' structures. Unsupervised DR techniques attempt to preserve original data similarities, but they do not consider their class label hence they can map originally separated classes as overlapping ones. Conversely, the state-of-the-art so-called supervised DR techniques naturally handle labeled data, but they do so in a predictive modeling framework where they attempt to separate the classes in order to improve a classification accuracy measure in the low-dimensional space, hence they can map as separated even originally overlapping classes. We propose ClassiMap, a DR technique which optimizes a new objective function enabling Exploratory Data Analysis (EDA) of labeled data. Mapping distortions known as tears and false neighborhoods cannot be avoided in general due to the reduction of the data dimension. ClassiMap intends primarily to preserve data similarities but tends to distribute preferentially unavoidable tears among the different-label data and unavoidable false neighbors among the same-label data. Standard quality measures to evaluate the quality of unsupervised mappings cannot tell about the preservation of within-class or between-class structures, while classification accuracy used to evaluate supervised mappings is only relevant to the framework of predictive modeling. We propose two measures better suited to the evaluation of DR of labeled data in an EDA framework. We use these two label-aware indices and four other standard unsupervised indices to compare ClassiMap to other state-of-the-art supervised and unsupervised DR techniques on synthetic and real datasets. ClassiMap appears to provide a better tradeoff between pairwise similarities and class structure preservation according to these new measures.