Studies in Pattern Recognition

The following sections are included:

• Contents

• Preface

The purpose of this paper is two-fold: to give a synoptic description of favored neural networks and to characterize the potency of these neural networks as pattern classifiers, against the background of the familiar nearest neighbors classification. We limit the study to those neural network structures most commonly used for pattern classification: the multilayer perceptron, the Kohonen associative memory, and the Carpenter–Grossberg clustering network, for which we give a tutorial description with the aim of making the driving concepts apparent. The nearest neighbors rule is presented with improved nearest neighbor search and reference data sample pruning. To gain some familiarity with the classifiers, we expound the sequence of computations implicated in pattern category assignment by each classifier. A characterization of the classifiers is drawn from observed and expected properties and from experiments in automatic target recognition and optical character recognition as summarized in comparative tables of performance. This characterization supports the suggestion that nearest neighbors classification always be considered before endorsing alternative pattern classifiers such as neural networks.

Please refer to full text.

A robotic agent operating in an unknown and complex environment may employ a search strategy of some kind to perform a navigational task such as reaching a given goal. In the process of performing the task, the agent can attempt to discover characteristics of its environment that enable it to choose a more efficient search strategy for that environment. If the agent is able to do this, we can say that it has “learned to navigate” — i.e., to improve its navigational performance. This paper describes how an agent can learn to improve its goal-finding performance in a class of discrete spaces, represented by graphs embedded in the plane. We compare several basic search strategies on two different classes of “random” graphs and show how information collected during the traversal of a graph can be used to classify the graph, thus allowing the agent to choose the search strategy best suited for that graph.

In speech and image processing, many good algorithms do not perform well in noisy conditions. Recently, robust algorithms have been proposed to improve the performance and robustness of those algorithms. However, many of them are either computationally very expensive or break down quickly. A generalized maximum-likelihood-based robust algorithm called Subset Least Squares (SLS) has been proposed by Kashyap to deal with contaminated data with less complexity and higher breakdown. In this paper, we first illustrate the basic framework of SLS and the use of SLS for robust regression. We then present the use of SLS method in several image and speech applications. Finally, we discuss the possible use of SLS in future applications.

Pattern recognition and object detection systems so far developed required the algorithmic description of every detail of the objects to be recognized by bottom-up process from pixel-to-pixel relation to line, corner, and structural description. Because this low-level process does not see global information, feature detection is highly sensitive to noise. To overcome this problem and to give human-like flexibility to machine recognition process, we developed a new system which had non-algorithmic feature detection functions by seeing a comparatively large area at once. It uses a variable size window which is applied to the most plausible parts in an image by a top-down command from an object model, and obtains characteristic features of object parts. This window application is realized mostly in hardware, and has some autonomic ability to detect the best features by a sort of random trial and error search. The system has some other hardware functions such as mutual correlation of one- and two-dimensional patterns, which are also flexible according to the variable size window. The system interprets user's declarative description of objects, and activates the window application functions to obtain characteristic features of the description. This new flexible approach of object detection can be used as a robot eye to recognize many simple two-dimensional shapes.

This paper describes some issues in building a 3-D human face modeling system which mainly consists of three parts:

• Modeling human faces;

• Analyzing facial motions;

• Synthesizing facial expressions.

A variety of techniques developed for this system are described in detail in this paper. Some preliminary results of applying this system to computer animation, video sequence compression and human face recognition are also shown.

The subject matter of this paper is one of long-standing interest to the Pattern Recognition and Artificial Intelligence research communities, namely that of “feature extraction” for facilitating the task of classification or various other tasks. We show that internal representations of neural networks do not yield unique feature values but can provide the basis for facilitating a number of useful information management tasks, such as memorization, categorization, discovery, associative recall and others. These matters are illustrated with three sets of data, one of a benchmark nature, another of the nature of real-world sensor data, and a third set consisting of semiconductor crystal structure parameters.

Given a 3-D object (solid), it is possible to divide the exterior viewing space of the solid into maximally connected subspaces of vantage points such that the perspective views of the solid from all vantage points within a subspace have identical labeled line-junction graphs. We refer to such subspaces as the characteristic-view domains (CVDs) of the solid, and to a representative perspective view from within a domain, as a characteristic view (CV). The concept suggests a scheme for modeling 3-D objects in which the infinite number of possible views of a given solid are represented by a finite set of characteristic views (together with their respective projection -defining parameters).

The concept was introduced more than 15 years ago as a modeling scheme to facilitate real-time recognition of 3-D objects. However, the difficulty of determining the CVs hindered its application. This article describes a technique for computing the characteristic views of general solids bounded by quadric surfaces (including polyhedra as a special case), and suggests how the technique can be used for 3-D object recognition.

This paper describes how to propagate approximately additive random perturbations through any kind of vision algorithm step in which the appropriate random perturbation model for the estimated quantity produced by the vision step is also an additive random perturbation. We assume that the vision algorithm step can be modeled as a calculation (linear or non-linear) that produces an estimate that minimizes an implicit scaler function of the input quantity and the calculated estimate. The only assumption is that the scaler function has finite second partial derivatives and that the random perturbations are small enough so that the relationship between the scaler function evaluated at the ideal but unknown input and output quantities and the observed input quantity and perturbed output quantity can be approximated sufficiently well by a first order Taylor series expansion.

The paper finally discusses the issues of verifying that the derived statistical behavior agrees with the experimentally observed statistical behavior.

In the syntactic approach to pattern recognition, patterns are represented as strings, where each pattern is expressed as a composition of its component patterns, called sub-patterns and pattern primitives. This approach draws an analogy between the structure of patterns and the syntax of a language. The patterns are considered at a single resolution level, and the recognition of each pattern is usually made by parsing the pattern structure according to a given set of syntax rules, obtained in the first stage of the analysis. This paper describes a pyramidal approach to 2-D object representation which relies on a linguistic description of the object contour at different resolution levels. Our approach, which is parallel and context-sensitive, implies the use of higher dimensional grammars for defining production rules which describe the evolution of the contour between levels. This method is computationally efficient (due to the fast coarse-fine search), rotationally invariant, as well as rather insensitive to noise.

Piecewise-linear mathematical structures form a convenient and important framework for implementing trainable and adaptive pattern classifiers. Neural networks and genetic algorithms offer additional approaches with important benefits for the design of such classifiers. In this paper we show how neural modeling and genetic selection can be applied to piecewise-linear structures to optimize both the topology and the parameter values of the network forming the classifier. Such a classifier will tend to have a low error rate and high robustness. We describe applications of these techniques to an adaptive detector of abnormal tissue in mammograms and a detector of straight lines and edges in noisy aerial images.