Neural Networks in Pattern Recognition and Their Applications

The following sections are included:

• INTRODUCTION

• CONTENTS

The control of small-scale systems using either knowledge-based or neural net methods is quite feasible. Large scale systems, however, introduce complexities in modeling and excessive computation time. This paper attacks these difficulties by breaking down the problem into a hierarchy of control contexts. The lowest level of this hierarchy is implemented as rule sets and/or neural networks. A method using "hints" is shown to greatly reduce training time in back-propagation neural nets.

A connectionist expert system model is proposed in this paper. The system combines the capability of production systems and associative memory. Rules are explicitly represented within a network structure. The Perceptron algorithm is generalized to include samples with uncertain components. A gradually-augmented-node learning algorithm is used to guarantee fast memorization of all rules. The convergence of the learning algorithms is analyzed. The system has a dynamic knowledge organization and adapts in real time to acquire new knowledge, or to relearn existing knowledge, through interaction with the user. This allows the system to be built incrementally. An example system is presented.

The output layer of a feedforward neural network approximates nonlinear functions as a linear combination of a fixed set of basis functions, or "features". These features are learned by the hidden-layer units, often by a supervised algorithm such as a back-propagation algorithm. This paper investigates features which are optimal for computing desired output functions from a given distribution of input data, and which must therefore be learned using a mixed supervised and unsupervised algorithm. A definition is proposed for optimal nonlinear features, and a constructive method, which has an iterative implementation, is derived for finding them. The learning algorithm always converges to a global optimum and the resulting network uses two layers to compute the hidden units. The general form of the features is derived for the case of continuous signal input, and this result is related to transmission of information through a bandlimited channel. The results of other algorithms can be compared to the optimal features, which in some cases have easily computed closed-form solutions. The application of this technique to the inverse kinematics problem for a simulated planar two-joint robot arm is demonstrated here.

A two-stage design method for artificial neural networks is presented. The first stage is an evolutionary RLS (recursive least squares) algorithm which determines the optimal configuration of the net based on the concept of optimal interpolation. During this stage, the members of a given sample set are processed sequentially and a small consistent subset, constituting what we call prototypes, is selected to form the building blocks of the net. The synaptic weights as well as the internal dimensions of the net are updated recursively as each new prototype is selected. The evolving net at each intermediate step is a modified version of the Optimal Interpolative (OI) net derived in a recent paper by one of the authors. The concept of an evolving network configuration is attractive since it does not require the prescription of a fixed configuration in order to learn the optimal synaptic weights. This can eventually lead to a network architecture which is only as complex as it needs to achieve a required interpolation function.

The second stage is for the fine adjustment of the synaptic weights of the network structure acquired during the first stage. This stage is a two-step iterative optimization procedure using the method of steepest descent. The initial values of the synaptic weights in the iterative search are obtained from the first stage. It is seen that they are indeed very close to the optimal values. Hence, fast convergence during the second stage is guaranteed.

Asymptotic properties of recurrent neural networks for optimization are analyzed. Specifically, asymptotic stability of recurrent neural networks with monotonically time-varying penalty parameters for optimization is proven; sufficient conditions of feasibility and optimality of solutions generated by the recurrent neural networks are characterized. Design methodology of the recurrent neural networks for solving optimization problems is discussed. Operating characteristics of the recurrent neural networks are also presented using illustrative examples.

An image segmentation system which uses a connectionist classifier architecture as a central building block is described in this paper. The complete system, which consists of a feature extraction module and the connectionist classifier module, has been designed and implemented in digital VLSI; system architectural aspects as well as the procedure of adaptation of the system to different segmentation problems are discussed. The performance of the segmentation system on real world problems is demonstrated using scenes from industrial inspection, texture recognition, and combustion chamber research tasks.

Neural networks differ from traditional approaches to image processing in terms of their ability to adapt to regularities in image structure and to self-organize so as to implement directed transformations. Biomedical ultrasonic images are often degraded in quality by noise and other factors, making enhancement techniques particularly important. This paper describes use of back propagation and competitive learning for enhancement and segmentation of ultrasonic images of the eye. Of particular interest is the extension of these techniques to segmentation of three-dimensional data sets, where simple thresholding and gradient operations are not entirely successful.

Image binarization is a task to convert gray-level images into bi-level ones. Its underlying notion can be simply thought of as threshold selection. However, the result of binarization will cause significant influence on the process of image recognition or understanding. In this paper we discuss a new binarization method, named CMB (connectionist model binarization), which uses the connectionist model. In the method a gray-level histogram is input to a multilayer network trained with the back-propagation algorithm to obtain a threshold which gives a visually suitable binarized image. From the experimental results, it was verified that CMB is an effective binarization method in comparison with other methods.

We have spent the last several years investigating likely roles for neural network technology in the automatic active underwater sonar classifier development process, including exploration of both the improvement of in-service sonar systems and the creation of new ones. Currently, numerous classifier studies place emphasis on comparing class distinction performance figures of neural networks to those of traditional classifiers. We find that, for active sonar classifiers, this approach provides an incomplete and sometimes misleading assessment of the value of neural network technology. It is demonstrated that it is meaningful to divide the classifier development process into its phases and evaluate the influence that the neural network approach has on each phase. Conclusions are drawn on the potential impact of neural network technology on the active sonar classifier development field, which may help provide motivation for areas of future research.

Although statistical pattern recognition and artificial neural networks were initially developed independently, they are now closely related. Such relationships are examined in detail in this paper. Some experimental comparison results are also presented.