Neural Networks for Intelligent Signal Processing

The following sections are included:

• Dedication

• Acknowledgments

• Foreword

• Preface

• Contents

The following sections are included:

• Motivation for ANNs

• ANN Definitions and Main Types

• Specific ANN Models

• ANN Black Box Model

• ANN Implementation

• When To Use an ANN

• How To Use an ANN

• General Applications

• Pattern Recognition Examples
  • Sheep Eating Phase Identification from Jaw Sounds
  • Particle Isolation in SEM Images
  • Oxalate Needle Detection in Microscope Images

• Function Mapping and Filtering Examples
  • Water Level from Resonant Sound Analysis
  • Nonlinear Signal Filtering

• Motor Control Example

• ANN Summary

• References

The following sections are included:

• ANN History to 1970
  • Key Events prior to 1970

• ANN History after 1970
  • Key Events after 1970 to the Mid 1980s
  • Developments after the Mid 1980s
  • Nonparametric Learning From Finite Data

• Reasons for the Resurgence of Interest in ANNs

• Historical Summary

• References

The following sections are included:

• The Basic Model of the Neuron

• Activation Functions

• Topologies

• Learning
  • A Basic Supervised Learning Algorithm
  • A Basic Unsupervised Learning Algorithm

• The Basic McCulloch Pitts and Perceptron Models

• Vectors Spaces and Matrix Models
  • ANN Classifiers
  • Vectors and Feature Spaces
  • Representation of Multivariate Data

• Basic Structure of a Neural Network

• Basic ANN Operations in terms of Matrices

• Why Use Matrices in ANNs?
  • Subspace
  • Multiplication of Matrices and Vectors
  • Line Subspace Example
  • The XOR Problem

• References

The following sections are included:

• Confusion Matrix

• Error Measures including the Square Error
  • The Significance of Second Order Statistics

• Receiver-Operating-Characteristic (ROC)

• Chi-Squared Goodness of Fit

• References

The following sections are included:

• Data Pre-processing
  • Input Scaling and Normalisation
  • Feature Extraction

• Feature Measurement Types
  • Nominal Feature Variables
  • Ordinal Feature Variables
  • Interval Feature Variables
  • Ratio Feature Variables
  • 2-D Shape Feature Example

• Classification
  • K-Nearest Neighbour Classifier
  • Distance Measures in a Feature Space
  • Statistical Classifiers
  • ANN Classifiers

• Decision Criteria and Output Thresholds

• Design Procedure for an ANN Classifier
  • A Simple Classification Example

• Principle Component Analysis (PCA)

• References

The following sections are included:

• Adaptive Linear Combiner and ADALINE
  • Derivation of the LMS Algorithm
  • The ADALINE and MADALINE
  • Channel Equalisation Example
  • ADALINE Summary

• General MLP Networks
  • A Detailed Three-Layer MLP Model
  • Derivation of Backpropagation-of-error Learning
  • An Illustrative Worked Example
  • MLP Application Notes
  • Multi-Layer Perceptron Construction
  • MLP Summary

• References

The following sections are included:

• PNN Theory

• Bayes' Decision Strategy

• PDF Estimators and Radial Basis Functions

• PNN Architecture
  • Relation of the Dot Product to Radial Distance

• Features and Application Issues

• Applications of the PNN

• Gong Classification Application Example
  • Gong Description
  • Data Sampling
  • Primary Analysis and Feature Selection
  • PNN Training and Classification Performance

• Particle Isolation Application Example

• FPGA PNN Design
  • Proposed Design
  • Test Results
  • Test Conclusions

• References

The following sections are included:

• The Bayes Theorem and Regression Theory

• The General Regression Neural Network

• Short Wave Signal Filtering Application Example

• References

The following sections are included:

• MPNN Theory
  • MPNN Method A
  • MPNN Method B
  • Other MPNN Network Construction Methods
  • Automated Sample Size Reduction

• Other MPNN Characteristics
  • Noise Variance in Training and Network Size

• MPNN and GRNN Adaptation and Learning Scheme

• Signal Processing Application Examples

• MPNN Hardware Implementation Schemes
  • Optoelectronic Implementation
  • VLSI Implementation
  • A Virtual Digital VLSI Hardware Design
  • A Parallel VLSI Hardware Design

• MPNN Summary
  • Relationship to GRNN

• References

The following sections are included:

• A Tuneable Approximate Piecewise Linear Model
  • The New Model
  • Example Results
  • Discussion

• Integrated Sensory Intelligent System (ISIS)
  • Integrated Sensory Intelligent System Model
  • Doppler Shifted Chirp Detection
  • Test Results and Analysis of Initial Methods
  • Test Results and Analysis for ISIS Method

• Future Directions for the MPNN
  • Hyperspace Signal Processing
  • Improvements to the MPNN
  • Other Engineering Applications for the MPNN
  • The MPNN as a Building Block to a General Parallel Computer

• References

The following sections are included:

• Radial Basis Function Neural Network
  • Vector Quantisation and K-means Clustering
  • Least Squares Estimation

• Cerebellar Model Articulation Controller
  • Theory of the CMAC Network
  • CMAC Network Architecture
  • CMAC Network Operation
  • CMAC Applications

• References

The following sections are included:

• Kohonen's Self-Organising Map
  • Kohonen's Training Algorithm
  • Kohonen's Recognition Algorithm
  • A Character Clustering Example
  • Another Simple Example

• Adaptive Resonance Theory
  • ART1 and ART2 Network Characteristics
  • ART1 and ART2 Network Operation
  • The ART Algorithm
  • Theory of the ART1 and ART2 Networks
  • ART Applications

• References

The following sections are included:

• Hopfield Neural Network
  • Hopfield Network Characteristics
  • Hopfield Network Operation
  • Hopfield Network Equations
  • Theory of the Hopfield Network
  • Hopfield Network Applications

• Boltzmann Machine
  • Simulated Annealing
  • Boltzmann Network Characteristics
  • Boltzmann Network Operation
  • Theory of the Boltzmann Network
  • Boltzmann Machine Applications

• Bidirectional Associative Memory
  • BAM Network Characteristics
  • BAM Network Operation
  • BAM Network Equations
  • Theory of the BAM Network
  • BAM Network Applications

• Neocognitron
  • Neocognitron Network Structure
  • Neocognitron Network Equations
  • Neocognitron Network Training
  • Neocognitron Applications

• References

The following sections are included:

• Learning and Regularisation
  • Dimensionality Problems in Learning
  • Learning Functions and Complexity
  • Approaches to Complexity Control

• Vapnik's Statistical Learning Theory
  • Consistency and Convergence of ERM
  • VC-Dimension
  • Structural Risk Minimisation

• Support Vector Learning Machines
  • Fundamental Ideas
  • Hyperplane for Optimal Linear Separability
  • Optimal Hyperplane for Nonseparable Data
  • Pattern Recognition SVM Design
  • Summary of SVM Learning Method

• References

The following sections are included:

• Estimation or Approximation Theory

• General Signal Processing Model

• Static ANN Models

• Dynamic ANN Models

• Application and Signal Pre-processing Issues

• Signal Processing Examples

• MPNN Comparisons with Other Important ANNs

• References

The following sections are included:

• Utility of ANNs for Control

• Generic Approaches for Controller Design
  • Hybrid Controllers
  • ANN Controllers

• Neural Control Principles

• A Fast Adaptive Neural Network System
  • An Illustrative Example
  • Discussion

• References

The following sections are included:

• ANNs for Intelligent Engineering Systems

• Signal Processing

• Possible Generic Approaches

• References

The following sections are included:

• Appendix

• Subject Index