Vision Models for Target Detection and Recognition

The following sections are included:

• IN MEMORY OF ARTHUR R. MENENDEZ (1953–1992)

• PREFACE

• ACKNOWLEDGMENTS

• CONTENTS

Psychophysical research has resulted in the development of a quantitative model of human spatial vision that accurately predicts both threshold detection and discrimination of localized visual patterns in the fovea. This model incorporates both oriented filters and response nonlinearities characteristic of cortical simple cells. With appropriate modifications, this model is applicable to peripheral vision as well. Extension of such models to include nonlinear (non-Fourier) processing enables them to predict aspects of texture discrimination and illusory contour perception as well.

Vision can be understood as the application of decision processes to the visual representation of a stimulus. This contention implies at least two processing stages: the creation of the visual representation (early visual processing) and the subsequent use of that representation in order to achieve visual tasks (late visual processing). By assuming that the visual system comprises a number of linear visual channels, we show that early visual processing results in the creation of a joint spatial/spatial-frequency representation whose characteristics are determined by the shape of the channel transfer functions. We specifically consider two plausible choices for these transfer functions which respectively give rise to a windowed Fourier representation and a wavelet representation. Subsequent processing of each type of representation for the achievement of the late visual processing tasks of detection and identification shows that empirical data on performance in these tasks are consistent with the wavelet representation.

Simulation may provide insights into the factors that control the appearance of images to normal and visually impaired observers. The linear spectral methods that have been useful in interpreting low contrast, threshold phenomena with periodic stimuli are inappropriate for analyzing the local contrast in images. Linear simulations commonly use the normalized contrast sensitivity function (CSF) as a modulation transfer function (MTF) applied to the image amplitude rather than to contrast. Simulations within the linear model are limited by a number of problems discussed here. To achieve a valid simulation, a measure of local band-limited contrast in images is needed. Such a measure was developed and was used to simulate the threshold nonlinear characteristics of the visual system and suprathreshold contrast constancy. This approach was used to simulate both central vision and vision with peripheral retina. In the latter, the simulations were based on CSF measurements across the retina. Methods to test the validity of these simulations were implemented. The results of preliminary testing confirm our ability to evaluate important parameters of the model and point to previously unnoted image dependent characteristics of visual perception.

Experiments to determine the apparent contrast of suprathreshold stimuli give a different picture of the gains and response pooling properties of tuned spatial filters than experiments designed to measure contrast thresholds. A model is derived that accounts for these differences and simulates both detection thresholds and suprathreshold contrast perception for stimuli composed of multiple sinewave components. The filters and contrast transducer functions are also used to explore possible models for human image sharpness estimates.

Vision is based upon detecting and identifying differences in the radiance spectra arriving at the eye or camera from different directions. Radiance spectra can differ in total radiance (intensity), in shape (color), or both. Ideal-observer theory was used to measure the relative amounts of intensity and chromatic information available to discriminate pairs of natural radiance spectra created by illuminating natural surfaces (Krinov, 1947) with a natural daylight source (D65). Two ideal observers were evaluated -- one operating at the input to the eye or camera, and one at the level of the photopigments in the human cone photoreceptors. The relative amount of intensity and chromatic information in each discrimination pair was quantified by the effective increase in contrast due to the chromatic information. The analyses showed that chromatic information increases the effective contrast by an average of 30% to 70% at the input to the eye, but only by 5% to 10% at the level of the photopigments. It appears that much of the chromatic information in natural radiance spectra is lost between the cornea and photopigments, and that the chromatic information available at the level of the photopigments, for the purpose of discriminating between regions or detecting region boundaries, is usually small relative to intensity information.

The ORACLE model has been developed to represent as many features as possible of human visual processing in a simplified mathematical form, and can be applied to practical problems such as those involved with the prediction of visual task performance when using optical and electro-optical aids. As knowledge about specific processes of human vision grows, attempts are made to update the appropriate parts of the model. This chapter describes some of the component parts of the model as they are currently represented. It describes the treatment of detection and recognition of image components as a function of scene parameters such as background luminance. The initial calculation is of the 50% detection threshold for a particular target. However the model exploits aspects of observer variance to generate a performance prediction for a particular task and contrast relative to the detection threshold contrast for that target. The statistical variance for a population of observers is assumed to be a constant ratio relative to the threshold (50% probability of detection) value and enables probabilities of acquisition to be calculated for any target position in the visual field(the visual lobe). Acquisition probabilities for successive fixations within a field of view can be determined from consideration of the common area of the visual lobe and the search field, but adequate modelling of search is only achieved by acknowledging the time course of observer variation, and the use of 'soft shell' visual lobes.

The chapters in this book are based, to a large extent, on presentations made at the Armstrong Laboratory Advisory Group Conference "Applied Spatial Models for Target Detection and Recognition" organized by Dr. Arthur Menendez in May 1991. The proceedings of the conference were recorded and transcribed. This chapter is, with minor changes, the transcript of Dr. van Meeteren's presentation at the conference. Presenting his work in this format is intended to preserve for the reader the free spirit and uninhibited exchange of ideas that was fostered by Dr. Menendez at this meeting. This spirit of scientific camaraderie was the reason and cause for the publication of this book.

This chapter discusses insights into the complex activity of verifying and improving the accuracy of target acquisition models for predicting range performance with thermal imaging devices. Historically, range performance predictions for electro-optical sensors have used an equivalent bar pattern methodology developed at the U. S. Army Night Vision Laboratory in 1959. FLIR92 and ACQUIRE, current models for thermal imagers, continue in this tradition, but are being continuously improved and applied to changing system designs and operational requirements. In redefining parameters of the thermal range performance prediction models to support systems under development, we have chosen an experimental approach employing appropriate users as subject observers. Field tests and laboratory perception experiments using a variety of thermal imagery generation techniques have given us experiences in the improvement and testing of these models. In addition, other organizations have performed experiments with thermal imagers and asked for our participation either before, during or after having conducted a field test to evaluate the applicability of range predictions to their specific sensors and scenarios. All of these experiments are the subject of tbe pages which follow. While no experiment is perfect, and field tests especially are Subject to many uncontrollable circumstances, much can be learned from the errors as well as tbe successes. Those running field tests are often not versed in experimental design and analysis and may find this present effort helpful when confronted with a question to be answered by means of testing.

Human detection and identification thresholds were measured using real-world targets to test Wilson's¹ spatial vision model. The Wilson¹ model was installed on an image-processing system along with a digitized image of a B-1B aircraft target (i.e., airplane image target (AIT). The computerized version of the Wilson¹ model was used to filter the AIT and determine the magnitudes of the responses of the individual basis filters within the model to the AIT. Two targets were generated by combining the response outputs from three basis filters contained within the computer-implemented Wilson¹ model. The first filter target (FT1) was generated from the combined outputs of the three basis filters that had highest response magnitudes to the AIT. The second filter target (FT2) was created from the response outputs of three basis filters whose response magnitudes to AIT were at one-half of the overall maximum level of all of the filters contained within the computer-implemented Wilson¹ model. To obtain detection and identification thresholds for three subjects. AIT and FT1 were presented on a homogeneous photopic background and photopic backgrounds containing either static or dynantic Gaussian "white" noise. Detection and identification thresholds measured for AIT and FT2 only used the homogeneous background. The mean detection and identification thresholds for AIT and the two filter-generated targets were compared to determine if the information contained within the filters' outputs is that which is required for detection and identification of the AIT. Mean AIT and FT1 detection and identification thresholds were only statistically significantly different when measured using dynantic Gaussian noise backgrounds. Mean AIT detection and identification thresholds were statistically significantly different from mean FT2 thresholds. The results indicate that applications of Wilson's¹ model can be used to predict the spatial information contain within real-world static targets that is required for detection and identification. Dynantic noise results provided information about the temporal sampling rate of the human visual system.0

This report describes the structure, performance, and some applications of the Sarnoff Visual Discrimination Model. The model has been designed for physiological plausibility as well as speed and simplicity of operation, and has been shown to be highly accurate in predicting performance in a large number of visual discrimination and image quality rating tasks. The model can be used to assess the visual effects of variations in imaging system parameters, and is thus well-suited for applications in the design and evaluation of imaging systems and their components.

This chapter describes a novel method for template matching based on expanding the image with respect to basis functions that are shifted versions of the template. The coefficients of such a non-orthogonal expansion directly signify the presence of the template at the corresponding location in the image. This method, known as Expansion Matching (EXM), optimizes a new similarity measure called the Discriminative Signal-to-Noise Ratio (DSNR). The DSNR is a more practical criterion for matching tasks than the conventional SNR, since it penalizes all off-center responses, including off-center responses to the pattern itself. Thus, in comparison to the widely used correlation or matched filtering approach, EXM yields significantly better matching results with sharp peaks and minimal off-center response, even in conditions of noise, superposition and occlusion. EXM is shown to be equivalent to minimal squared error restoration via Wiener filtering, thus offering an efficient method for computing the expansion coefficients. EXM can also be implemented by a set of neural lattices. These parallel architectures are capable of multiple- scale Gaussian smoothing of multi-dimensional signals, and are used in an adaptive configuration to expand the image into template-similar basis functions. EXM is also extended for the simultaneous recognition of multiple templates using a single filter. Here, a single filter is designed to elicit user-defined responses from a given set of templates, while optimizing the DSNR criterion.

Most of our knowledge about human visual performance describes foveal vision with a well centered pupil. However, these studies of "on-axis" vision do not provide much insight into the effects of moving the retinal image or the pupil off-axis. In this chapter we define the inherent optical and neural axes of the human eye, and we examine the optical and visual effects of decentering either the retinal image or the entrance pupil. The optical axis of the eye is usually well centered on the eye's neural axis, which provides best image quality at the fovea. Decentration of the eye's entrance pupil can occur when using any visually-coupled electro-optical device with a defined exit pupil. This decentration can lead to reduced light sensitivity, reduced spatial resolution, reduced contrast sensitivity, color distortions, unpredictable changes in refractive error, wavelength-specific distortions of visual direction and wavelength-specific errors in the judgment of apparent depth and size. We then examine associated problem of a centered pupil and an off-axis object and conclude with a simple computational model that accounts for the majority of the effects on contrast sensitivity and spatial resolution.

In Section 1 of this chapter we review the existing models of peripheral vision, most of which are based on the appealing conception that peripheral vision is just a spatially scaled version of central vision. We argue against scaling models, however, on the grounds that they cannot account for the primary factor which limits resolving power in the periphery: neural undersampling of the optical image formed on the retina. In order to account for sampling effects, we adopt an engineering perspective to develop in Section 2 a simple model of optical and neural processing of the retinal image. In Section 3, we apply our neuro-optical model to human eyes in order to discover the relative importance of the optical and neural limits to pattern detection and resolution. The results show that, although the optical system of the eye is the dominant factor limiting central vision, spatial undersampling by the optic nerve cells of the retina limits resolution in the periphery. Neural undersampling of the retinal image leads to perceptual aliasing and spurious detection of patterns up to an order of magnitude finer than the Nyquist limit. Aliasing is curtailed at very high frequencies by a combination of optical filtering and spatial summation over the finite aperture of cone photoreceptors. We conclude with some comments on practical applications of the model.

Effective models of target acquisition must address issues of detecting targets against spatially structured backgrounds. This research focused on how acquisition performance is affected by backgrounds of varying complexity, target contrast, and motion. A metric was devised that allowed quantitative control of target contrast independently of the background. Observers searched for an aircraft embedded in backgrounds of differing complexity. The results showed that acquisition time increased as background complexity increased and target contrast decreased. Target motion facilitated acquisition performance. Although contrast had a systematic effect on acquisition performance, it was clear that factors other than contrast contributed to target visibility. Other factors that may have affected visibility included distracters and camouflaging effects introduced by the structured backgrounds.

The Georgia Tech Vision (GTV) model is a simulation of human visual search and detection based on research and models from the field of computational vision, including texture segregation and pop-out. The model is consistent with recent psychophysical and neurophysiological research on low-level visual processes. The model also incorporates findings from research on visual search, selective attention, and signal detection theory. A learning algorithm is used to simulate the effects of extended training on pop-out and visual search performance. The structure and algorithms of the model are described and the results of a validation test with human observers are presented.