International Journal of Pattern Recognition and Artificial Intelligence

Intelligent surveillance refers to using Artificial Intelligence techniques in order to improve surveillance and deal with semantic information obtained from low-level security devices. In this context, the use of expert knowledge may offer a more realistic solution when dealing with the design of a surveillance system. In this work, a conceptual framework based on normality analysis to detect abnormal behaviors by means of normality concepts is presented. A normality concept specifies how a certain object should ideally behave in a concrete environment depending on such a concept. The definition of the normal path concept is studied in depth in order to analyze behaviors in an outdoor environment.

A novel object tracking algorithm is presented in this paper by using the joint color-texture histogram to represent a target and then applying it to the mean shift framework. Apart from the conventional color histogram features, the texture features of the object are also extracted by using the local binary pattern (LBP) technique to represent the object. The major uniform LBP patterns are exploited to form a mask for joint color-texture feature selection. Compared with the traditional color histogram based algorithms that use the whole target region for tracking, the proposed algorithm extracts effectively the edge and corner features in the target region, which characterize better and represent more robustly the target. The experimental results validate that the proposed method improves greatly the tracking accuracy and efficiency with fewer mean shift iterations than standard mean shift tracking. It can robustly track the target under complex scenes, such as similar target and background appearance, on which the traditional color based schemes may fail to track.

Traffic flow information can be employed in an intelligent transportation system to detect and manage traffic congestion. One of the key elements in determining the traffic flow information is traffic density estimation. The goal of traffic density estimation is to determine the density of vehicles on a given road from loop detectors, traffic radars, or surveillance cameras. However, due to the inflexibility of deploying loop detectors and traffic radars, there is a growing trend of using video-content-understanding technique to determine the traffic flow from a surveillance camera. But difficulties arise when attempting to do this in real-time under changing illumination and weather conditions as well as heavy traffic congestions. In this paper, we attempt to address the problem of real-time traffic density estimation by using a stochastic model called Hidden Markov Models (HMM) to probabilistically determine the traffic density state. Choosing a good set of model parameters for HMMs has a significant impact on the accuracy of traffic density estimation. Thus, we propose a novel feature extraction scheme to represent traffic density, and a novel approach to initialize and construct the HMMs by using an unsupervised clustering technique called AutoClass. We show through extensive experiments that our proposed real-time algorithm achieves an average traffic density estimation accuracy of 96.6% over various different illumination and weather conditions.

This paper presents an approach to extract semantic layers from aerial surveillance videos for scene understanding and object tracking. The input videos are captured by low flying aerial platforms and typically consist of strong parallax from non-ground-plane structures as well as moving objects. Our approach leverages the geo-registration between video frames and reference images (such as those available from Terraserver and Google satellite imagery) to establish a unique geo-spatial coordinate system for pixels in the video. The geo-registration process enables Euclidean 3D reconstruction with absolute scale unlike traditional monocular structure from motion where continuous scale estimation over long periods of time is an issue. Geo-registration also enables correlation of video data to other stored information sources such as GIS (Geo-spatial Information System) databases. In addition to the geo-registration and 3D reconstruction aspects, the other key contributions of this paper also include: (1) providing a reliable geo-based solution to estimate camera pose for 3D reconstruction, (2) exploiting appearance and 3D shape constraints derived from geo-registered videos for labeling of structures such as buildings, foliage, and roads for scene understanding, and (3) elimination of moving object detection and tracking errors using 3D parallax constraints and semantic labels derived from geo-registered videos. Experimental results on extended time aerial video data demonstrates the qualitative and quantitative aspects of our work.

In this paper, we present two approaches to recognize a special type of human action, e.g. climbing a fence, from continuous monocular videos. Both approaches represent human activities as time series of discrete observations. In the first approach, we extract a feature vector from each frame, based on the relative configuration of human body parts against a fence. In the second approach, we propose a novel concept called "stable contact", to decompose the continuous human activities into adjacent but disjoint time intervals called primitive intervals. We then define a primitive motion unit (PMU) on each primitive interval with three attributes, thus converting each continuous activity into a sequence of PMUs. With the time series representations, we utilize Hidden Markov Model (HMM) techniques for recognition, including decoding in the first approach and searching in the second approach. Experimental results, on a set of image sequences with mixed human actions including walking and climbing a fence, show the effectiveness of the proposed methods. We compare the two approaches and discuss their applicability in different scenarios.

This paper presents a thorough study of some particle filter (PF) strategies dedicated to human motion capture from a trinocular vision surveillance setup. An experimental procedure is used, based on a commercial motion capture ring to provide ground truth. Metrics are proposed to assess performances in terms of accuracy, robustness, but also estimator dispersion which is often neglected elsewhere. Relative performances are discussed through some quantitative and qualitative evaluations on a video database. PF strategies based on Quasi Monte Carlo sampling, a scheme which is surprisingly seldom exploited in the Vision community, provide an interesting way to explore. Future works are finally discussed.

We propose a matching method for images captured at different times and under different capturing conditions. Our method is designed for change detection in streetscapes using normal automobiles that have an off-the-shelf car mounted camera and a GPS. Therefore, we are able to analyze low-resolution and low frame-rate images captured asynchronously. To cope with this difficulty, previous and current panoramic images are created from sequential images which are rectified based on the view direction of a camera, and are then compared. In addition, in order to allow the matching method to be applicable to images captured under varying conditions, (1) for different lanes, enlarged/reduced panoramic images are compared with each other, and (2) robustness to noises and changes in illumination is improved by the edge features. To confirm the effectiveness of the proposed method, we conducted experiments matching real images captured under various capturing conditions.

Traditional background subtraction methods perform poorly when scenes contain dynamic backgrounds such as waving tree branches, spouting fountain, illumination changes, camera jitters, etc. In this paper, from the view of spatial context, we present a novel and effective dynamic background method with three contributions. First, we present a novel local dependency descriptor, called local dependency histogram (LDH), to effectively model the spatial dependencies between a pixel and its neighboring pixels. The spatial dependencies contain substantial evidence for differentiating dynamic background regions from moving objects of interest. Second, based on the proposed LDH, an effective approach to dynamic background subtraction is proposed, in which each pixel is modeled as a group of weighted LDHs. Labeling a pixel as foreground or background is done by comparing the LDH computed in current frame against its model LDHs. The model LDHs are adaptively updated by the current LDH. Finally, unlike traditional approaches using a fixed threshold to judge whether a pixel matches to its model, an adaptive thresholding technique is also proposed. Experimental results on a diverse set of dynamic scenes validate that the proposed method significantly outperforms traditional methods for dynamic background subtraction.

This paper presents an efficient algorithm to track the object contour in image sequences. Firstly, the probability distributions of color feature and texture feature are approximated by different schemes. Secondly, the object and the background pixels are distinguished by calculating their posterior probability in light of Bayesian formula based on the two models. Finally, an energy functional is proposed and solved by gradient descent flow to evolve the initial contour which is estimated by motion feature, so that minimum energy contour corresponds to the object contour. In order to save the computational resource, a new texture descriptor is applied to analyze pixels merely in the neighborhood of a contour rather than the whole image. Furthermore, the algorithm's implementation based on the narrow band level set scheme can help to reduce the processing time further. In some experiments, the contours of various objects such as human, car and body organ are successfully tracked by our algorithm and the processing time is short enough for practical application.

In this paper, we propose an approach for surveillance video indexing and retrieval. The objective of this approach is to answer five main challenges we have met in this domain: (1) the lack of means for finding data from the indexed databases, (2) the lack of approaches working at different abstraction levels, (3) imprecise indexing, (4) incomplete indexing, (5) the lack of user-centered search. We propose a new data model containing two main types of extracted video contents: physical objects and events. Based on this data model, we present a new rich and flexible query language. This language works at different abstraction levels, provides both exact and approximate matching and takes into account users' interest. In order to work with the imprecise indexing, two new methods respectively for object representation and object matching are proposed. Videos from two projects which have been partially indexed are used to validate the proposed approach. We have analyzed both query language usage and retrieval results. The obtained retrieval results analyzed by the average normalized ranks are promising. The retrieval results at the object level are compared with another state of the art approach.

Visual surveillance in dynamic scenes is currently one of the most active research topics in computer vision, many existing applications are available. However, difficulties in realizing effective video surveillance systems that are robust to the many different conditions that arise in real environments, make the actual deployment of such systems very challenging.

In this article, we present a real, unique and pioneer video surveillance system for boat traffic monitoring, ARGOS. The system runs continuously 24 hours a day, 7 days a week, day and night in the city of Venice (Italy) since 2007 and it is able to build a reliable background model of the water channel and to track the boats navigating the channel with good accuracy in real-time. A significant experimental evaluation, reported in this article, has been performed in order to assess the real performance of the system.

This paper proposes an efficient method for detecting ghost and left objects in surveillance video, which, if not identified, may lead to errors or wasted computational power in background modeling and object tracking in video surveillance systems. This method contains two main steps: the first one is to detect stationary objects, which narrows down the evaluation targets to a very small number of regions in the input image; the second step is to discriminate the candidates between ghost and left objects. For the first step, we introduce a novel stationary object detection method based on continuous object tracking and shape matching. For the second step, we propose a fast and robust inpainting method to differentiate between ghost and left objects by reconstructing the real background using the candidate's corresponding regions in the current input and background image. The effectiveness of our method has been validated by experiments over a variety of video sequences and comparisons with existing state-of-art methods.