Narrow Results

In real-life scenarios, Human Activity Recognition (HAR) from video data is prone to occlusion of one or more body parts of the human subjects involved. Although it is common sense that the recognition of the majority of activities strongly depends on the motion of some body parts, which when occluded compromise the performance of recognition approaches, this problem is often underestimated in contemporary research works. Currently, training and evaluation is based on datasets that have been shot under laboratory (ideal) conditions, i.e. without any kind of occlusion. In this work, we propose an approach for HAR in the presence of partial occlusion, in cases wherein up to two body parts are involved. We assume that human motion is modeled using a set of 3D skeletal joints and also that occluded body parts remain occluded during the whole duration of the activity. We solve this problem using regression, performed by a novel deep Convolutional Recurrent Neural Network (CRNN). Specifically, given a partially occluded skeleton, we attempt to reconstruct the missing information regarding the motion of its occluded part(s). We evaluate our approach using four publicly available human motion datasets. Our experimental results indicate a significant increase of performance, when compared to baseline approaches, wherein networks that have been trained using only nonoccluded or both occluded and nonoccluded samples are evaluated using occluded samples. To the best of our knowledge, this is the first research work that formulates and copes with the problem of HAR under occlusion as a regression task.

A partially occluded scene in an image consists of a number of objects that are partially obstructed by others. By validating a partially occluded image one aims to generate a sequence of concatenated and possibly overlapping objects that corresponds to the input image.

This is a theoretical study of partially occluded strings (considered as one-dimensional images) allowing for the presence of errors in each occluded object appearing in the input. Using the unit cost edit distance as our measure of errors, for some small integer k ≥ 0, we present a sequential algorithm for validating a k-approximate one-dimensional image x of length n over a dictionary of m objects each having equal length τ in O(nd) time where d = mτ is the size of the dictionary.

Stereo matching is the central problem of stereovision paradigm. Area-based techniques provide the dense disparity maps and hence they are preferred for stereo correspondence. Normalized cross correlation (NCC), sum of squared differences (SSD) and sum of absolute differences (SAD) are the linear correlation measures generally used in the area-based techniques for stereo matching. In this paper, similarity measure for stereo matching based on fuzzy relations is used to establish the correspondence in the presence of intensity variations in stereo images. The strength of relationship of fuzzified data of two windows in the left image and the right image of stereo image pair is determined by considering the appropriate fuzzy aggregation operators. However, these measures fail to establish correspondence of the pixels in the stereo images in the presence of occluded pixels in the corresponding windows. Another stereo matching algorithm based on fuzzy relations of fuzzy data is used for stereo matching in such regions of images. This algorithm is based on weighted normalized cross correlation (WNCC) of the intensity data in the left and the right windows of stereo image pair. The properties of the similarity measures used in these algorithms are also discussed. Experiments with various real stereo images prove the superiority of these algorithms over normalized cross correlation (NCC) under nonideal conditions.

In stereo research the construction of a dense disparity map is a complicated task when the scene contains a lot of occlusions. In this case in the neighborhood of occlusions, we could consider that the images have a non-stationary behavior. In this paper we propose a new method for computing a dense disparity map using the decomposition of the 2D wavelet transform in four quarters allowing us to find a corresponding pixel in the case of occlusion as well. Our algorithm constructs in each pixel of two images four estimators corresponding to each quarter. The matching of the four wavelet coefficient estimators in the right image with the other four in the left image allow us to construct a dense map disparity map in each pixel of an image.

This paper proposes the prediction based occluded multitarget tracking method using spatio-temporal attention mechanism. To cope with occlusion between targets, the proposed method provides an efficient method for more complex analysis by combining object association with partial probability model in spatially attentive window and occlusion activity detection in predicted temporal location. While multiple objects are moving or occluding between them in areas of visual field, a simultaneous tracking of multiple objects tends to fail. This is due to the fact that incompletely estimated feature vectors such as location, color, velocity, and acceleration of a target can provide only ambiguous and missing information. Thus, the spatially and temporally considered mechanism is proposed to track each target before, during, and after occlusion. Robustness of the proposed method is demonstrated with representative simulations.

The paper describes an integrated recognition-by-parts architecture for reliable and robust face recognition. Reliability and robustness are characteristic of the ability to deploy full-fledged and operational biometric engines, and handling adverse image conditions that include among others uncooperative subjects, occlusion, and temporal variability, respectively. The architecture proposed is model-free and non-parametric. The conceptual framework draws support from discriminative methods using likelihood ratios. At the conceptual level it links forensics and biometrics, while at the implementation level it links the Bayesian framework and statistical learning theory (SLT). Layered categorization starts with face detection using implicit rather than explicit segmentation. It proceeds with face authentication that involves feature selection of local patch instances including dimensionality reduction, exemplar-based clustering of patches into parts, and data fusion for matching using boosting driven by parts that play the role of weak-learners. Face authentication shares the same implementation with face detection. The implementation, driven by transduction, employs proximity and typicality (ranking) realized using strangeness and p-values, respectively. The feasibility and reliability of the proposed architecture are illustrated using FRGC data. The paper concludes with suggestions for augmenting and enhancing the scope and utility of the proposed architecture.

The appearance model is very effective in tracking multiple persons. The main difficulty in tracking persons is to represent appearance reliably and effectively, especially in the presence of occlusions. In this paper, an effective Attributed Relational Graph (ARG) based tracking algorithm is presented to track multiple persons even under occlusions. The appearance of each person is expressed by an ARG model which not only combines color feature with spatial information but also illustrates the relations among body parts. The similarity of ARG models is computed to build a matching matrix in consecutive frames. Four tracking situations are determined according to the matching matrix. In addition, to track persons under occlusions, probabilistic relaxation labeling in the ARG models of body parts is deduced to label occluded persons optimally. Experimental validation of the proposed tracking method is verified and presented on indoor and outdoor sequences.

In this paper, we consider the problem of robust face recognition using color information. In this context, sparse representation-based algorithms are the state-of-the-art solutions for gray facial images. We will integrate the existing sparse representation-based algorithms with color information and this integration can improve the previous performances significantly. Furthermore, we propose a new performance metric, namely the discriminativeness (DIS) to describe the recognition effectiveness for sparse representation algorithms. We find out that the richer information in color space can be used to increase the DIS, i.e. enhancing the robustness in face recognition. Extensive experiments have been conducted under different conditions, including various feature extractors, random pixel corruptions and occlusions on AR and GT databases, to demonstrate the advantages of using color information in robust face recognition. Detailed analysis is also included for each experiment to explain why and how color improve the robustness of different sparse representation-based methods.

The manifold learning technique is a class of machine learning techniques that converts the intrinsic geometry of the data from higher to lower dimensional representation. The manifold learning technique in image analysis is used to view as a single point in a very high-dimensional space, a set of such points for population of images that may be well represented by a sub-manifold of the space that is likely to be nonlinear and of significantly lower dimensions. In this paper, we presented a new method to generate optimal random fields for image segmentation using local linear embedding manifold learning technique. This method gives better segmentation results compared to entropy classification and occlusion reasoning techniques.

Industrial vision systems should be capable of recognising noisy objects, partially occluded objects and randomly located and/or oriented objects. This paper considers the problem of recognition of partially occluded planar shapes using contour segment-based features. None of the techniques suggested in the literature for solving the above problem guarantee reliable results for problem instances which require memory in excess of what is available. In this paper, a heuristic search-based recognition algorithm is presented, which guarantees reliable recognition results even when memory is limited. This algorithm identifies an object, the maximum portion of whose contour is visible in a conglomerate of objects. For increasing efficiency of the method, a two-stage recognition scheme has been designed. In the first phase, a relevant subset of the known model shapes is chosen and in the second stage, matching between the unknown shape and elements of the relevant subset is attempted using the above approach. The technique is general in the sense that it can be used with any kind of contour features. To evaluate the efficiency of the method, experimentation was carried out using polygonal approximations of the object contours. Results are cited for establishing the effectiveness of the approach.

The problem of occlusion in a two-dimensional scene introduces errors into many existing vision algorithms that cannot be resolved. Occlusion occurs where two or more objects in a given image touch or overlap one another. Since occlusion will be present in all but the most constrained environment, the recognition of partially occluded objects is important for industrial machine vision applications to solve real problems in the military domain and in factory automation. A new method is proposed in this paper to identify and locate objects lying on a flat surface. The method is based on a local and compact description of the objects' boundaries and a new fast recognition method involving neural networks. The merit of such approach is that it provides strong robustness for partially occluded object recognition. The method is integrated into a vision system that couples with an industrial robot arm to provide automatic picking and repositioning of partially occluded industrial parts.

The Hausdorff distance (HD) between planar sets of points is known to be an effective measure for determining the degree of resemblance between binary images. In this paper, we analyze the conventional HD measure and propose a new Robust HD (RHD) measure. The proposed RHD measure takes into account not only the location information of the edge points, but also other factors such as the total number of the edge points whose nearest neighbors are within a specified directed distance, spurious edge segments defined by a small number of points, outliers, and occlusions. Experimental results for both synthetic and real images show that the proposed RHD measure is more efficient than the conventional HD measure.

We present an automated and complete camera-based monitoring system that makes use of low-level color features to perform detection, tracking and recognition of multiple people in video sequence. Specifically, the system employs a novel coverage check-up method to segment detected foreground regions into isolated people and then localize each of them. During tracking, the appearances of people are modeled by their color histograms so that the system can keep aware of their identities and recognize them after occlusions by maximizing the joint likelihood. To make the recognition more robust against shadows or changes of background illumination, the system also incorporates a shadow removal scheme to suppress shadow effects and hence improve the quality of color histogram. The proposed system has been used to identify people who re-enter the field of view of a monitoring camera in a closed-environment. Experimental results of real video data demonstrate the efficacy of the proposed people monitoring system.

It is an important way for segmentations of CT images to extract contours of objects slice-by-slice. For such a way, an important idea is analogy. That is to say, correct the contour in current slice (current contour) according to the contour in previous slice (previous contour). The key to properly correct the current contour is the ability to recognize occlusions (or say leaking parts) in the current contour. We present a curve-based curve parameterization method to recognize occlusions. The previous contour is evolved to the current contour using line projections. In the process of evolution, the parameterization is realized, which includes two types of information for every point in the evolved contour: the arc length parameter on the previous contour, and distance moved from the initial position to the present position. Using these two parameters, we are able to recognize occlusions in the current contour. Many experiments indicate that the method can recognize all of the occlusions in a given contour. Consequently, the method is robust and can be used as a part of an algorithm to automatically extract contours for CT images.

The foreground–background separation is an essential part of any video-based surveillance system. Gaussian Mixture Models (GMM) based object segmentation method accurately segments the foreground, but it is computationally expensive. In contrast, single Gaussian-based segmentation is computationally inexpensive but inaccurate because it can not handle the variations in the background. There is a trade-off between computation efficiency and precision in the segmentation approach. From the experimental observations, the variations such as lighting variations, shadows, background motion, etc., affect only a few pixels in the frames in temporal direction. So, unaffected pixel can be modeled by single Gaussian in temporal direction while the affected pixels may need GMM to handle the variations in the background. We propose an adaptive algorithm which models pixel dynamically in terms of number of Gaussians in temporal direction. The proposed method is computationally inexpensive and precise. The flexibility in terms of number of Gaussians used to model each pixel, along with adaptive learning approach, reduces the time complexity of the algorithm significantly. To resolve spacial occlusion problem, a spatial smoothing is carried out by weighted $K_n$ nearest neighbors which improves the overall accuracy of proposed algorithm. To avoid false detection due to illumination variations and shadows in a particular image, illumination invariant representation is used.

Ramanujan Sums (RS) have been found to be very successful in signal processing recently. However, as far as we know, the RS have not been applied to image analysis. In this paper, we propose two novel algorithms for image analysis, including moment invariants and pattern recognition. Our algorithms are invariant to the translation, rotation and scaling of the 2D shapes. The RS are robust to Gaussian white noise and occlusion as well. Our algorithms compare favourably to the dual-tree complex wavelet (DTCWT) moments and the Zernike's moments in terms of correct classification rates for three well-known shape datasets.

Most of the current computational models for splice junction prediction are based on the identification of canonical splice junctions. However, it is observed that the junctions lacking the consensus dimers GT and AG also undergo splicing. Identification of such splice junctions, called the non-canonical splice junctions, is also essential for a comprehensive understanding of the splicing phenomenon. This work focuses on the identification of non-canonical splice junctions through the application of a bidirectional long short-term memory (BLSTM) network. Furthermore, we apply a back-propagation-based (integrated gradient) and a perturbation-based (occlusion) visualization techniques to extract the non-canonical splicing features learned by the model. The features obtained are validated with the existing knowledge from the literature. Integrated gradient extracts features that comprise contiguous nucleotides, whereas occlusion extracts features that are individual nucleotides distributed across the sequence.

We propose a self collision avoidance system for humanoid robots designed for interacting with the real world. It protects not only the humanoid robots' hardware but also expands its working range while keeping smooth motions. It runs in real-time in order to handle unpredictable reactive tasks such as reaching to moving targets tracked by vision during dynamic motions like e.g. biped walking.

The collision avoidance is composed of two important elements. The first element is reactive self collision avoidance which controls critical segments in only one direction — as opposed to other methods which use 3D position control. The virtual force for the collision avoidance is applied to this direction and therefore the system has more redundant degrees of freedom which can be used for other criteria. The other second element is a dynamic task prioritization scheme which blends the priority between target reaching and collision avoidance motions in a simple way. The priority between the two controllers is changed depending on current risk.

We test the algorithm on our humanoid robot ASIMO and works while the robot is standing and walking. Reaching motions from the front to the side of the body without the arm colliding with the body are possible. Even if the target is inside the body, the arm stops at the closest point to the target outside the body. The collision avoidance is working as one module of a hierarchical reactive system and realizes reactive motions. The proposed scheme can be used for other applications: We also apply it to realizing a body schema and occlusion avoidance.

This paper addresses the problem of stereo vision from a new viewpoint. The problems of both depth determination and scene segmentation are considered as a whole. Based on the facts from psychophysical observations, we propose a multi-layered representation model for stereo vision and show that the problems of both depth determination and image segmentation can be solved simultaneously. The constraints and algorithms for formulating the model are described. A mechanism of interaction between depth perception and surface completion is used to integrate imperfect disparity data and image data to get the correct solution about depth maps of a scene. The experimental results show that the model is a useful as well as an effective one.

Successful image reconstruction requires the recognition of a scene and the generation of a clean image of that scene. We propose to use recurrent neural networks for both analysis and synthesis.

The networks have a hierarchical architecture that represents images in multiple scales with different degrees of abstraction. The mapping between these representations is mediated by a local connection structure. We supply the networks with degraded images and train them to reconstruct the originals iteratively. This iterative reconstruction makes it possible to use partial results as context information to resolve ambiguities.

We demonstrate the power of the approach using three examples: superresolution, fill-in of occluded parts, and noise removal/contrast enhancement. We also reconstruct images from sequences of degraded images.