International Journal of Semantic Computing

An efficient access and indexing framework, called Affinity Hybrid Tree (AH-Tree), is proposed which combines feature and metric spaces in a novel way. The proposed framework helps to organize large image databases and support popular multimedia retrieval mechanisms like Content-Based Image Retrieval (CBIR). It is efficient in terms of computational overhead and fairly accurate in producing query results close to human perception. AH-Tree, by being able to introduce the high level semantic image relationship as it is in its index structure, solves the problem of translating the content-similarity measurement into feature level equivalence which is both painstaking and error-prone. Algorithms for similarity (range and k-nearest neighbor) queries are implemented and extensive experiments are performed which produces encouraging results with low I/O and distance computations and high precision of query results.

This paper describes an approach for creating high-resolution video panoramas of large-scale sport events. In an exemplary football scenario, we used two ARRIFLEX D-20 "film-style" digital cameras to capture both sides of the playing field as well as large parts of the stands. From the recorded left and right view images, we created a joint panoramic view with a resolution of 5016 × 1400 pel (5k) using sophisticated image processing algorithms. These highly immersive football panoramas were screened with our modular, high-resolution multi-projection system during the FIFA World Cup 2006 in a 600 seat CinemaxX movie theater in Berlin, providing the viewers with the feeling of actually watching the game from a good seat on the stands.

Retrieving information about the occurrences of persons in a video is an important task in many video indexing and retrieval applications. The problem is to answer the question "In which shots and scenes does person X appear?". In this paper, we present an automatic video annotation system with respect to a person's appearance based on state-of-the-art algorithms for face detection, tracking and recognition. In contrast to many related approaches, knowledge about the persons in a given video is not assumed in advance. Adaboost is employed after an initial clustering of faces to select the best features describing a person's face. These features are then used to train new classifiers based only on the faces extracted from the video under consideration. Several possibilities to train Adaboost and Support Vector Machine (ensemble) classifiers directly on a video are compared. Finally, experimental results demonstrate the effectiveness of correcting in-plane face rotation and of the employed self-supervised learning method.

In this paper, a framework of automatic human action segmentation and recognition in continuous action sequences is proposed. A star figure enclosed by a bounding convex polygon is used to effectively represent the extremities of the silhouette of a human body. The human action, thus, is recorded as a sequence of the star-figure's parameters, which is used for action modeling. To model human actions in a compact manner while characterizing their spatio-temporal distributions, the star-figure's parameters are represented by Gaussian mixture models (GMM). In addition, to address the intrinsic nature of temporal variations in a continuous action sequence, we transform the time sequence of star-like figure parameters into frequency domain by discrete cosine transform (DCT) and use only the first few coefficients to represent different temporal patterns with significant discriminating power. The performance shows that the proposed framework can recognize continuous human actions in an efficient way.

Most image and video editing applications implement only a set of low-level operations, such as linear and non-linear filters, scaling, and simple rectangular cropping. A skilled user is required to combine these traditional tools creatively to perform a useful task. This article presents an example of an approach that takes into account the semantics of the image and is therefore more powerful than traditional tools. Using a model of the foreground and the background of a given image, the SIOX tool (Simple Interactive Object Extraction) allows a user to cut out a certain object from an image or video with very little user interaction. The core of the algorithm is a color clustering approach that has been derived from image retrieval techniques and evaluated against a benchmark proposed by Microsoft Research. The method allows for sub-pixel accuracy, allows video segmentation in realtime (640 × 480 × 25), and is noise robust. SIOX has recently been integrated into the development versions of several open source image and video editing applications such as GIMP, Blender, and Inkscape.

Semantic computing addresses the transformation of data, both structured and unstructured into information that is useful in application domains. One domain where semantic computing would be extremely effective is evacuation route planning, an area of critical importance in disaster emergency management and homeland defense preparation. Evacuation route planning, which identifies paths in a given transportation network to minimize the time needed to move vulnerable populations to safe destinations, is computationally challenging because the number of evacuees often far exceeds the capacity, i.e. the number of people that can move along the road segments in a unit time. A semantic computing framework would help further the design and development of effective tools in this domain, by providing a better understanding of the underlying data and its interactions with various design techniques. Traditional Linear Programming(LP) based methods using time expanded networks can take hours to days of computation for metropolitan sized problems. In this paper, we propose a new approach, namely a capacity constrained routing planner for evacuation route planning which models capacity as a time series and generalizes shortest path algorithms to incorporate capacity constraints. We describe the building blocks and discuss the implementation of the system. Analytical and experimental evaluations that compare the performance of the proposed system with existing route planners show that the capacity constrained route planner produces solutions that are comparable to those produced by LP based algorithms while significantly reducing the computational cost.