International Journal of Image and Graphics

Retrieval of color images has become an important application in recent years. We make a concise analysis of methodologies for interactive retrieval of color images. Two issues are of importance. First, the domain, which can be broad or narrow. Second, the search method, which can be object search, target search, category search or associative search. On the basis of these, we give complete guidelines for choosing and designing methods for interactive color image retrieval based on the domain and search goal characteristics.

In this paper, we propose a new framework for the dynamic construction of structured visual object/scene detectors for content-based retrieval. In the Visual Apprentice, a user defines visual object/scene models via a multiple-level Definition Hierarchy: a scene consists of objects, which consist of object-parts, which consist of perceptual-areas, which consist of regions. The user trains the system by providing example images/videos and labeling components according to the hierarchy she defines (e.g., image of two people shaking hands contains two faces and a handshake). As the user trains the system, visual features (e.g., color, texture, motion, etc.) are extracted from each example provided, for each node of the hierarchy (defined by the user). Various machine learning algorithms are then applied to the training data, at each node, to learn classifiers. The best classifiers and features are then automatically selected for each node (using cross-validation on the training data). The process yields a Visual Object/Scene Detector (e.g., for a handshake), which consists of an hierarchy of classifiers as it was defined by the user. The Visual Detector classifies new images/videos by first automatically segmenting them, and applying the classifiers according to the hierarchy: regions are classified first, followed by the classification of perceptual-areas, object-parts and objects. We discuss how the concept of Recurrent Visual Semantics can be used to identify domains in which learning techniques such as the one presented can be applied. We then present experimental results using several hierarchies for classifying images and video shots (e.g., Baseball video, images that contain handhakes, skies, etc.). These results, which show good performance, demonstrate the feasibility and usefulness of dynamic approaches for constructing structured visual object/scene detectors from user input at multiple levels.

In this paper, we present major issues in video parsing, abstraction, retrieval and semantic analysis. We discuss the success, the difficulties and the expectations in these areas. In addition, we identify important opened problems that can lead to more sophisticated ways of video content analysis.

For video parsing, we discuss topics in video partitioning, motion characterization and object segmentation. The success in video parsing, in general, will have a great impact on video representation and retrieval. We present three levels of abstracting video content by scene, keyframe and key object representations. These representation schemes in overall serve as a good start for video retrieval. We then describe visual features, in particular motion, and similarity measures adopted for retrieval. Next, we discuss the recent computational approaches in bridging the semantic gap for video content understanding.

A large number of shot boundary detection, or equivalently, transition detection techniques have been developed in recent years. They all can be classified based on a few core concepts underlying the different detection schemes. This survey emphasizes those different core concepts underlying the different detection schemes for the three most widely used video transition effects: hard cuts, fades and dissolves. Representative of each concept one or a few very sound and thoroughly tested approaches are present in detail, while others are just listed. Whenever reliable performance numbers could be found in the literature, they are mentioned. Guidelines for practitioners in video processing are also given.

Content description becomes important in the ubiquity of video content on the Web and consumer devices. Video classification is needed so that more appropriate description and search methods can be applied. This paper describes two methods for video content classification: a Nearest Neighbor (NN) method relying on domain knowledge and Hidden Markov Model (HMM) based method. Our approach stems from the observation that in different TV categories, there are different objects (e.g., face and text) trajectory patterns. Face and text tracking is applied to video segments to extract face and text trajectories. We used NN and HMM to classify a given video segment into predefined classes, e.g., commercial, news, situation comedy and soap. Our preliminary experimental results show classification accuracy of 75% for NN and over 80% for HMM based method on short video segments.

In this paper, we present a novel scheme on video content representation by exploring the spatio-temporal information. A pseudo-object-based shot representation containing more semantics is proposed to measure shot similarity and force competition approach is proposed to group shots into scene based on content coherences between shots. Two content descriptors, color objects: Dominant Color Histograms (DCH) and Spatial Structure Histograms (SSH), are introduced. To represent temporal content variations, a shot can be segmented into several subshots that are of coherent content, and shot similarity measure is formulated as subshot similarity measure that serves to shot retrieval. With this shot representation, scene structure can be extracted by analyzing the splitting and merging force competitions at each shot boundary. Experimental results on real-world sports video prove that our proposed approach for video shot retrievals achieve the best performance on the average recall (AR) and average normalized modified retrieval rank (ANMRR), and Experiment on MPEG-7 test videos achieves promising results by the proposed scene extraction algorithm.

Multimedia information is getting more abundant and the means to produce it are becoming a commodity, but finding and managing multimedia content is getting harder and harder. In 1996, MPEG (Moving Picture Experts Group) started a project formally named "Multimedia Content Description Interface", but better known as MPEG-7, acknowledging the need to efficiently and effectively describe and retrieve multimedia information and recognizing the substantial technological developments in the area of multimedia content description.

MPEG-7 sets a standard for multimedia description tools, notably so-called descriptors, description schemes, systems tools and a description definition language. MPEG-7 is generic in the sense that it is not specially designed or optimized for a particular application domain. It is however clear that image and video database applications are among its most important application domains.

This paper intends to overview the context, objectives, technical approach, workplan and achievements of the MPEG-7 standard. The first version and main body of this standard should be ready by July 2001.

A texture description is useful in many applications including similarity based image search and browsing. We present here three texture descriptors that are being considered for the final committee draft of the ISO/MPEG-7 standard. A comprehensive overview of the syntax and semantics of these texture descriptors is provided. The Homogeneous Texture Descriptor (HTD) and the Edge Histogram Descriptor (EHD) are useful in similarity search. The HTD characterizes homogeneous texture regions and is also useful in texture classification and recognition. The EHD is applicable when the underlying texture is not homogeneous and can also be used in sketch based retrieval. In addition, a compact descriptor that facilitates browsing applications is also defined. These descriptors are selected after a highly competitive test and evaluation phase within the MPEG group and we briefly summarize the evaluation criteria, the datasets used and the experimental results.