Narrow Results

In this paper, we propose a system for automatic image annotation that has two main components. The first component consists of a novel semi-supervised possibilistic clustering and feature weighting algorithm based on robust modeling of the generalized Dirichlet (GD) finite mixture. This algorithm is used to group image regions into prototypical region clusters that summarize the training data and can be used as the basis of annotating new test images. The constraints consist of pairs of image regions that should not be included in the same cluster. These constraints are deduced from the irrelevance of all concepts annotating the training images to help in guiding the clustering process. The second component of our system consists of a probabilistic model that relies on the possibilistic membership degrees, generated by the clustering algorithm, to annotate unlabeled images. The proposed system was implemented and tested on a data set that include thousands of images using four-fold cross validation.

In this paper, we propose a novel automatic image annotation model by mining the web. In our approach, the terms or words appearing in the associated text are extracted and filtered as labels or annotations for the corresponding web images. Sure, much noise exists in those selected labels. In order to reduce the influence caused by the noisy labels, for each label or potential word, we improve web image-word relationships using Mixture Gaussian Distribution Model. By doing so, the relationships between words and images are re-weighted both in terms of sematic relevance and in terms of visual feature similarity. In fact, all the words associated to an image are not semantically independent. We use co-occurrences between two words to describe their semantic relevance. Thus, we further use a method, called Word Promotion, to co-enhance the weights of all the words associated to a given image based on their co-occurrences. Our experiments are conducted in several ways and the results show that our annotation method can achieve a satisfactory performance in respects of system scalability and sematic evolution.

Keyword-based image retrieval is more comfortable for users than content-based image retrieval. Because of the lack of semantic description of images, image annotation is often used a priori by learning the association between the semantic concepts (keywords) and the images (or image regions). This association issue is particularly difficult but interesting because it can be used for annotating images but also for multimodal image retrieval. However, most of the association models are unidirectional, from image to keywords. In addition to that, existing models rely on a fixed image database and prior knowledge. In this paper, we propose an original association model, which provides image-keyword bidirectional transformation. Based on the state-of-the-art Bag of Words model dealing with image representation, including a strategy of interactive incremental learning, our model works well with a zero-or-weak-knowledge image database and evolving from it. Some objective quantitative and qualitative evaluations of the model are proposed, in order to highlight the relevance of the method.

With the rapid growth of image collections, image classification and annotation has been active areas of research with notable recent progress. Bag-of-Visual-Words (BoVW) model, which relies on building visual vocabulary, has been widely used in this area. Recently, attention has been shifted to the use of advanced architectures which are characterized by multi-level processing. Hierarchical Max-Pooling (HMAX) model has attracted a great deal of attention in image classification. To improve image classification and annotation, several approaches based on ontologies have been proposed. However, image classification and annotation remain a challenging problem due to many related issues like the problem of ambiguity between classes. This problem can affect the quality of both classification and annotation results. In this paper, we propose an ontology-based image classification and annotation approach. Our contributions consist of the following: (1) exploiting ontological relationships between classes during both image classification and annotation processes; (2) combining the outputs of hypernym–hyponym classifiers to lead to a better discrimination between classes; and (3) annotating images by combining hypernym and hyponym classification results in order to improve image annotation and to reduce the ambiguous and inconsistent annotations. The aim is to improve image classification and annotation by using ontologies. Several strategies have been experimented, and the obtained results have shown that our proposal improves image classification and annotation.

The interest in image annotation and recommendation has been increased due to the ever rising amount of data uploaded to the web. Despite the many efforts undertaken so far, accuracy or efficiency still remain open problems. Here, a complete image annotation and tourism recommender system is proposed. It is based on the probabilistic latent semantic analysis (PLSA) and hypergraph ranking, exploiting the visual attributes of the images and the semantic information found in image tags and geo-tags. In particular, semantic image annotation resorts to the PLSA, exploiting the textual information in image tags. It is further complemented by visual annotation based on visual image content classification. Tourist destinations, strongly related to a query image, are recommended using hypergraph ranking enhanced by enforcing group sparsity constraints. Experiments were conducted on a large image dataset of Greek sites collected from Flickr. The experimental results demonstrate the merits of the proposed model. Semantic image annotation by means of the PLSA has achieved an average precision of 92% at 10% recall. The accuracy of content-based image classification is 82, 6%. An average precision of 92% is measured at 1% recall for tourism recommendation.

This research deals with semantic interpretation of Remote Sensing Images (RSIs) using ontologies which are considered as one of the main challenging methods for modeling high-level knowledge, and reducing the semantic gap between low-level features and high-level semantics of an image. In this paper, we propose a new ontology which allows the annotation as well as the interpretation of RSI with respect to natural risks, while taking into account uncertainty of data, object dynamics in natural scenes, and specificities of sensors. In addition, using this ontology, we propose a methodology to (i) annotate the semantic content of RSI, and (ii) deduce the susceptibility of the land cover to natural phenomena such as erosion, floods, and fires, using case-based reasoning supported by the ontology. This work is tested using LANDSAT and SPOT images of the region of Kef which is situated in the north-west of Tunisia. Results are rather promising.

Advanced digital capturing technologies have led to the explosive growth of images on the Web. To retrieve the desired image from a huge amount of images, textual query is handier to represent the user's interest than providing a visually similar image as a query. Semantic annotation of images' has been identified as an important step towards more efficient manipulation and retrieval of images. The aim of the semantic annotation of images is to annotate the existing images on the Web so that the images are more easily interpreted by searching programs. To annotate the images effectively, extensive image interpretation techniques have been developed to explore the semantic concept of images. But, due to the complexity and variety of backgrounds, effective image annotation is still a very challenging and open problem. Semantic annotation of Web contents manually is not feasible or scalable too, due to the huge amount and rate of emerging Web content. In this paper, we have surveyed the existing image annotation models and developed a hierarchical classification-based image annotation framework for image categorization, description and annotation. Empirical evaluation of the proposed framework with respect to its annotation accuracy shows high precision and recall compared with other annotation models with significant time and cost. An important feature of the proposed framework is that its specific annotation techniques, suitable for a particular image category, can be easily integrated and developed for other image categories.

Brain tumor is a fatal central nervous system disease that occurs in around 250,000 people each year globally and it is the second cause of cancer in children. It has been widely acknowledged that genetic factor is one of the significant risk factors for brain cancer. Thus, accurate descriptions of the locations of where the relative genes are active and how these genes express are critical for understanding the pathogenesis of brain tumor and for early detection. The Allen Developing Mouse Brain Atlas is a project on gene expression over the course of mouse brain development stages. Utilizing mouse models allows us to use a relatively homogeneous system to reveal the genetic risk factor of brain cancer. In the Allen atlas, about 435,000 high-resolution spatiotemporal in situ hybridization images have been generated for approximately 2,100 genes and currently the expression patterns over specific brain regions are manually annotated by experts, which does not scale with the continuously expanding collection of images. In this paper, we present an efficient computational approach to perform automated gene expression pattern annotation on brain images. First, the gene expression information in the brain images is captured by invariant features extracted from local image patches. Next, we adopt an augmented sparse coding method, called Stochastic Coordinate Coding, to construct high-level representations. Different pooling methods are then applied to generate gene-level features. To discriminate gene expression patterns at specific brain regions, we employ supervised learning methods to build accurate models for both binary-class and multi-class cases. Random undersampling and majority voting strategies are utilized to deal with the inherently imbalanced class distribution within each annotation task in order to further improve predictive performance. In addition, we propose a novel structure-based multi-label classification approach, which makes use of label hierarchy based on brain ontology during model learning. Extensive experiments have been conducted on the atlas and results show that the proposed approach produces higher annotation accuracy than several baseline methods. Our approach is shown to be robust on both binary-class and multi-class tasks and even with a relatively low training ratio. Our results also show that the use of label hierarchy can significantly improve the annotation accuracy at all brain ontology levels.