Computer VisionNo Access

AUTOMATIC IMAGE ANNOTATION BASED ON SEMI-SUPERVISED CLUSTERING AND MEMBERSHIP-BASED CROSS MEDIA RELEVANCE MODEL

MOHAMED MAHER BEN ISMAIL

King Saud University, KSA, Saudi Arabia

Author for Correspondence.

Search for more papers by this author

and

OUIEM BCHIR

King Saud University, KSA, Saudi Arabia

Search for more papers by this author

https://doi.org/10.1142/S0218001412550099Cited by:1 (Source: Crossref)

Abstract

In this paper, we propose a system for automatic image annotation that has two main components. The first component consists of a novel semi-supervised possibilistic clustering and feature weighting algorithm based on robust modeling of the generalized Dirichlet (GD) finite mixture. This algorithm is used to group image regions into prototypical region clusters that summarize the training data and can be used as the basis of annotating new test images. The constraints consist of pairs of image regions that should not be included in the same cluster. These constraints are deduced from the irrelevance of all concepts annotating the training images to help in guiding the clustering process. The second component of our system consists of a probabilistic model that relies on the possibilistic membership degrees, generated by the clustering algorithm, to annotate unlabeled images. The proposed system was implemented and tested on a data set that include thousands of images using four-fold cross validation.

Keywords:

References

K. Barnard and D. Forsyth, Learning the semantics of words and pictures, Proc. Int. Conf. Computer Vision2 (2001) pp. 408–415. Google Scholar
J. C. Bezdek , Pattern Recognition with Fuzzy Objective Function Algorithms ( Plenum Press , New York , 1981 ) . Crossref, Google Scholar
N. Bouguila and D. Ziou, A powerful finite mixture model based on the generalized Dirichlet distribution: Unsupervised learning and applications, Proc. 17th Int. Conf. Pattern Recognition (ICPR '04) (2004) pp. 280–283. Google Scholar
N. Bouguila and D. Ziou, IEEE Trans. Knowl. Data Eng. 18(8), 993 (2006). Crossref, Web of Science, Google Scholar
N. Bouguila and D. Ziou, J. Vis. Commun. Image Represent. 18(4), 295 (2007). Crossref, Web of Science, Google Scholar
R. Datta, D. Joshi, J. Li and J. Z. Wang, Image retrieval: Ideas, influences, and trends of the new age, Penn State University Technical Report CSE (2006) . Google Scholar
A. Del Bimbo , Visual Information Retrieval Book ( Morgan Kaufmann , 1999 ) . Google Scholar
P. Duyguluet al., Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary, Proc. Seventh European Conf. Computer Vision (2002) pp. 97–112. Google Scholar
M. Flickneret al., IEEE Comput. 28(9), 23 (1995). Crossref, Web of Science, Google Scholar
H. Frigui and J. Caudill, Region based image annotation, Proc. Int. Conf. Image Processing (ICPR'06) (2006) pp. 953–956. Google Scholar
H. Frigui and R. Krishnapuram, Pattern Recogn. Lett. 17(12), 1223 (1996). Crossref, Web of Science, Google Scholar
H. Frigui and R. Krishnapuram, Pattern Recogn. 30(7), 1223 (1997). Crossref, Web of Science, Google Scholar
H. Frigui and R. Mahdi, Semi-supervised clustering and feature discrimination with instance-level constraints, Proc. Fuzzy Systems Conf. FUZZ-IEEE 2007 (2007) pp. 1–6. Google Scholar
H. Frigui and O. Nasraoui , Pattern Recogn. 37 , 567 ( 2004 ) . Crossref, Web of Science, Google Scholar
J. S. Hare et al. , Mind the gap: Another look at the problem of the semantic gap in image retrieval , Proc. SPIE Multimedia Content Analysis, Management and Retrieval 6073 ( California, USA , 2006 ) . Google Scholar
N. Hervé and N. Boujemaa, Image annotation : Which approach for realistic databases, Proc. 6th ACM Int. Conf. Image and Video Retrieval. (2007) pp. 170–177. Google Scholar
J. Jeon, V. Lavrenko and R. Manmatha, Automatic image annotation and retrieval using cross media relevance models, Proc. SIGIR'03 (2003) pp. 119–126. Google Scholar
R. Krishnapuram and J. Keller , IEEE Trans. Fuzzy Syst. 1 , 98 ( 1993 ) . Crossref, Google Scholar
J. Li and J. Z. Wang, IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 985 (2008). Crossref, Web of Science, Google Scholar
ludicorp, "flickr", http://www.flickr.com . Google Scholar
D. J. C. Mackay and L. Peto, Nat. Lang. Eng. 1(3), 1 (1994). Google Scholar
M. Maher Ben Ismail and H. Frigui, Possibilistic clustering based on robust modeling of finite generalized Dirichlet mixture, Proc. Twentieth Conf. Int. Association for Pattern Recognition (IAPR) (ICPR, 2010) pp. 573–576. Google Scholar
M. Maher Ben Ismail and H. Frigui, Image database categorization using robust unsupervised learning of finite generalized Dirichlet mixture models, Proc. Int. Conf. on Image Processing (ICIP) (2011) pp. 2457–2460. Google Scholar
B. S. Manjunth , T. Sikora and P. Salembier , Introduction to MPEG-7: Multimedia Content Description Interface ( John Wiley , 2002 ) . Google Scholar
G. McLachlan and D. Peel , Finite Mixture Models ( John Wiley & Sons , New York , 2000 ) . Crossref, Google Scholar
Y. Mori , H. Takahashi and R. Oka , Image-to-word transformation based on dividing and vector quantizing images with words , MISRM'99. First Int. Workshop on Multimedia Intelligent Storage and Retrieval Management ( Florida, USA , 1999 ) . Google Scholar
P. Pudil, J. Novovicova and J. Kittler, Pattern Recogn. 28(9), 1389 (1995). Crossref, Web of Science, Google Scholar
P. Rao , Wiley Series in Probability and Mathematical Statistics ( Wiley , New York , 1987 ) . Google Scholar
S. Rui, W. Jin and T.-S. Chua, A novel approach to auto image annotation based on pairwise constrained clustering and semi-naïve Bayesian Model, Proc. 11th Int. Multimedia Modeling Conf. (MMM'05) (2005) pp. 322–327. Google Scholar
A. W. M. Smeulderset al., IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349 (2000). Crossref, Web of Science, Google Scholar
M. Szummer and R. Picard, Indoor-Outdoor Image Classification, Proc. Workshop Content-Based Access to Image and Video Databases (1998) pp. 42–51. Google Scholar
University of California at Berkeley, Corel database, Available at http://elib.cs.berkeley.edu/photos/corel/ . Google Scholar
A. Vailaya , A. Jain and H. Zhang , Pattern Recogn. 31 , 1921 ( 1998 ) . Crossref, Web of Science, Google Scholar
T. Wong , Appl. Math. Comput. 97 , 165 ( 1998 ) . Crossref, Web of Science, Google Scholar
J. Zhanget al., Int. J. Pattern Recogn. Artif. Intell. 24(1), 401 (2010). Link, Web of Science, Google Scholar
X. S. Zhou and T. S. Huang, IEEE Multimedia 9(2), 23 (2002). Crossref, Web of Science, Google Scholar