No Access

ACTIVE SEGMENTATION

AJAY K. MISHRA

Computer Vision Laboratory, Institute of Advanced Computer Studies, University of Maryland, College Park, USA

Search for more papers by this author

and

YIANNIS ALOIMONOS

Computer Vision Laboratory, Institute of Advanced Computer Studies, University of Maryland, College Park, USA

Search for more papers by this author

https://doi.org/10.1142/S0219843609001784Cited by:24 (Source: Crossref)

Abstract

The human visual system observes and understands a scene/image by making a series of fixations. Every "fixation point" lies inside a particular region of arbitrary shape and size in the scene which can either be an object or just a part of it. We define as a basic segmentation problem the task of segmenting that region containing the "fixation point". Segmenting the region containing the fixation is equivalent to finding the enclosing contour — a connected set of boundary edge fragments in the edge map of the scene — around the fixation. This enclosing contour should be a depth boundary.

We present here a novel algorithm that finds this bounding contour and achieves the segmentation of one object, given the fixation. The proposed segmentation framework combines monocular cues (color/intensity/texture) with stereo and/or motion, in a cue independent manner. The semantic robots of the immediate future will be able to use this algorithm to automatically find objects in any environment. The capability of automatically segmenting objects in their visual field can bring the visual processing to the next level. Our approach is different from current approaches. While existing work attempts to segment the whole scene at once into many areas, we segment only one image region, specifically the one containing the fixation point. Experiments with real imagery collected by our active robot and from the known databases¹ demonstrate the promise of the approach.

Keywords:

References

Semantic robot vision challenge, http://www.semantic-robot-vision-challenge.org/ . Google Scholar
G. Adiv, T-PAMI 7, 384 (1985). Google Scholar
G. Adiv, IEEE Trans. Patt. Anal. Mach. Intell. 11(5), 477 (1989), DOI: 10.1109/34.24780. Crossref, Web of Science, Google Scholar
J. Aloimonos, I. Weiss and A. Bandyopadhyay, IJCV 1(4), 333 (1988), DOI: 10.1007/BF00133571. Crossref, Web of Science, Google Scholar
S. Alpert et al. , Image segmentation by probabilistic bottom-up aggregation and cue integration , Proc. IEEE Conf. on Computer Vision and Pattern Recognition ( 2007 ) . Google Scholar
P. Arbelaez and L. Cohen, Constrained image segmentation from hierarchical boundaries, CVPR (2008) pp. 454–467. Google Scholar
S. Ayer, P. Schroeter and J. Bigün, Segmentation of moving objects by robust motion parameter estimation over multiple frames, ECCV (Springer-Verlag, 1994) pp. 316–327. Google Scholar
S. Bagon, O. Boiman and M. Irani, What is a good image segment? A unified approach to segment extraction, Computer Vision — ECCV 20085305, LNCS, eds. D. Forsyth, P. Torr and A. Zisserman (Springer, 2008) pp. 30–44. Google Scholar
R. Bajcsy, Proc. of the IEEE Special Issue on Computer Vision 76(8), 966 (1988). Google Scholar
D. Ballard, Artif. Intell. J. 48(8), 57 (1991), DOI: 10.1016/0004-3702(91)90080-4. Crossref, Web of Science, Google Scholar
W. A. Barrett and E. N. Mortensen, Medical Image Analysis 1, 331 (1997), DOI: 10.1016/S1361-8415(97)85005-0. Crossref, Google Scholar
A. Blakeet al., Interactive image segmentation using an adaptive gmmrf model, ECCV (2004) pp. 428–441. Google Scholar
M. Bober and J. Kittler, Robust motion analysis, CVPRII (1994) pp. 947–952. Google Scholar
Y. Boykov and M. Jolly, Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images, ICCVI (2001) pp. 105–112. Google Scholar
Y. Boykov and V. Kolmogorov, IEEE Trans. Patt. Anal. Mach. Intell. 26, 359 (2004). Google Scholar
Y. Boykov and V. Kolmogorov, An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision, PAMI26 (2004) pp. 1124–1137. Google Scholar
T. Broxet al., High accuracy optical flow estimation based on a theory for warping (Springer, 2004) pp. 25–36. Google Scholar
P. Burtet al., Object tracking with a moving camera, Visual Motion 1989 Proceedings Workshop onII (1989) pp. 2–12. Google Scholar
M. Cerfet al., Predicting human gaze using low-level saliency combined with face detection, Advances in Neural Information Processing Systems (NIPS) (MIT Press, 2008) p. 20. Google Scholar
J. Costeira and T. Kanade, A multi-body factorization method for motion analysis, ICCV (IEEE Computer Society, 1995) p. 1071. Google Scholar
E. Craftet al., J. Neurophysio. 6(97), 4310 (2007). Google Scholar
K. Daniilidis, Comput. Vis. Image Underst. 68(2), 158 (1997), DOI: 10.1006/cviu.1997.0535. Crossref, Web of Science, Google Scholar
P. F. Felzenszwalb and D. P. Huttenlocher, Efficient graph-based image segmentation, IJCV59 (2004) pp. 167–181. Google Scholar
C. Fowlkes, D. R. M. and J. Malik, Local figure/ground cues are valid for natural images, JV 7(8) (2007) 1–9 . Google Scholar
M. Irani and P. Anandan, IEEE Trans. Pattern Anal. Mach. Intell. 20(6), 577 (1998), DOI: 10.1109/34.683770. Crossref, Web of Science, Google Scholar
L. Itti, C. Koch and E. Niebur, T-PAMI 20(11), 1254 (1998). Crossref, Web of Science, Google Scholar
M. Kass, A. Witkin and D. Terzopoulos, Int. J. Comput. Vision 1, 321 (1988), DOI: 10.1007/BF00133570. Crossref, Web of Science, Google Scholar
D. G. Lowe, Int. J. Comput. Vision 60(2), 91 (2004), DOI: 10.1023/B:VISI.0000029664.99615.94. Crossref, Web of Science, Google Scholar
J. Maliket al., IJCV 43(1), 7 (2001), DOI: 10.1023/A:1011174803800. Crossref, Web of Science, Google Scholar
D. Martin, C. Fowlkes and J. Malik, T-PAMI 26(5), 530 (2004). Crossref, Web of Science, Google Scholar
K. Mikolajczyk and C. Schmid , An affine invariant interest point detector , Proc. Eur. Conf. on Computer Vision (ECCV) ( Springer-Verlag , 2002 ) . Google Scholar
E. N. Mortensen and W. A. Barrett, Intelligent scissors for image composition, SIGGRAPHI (1995) pp. 191–198. Google Scholar
R. Nelson, IJCV 7, 33 (1991), DOI: 10.1007/BF00130488. Crossref, Web of Science, Google Scholar
J.-M. Odobez and P. Bouthemy, MRF-based motion segmentation exploiting a 2D motion model robust estimation, ICIP (IEEE Computer Society, 1995) p. 3628. Google Scholar
K. Pahlavan, T. Uhlin and J.-O. Eklundh, Int. J. Comput. Vision 17(2), 113 (1996). Crossref, Web of Science, Google Scholar
X. Ren and J. Malik, A probabilistic multi-scale model for contour completion based on image statistics, ECCV '02: Proc. 7th Eur. Conf. on Computer Vision-Part I (Springer-Verlag, 2002) pp. 312–327. Google Scholar
C. Rother, V. Kolmogorov and A. Blake, ACM Trans. Graph. 23(3), 309 (2004), DOI: 10.1145/1015706.1015720. Crossref, Web of Science, Google Scholar
H. S. Sawhney, Y. Guo and R. Kumar, IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1191 (2000), DOI: 10.1109/34.879803. Crossref, Web of Science, Google Scholar
J. T. Serences and S. Yantis, Trends in Cognitive Sciences 10(1), 38 (2006), DOI: 10.1016/j.tics.2005.11.008. Crossref, Web of Science, Google Scholar
J. Shi and J. Malik, Normalized cuts and image segmentation, PAMI22 (2000) pp. 888–905. Google Scholar
D. Sinclair, Motion segmentation and local structure, ICCV93 pp. 366–373. Google Scholar
A. K. Sinop and L. Grady, A seeded image segmentation framework unifying graph cuts and random walker which yields a new algorithm, ICCV (2007) pp. 1–8. Google Scholar
W. Thompson and T. Pong, IJCV 4, 39 (1990), DOI: 10.1007/BF00137442. Crossref, Web of Science, Google Scholar
P. H. S. Torret al., Philosophical Transactions: Mathematical, Physical and Engineering Sciences 356(1740), 1321 (1998), DOI: 10.1098/rsta.1998.0224. Crossref, Web of Science, Google Scholar
P. H. S. Torr and D. W. Murray, Stochastic motion clustering, ECCV (Springer-Verlag New York, Inc, 1994) pp. 328–337. Google Scholar
B. Triggset al., Bundle adjustment — A modern synthesis, ICCV '99: Proc. Int. Workshop on Vision Algorithms (Springer-Verlag, 2000) pp. 298–372. Google Scholar
Z. Tu and S. Zhu, T-PAMI 24(5), 603 (2002). Web of Science, Google Scholar
Z. Tu and S.-C. Zhu, IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 657 (2002). Web of Science, Google Scholar
O. Veksler, Star shape prior for graph-cut image segmentation, ECCV3 (2008) pp. 454–467. Google Scholar
D. Walther and C. Koch, Neural Networks 19(4), 1395 (2006), DOI: 10.1016/j.neunet.2006.10.001. Crossref, Web of Science, Google Scholar
J. Weber and J. Malik, IEEE Trans. Pattern Anal. Mach. Intell. 19(2), 139 (1997), DOI: 10.1109/34.574794. Crossref, Web of Science, Google Scholar
Y. Weiss, Smoothness in layers: Motion segmentation using nonparametric mixture estimation, CVPR (IEEE Computer Society, 1997) p. 520. Google Scholar
C. S. Wiles and M. Brady, Closing the loop on multiple motions, ICCV '95: Proc. 5th Int. Conf. on Computer Vision (IEEE Computer Society, 1995) p. 308. Google Scholar
L. R. Williams and D. W. Jacobs, Neural Comput. 9(4), 837 (1997), DOI: 10.1162/neco.1997.9.4.837. Crossref, Web of Science, Google Scholar
C. Xu and J. L. Prince, IEEE Trans. on Image Processing 7(3), 359 (1998). Web of Science, Google Scholar
S. X. Yu and J. Shi , Grouping with bias , NIPS ( 2001 ) . Google Scholar
C. Zahn, IEEE Trans. on Computers 20(1), 68 (1971). Web of Science, Google Scholar
Z. Zhang, O. Faugeras and N. Ayache, Analysis of a sequence of stereo scenes containing multiple moving objects using rigidity constraints, ICCV (1988) pp. 177–186. Google Scholar
Q. Zheng and R. Chellapa, Motion detection in image sequences acquired from a moving platform, ICASSP (1993) pp. 201–204. Google Scholar