No Access

TOWARDS GRASP-ORIENTED VISUAL PERCEPTION FOR HUMANOID ROBOTS

Computational Vision and Active Perception Laboratory, Centre for Autonomous Systems, Royal Institute of Technology, Stockholm, Sweden

Search for more papers by this author

CARL BARCK-HOLST

Computational Vision and Active Perception Laboratory, Centre for Autonomous Systems, Royal Institute of Technology, Stockholm, Sweden

Search for more papers by this author

KAI HUEBNER

Computational Vision and Active Perception Laboratory, Centre for Autonomous Systems, Royal Institute of Technology, Stockholm, Sweden

Search for more papers by this author

MARIA RALPH

Computational Vision and Active Perception Laboratory, Centre for Autonomous Systems, Royal Institute of Technology, Stockholm, Sweden

Search for more papers by this author

BABAK RASOLZADEH

Computational Vision and Active Perception Laboratory, Centre for Autonomous Systems, Royal Institute of Technology, Stockholm, Sweden

Search for more papers by this author

DAN SONG

Computational Vision and Active Perception Laboratory, Centre for Autonomous Systems, Royal Institute of Technology, Stockholm, Sweden

Search for more papers by this author

, and

DANICA KRAGIC

Computational Vision and Active Perception Laboratory, Centre for Autonomous Systems, Royal Institute of Technology, Stockholm, Sweden

Search for more papers by this author

https://doi.org/10.1142/S0219843609001796Cited by:16 (Source: Crossref)

Abstract

A distinct property of robot vision systems is that they are embodied. Visual information is extracted for the purpose of moving in and interacting with the environment. Thus, different types of perception-action cycles need to be implemented and evaluated.

In this paper, we study the problem of designing a vision system for the purpose of object grasping in everyday environments. This vision system is firstly targeted at the interaction with the world through recognition and grasping of objects and secondly at being an interface for the reasoning and planning module to the real world. The latter provides the vision system with a certain task that drives it and defines a specific context, i.e. search for or identify a certain object and analyze it for potential later manipulation. We deal with cases of: (i) known objects, (ii) objects similar to already known objects, and (iii) unknown objects. The perception-action cycle is connected to the reasoning system based on the idea of affordances. All three cases are also related to the state of the art and the terminology in the neuroscientific area.

Keywords:

References

J. Grayet al., Action parsing and goal inference using self as simulator, IEEE Workshop on Robot and Human Interactive Communication (2005) pp. 202–209. Google Scholar
T. Kyriacou, G. Bugmann and S. Lauria, Robotics and Autonomous Systems 51(1), 69 (2005). Web of Science, Google Scholar
P. McGuireet al., Multi-modal human-machine communication for instructing robot grasping tasks, IEEE/RSJ Int. Conf. on Intell. Robots and Systems (2002) pp. 1082–1089. Google Scholar
G. Rizzolattiet al., Experimental Brain Research 111(2), 246 (1996). Web of Science, Google Scholar
G. Rizzolatti, G. Luppino and M. Matelli, Electroencephalography and Clinical Neurophysiology 106(4), 283 (1998). Google Scholar
A. H. Fagg and M. A. Arbib, Neural Networks 11(8), 1277 (1998). Web of Science, Google Scholar
V. Raoset al., J. of Neurophysiology 95(2), 709 (2006). Web of Science, Google Scholar
Z. Li, Trends in Cognitive Sciences 6(1), 9 (2002). Web of Science, Google Scholar
M. Goodale, Trends in Neurosciences 15(1), 20 (1992). Web of Science, Google Scholar
M. Lu, J. Preston and P. Strick, J. of Comparative Neurology 341(3), 375 (1994). Web of Science, Google Scholar
G. Luppino, M. Matelli and G. Rizzolatti, Experimental Brain Research 82(1), 214 (1990). Web of Science, Google Scholar
D. Ingleet al., Psychological Research 31(1), 42 (1967). Web of Science, Google Scholar
E. Chinellato and A. del Pobil, Neural Coding in the Dorsal Visual Stream (Springer-Verlag Berlin/Heidelberg, 2008) pp. 230–239. Google Scholar
A. Murataet al., J. of Neurophysiology 83(5), 2580 (2000). Web of Science, Google Scholar
T. Jameset al., Neuron 35(4), 793 (2002). Web of Science, Google Scholar
E. Chinellato, Y. Demiris and A. P. del Pobil, Studying the human visual cortex for achieving action-perception coordination with robots, IASTED Int. Conf. Artif. Intell. Soft Comput. (2006) pp. 184–189. Google Scholar
U. Castiello and M. Jeannerod, Neuroreport 2(12), 797 (1991). Web of Science, Google Scholar
M. J. Webster, J. Bachevalier and L. G. Ungerleider, Cerebral Cortex 4(5), 470 (1994). Web of Science, Google Scholar
G. Rizzolatti and G. Luppino, Neuron 31(6), 889 (2001). Web of Science, Google Scholar
M. Petrides and D. N. Pandya, J. of Comparative Neurology 228(1), 105 (1984). Web of Science, Google Scholar
M. A. Arbib, A. R. Iberall and D. Lyons, Hand Function and the Neocortex, Experimental Brain Research Supplemental 10, eds. A. W. Goodwin and I. Darian-Smith (Springer-Verlag, Berlin, 1985) pp. 111–129. Google Scholar
T. M. Preuss and P. S. Goldman-Rakic, J. of Comparative Neurology 282(2), 293 (1989). Web of Science, Google Scholar
G. Recataláet al., Autonomous Robots 25(1), 59 (2008). Web of Science, Google Scholar
R. S. Johansson and G. Westling, Experimental Brain Research 56(3), 550 (1984). Web of Science, Google Scholar
A. Ude, C. Gaskett and G. Cheng, Foveated vision systems with two cameras per eye, IEEE Int. Conf. on Robotics and Automation (2006) pp. 3457–3462. Google Scholar
M. Björkman and J.-O. Eklundh , Attending, foveating and recognizing objects in real world scenes , British Machine Vision Conference, BMVC'04 ( 2004 ) . Google Scholar
J. K. Tsotsos, Analyzing vision at the complexity level: constraints on an architecture, an explanation for visual search performance, and computational justification for attentive processes, Technical Report, University of Toronto Vision Laboratory (1987) . Google Scholar
J. K. Tsotsos, Int. J. of Computer Vision 1(4), 303 (1988). Web of Science, Google Scholar
J. K. Tsotsos, The complexity of perceptual search tasks, Technical Report, University of Toronto Vision Laboratory (1989) . Google Scholar
S. Ekvall and D. Kragic, Learning and evaluation of the approach vector for automatic grasp generation and planning, IEEE Int. Conf. on Robotics and Automation (2007) pp. 4715–4720. Google Scholar
A. Moraleset al., An anthropomorphic grasping approach for an assistant humanoid robot, 37th International Symposium on Robotics (2006) pp. 149–152. Google Scholar
L. Itti, Models of bottom-up and top-down visual attention, Ph.D. thesis, California Institute of Technology (2000) . Google Scholar
B. Olshausen, C. Anderson and D. van Essen, J. of Neurosci. 13, 4700 (1993). Web of Science, Google Scholar
A. M. Treisman and G. Gelade, Cognitive Psychology 12, 97 (1980). Web of Science, Google Scholar
C. Koch and S. Ullman, Human Neurobiology 4, 219 (1985). Google Scholar
T. Koike and J. Saiki, Biologically Motivated Computer Vision 408 (2002). Google Scholar
O. Ramström and H. I. Christensen, Object detection using background context, Int. Conf. Patt. Recogn. (2004) pp. 45–48. Google Scholar
S. B. Choi, S. W. Ban and M. Lee, Neural Information Processing — Letters and Review 2, 19 (2004). Google Scholar
L. Itti, C. Koch and E. Niebur, IEEE Trans. Patt. Anal. Mach. Intell. 20(11), 1254 (1998). Web of Science, Google Scholar
V. Navalpakkam and L. Itti , Sharing resources: Buy attention, get recognition , International Workshop Attention and Performance in Computer Vision ( 2003 ) . Google Scholar
K. Lee, H. Buxton and J. Feng, Selective attention for cueguided search using a spiking neural network, International Workshop on Attention and Performance in Computer Vision (2003) pp. 55–62. Google Scholar
A. Olivaet al., Top-down control of visual attention in object detection, Int. Conf. Image Process. (2003) pp. 253–256. Google Scholar
S. Frintrop , VOCUS: A Visual Attention System for Object Detection and Goal-Directed Search 3899 ( Springer , 2006 ) . Google Scholar
J. Morenet al., Int. J. of Humanoid Robotics 3 (2008). Web of Science, Google Scholar
A. M. Okamura, N. Smaby and M. R. Cutkosky, An overview of dexterous manipulation, IEEE Int. Conf. on Robotics and Automation (2000) pp. 255–262. Google Scholar
K. B. Shimoga, Int. J. Robotic Research 15(3), 230 (1996). Web of Science, Google Scholar
M. Ciorcarlie , C. Goldfeder and P. Allen , Dexterous grasping via eigengrasps: A low-dimensional approach to a high-complexity problem , Robotics: Science and Systems Manipulation Workshop ( 2007 ) . Google Scholar
C. Borst, M. Fischer and G. Hirzinger, Grasping the dice by dicing the grasp, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (2003) pp. 3692–3697. Google Scholar
T. Asfouret al., ARMAR-III: An integrated humanoid platform for sensory-motor control, 6th IEEE-RAS Int. Conf. on Humanoid Robots (2006) pp. 169–175. Google Scholar
J. Teginet al., Journal of Intelligent Service Robotics (2008). Google Scholar
A. Saxenaet al., Neural Inform. Process. Syst. 19, 1209 (2007). Google Scholar
A. Moraleset al., Int. J. of Humanoid Robotics 1(4), 671 (2004). Link, Google Scholar
M. Starket al., Functional object class detection based on learned affordance cues, 6th Int. Conf. on Computer Vision Systems5008, LNAI (Springer-Verlag, 2008) pp. 435–444. Google Scholar
J. Grezes and J. Decety, Neuropsychologia 40(2), 212 (2002). Web of Science, Google Scholar
M. Tucker and R. Ellis, Journal of Experimental Psychology: Human Perception and Performance 24(3), 830 (1998). Web of Science, Google Scholar
A. M. Borghi , Grounding Cognition: The Role of Perception and Action in Memory, Language, and Thinking , Chapter Object Concepts and Action ( Cambridge University Press , 2005 ) . Google Scholar
S. H. Creem and D. R. Proffitt, Journal of Experimental Psychology: Human Perception and Performance 27(1), 218 (2001). Web of Science, Google Scholar
J. J. Gibson , The Ecological Approach to Visual Perception ( Lawrence Erlbaum Associates , 1979 ) . Google Scholar
M. A. Goodaleet al., Current Biology 4(7), 604 (1994). Web of Science, Google Scholar
R. H. Cuijpers, J. B. J. Smeets and E. Brenner, Journal of Neurophysiology 91, 2598 (2004). Web of Science, Google Scholar
M. Gentilucci, Neuropsychologia 40(8), 1139 (2002). Web of Science, Google Scholar
A. T. Milleret al., Automatic grasp planning using shape primitives, IEEE Int. Conf. on Robotics and Automation (2003) pp. 1824–1829. Google Scholar
C. Goldfederet al., Grasp planning via decomposition trees, IEEE Int. Conf. on Robotics and Automation (2007) pp. 4679–4684. Google Scholar
G. Biegelbauer and M. Vincze, Efficient 3D object detection by fitting superquadrics to range image data for robot's object manipulation, IEEE Int. Conf. on Robotics and Automation (2007) pp. 1086–1091. Google Scholar
L. Chevalier, F. Jaillet and A. Baskurt, Journal of Winter School of Computer Graphics, WSCG'03 (2003). Google Scholar
D. Katsoulas, Reliable recovery of piled box-like objects via parabolically deformable superquadrics, 9th IEEE Int. Conf. on Computer Vision2 (2003) pp. 931–938. Google Scholar
E. Lopez-Damian, Grasp planning for object manipulation by an autonomous robot, Ph.D. thesis, Laboratoire d'Analyse et d'Architecture des Systmes du CNRS (2006) . Google Scholar
E. Lopez-Damian , D. Sidobre and R. Alami , Grasp planning for non-convex objects , 36th Int. Symp. Robotics ( 2005 ) . Google Scholar
S. El-Khoury and Anis Sahbani , Handling objects by their handles , IROS-2008 Workshop on Grasp and Task Learning by Imitation ( 2008 ) . Google Scholar
N. Amenta, S. Choi and R. Kolluri, The power crust, 6th ACM Symposium on Solid Modeling and Applications (2001) pp. 249–260. Google Scholar
G. Barequet and S. Har-Peled, Journal of Algorithms 38, 91 (2001). Web of Science, Google Scholar
B. Draper and A. Lionelle, Evaluation of selective attention under similarity transforms, International Workshop on Attention and Performance in Computer Vision (2003) pp. 31–38. Google Scholar
A. Tavakoli Targhiet al., The eigen-transform and applications, Asian Conference on Computer Vision (2006) pp. 70–79. Google Scholar
B. Rasolzadeh, A. Tavakoli Targhi and J.-O. Eklundh, An attentional system combining top-down and bottom-up influences, Workshop on Attention and Performance in Computational Vision (2007) pp. 123–140. Google Scholar
B. Rasolzadeh, Interaction of bottom-up and top-down influences for attention in an active vision system, Master's thesis, Royal Institute of Technology, Stockholm, Sweden (2006) . Google Scholar
A. Rabinovichet al., Objects in context, Int. Conf. on Computer Vision (2007) pp. 1–8. Google Scholar
T. M. Strat and M. A. Fischler, Context-based vision: Recognition of natural scenes, Asilomar Conference on Signals, Systems and Computers (1989) pp. 532–536. Google Scholar
T. M. Strat and M. A. Fischler , The use of context in vision , Workshop on Context-Based Vision ( 1995 ) . Google Scholar
Y. Huet al., Salient region detection using weighted feature maps based on the human visual attention model, IEEE Pacific-Rim Conference on Multimedia (2004) pp. 993–1000. Google Scholar
A. K. C. Wong and P. K Sahoo, IEEE Trans. Systems Man and Cybernetics 19, 866 (1989). Web of Science, Google Scholar
J. Bohg and D. Kragic , Grasping familiar objects using shape context , Int. Conf. on Advanced Robotics ( 2009 ) . Google Scholar
S. Belongie, J. Malik and J. Puzicha, IEEE Trans. Patt. Anal. Mach. Intell. 24(4), 509 (2002). Web of Science, Google Scholar
M. Craven and J. Shavlik, Extracting tree-structured representations of trained networks, Advances in Neural Information Processing Systems (NIPS-8) (MIT Press, 1995) pp. 24–30. Google Scholar
D. Martenset al., Eur. J. Operational Research 183(3), 1466 (2007). Web of Science, Google Scholar
W. T. Townsend, Industrial Robot: An International Journal 27(3), 181 (2000). Web of Science, Google Scholar
K. Huebner, S. Ruthotto and D. Kragic, Minimum volume bounding box decomposition for shape approximation in robot grasping, IEEE Int. Conf. on Robotics and Automation (2008) pp. 1628–1633. Google Scholar
R. B. Rusuet al., Functional object mapping of kitchen environments, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (2008) pp. 3525–3532. Google Scholar
R. Triebelet al., Instance-based AMN classification for improved object recognition in 2D and 3D laser range data, Int. Joint Conf. Artif. Intell. (2007) pp. 2225–2230. Google Scholar
K. Huebneret al., Integration of visual and shape attributes for object action complexes, 6th Int. Conf. on Computer Vision Systems5008, LNAI (Springer-Verlag, 2008) pp. 13–22. Google Scholar
A. Saxena, L. Wong and A. Y. Ng, Learning grasp strategies with partial shape information, 23rd AAAI Conf. Artif. Intell. (2008) pp. 1491–1494. Google Scholar
D. Aarnoet al., Early reactive grasping with second order 3D feature relations, ICRA Workshop: From Features to Actions (2007) pp. 319–325. Google Scholar
C. Borst, M. Fischer and G. Hirzinger, Grasp planning: How to choose a suitable task wrench space, IEEE Int. Conf. on Robotics and Automation (2004) pp. 319–325. Google Scholar
K. Huebner and D. Kragic, Selection of robot pre-grasps using box-based shape approximation, IEEE Int. Conf. on Intelligent Robots and Systems (2008) pp. 1765–1770. Google Scholar
A. Nealen et al. , FiberMesh: Designing freeform surfaces with 3D curve , ACM Transactions on Computer Graphics, SIGGRAPH 2007 23 ( 2007 ) . Google Scholar
A. T. Miller and P. K. Allen, IEEE Robotics & Automation Magazine 11(4), 110 (2004). Web of Science, Google Scholar
A. Morales, Learning to predict grasp reliability with a multifinger robot hand by using visual features, Ph.D. thesis, Dep. of Computer and Engineering Science, Univ. Jaume I (2004) . Google Scholar
J. Speth, A. Morales and P. J. Sanz, Vision-based grasp planning of 3D objects by extending 2D contour based algorithms, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (2008) pp. 2240–2245. Google Scholar
S. Geidenstam et al. , Learning of grasping strategies from box-based 3D object approximations , Proceedings of Robotics: Science and Systems ( 2009 ) . Google Scholar
C. Geibet al., IEEE-RAS Humanoids-06 Workshop: Toward Cognitive Humanoid Robots (2006). Google Scholar
J. Gibson, Perceiving, Acting, and Knowing: Toward an Ecological Psychology, eds. R. E. Shaw and J. Bransford (Erlbaum, NJ, 1977) pp. 67–82. Google Scholar
M. Bar, Nature Reviews Neuroscience 5(8), 617 (2004). Web of Science, Google Scholar
M. Bar and S. Ullman, Spatial context in recognition, Technical Report, Mathematics & Computer Science, Weizmann Institute of Science (1993) . Google Scholar
A. Torralba and P. Sinha, Statistical context priming for object detection, Int. Conf. on Computer Vision (2001) pp. 763–770. Google Scholar
L. Montesanoet al., IEEE Trans. on Robotics 24(1), 15 (2008). Web of Science, Google Scholar
M. Lopes and J. Santos-Victor, IEEE Transactions on Systems, Man, and Cybernetics, Part B 37(2), 308 (2007). Web of Science, Google Scholar