International Journal of Humanoid Robotics

We present an algorithm enabling a humanoid robot to visually learn its body schema, knowing only the number of degrees of freedom in each limb. By "body schema" we mean the joint positions and orientations and thus the kinematic function. The learning is performed by visually observing its end-effectors when moving them. With simulations involving a body schema of more than 20 degrees of freedom, results show that the system is scalable to a high number of degrees of freedom. Real robot experiments confirm the practicality of our approach. Our results illustrate how subjective space representation can develop as a result of sensorimotor contingencies.

In this paper, we present an approach to imitation learning of arm movements in humanoid robots. Continuous hidden Markov models (HMMs) are used to generalize movements demonstrated to a robot multiple times. Characteristic features of the perceived movement, so-called key points, are detected in a preprocessing stage and used to train the HMMs. For the reproduction of a perceived movement, key points that are common to all (or almost all) demonstrations, so-called common key points, are used. These common key points are determined by comparing the HMM state sequences and selecting only those states that appear in every sequence. We also show how the HMM can be used to detect temporal dependencies between the two arms in dual-arm tasks. Experiments reported in this paper have been performed using a kinematics model of the human upper body to simulate the reproduction of arm movements and the generation of natural-looking joint configurations from perceived hand paths. Results are presented and discussed.

Understanding intent is an important aspect of communication among people and is an essential component of the human cognitive system. This capability is particularly relevant to situations that involve collaboration among multiple agents or detection of situations that can pose a particular threat. In this paper, we propose an approach that allows a physical robot to detect the intent of others based on experience acquired through its own sensory–motor capabilities, then use this experience while taking the perspective of the agent whose intent should be recognized. Our method uses a novel formulation of hidden Markov models (HMMs) designed to model a robot's experience and interaction with the world when performing various actions. The robot's capability to observe and analyze the current scene employs a novel vision-based technique for target detection and tracking, using a nonparametric recursive modeling approach. We validate this architecture with a physically embedded robot, detecting the intent of several people performing various activities.

The saccadic selection of relevant visual input for preferential processing allows the efficient use of computational resources. Based on saccadic active human vision, we aim to develop a plausible saccade-based visual cognitive system for a humanoid robot. This paper presents two initial steps toward our objective by extending the saccade-based model of human memory called NIM¹ to a plausible model of natural visual classification. NIM builds feature-vector representations from selected local image samples and uses these to make memory-based decisions. As a first step, we adapt NIM to a straightforward saccade-based model for the classification of natural visual input called NIM-CLASS and evaluate the model in a face-classification experiment. As a second step, we aim to approach the interactive nature of human vision by extending NIM-CLASS to NIM-CLASS^TD by adding active top-down saccadic control. We then assess to what extent top-down control enhances classification performance. The results show that the incorporation of top-down saccadic control benefits classification performance compared to the purely bottom-up control, reducing the amount of visual input required for correct classification. We conclude that NIM-CLASS^TD may provide a fruitful basis for an active visual cognitive system in a humanoid robot that enables efficient visual processing.

We describe a process in which the segmentation of objects as well as the extraction of the object shape becomes realized through active exploration of a robot vision system. In the exploration process, two behavioral modules that link robot actions to the visual and haptic perception of objects interact. First, by making use of an object independent grasping mechanism, physical control over potential objects can be gained. Having evaluated the initial grasping mechanism as being successful, a second behavior extracts the object shape by making use of prediction based on the motion induced by the robot. This also leads to the concept of an "object" as a set of features that change predictably over different frames.

The system is equipped with a certain degree of generic prior knowledge about the world in terms of a sophisticated visual feature extraction process in an early cognitive vision system, knowledge about its own embodiment as well as knowledge about geometric relationships such as rigid body motion. This prior knowledge allows the extraction of representations that are semantically richer compared to many other approaches.

The exploration and learning of new objects is an essential capability of a cognitive robot. In this paper we focus on making use of the robot's manipulation abilities to learn complete object representations suitable for 3D object recognition. Taking control of the object allows the robot to focus on relevant parts of the images, thus bypassing potential pitfalls of purely bottom-up attention and segmentation. The main contribution of the paper consists in integrated visuomotor processes that allow the robot to learn object representations by manipulation without having any prior knowledge about the objects. Our experimental results show that the acquired data is of sufficient quality to train a classifier that can recognize 3D objects independently of the viewpoint.

Aiming at building versatile humanoid systems, we present in this paper the real-time implementation of behaviors which integrate walking and vision to achieve general functionalities. The paper describes how real-time — or high-bandwidth — cognitive processes can be obtained by combining vision with walking. The central point of our methodology is to use appropriate models to reduce the complexity of the search space. We will describe the models introduced in the different blocks of the system and their relationships: walking pattern, self-localization and map building, real-time reactive vision behaviors, and planning.

Knowledge representation is a traditional field in artificial intelligence. Researchers have developed various ways to represent and share information among intelligent agents. Agents that share resources, data, information, and knowledge perform better than agents working alone. However, previous research has also revealed that sharing knowledge among a large number of entities in an open environment is a problem yet to be solved. Intelligent robots are designed and produced by different manufacturers. They have various physical attributes and employ different knowledge representations. Therefore, any nonstandard or non-widely-adopted technology is unsuitable for providing a satisfactory solution to the knowledge sharing problem. In this research, we pose robot knowledge sharing as an activity to be developed in an open environment — the World Wide Web. Just as search engines like Google provide enormous power for information exchange and sharing for humans, we believe that a searching mechanism designed for intelligent agents can provide a robust approach to sharing knowledge among robots. We have developed: (1) a knowledge representation for robots that allows Internet access, (2) a knowledge organization and search indexing engine, and (3) a query/reply mechanism between robots and the search engine.