International Journal of Humanoid Robotics

What is available to developmental programs in autonomous mental development, and what should be learned at the very early stages of mental development? Our observation is that sensory and motor primitives are the most basic components present at the beginning, and what developmental agents need to learn from these resources is what their internal sensory states stand for. In this paper, we investigate the question in the context of a simple biologically motivated visuomotor agent. We observe and acknowledge, as many other researchers do, that action plays a key role in providing content to the sensory state. We propose a simple, yet powerful learning criterion, that of invariance, where invariance simply means that the internal state does not change over time. We show that after reinforcement learning based on the invariance criterion, the property of action sequence based on an internal sensory state accurately reflects the property of the stimulus that triggered that internal state. That way, the meaning of the internal sensory state can be firmly grounded on the property of that particular action sequence. We expect the framing of the problem and the proposed solution presented in this paper to help shed new light on autonomous understanding in developmental agents such as humanoid robots.

The new field of developmental robotics faces the formidable challenge of implementing effective learning mechanisms in complex, dynamic environments. We make a case that knowledge-based learning algorithms might help to meet this challenge. A constructive neural learning algorithm, knowledge-based cascade-correlation (KBCC), autonomously recruits previously-learned networks in addition to the single hidden units recruited by ordinary cascade-correlation. This enables learning by analogy when adequate prior knowledge is available, learning by induction from examples when there is no relevant prior knowledge, and various combinations of analogy and induction. A review of experiments with KBCC indicates that recruitment of relevant existing knowledge typically speeds learning and sometimes enables learning of otherwise impossible problems. Some additional domains of interest to developmental robotics are identified in which knowledge-based learning seems essential. The characteristics of KBCC in relation to other knowledge-based neural learners and analogical reasoning are summarized as is the neurological basis for learning from knowledge. Current limitations of this approach and directions for future work are discussed.

Currently, there is a lack of general-purpose, in-place learning engines that incrementally learn multiple tasks, to develop "soft" multi-task-shared invariances in the intermediate internal representation while a developmental robot interacts with its environment. In-place learning is a biologically inspired concept, rooted in the genomic equivalence principle, meaning that each neuron is responsible for its own development while interacting with its environment. With in-place learning, there is no need for a separate learning network. Computationally, biologically inspired, in-place learning provides unusually efficient learning algorithms whose simplicity, low computational complexity, and generality are set apart from typical conventional learning algorithms. We present in this paper the multiple-layer in-place learning network (MILN) for this ambitious goal. As a key requirement for autonomous mental development, the network enables both unsupervised and supervised learning to occur concurrently, depending on whether motor supervision signals are available or not at the motor end (the last layer) during the agent's interactions with the environment. We present principles based on which MILN automatically develops invariant neurons in different layers and why such invariant neuronal clusters are important for learning later tasks in open-ended development. From sequentially sensed sensory streams, the proposed MILN incrementally develops a hierarchy of internal representations. The global invariance achieved through multi-layer invariances, with increasing invariance from early layers to the later layers. Experimental results with statistical performance measures are presented to show the effects of the principles.

It is the goal of the Language Acquisition Group at the University of Illinois at Urbana-Champaign (LAR-UIUC) to build a robot that is able to learn language as well as humans through embodied sensori-motor interaction with the physical world. This paper proposes cognitive structures to enable an autonomous robot to learn the syntax of two-word sentences using its understanding of lexical semantics. A production rule of syntax in Chomsky Normal Form will be explicitly represented using a hidden Markov Model. Results of robotic experiments show that these models can learn representations of syntax in this form and that they can be used to produce novel sentences.

This paper proposes a learning model that enables a robot to acquire a body image for parts of its body that are invisible to itself. The model associates spatial perception based on motor experience and motor image with perception based on the activations of touch sensors and tactile image, both of which are supported by visual information. The tactile image can be acquired with the help of the motor image, which is thought to be the basis for spatial perception, because all spatial perceptions originate in motor experiences. Based on the proposed model, a robot estimates invisible hand positions using the Jacobian between the displacement of the joint angles and the optical flow of the hand. When the hand touches one of the invisible tactile sensor units on the face, the robot associates this sensor unit with the estimated hand position. The simulation results show that the spatial arrangement of tactile sensors is successfully acquired by the proposed model.

There is currently a division between real-world human performance and the decision making of socially interactive robots. This circumstance is partially due to the difficulty in estimating human cues, such as pose and gesture, from robot sensing. Towards bridging this division, we present a method for kinematic pose estimation and action recognition from monocular robot vision through the use of dynamical human motion vocabularies. Our notion of a motion vocabulary is comprised of movement primitives that structure a human's action space for decision making and predict human movement dynamics. Through prediction, such primitives can be used to both generate motor commands for specific actions and perceive humans performing those actions. In this paper, we focus specifically on the perception of human pose and performed actions using a known vocabulary of primitives. Given image observations over time, each primitive infers pose independently using its expected dynamics in the context of a particle filter. Pose estimates from a set of primitives inferencing in parallel are arbitrated to estimate the action being performed. The efficacy of our approach is demonstrated through interactive-time pose and action recognition over extended motion trials. Results evidence our approach requires small numbers of particles for tracking, is robust to unsegmented multi-action movement, movement speed, camera viewpoint and is able to recover from occlusions.

Several algorithms and models have recently been proposed for imitation learning in humans and robots. However, few proposals offer a framework for imitation learning in noisy stochastic environments where the imitator must learn and act under real-time performance constraints. We present a novel probabilistic framework for imitation learning in stochastic environments with unreliable sensors. Bayesian algorithms, based on Meltzoff and Moore's AIM hypothesis for action imitation, implement the core of an imitation learning framework. Our algorithms are computationally efficient, allowing real-time learning and imitation in an active stereo vision robotic head and on a humanoid robot. We present simulated and real-world robotics results demonstrating the viability of our approach. We conclude by advocating a research agenda that promotes interaction between cognitive and robotic studies of imitation.

The inherent value system of a developmental agent enables autonomous mental development to take place right after the agent's "birth." Biologically, it is not clear what basic components constitute a value system. In the computational model introduced here, we propose that inherent value systems should have at least three basic components: punishment, reward and novelty with decreasing weights from the first component to the last. Punishments and rewards are temporally sparse but novelty is temporally dense. We present a biologically inspired computational architecture that guides development of sensorimotor skills through real-time interactions with the environments, driven by an inborn value system. The inherent value system has been successfully tested on an artificial agent in a simulation environment and a robot in the real world.

Computational models of development aim to describe the mechanisms that underlie the acquisition of new skills or the emergence of new capabilities. The strength of a model is judged by both its ability to explain the phenomena in question as well as its ability to generate new hypotheses, generalize to new situations, and provide a unifying conceptual framework. Although often constructed using traditional engineering methodologies, evaluating the performance of a computational model of development in terms of traditional perspectives is a flawed approach. This paper addresses the fundamental issues that confound quantitative analysis of computational models of developmental systems. In particular, we focus on the following recommendations: (i) do not equate the success of a developmental model with its peak performance at some task; (ii) do not employ purely subjective or vague measures of model fitness; and (iii) do not hide or reject variation as found in the computational model. Along the way, we discuss the aspects of computational models of development that lead to the requirements for specialized methods of analysis.