Interdisciplinary Approaches to Robot Learning

The following sections are included:

• Program Committee

• CONTENTS

Please refer to full text.

We discuss three mechanisms for bootstrapping a robot towards higher complexity through embodied interaction dynamics with the environment. With self-exploration, the system learns a structured lower-complexity sensorimotor space, shaped by its natural body dynamics and its reflexes. Categorization of the sensorimotor patterns obtained over the system's interaction enables the system to learn about its basic sensorimotor couplings, refine them and hence enable the emergence of stable sensorimotor configurations over time. Finally, by entraining other environmental dynamics, the system widens its repertoire of interactive dynamics. The integration of the above mechanisms forms the filter hypothesis. We describe the neural implementation of the filter and report our recent experiments on a 4DOF redundant foveated robot-head.

Accurate oculomotor control is one of the essential pre-requisites for successful visuomotor coordination. In this paper, we suggest a biologically inspired control system for learning gaze stabilization with a biomimetic robotic oculomotor system. In a stepwise fashion, we develop a control circuit for the vestibulo-ocular reflex (VOR) and the opto-kinetic response (OKR), and add a nonlinear learning network to allow adaptivity. We discuss the parallels and differences of our system with biological oculomotor control and suggest solutions how to deal with nonlinearities and time delays in the control system. In simulation and actual robot studies, we demonstrate that our system can learn gaze stabilization in real time in only a few seconds with high final accuracy.

Taking inspiration from animal strategies, we propose an artificial neural network model for robot visual navigation in a maze-like environment. It is composed of two structures. The first one enables visual homing. The second one builds a “cognitive map” of the environment in order to plan a path. This architecture is successfully tested on various action selection problems involving the satisfaction of several drives. The importance to never uncouple the motivations and the planning system during the cognitive map building will be emphasized for learning complex and partially contradictory strategies.

We aim to show in this paper that a cortically-inspired model applied to the control of an autonomous robot can provide it with strong capabilities. First, such an approach enables a robot to extract spatio-temporal regularities during its “life” in an unknown environment, these regularities being stored in a more expressive way than the Q-values. Second, a compromise between perception and drives emerges from the distributed mechanism of spreading activities through the cortical net. Endly, the multi-modal management of information flows endows the robot with capabilities to generalize sequences of perceptions and to use this knowledge to transfer learning from those learned sequences to further behavior.

We insist in the paper on three learning mechanisms used in the model, and more precisely on the temporal causality learning rule, which is an original contribution of our work and whose robustness is tested.

The cost of gathering and labelling data is remaining constant, yet computing power is becoming cheaper. This makes “lifelong learning” agents attractive. A lifelong learning agent is a system that learns knowledge and, when thereafter learning novel tasks, can transfer this knowledge to increase the accuracy of what is learned subsequently or to reduce the number of examples necessary for subsequent learning. In this article some open problems in lifelong learning are reviewed including whether an agent can exploit multiple sources of learned knowledge, would an agent benefit from different types of learned knowledge, and how should such an agent be built? Two contributions are described – the sequential transfer of a particular type of learned knowledge, and combining multiple mechanisms for transfer in a single agent – in addition to ongoing work toward answering these questions in the context of a lifelong learning mobile robot agent.

Imitation is a powerful mechanism for efficient learning of novel behaviors that both supports and takes advantage of sociality. A fundamental problem for imitation is to create an appropriate (partial) mapping between the body of the system being imitated and the imitator. By considering for each of these two systems an associated automata (resp. transformation semigroup) structure, attempts at such mapping can be considered (partial) relational homomorphisms. This chapter shows how mathematical techniques can be applied to characterize how far a behavior is from a successful imitation and to evaluate attempts at imitation arising from a particular correspondence between the imitator's and model's transformation semigroups.

For the imitator and the imitated, affordances in the agent-environment structural coupling are likely to be different, all the more so in the case of dissimilar embodiment. We argue that the use of what is afforded to the imitator to attain corresponding effects or, as in dance, sequences of effects, is necessary and sufficient for successful imitation.

Moreover, the judged degree of success or failure of an imitation depends on some externally imposed or — in the case of automonous agents — internally determined criteria on effects of the attempted imitative behavior (including effects attained successively as well as final effects). We discuss how identification of states in the system-environment coupling which are equivalent for such ‘observer-dependent’ purposes allows one to formally define successful imitation with respect to such criteria. The resulting measures can be used to compare various candidate mappings (e.g. body plan or perception-action correspondences). Additionally, this may be applied in the automated construction of mappings to be used in imitation for artificial, hardware and software systems.

Shaping is a way in which a human designer can provide assistance to a learning system to enable it to solve problems that would otherwise defeat it. The experiments reported here explore a variety of shaping techniques used in conjunction with a genetic programming based system, in order to develop controllers for visual tracking tasks. Controllers are evolved in simulation using shaping, and are then transferred successfully to a real robot head.

Many robot control systems have been evolved within simulated environments. Some of these have been transferred to real robots. Several have been evolved completely on real robots. Each of these methods has its drawbacks. This paper presents a system, Preston, which evolves robot controllers in the form of sequences of behaviours. These sequences are tested on a real robot and the successful results are fed back into another loop of the evolution. This paper concentrates on the development of some simple low level primitive behaviours, and how evolved sequences of these act in the real world. The repeatability of the low level behaviours is examined, along wkh the effect this has on the evolved behavioural sequences.