International Journal of Humanoid Robotics

In this paper, we present Furhat — a back-projected human-like robot head using state-of-the art facial animation. Three experiments are presented where we investigate how the head might facilitate human–robot face-to-face interaction. First, we investigate how the animated lips increase the intelligibility of the spoken output, and compare this to an animated agent presented on a flat screen, as well as to a human face. Second, we investigate the accuracy of the perception of Furhat's gaze in a setting typical for situated interaction, where Furhat and a human are sitting around a table. The accuracy of the perception of Furhat's gaze is measured depending on eye design, head movement and viewing angle. Third, we investigate the turn-taking accuracy of Furhat in a multi-party interactive setting, as compared to an animated agent on a flat screen. We conclude with some observations from a public setting at a museum, where Furhat interacted with thousands of visitors in a multi-party interaction.

This paper describes findings from a Human-to-Human Interaction experiment that examines human communicative non-verbal facial behaviour. The aim was to develop a more comfortable and effective model of social human-robot communication. Analysis of the data revealed a strong co-occurrence between human blink production and non-verbal communicative behaviours of own speech instigation and completion, interlocutor speech instigation, looking at/away from the interlocutor, facial expression instigation and completion, and mental communicative state changes. Seventy-one percent of the total 2007 analysed blinks co-occurred with these behaviours within a time window of +/- 375 ms, well beyond their chance co-occurrence probability of 23%. Thus between 48% and 71% of blinks are directly related to human communicative behaviour and are not simply "physiological" (e.g., for cleaning/humidifying the eye). Female participants are found to blink twice as often as male participants, in the same communicative scenario, and have a longer average blink duration. These results provide the basis for the implementation of a blink generation system as part of a social cognitive robot for human-robot interaction.

Our research focuses on the social aspects of teleoperated androids as new media for human relationships and explores how they can contribute and encourage people to associate with others. We introduced Telenoid, a teleoperated android with a minimalistic human design, to elementary school classrooms to see how children respond to it. We found that Telenoid encourages children to work cooperatively and facilitates communication with senior citizens with dementia. Children differentiated their roles spontaneously and cooperatively participated in group work. In another class, we applied Telenoid to remote communication between schoolchildren and assisted living residents. The children felt relaxed about continuing their conversations with the elderly and positively participated in them. The results suggest that limited functionality may facilitate cooperation among participants, and varied embodiments may promote the learning process of the association with others, even those who are unfamiliar. We propose a teleoperated android as an educational tool to promote socialization.

The ability of robots to understand human characteristics and make themselves socially accepted by humans are important issues if smooth collision avoidance between humans and robots is to be achieved. When discussing smooth collision avoidance, robot should understand not only physical components such as human position, but also social components such as body pose, face orientation and proxemics (personal space during motion). We integrated these components in a modified social force model (MSFM) which allows robots to predict human motion and perform smooth collision avoidance. In the modified model, short-term intended direction is described by body pose, and a supplementary force related face orientation is added for intention estimation. Face orientation is also the best indication of the direction of personal space during motion, which was verified in preliminary experiments. Our approach was implemented and tested on a real humanoid robot in a situation in which a human is confronted with the robot in an indoor environment. Experimental results showed that better human motion tracking was achieved with body pose and face orientation tracking. Being provided with the face orientation as an indication of the intended direction, and observing the laws of proxemics in a human-like manner, the robot was able to perform avoidance motions that were more human-like when compared to the original social force model (SFM) in a face-to-face confrontation.

Head motion occurs naturally and in synchrony with speech during human dialogue communication, and may carry paralinguistic information, such as intentions, attitudes and emotions. Therefore, natural-looking head motion by a robot is important for smooth human–robot interaction. Based on rules inferred from analyses of the relationship between head motion and dialogue acts, this paper proposes a model for generating head tilting and nodding, and evaluates the model using three types of humanoid robot (a very human-like android, "Geminoid F", a typical humanoid robot with less facial degrees of freedom, "Robovie R2", and a robot with a 3-axis rotatable neck and movable lips, "Telenoid R2"). Analysis of subjective scores shows that the proposed model including head tilting and nodding can generate head motion with increased naturalness compared to nodding only or directly mapping people's original motions without gaze information. We also find that an upward motion of a robot's face can be used by robots which do not have a mouth in order to provide the appearance that utterance is taking place. Finally, we conduct an experiment in which participants act as visitors to an information desk attended by robots. As a consequence, we verify that our generation model performs equally to directly mapping people's original motions with gaze information in terms of perceived naturalness.

Affect recognition for socially perceptive robots relies on representative data. While many of the existing affective corpora and databases contain posed and decontextualized affective expressions, affect resources for designing an affect recognition system in naturalistic human–robot interaction (HRI) must include context-rich expressions that emerge in the same scenario of the final application. In this paper, we propose a context-based approach to the collection and modeling of representative data for building an affect-sensitive robotic game companion. To illustrate our approach we present the key features of the Inter-ACT (INTEracting with Robots–Affect Context Task) corpus, an affective and contextually rich multimodal video corpus containing affective expressions of children playing chess with an iCat robot. We show how this corpus can be successfully used to train a context-sensitive affect recognition system (a valence detector) for a robotic game companion. Finally, we demonstrate how the integration of the affect recognition system in a modular platform for adaptive HRI makes the interaction with the robot more engaging.

While much of the state-of-the-art research in human–robot interaction (HRI) investigates task-oriented interaction, this paper aims at exploring what people talk about to a robot if the content of the conversation is not predefined. We used the robot head Furhat to explore the conversational behavior of people who encounter a robot in the public setting of a robot exhibition in a scientific museum, but without a predefined purpose. Upon analyzing the conversations, it could be shown that a sophisticated robot provides an inviting atmosphere for people to engage in interaction and to be experimental and challenge the robot's capabilities. Many visitors to the exhibition were willing to go beyond the guiding questions that were provided as a starting point. Amongst other things, they asked Furhat questions concerning the robot itself, such as how it would define a robot, or if it plans to take over the world. People were also interested in the feelings and likes of the robot and they asked many personal questions — this is how Furhat ended up with its first marriage proposal. People who talked to Furhat were asked to complete a questionnaire on their assessment of the conversation, with which we could show that the interaction with Furhat was rated as a pleasant experience.

Communication between humans and robots is a very important aspect in the field of Humanoid Robotics. For a natural interaction, robots capable of nonverbal communication must be developed. However, despite the most recent efforts, robots still can show only limited expression capabilities. The purpose of this work is to create a facial expression generator that can be applied to the 24 DoF head of the humanoid robot KOBIAN-R. In this manuscript, we present a system that based on relevant studies of human communication and facial anatomy can produce thousands of combinations of facial and neck movements. The wide range of expressions covers not only primary emotions, but also complex or blended ones, as well as communication acts that are not strictly categorized as emotions. Results showed that the recognition rate of expressions produced by this system is comparable to the rate of recognition of the most common facial expressions. Context-based recognition, which is especially important in case of more complex communication acts, was also evaluated. Results proved that produced robotic expressions can alter the meaning of a sentence in the same way as human expressions do. We conclude that our system can successfully improve the communication abilities of KOBIAN-R, making it capable of complex interaction in the future.

This paper presents a natural and comfortable communication system between human and robot based on synchronization to human emotional state using human facial expression recognition. The system consists of three parts: human emotion recognition, robotic emotion generation, and robotic emotion expression. The robot recognizes human emotion through human facial expressions, and robotic emotion is generated and synchronized with human emotion dynamically using a vector field of dynamics. The robot makes dynamically varying facial expressions to express its own emotions to the human. A communication experiment was conducted to examine the effectiveness of the proposed system. The authors found that subjects became much more comfortable after communicating with the robot with synchronized emotions. Subjects felt somewhat uncomfortable after communicating with the robot with non-synchronized emotions. During emotional synchronization, subjects communicated much more with the robot, and the communication time was double that during non-synchronization. Furthermore, in the case of emotional synchronization, subjects had good impressions of the robot, much better than the impressions in the case of non-synchronization. It was confirmed in this study that emotional synchronization in human–robot communication can be effective in making humans comfortable and makes the robot much more favorable and acceptable to humans.

In human–human interactions, a consciously perceived high degree of self–other overlap is associated with a higher degree of integration of the other person's actions into one's own cognitive representations. Here, we report data suggesting that this pattern does not hold for human–robot interactions. Participants performed a social Simon task with a robot, and afterwards indicated the degree of self–other overlap using the Inclusion of the Other in the Self (IOS) scale. We found no overall correlation between the social Simon effect (as an indirect measure of self–other overlap) and the IOS score (as a direct measure of self–other overlap). For female participants we even observed a negative correlation. Our findings suggest that conscious and unconscious evaluations of a robot may come to different results, and hence point to the importance of carefully choosing a measure for quantifying the quality of human–robot interactions.