We aim natural and expressive communication on robotics and agents as same as human-human communication. There are many method to express your internal mind. Especially we use non-verbal expressions; gestures, eye-motions, or vocal intonations including vocal timbre, tone, strength. In this research we especially focus on Stuffed-toys to be natural and familiar partners.
Keywords: stuffed-toy interface, speech and music information processing, communication, multimodality

• Gaze-communicative stuffed-toy robot system based on remote gaze-tracking

  The stuffed-toys have possibility of evoking communication for elderly people. We proposes a gaze-communicative stuffed-toy robot system with joint attention and eye-contact reactions based on our remote gaze-tracking method for free and natural interaction. Corresponding to the user's gaze, the gaze-reactive stuffed-toy robot is designed to gradually establish 1) joint attention using the direction of the robot's head and 2) eye-contact reactions from several sets of motion.

• Hand-puppet interface for expressive singing voice control

  The HandySinger system is a personified tool developed to naturally express a singing voice controlled by the gestures of a hand puppet. Assuming that a singing voice is a kind of musical expression, the natural expressions of the singing voice are important for personification. We adopted a singing voice morphing algorithm that effectively smoothes out the strength of expressions delivered with a singing voice. The system consists of a hand puppet and a glove with seven bend sensors and two pressure sensors for sensitively capturing the personified puppet's gesture.

  • Crossmodality on expressive coordination of voice and gestures

    We presume that it is necessary to clarify the relationship between the expressive strengths of gestures and voice for embodied and personified representations. The results of crossmodal perceptual tests propose balancing a singing voice and gestural expressions by expanding and correcting the width and shape of the curve of expressive strength in the singing voice.

  • Expressive singing voice morphing for particular singer

    Voice is one of expressive modalities in multimodal representation for both human and anthropomorphic media. We propose to interpolate vocal expressions with adopting Speech Morphing based on STRAIGHT for smooth and natural expression of vocal timbre in singing voice. The perceptual tests shows the advantage of proposing method in comparison with the binary discrete transformation between different expressions.

I. Automatic Segmentation of Phonemes based on HMM for Database of Speech Synthesis @NTT SP Lab.

• Minimal Data Amount of Phoneme Items for HMM Model Learning (2002-03)
• Investigation on Parameters for Accurate Segmentation using HMM (2001-02)