Abstract
Communication between robots and humans is a challenge in complex built environments. In this project, we are exploring how previous achievements in this area apply to human/robot communication within immersive virtual displays. In our concrete example, we host human-scale robots that interact with humans in our Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab). The CRAIVE-Lab provides a physical/digital environment for collaborative tasks using seamless video from multiple projectors and a 128-channel wave-field system. The system is designed to monitor the whole floor-space area (10 m × 12 m) with a camera network mounted to the ceiling grid and a microphone array combining shotgun, spherical and possibly people-worn microphones. Based on the sensor data, a visual analysis and an auditory scene analysis are performed. The latter includes sound localization, speech and musical-feature recognition. The robots, a Rethink Baxter robot mounted on an electric wheelchair and a Robokind Zeno, are used to perform assistive technology and social communication tasks. Both robots have direct access to these data using a digital feed over a wireless network to augment their own sensor systems of built-in cameras and microphones. [Work supported by NSF grant #1229391.]
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
