Robot Perception Research Articles

Perceived robotic server qualities and functions explaining customer loyalty in the theme park context

Purpose This paper aims to explore theme park visitors’ attitudes toward interacting with robots and investigated the qualities and functions of robotic servers and their influence on customers’ loyalty. A structural equation modeling approach was used to identify the complex relationships among variables in the entire network. Design/methodology/approach An online survey randomly assigned respondents to four different robotic server scenarios with robots that look like humans, animals, cartoon characters and anime features. The influence of robot types was investigated by manipulating robot type with four different pictures; however, the data were analyzed with a structural equation modeling model to identify the complex relationships rather than one-way analysis of variance to identify influences of robot types on different variables in separate analyzes. Findings The data collected from the 385 experienced theme park visitors revealed that perception of robots with human orientation and safety qualities had the strongest effect on the perceived robotic functionality, while emotions and co-creation qualities hardly had any effect on the perceived functionality, which included utilitarian rather than experiential functions such as excitement. Human orientation qualities, regardless of the specific robotic design, had a significant impact on perceived robotic functionality. The study also revealed a strong positive influence of perceived robotic functionality on customer loyalty. Originality/value The debate of whether or not to introduce and blend the growing robotic technology into the theme park experience is in its infancy. The study contributes to the theory of how robotics qualities and functions can augment customer loyalty.

Autonomous dishwasher loading from cluttered trays using pre‐trained deep neural networks

AbstractAutonomous dishwasher loading is a benchmark problem in robotics that highlights the challenges of robotic perception, planning, and manipulation in an unstructured environment. Current approaches resort to a specialized solution, however, these technologies are not viable in a domestic setting. Learning‐based solutions seem promising for a general purpose solutions; however, they require large amounts of catered data to be applied in real‐world scenarios. This article presents a novel learning‐based solution without a training phase using pre‐trained object detection networks. By developing a perception, planning, and manipulation framework around an off‐the‐shelf object detection network, we are able to develop robust pick‐and‐place solutions that are easy to develop and general purpose requiring only a RGB feedback and a pinch gripper. Analysis of a real‐world canteen tray data is first performed and used for developing our in‐lab experimental setup. Our results obtained from real‐world scenarios indicate that such approaches are highly desirable for plug‐and‐play domestic applications with limited calibration. All the associated data and code of this work are shared in a public repository.

Neural network‐based semantic segmentation model for robot perception of driverless vision

Driverless vision is one of the important applications of robot perception. With the development of driverless vehicles, the perception and understanding of the surrounding environment are becoming more and more important. When the types of surrounding objects are too complex, the ability of the computer to recognise the environment is poor. To improve the recognition accuracy of the computer and enhance the ability of segmentation, in this study, depth estimation is used to predict depth information to assist semantic segmentation, and then edge features of objects are introduced to enhance the contour of objects. A neural network-based semantic segmentation model is proposed. Finally, the intrinsic mechanism of attention is used to increase the correlation between channels. The experimental results on the CamVid data set show that this model can obtain better evaluation results and improve the segmentation accuracy of images compared with other models.

Crop Height Measurement System Based on 3D Image and Tilt Sensor Fusion

Machine-vision-based crop detection is a central issue for digital farming, and crop height is an important factor that should be automatically measured in robot-based cultivations. Three-dimensional (3D) imaging cameras make it possible to measure actual crop height; however, camera tilt due to irregular ground conditions in farmland prevents accurate height measurements. In this study, stereo-vision-based crop height was measured with compensation for the camera tilt effect. For implementing the tilt of the camera installed on farm machines (e.g., tractors), we developed a posture tilt simulator for indoor testing that could implement the camera tilt by pitch and roll rotations. Stereo images were captured under various simulator tilt conditions, and crop height was measured by detecting the crop region in a disparity map, which was generated by matching stereo images. The measured height was compensated for by correcting the position of the region of interest (RoI) in the 3D image through coordinate transformation between camera coordinates and simulator coordinates. The tests were conducted by roll and pitch rotation around the simulator coordinates. The results showed that crop height could be measured using stereo vision, and that tilt compensation reduced the average error from 15.6 to 3.9 cm. Thus, the crop height measurement system proposed in this study, based on 3D imaging and a tilt sensor, can contribute to the automatic perception of agricultural robots.

Robots are friends as well as foes: Ambivalent attitudes toward mindful and mindless AI robots in the United States and China

In light of the ongoing and rapid development of innovative technologies, two intriguing issues arise: do people have more positive or more negative attitudes toward robots with high (versus low) mental capabilities, and do attitudes toward robots differ between Western and East Asian cultures? Past work on these topics has produced contradictory results. Inspired by the perspective that attitudes are ambivalent rather than bipolar, we argue that these controversial findings stem from people's ambivalent attitudes toward robots. To test the assumption that ambivalent attitudes toward robots differ by type of robots and by cultural background, we conducted an experimental study. By manipulating the level of robot mind and recruiting both American and Chinese participants, we examined how robot mind and culture influence ambivalent attitudes toward robots. We simultaneously measured participants' perceptions of robots as “ally” or “enemy”. The results revealed that robots with high (versus low) mental abilities elicited more ambivalent attitudes and that American participants reported more ambivalence toward robots than Chinese participants. These findings enhance our understanding of human–robot interaction and provide guidance for modulating people's attitudes toward robots.

Nengo and Low-Power AI Hardware for Robust, Embedded Neurorobotics.

In this paper we demonstrate how the Nengo neural modeling and simulation libraries enable users to quickly develop robotic perception and action neural networks for simulation on neuromorphic hardware using tools they are already familiar with, such as Keras and Python. We identify four primary challenges in building robust, embedded neurorobotic systems, including: (1) developing infrastructure for interfacing with the environment and sensors; (2) processing task specific sensory signals; (3) generating robust, explainable control signals; and (4) compiling neural networks to run on target hardware. Nengo helps to address these challenges by: (1) providing the NengoInterfaces library, which defines a simple but powerful API for users to interact with simulations and hardware; (2) providing the NengoDL library, which lets users use the Keras and TensorFlow API to develop Nengo models; (3) implementing the Neural Engineering Framework, which provides white-box methods for implementing known functions and circuits; and (4) providing multiple backend libraries, such as NengoLoihi, that enable users to compile the same model to different hardware. We present two examples using Nengo to develop neural networks that run on CPUs and GPUs as well as Intel's neuromorphic chip, Loihi, to demonstrate two variations on this workflow. The first example is an implementation of an end-to-end spiking neural network in Nengo that controls a rover simulated in Mujoco. The network integrates a deep convolutional network that processes visual input from cameras mounted on the rover to track a target, and a control system implementing steering and drive functions in connection weights to guide the rover to the target. The second example uses Nengo as a smaller component in a system that has addressed some but not all of those challenges. Specifically it is used to augment a force-based operational space controller with neural adaptive control to improve performance during a reaching task using a real-world Kinova Jaco2 robotic arm. The code and implementation details are provided1, with the intent of enabling other researchers to build and run their own neurorobotic systems.

Event-Based Robotic Grasping Detection With Neuromorphic Vision Sensor and Event-Grasping Dataset.

Robotic grasping plays an important role in the field of robotics. The current state-of-the-art robotic grasping detection systems are usually built on the conventional vision, such as the RGB-D camera. Compared to traditional frame-based computer vision, neuromorphic vision is a small and young community of research. Currently, there are limited event-based datasets due to the troublesome annotation of the asynchronous event stream. Annotating large scale vision datasets often takes lots of computation resources, especially when it comes to troublesome data for video-level annotation. In this work, we consider the problem of detecting robotic grasps in a moving camera view of a scene containing objects. To obtain more agile robotic perception, a neuromorphic vision sensor (Dynamic and Active-pixel Vision Sensor, DAVIS) attaching to the robot gripper is introduced to explore the potential usage in grasping detection. We construct a robotic grasping dataset named Event-Grasping dataset with 91 objects. A spatial-temporal mixed particle filter (SMP Filter) is proposed to track the LED-based grasp rectangles, which enables video-level annotation of a single grasp rectangle per object. As LEDs blink at high frequency, the Event-Grasping dataset is annotated at a high frequency of 1 kHz. Based on the Event-Grasping dataset, we develop a deep neural network for grasping detection that considers the angle learning problem as classification instead of regression. The method performs high detection accuracy on our Event-Grasping dataset with 93% precision at an object-wise level split. This work provides a large-scale and well-annotated dataset and promotes the neuromorphic vision applications in agile robot.

LIVING WITH ROBOTS: AN ONTOLOGICAL LEAP?

Artificial intelligence (AI) and robotics applications have proliferated primarily in the industrial sphere, and social scientific studies emphasized robots’ functionality and appropriateness for certain roles, especially those related to work and most particularly to robots replacing humans’ jobs. Notably, robot studies are often premised on negative prognostications, emphasizing how robots threaten livelihoods and are disruptive. As AI and robot technologies advance, however, more positive possibilities arise for robots’ social integration. However, there is an ontological divide between humans and machines that will likely influence people’s responses to and interactions with these emerging technologies. People may logically know and intend to treat robots as mere technological tools, but reflexively respond to them socially. This study explores these possible dynamics and examines how people perceive robots as social and human-like entities. A qualitative analysis of open-ended comments (N=591) collected through a survey on robot perceptions was conducted. Five main themes about social life with robots were revealed: robot as tool/machine (32.5% of comments), human-robot relationships (26.9%), social adjustment (19.0%), robot rights (10.7%), and robots’ aliveness/appearance (10.0%). The findings show that human-robot ontology is an important consideration for robots’ social acceptance and integration. AI-supported robotic technology presents great promise; however, its advancement challenges the ontological divide that has implications not only for human-machine interaction but also for self-identity and ultimately human-human relations. In terms of suggestions, these dynamics should be explored in tandem with ways to improve human-machine communication and considered from usability and design standpoints.

Interact With Me: An Exploratory Study on Interaction Factors for Active Physical Human-Robot Interaction

In future robotic applications in environments such as nursing houses, construction sites, private homes, etc, robots might need to take unpredicted physical actions according to the state of the users to overcome possible human errors. Referring to these actions as active physical human-robot interactions ( active pHRI ), in this letter, the goal is to verify the possibility of identifying measurable interaction factors that could be used in future active pHRI controllers, by exploring and analyzing the state of the users during active pHRI. We hypothesize that active physical robot actions can cause measurable alterations in the physical and physiological data of the users, and that these measurements could be interpreted with users’ personality and perceptions. We design an experiment where the participant uses the robot to play a visual puzzle game, during which, the robot takes unanticipated physical actions. We collect physiological and physical data, as well as outcomes of two state-of-the-art questionnaires on the perceptions of robots, CH-33 and Godspeed Series Questionnaires (GSQ), and a pre-experiment personality questionnaire, to relate the collected data with the users’ perceptions and personality. The experiment outcomes show that we can extract a few factors related to personality, perception, physiological, and physical measurements. Even though we could not draw very clear correlations, these outcomes give fundamental insights for the design of novel pHRI experiments.

Near-hysteresis-free soft tactile electronic skins for wearables and reliable machine learning.

Electronic skins are essential for real-time health monitoring and tactile perception in robots. Although the use of soft elastomers and microstructures have improved the sensitivity and pressure-sensing range of tactile sensors, the intrinsic viscoelasticity of soft polymeric materials remains a long-standing challenge resulting in cyclic hysteresis. This causes sensor data variations between contact events that negatively impact the accuracy and reliability. Here, we introduce the Tactile Resistive Annularly Cracked E-Skin (TRACE) sensor to address the inherent trade-off between sensitivity and hysteresis in tactile sensors when using soft materials. We discovered that piezoresistive sensors made using an array of three-dimensional (3D) metallic annular cracks on polymeric microstructures possess high sensitivities (> 107 Ω ⋅ kPa-1), low hysteresis (2.99 ± 1.37%) over a wide pressure range (0-20 kPa), and fast response (400 Hz). We demonstrate that TRACE sensors can accurately detect and measure the pulse wave velocity (PWV) when skin mounted. Moreover, we show that these tactile sensors when arrayed enabled fast reliable one-touch surface texture classification with neuromorphic encoding and deep learning algorithms.

Technology acceptance and perceptions of robotic assistive devices by older adults – implications for exoskeleton design

Aim This study explored and interpreted insights expressed by a cohort of older adults related to their life experience, their experiences using or assisting someone with assistive devices, and their perceptions of robots and robotic assistive devices, including lower limb exoskeletons. Method A grounded theory study was undertaken with 24 older adult participants over five months. Each participant participated in a structured interviewed regarding their experiences with technologies and in particular their perceptions of assistive technologies. Themes from the interviews were coded using Nvivo software. Results Five main themes emerged from this study – (1) Aging & life stage experiences, (2) Quality of Life, (3) Assistive Technologies, (4) Health Conditions & Care, (5) Products & Service Systems. These have influenced new constructs for a hybrid design tool that incorporates stages of Usability and TAMs (Technology Acceptance Models) to gauge (a) Perception, (b) Experience and (c) Perceived Impact by older adults of lower limb exoskeletons. Conclusions: Emerging technologies such as robotic assistive devices require a specific enquiry to understand how best to optimise acceptance by older adults and avoid feelings by them of frustration, embarrassment and ultimately abandonment of these devices. Implications for rehabilitation Older adults frequently require rehabilitation and assistance with ambulation Exoskeletons are forms of assistive technologies for rehabilitation, and they are moving from clinical use to more day care use, including as part of daily living These results help explain factors related to the perception of exoskeletons by older adults, which if considered during exoskeleton design, could improve the technology uptake and compliance with this technology use by these users

Photometric Stereo-Based Depth Map Reconstruction for Monocular Capsule Endoscopy

The capsule endoscopy robot can only use monocular vision due to the dimensional limit. To improve the depth perception of the monocular capsule endoscopy robot, this paper proposes a photometric stereo-based depth map reconstruction method. First, based on the characteristics of the capsule endoscopy robot system, a photometric stereo framework is established. Then, by combining the specular property and Lambertian property of the object surface, the depth of the specular highlight point is estimated, and the depth map of the whole object surface is reconstructed by a forward upwind scheme. To evaluate the precision of the depth estimation of the specular highlight region and the depth map reconstruction of the object surface, simulations and experiments are implemented with synthetic images and pig colon tissue, respectively. The results of the simulations and experiments show that the proposed method provides good precision for depth map reconstruction in monocular capsule endoscopy.

Hotel managers’ perceptions towards the use of robots: a mixed-methods approach

Adopting a supply-side perspective, the paper analyses Bulgarian hotel managers’ perceptions of service robots using a convergent mixed methods design. Structured quantitative data were collected from 79 managers using a questionnaire, while interviews were used for the collection of qualitative data from 20 managers. The findings indicate respondents feel that repetitive, dirty, dull, and dangerous tasks in hotels would be more appropriate for robots, while hotel managers would rather use employees for tasks that require social skills and emotional intelligence. The individual characteristics of respondents and the organisational characteristics of the hotels they currently worked in played little role in their perceptions of service robots. The managers considered that robots would decrease the quality of the service and were generally not ready to use robots. Additionally, the interviewees indicated that skilled and well-trained employees were more valuable and more adequate than robots for the hospitality and tourism industry. Theoretical and managerial implications are provided as well.

Measuring the Perceived Social Intelligence of Robots

Robotic social intelligence is increasingly important. However, measures of human social intelligence omit basic skills, and robot-specific scales do not focus on social intelligence. We combined human robot interaction concepts of beliefs, desires, and intentions with psychology concepts of behaviors, cognitions, and emotions to create 20 Perceived Social Intelligence (PSI) Scales to comprehensively measure perceptions of robots with a wide range of embodiments and behaviors. Participants rated humanoid and non-humanoid robots interacting with people in five videos. Each scale had one factor and high internal consistency, indicating each measures a coherent construct. Scales capturing perceived social information processing skills (appearing to recognize, adapt to, and predict behaviors, cognitions, and emotions) and scales capturing perceived skills for identifying people (appearing to identify humans, individuals, and groups) correlated strongly with social competence and constituted the Mind and Behavior factors. Social presentation scales (appearing friendly, caring, helpful, trustworthy, and not rude, conceited, or hostile) relate more to Social Response to Robots Scales and Godspeed Indices, form a separate factor, and predict positive feelings about robots and wanting social interaction with them. For a comprehensive measure, researchers can use all PSI 20 scales for free. Alternatively, they can select the most relevant scales for their projects.

Introducing service robotics in inpatient geriatric care\u2014a\xa0qualitative systematic review from a\xa0human resources perspective

This article in the journal Gruppe. Interaktion. Organisation. (GIO) presents a qualitative systematic review of nursing home staff perspectives on the usage of robotic devices in geriatric nursing. Robotic devices have been introduced to the field of inpatient geriatric care only recently. Although the elderly are seen as the primary users of many robotic devices in geriatric care, staff members are the driving force in the implementation and application of these devices. Hence, the Human Resources department has an important role to play in the introduction of service robotics in inpatient geriatric care. In the context of the present study, 22 empirical studies could be identified which were analysed based on the HR framework of Beer et al. (1984). Results hint at differences in staff perception of social robots and robots that support physical activities. Based on the discussion of the findings, five practical recommendations for the introduction of service robotics in inpatient geriatric care are made.

Multifunctional Self‐Powered E‐Skin with Tactile Sensing and Visual Warning for Detecting Robot Safety

AbstractFlexible electronic skins (e‐skins) are widely applied in human activities monitoring, medical health, and robotic perception. However, most of the reported e‐skins only focus on the physical force sensing. Herein, a novel multifunctional e‐skin possessing physical force sensing and real‐time visualization is proposed. The vertical‐separation triboelectric sensor is used as the tactile sensing units, showing excellent performance with the open‐circuit voltage (V0) of 376.8 V, the short‐circuit (Is) of 1.02 µA, and the maximum output power density of 7.225 W m−2, which also can carry out pressure detection with a wide detection range from 55.5 pa to 14 Kpa, a high sensitivity of 57.76 mV pa−1, and a fast response time of 5.6 ms. The electrochromic devices as the visualization window shows the excellent stability and the low driving voltage. This e‐skin can detect the instantaneous external force applied on the human arm and robotic arm and simultaneously shows the colors varying from light green to dark blue when experiencing the applied force. The proposed e‐skin shows a promising solution of realizing both the digitized physical force sensing and the direct visualization, which can greatly enhance the safety precaution in various fields, such as robotic production line.

Frontline robots in tourism and hospitality: service enhancement or cost reduction?

Robots are being implemented in many frontline services, from waiter robots in restaurants to robotic concierges in hotels. A growing number of firms in hospitality and tourism industries introduce service robots to reduce their operational costs and to provide customers with enhanced services (e.g. greater convenience). In turn, customers may consider that such a disruptive innovation is altering the established conditions of the service-provider relationship. Based on attribution theory, this research explores how customers’ attributions about the firm motivations to implement service robots (i.e. cost reduction and service enhancement) are affecting customers’ intentions to use and recommend this innovation. Following previous research on robot’s acceptance, our research framework analyzes how these attributions may be shaped by customers’ perceptions of robot’s human-likeness and their affinity with the robot. Structural equation modelling is used to analyze data collected from 517 customers evaluating service robots in the hospitality industry; results show that attributions mediate the relationships between affinity toward the robot and customer behavioral intentions to use and recommend service robots. Specifically, customer’s affinity toward the service robot positively affects service improvement attribution, which in turn has a positive influence on customer behavioral intentions. In contrast, affinity negatively affects cost reduction attribution, which in turn has a negative effect on behavioral intentions. Finally, human-likeness has a positive influence on affinity. This research provides practitioners with empirical evidence and guidance about the introduction of service robots and its relational implications in hospitality and tourism industries. Theoretical advances and future research avenues are also discussed.

On Robustness of Multi-Modal Fusion—Robotics Perspective

The efficient multi-modal fusion of data streams from different sensors is a crucial ability that a robotic perception system should exhibit to ensure robustness against disturbances. However, as the volume and dimensionality of sensory-feedback increase it might be difficult to manually design a multimodal-data fusion system that can handle heterogeneous data. Nowadays, multi-modal machine learning is an emerging field with research focused mainly on analyzing vision and audio information. Although, from the robotics perspective, haptic sensations experienced from interaction with an environment are essential to successfully execute useful tasks. In our work, we compared four learning-based multi-modal fusion methods on three publicly available datasets containing haptic signals, images, and robots’ poses. During tests, we considered three tasks involving such data, namely grasp outcome classification, texture recognition, and—most challenging—multi-label classification of haptic adjectives based on haptic and visual data. Conducted experiments were focused not only on the verification of the performance of each method but mainly on their robustness against data degradation. We focused on this aspect of multi-modal fusion, as it was rarely considered in the research papers, and such degradation of sensory feedback might occur during robot interaction with its environment. Additionally, we verified the usefulness of data augmentation to increase the robustness of the aforementioned data fusion methods.

Cross-View Semantic Segmentation for Sensing Surroundings

Sensing surroundings plays a crucial role in human spatial perception, as it extracts the spatial configuration of objects as well as the free space from the observations. To facilitate the robot perception with such a surrounding sensing capability, we introduce a novel visual task called Cross-view Semantic Segmentation as well as a framework named View Parsing Network (VPN) to address it. In the cross-view semantic segmentation task, the agent is trained to parse the first-view observations into a top-down-view semantic map indicating the spatial location of all the objects at pixel-level. The main issue of this task is that we lack the real-world annotations of top-down-view data. To mitigate this, we train the VPN in 3D graphics environment and utilize the domain adaptation technique to transfer it to handle real-world data. We evaluate our VPN on both synthetic and real-world agents. The experimental results show that our model can effectively make use of the information from different views and multi-modalities to understanding spatial information. Our further experiment on a LoCoBot robot shows that our model enables the surrounding sensing capability from 2D image input. Code and demo videos can be found at \url{https://view-parsing-network.github.io}.

유아의 성별에 따른 로봇의 기능과 외형에 대한 인식

본 연구의 목적은 로봇의 기능에 대한 인식 및 외형에 대한 선호가 유아의 성별에 따라 차이가 있는지를 알아보는 것이다. 이를 위하여 만 5세 유아들을 대상으로 로봇 그림그리기와 로봇 이미지를 사용한 면접을 하였다. 연구대상은 경상남도 J시에 소재한 2개 어린이집의 만 5세 남아 30명, 여아 30명으로 2019년 8월19일부터 8월30일 까지 실시하였다. 연구 결과는 다음과 같다. 첫째, 로봇의 기능에 대한 인식은 유아의 성별에 따라 차이가 있었으나, 가정용 로봇의 기능에 대한 기대는 유아의 성별에 따라 통계적으로 유의미한 차이는 없었다. 또한 로봇의 성별에 대한 인식은 남아와 여아 간에 차이가 없었다. 둘째, 로봇의 외형에 대한 선호 및 혐오는 유아의 성별에 따른 차이가 없어, 남아와 여아 모두 동물모습 로봇을 가장 선호하고 사람 모습 로봇을 가장 혐오하였다. 본 연구의 결과는 유아 대상 로봇 관련 교육 활동 개발 및 교육용 로봇 디자인을 위한 기초 자료로 활용될 수 있을 것이다.