Robot Perception Research Articles

A fuzzy-logic approach for longitudinal assessment of patients’ psychophysiological state: an application to upper-limb orthopedic robot-aided rehabilitation

Understanding the psychophysiological state during robot-aided rehabilitation is crucial for assessing the patient experience during treatments. This paper introduces a psychophysiological estimation approach using a Fuzzy Logic inference model to assess patients’ perception of robots during upper-limb robot-aided rehabilitation sessions. The patients were asked to perform nine cycles of 3D point-to-point trajectories toward different targets at varying heights with the assistance of an anthropomorphic robotic arm (i.e. KUKA LWR 4+). Physiological parameters, including galvanic skin response, heart rate, and respiration rate, were monitored across ten out of forty daily sessions. This data enabled the construction of an inference model to estimate patients’ perception states. Results expressed in terms of correlation coefficients between the patient state and the increasing number of sessions. Correlation coefficients showed statistically significant strong associations: a state of heightened engagement (formerly described as “Excited”) had ρ=-0.73\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\rho = -0.73$$\\end{document} (p-value=0.01), and a more calm and resting state (namely “Relaxed” state) had ρ=0.70\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\rho = 0.70$$\\end{document} (p-value=0.02) with the number of sessions completed. All patients had positive interaction with the robot, initially expressing curiosity and interest that gradually shifted to a more “Relaxed” state over time.

Large-Scale Knowledge Graphs as a Tool for Enhanced Robotic Perception

Autonomous robotic systems depend on their perception and understanding of their environment for informed decision-making. One of the goals of the Semantic Web is to make knowledge on the Web machine-readable, which can significantly aid robots by providing background knowledge, and thereby support their understanding. In this paper, we present a reasoning system that uses the Ontology for Robotic Knowledge Acquisition (ORKA) to integrate the sensory data and perception algorithms of the robot, thereby enhancing its autonomous capabilities. This reasoning system is subsequently employed to retrieve and integrate information from the Semantic Web, thereby improving the robot's comprehension of its environment. To achieve this, the system employs a Perceived-Entity Linking (PEL) pipeline that associates regions in the sensory data of the robotic agent with concepts in a target knowledge graph. As a use-case for the linking process, the Perceived-Entity Typing task is used to determine the more fine-grained subclass of the perceived entities. Specifically, we provide an analysis of the performance of different knowledge graph embedding methods on the task using a annotated observations and WikiData as a target knowledge graph. The experiments indicate that relying on pre-trained embedding methods results in an increased performance when using TransE as the embedding method for the observations of the robot. This contribution advances the field by demonstrating the potential of integrating Semantic Web technologies with robotic perception, thereby enabling more nuanced and context-aware decision-making in autonomous systems.

Preferences and Expectations for Home Robot Tasks: Comparison According to Age and Household Type in Republic of Korea

Most studies of consumer preferences and expectations for home robots focus on either older adults or single-person households (SPHs). However, with the rise in voluntary SPHs among young adults and seniors, it is critical to compare both age and household types in robot research. This study explored perceptions of home robots and willingness to use their features based on age and household type, in the context of the expanding home robot market in Republic of Korea. An online survey of 400 individuals was conducted, targeting young SPHs and multi-person households (MPHs) in their 20s and 30s as well as older SPHs and MPHs in their 50s and 60s. The survey covered four robot task categories, with 40 items derived from previous research: household chores (20 items), personal care (seven items), leisure/companion (nine items), and health (four items). The results helped predict the main target groups for each in-home robot task by identifying items that showed differences in responses between groups and interpreting these based on age, household type, and their combination. The study provides valuable data on consumer expectations, highlighting differences in responses according to both age and household type, offering insights for the robotics industry to effectively target their products.

Deep learning-assisted object recognition with hybrid triboelectric-capacitive tactile sensor

Tactile sensors play a critical role in robotic intelligence and human-machine interaction. In this manuscript, we propose a hybrid tactile sensor by integrating a triboelectric sensing unit and a capacitive sensing unit based on porous PDMS. The triboelectric sensing unit is sensitive to the surface material and texture of the grasped objects, while the capacitive sensing unit responds to the object’s hardness. By combining signals from the two sensing units, tactile object recognition can be achieved among not only different objects but also the same object in different states. In addition, both the triboelectric layer and the capacitor dielectric layer were fabricated through the same manufacturing process. Furthermore, deep learning was employed to assist the tactile sensor in accurate object recognition. As a demonstration, the identification of 12 samples was implemented using this hybrid tactile sensor, and an recognition accuracy of 98.46% was achieved. Overall, the proposed hybrid tactile sensor has shown great potential in robotic perception and tactile intelligence.

Augmenting Consumer Acceptance of Robot-Assisted Technologies in Retail Industry: An Interdisciplinary Approach

Since they have been incorporated into retail, robots have been discussed more frequently in academic publications and public debate. Customers in retail sector, as well as workers will be resistant to the rising employment of robots as their capabilities increase and they are employed to supplement or replace human labour. "Interdisciplinary" denotes a technique that amalgamates knowledge and methodologies from several academic or professional domains to attain a thorough comprehension of a particular subject or matter. As robots and AI become more common in society, this study examines the public's attitudes towards their use in retail. Consumer perceptions of retail robots are examined using consumer psychology and technology attitudes literature. To fill the research gap, 393 retailers' responses on service robot deployment in retail contexts are analyzed, with an emphasis on the East region. Service robot attitudes vary based on aspects such as robot benefits, social skills, and technology attitudes. Cluster analysis divides respondents into "high techies" and "high touches," depending on their service robot acceptance. The regression study clarifies how numerous factors affect retail robot-assisted service perceptions. Retailers and policymakers can learn from the survey that consumer opinion is significant for the future integration of robots in retail.

Research on Deep Learning Detection Model for Pedestrian Objects in Complex Scenes Based on Improved YOLOv7

Objective: Pedestrian detection is very important for the environment perception and safety action of intelligent robots and autonomous driving, and is the key to ensuring the safe action of intelligent robots and auto assisted driving. Methods: In response to the characteristics of pedestrian objects occupying a small image area, diverse poses, complex scenes and severe occlusion, this paper proposes an improved pedestrian object detection method based on the YOLOv7 model, which adopts the Convolutional Block Attention Module (CBAM) attention mechanism and Deformable ConvNets v2 (DCNv2) in the two Efficient Layer Aggregation Network (ELAN) modules of the backbone feature extraction network. In addition, the detection head is replaced with a Dynamic Head (DyHead) detector head with an attention mechanism; unnecessary background information around the pedestrian object is also effectively excluded, making the model learn more concentrated feature representations. Results: Compared with the original model, the log-average miss rate of the improved YOLOv7 model is significantly reduced in both the Citypersons dataset and the INRIA dataset. Conclusions: The improved YOLOv7 model proposed in this paper achieved good performance improvement in different pedestrian detection problems. The research in this paper has important reference significance for pedestrian detection in complex scenes such as small, occluded and overlapping objects.

Religiosity and Attitudes towards Robots: Results from a Global Survey

Religion is one lens in which people understand the world around them and interpret the world around them. However, it is unclear whether religion has an impact upon attitudes towards robots around the world. In this article, the authors investigate how an individual’s religiosity impacts upon perceptions of robots. The article investigates how an individual’s religiosity impacts attitudes towards robots, using data from a large-scale global survey of attitudes towards robots (N=1263). In order to investigate how religion impacts upon such perceptions, cluster, factor and regression analyses were used for data analysis. The findings illustrate that there are discernible clusters of individuals exhibiting different levels of religiosity and that these clusters have different perceptions of robots, showing as hypothesized, that more religious individuals are more likely to be robophobic than robophilic. The relationship between religiosity and attitudes towards robots is negative: less religious people show more positive attitudes towards robots. While there is clear indication that there is a relationship between religiosity and attitudes towards robots, religiosity does not have a particularly strong impact upon perceptions of robots. The analysis of the data illustrates that there are factors more clearly associated with perceptions of robots, meaning that while religiosity may have some impact on perceptions of robots, that there are more important attitudes and other factors that should be taken into consideration to encourage users to use robots in various service environments. In addition, in this particular research, there seems to be no notable relationship between perceptions of robots and demographics of individuals, illustrating that the age, gender, wealth, and education of an individual may not play a strong role in shaping perceptions of robots in service industries.

Artificial Skin Based on Visuo‐Tactile Sensing for 3D Shape Reconstruction: Material, Method, and Evaluation

AbstractArtificial skin has shown great potential in robot perception and human healthcare. It provides multifunctional tactile sensing, including 3D shape reconstruction, contact feedback, and temperature perception, where the 3D reconstruction function is indispensable for dexterous hands in tactile cognition and interaction. Vision‐based tactile sensor (VTS) is an innovative bionic tactile sensor and supports high‐resolution, high‐precision, and high‐density tactile reconstruction compared with electronic tactile sensors. Considering the unique contribution of visuo‐tactile sensing to artificial skin, this review focuses on the 3D reconstruction techniques of the VTS. 3D reconstruction methods are classified into five categories based on sensing modalities, hardware categories, and modeling approaches: 1) photometric stereo, 2) binocular depth calibration, 3) optical flow, 4) deep learning, and 5) ToF (time of flight). In addition, the association and difference of reconstruction methods are analyzed from the hardware perspective, and the development and technological details of 3D reconstruction are summarized. On this basis, the challenges and development direction are discussed. This review can be viewed as a technology guide to provide references for interested researchers. Furthermore, it is expected to promote the extensive application of the VTS in artificial skins.

Avian-inspired embodied perception in biohybrid flapping-wing robotics

Avian feather intricate adaptable architecture to wing deformations has catalyzed interest in feathered flapping-wing aircraft with high maneuverability, agility, and stealth. Yet, to mimic avian integrated somatic sensation within stringent weight constraints, remains challenging. Here, we propose an avian-inspired embodied perception approach for biohybrid flapping-wing robots. Our feather-piezoelectric mechanoreceptor leverages feather-based vibration structures and flexible piezoelectric materials to refine and augment mechanoreception via coupled oscillator interactions and robust microstructure adhesion. Utilizing convolutional neural networks with the grey wolf optimizer, we develop tactile perception of airflow velocity and wing flapping frequency proprioception. This method also senses pitch angle via airflow direction and detects wing morphology through feather collisions. Our low-weight, accurate perception of flapping-wing robot flight states is validated by motion capture. This investigation constructs a biomechanically integrated embodied perception system in flapping-wing robots, which holds significant promise in reflex-based control of complex flight maneuvers and natural bird flight surveillance.

A LiDAR SLAM Algorithm Considering Dynamic Extraction of Feature Points in Underground Coal Mine

Abstract. To address the absence of GNSS signals in underground coal mines and the susceptibility of mainstream LiDAR SLAM to degradation due to insufficient feature constraints, this paper proposes a tightly-coupled SLAM algorithm incorporating LiDAR and IMU for use in such environments. The paper initially introduces a dynamic feature point extraction strategy, where the number of corner feature points can be dynamically adjusted based on the occurrence of degradation in the underground coal mine environment. This approach constructs a constraint matrix with rich and robust feature information, enhancing pose estimation accuracy in environments with inadequate feature constraints, mitigating degradation effects, and reducing global cumulative error. Subsequently, the consistent building of the underground coal mine environment is achieved through back-end factor graph optimization. Finally, to validate the effectiveness of the method, experimental analysis is conducted in an underground coal mine. The results demonstrate that LeGO_LOAM exhibits poor robustness in underground coal mines and fails to construct a globally consistent pose estimation and map. Conversely, the pose estimation error of the proposed method in this paper is 50.93% lower in the plane direction and 42.13% lower in the elevation direction compared to LIO_SAM. This underscores the method's significance as crucial technical support for the intelligent perception and positioning of robots in underground coal mines.

A Survey of Multimodal Perception Methods for Human–Robot Interaction in Social Environments

Human–robot interaction (HRI) in human social environments (HSEs) poses unique challenges for robot perception systems, which must combine asynchronous, heterogeneous data streams in real time. Multimodal perception systems are well-suited for HRI in HSEs and can provide more rich, robust interaction for robots operating among humans. In this article, we provide an overview of multimodal perception systems being used in HSEs, which is intended to be an introduction to the topic and summary of relevant trends, techniques, resources, challenges, and terminology. We surveyed 15 peer-reviewed robotics and HRI publications over the past 10+ years, providing details about the data acquisition, processing, and fusion techniques used in 65 multimodal perception systems across various HRI domains. Our survey provides information about hardware, software, datasets, and methods currently available for HRI perception research, as well as how these perception systems are being applied in HSEs. Based on the survey, we summarize trends, challenges, and limitations of multimodal human perception systems for robots, then identify resources for researchers and developers and propose future research areas to advance the field.

DenseFusion-DA2: End-to-End Pose-Estimation Network Based on RGB-D Sensors and Multi-Channel Attention Mechanisms.

Notably, 6D pose estimation is a critical technology that enables robotics to perceive and interact with their operational environment. However, occlusion causes a loss of local features, which, in turn, restricts the estimation accuracy. To address these challenges, this paper proposes an end-to-end pose-estimation network based on a multi-channel attention mechanism, DA2Net. Firstly, a multi-channel attention mechanism, designated as "DA2Net", was devised using A2-Nets as its foundation. This mechanism is constructed in two steps. In the first step, the essential characteristics are extracted from the global feature space through the second-order attention pool. In the second step, a feature map is generated by the integration of position and channel attention. Subsequently, the extracted key features are assigned to each position of the feature map, enhancing both the feature representation capacity and the overall performance. Secondly, the designed attention mechanism is introduced into both the feature fusion and pose iterative refinement networks to enhance the network's capacity to acquire local features thus improving its overall performance. The experimental results demonstrated that the estimation accuracy of DenseFusion-DA2 on the LineMOD dataset was approximately 3.4% higher than that of DenseFusion. Furthermore, the estimation accuracy surpassed that of PoseCNN, PVNet, SSD6D, and PointFusion by 8.3%, 11.1%, 20.3%, and 23.8%, respectively. The estimation accuracy also shows a significant advantage on the Occluded LineMOD and HR-Vision datasets. This research not only presents a more efficient solution for robot perception but also introduces novel ideas and methods for technological advancements and applications in related fields.

Arrayed multi-layer piezoelectric sensor based on electrospun P(VDF-TrFE)/ZnO with enhanced piezoelectricity

Flexible pressure sensors can maintain good pressure sensing capabilities even under arbitrary degrees of deformation, which are widely used in fields such as robotic perception, physiological signal detection and wearable electronics. Currently, piezoelectric-based flexible pressure sensors face significant limitations in terms of large-area, high-sensitivity, and high-resolution pressure detection. Targeted research efforts primarily focus on enhancing the performance of piezoelectric sensors by optimizing the properties of the sensitive layer piezoelectric material and modulating device structural design. This study introduces a novel multilayer array design for flexible composite piezoelectric sensors. A high-voltage electrospinning selective deposition process is proposed to fabricate arrayed nanofiber mats. Patterned electrospun films can be automatically patterned by patterned AgNW electrodes, which allows manual alignment of patterns for subsequent stacking. Compared to sensor based on P(VDF-TrFE) prepared under the same conditions, the sensitivity of the multi-layer P(VDF-TrFE)/ZnO sensor increased from 2.82 mV/kPa to 8.30 mV/kPa, providing an effective approach for highly sensitivity wearable devices with short response time (∼5 ms).

Integrated Deep Learning Architectures for Perception, Control, and Decision-making in Robotics: a Framework for Sensing, Cognition, and Transparent Decision-making

The increasing complexity of robotic applications demands innovative approaches for addressing problems that lack analytical solutions, with deep learning (DL) emerging as a key tool for enabling robots to learn and adapt in dynamic environments. This survey reviews existing DL techniques in robotics, categorizing the major challenges and exploring successful solutions that leverage DL for perception, control, and decision-making. We discuss the use of modular versus end-to-end DL architectures, providing guidelines for selecting appropriate model structures and training strategies. The review also examines advancements in neuro-robotics systems (NRS), where the convergence of neuroscience and robotics is driving the development of robots with embodied intelligence, enabling more natural human-robot interactions. Recent progress in neural mechanisms for perception, cognition, learning, and control is highlighted, offering insights into creating future neuro-robots. Furthermore, we address the challenge of uncertainty estimation in neural networks, crucial for reliable robotic decision-making, and evaluate existing frameworks such as Bayesian belief networks and Monte Carlo sampling that improve uncertainty modeling without requiring architectural changes. Lastly, we explore efforts to enhance transparency in robotic systems through integrated reasoning and learning methods, focusing on architectures that combine logical reasoning with deep learning to provide explainable decision-making. This survey aims to offer a structured overview of current research and guide future developments in neuro-robotics.

Nurses' perceptions of medical service robots in negative-pressure isolated wards and in general wards: A cross-sectional survey.

To evaluate nurses' workload during the coronavirus disease 2019 (COVID-19) pandemic and to identify their perception of medical serviced robots (MSRs) that may help with or replace the tasks of nurses in negative-pressure isolated wards and general wards. Cross-sectional survey. A researcher-made questionnaire was applied that assessed nurses' current workload and their attitudes towards and perceptions of MSRs. A visual analogue scale (VAS) from 1 to 10 was used to assess workload. Perceptions evaluated on a 5-point Likert scale ranging from one point for 'strongly disagree' to five points for 'strongly agree'. A higher VAS score indicated a higher workload. On the other hands, a higher Likert score indicated a more positive perception and three points was neutral. The questionnaire was conducted on 150 nurses in negative-pressure isolated wards for the management of COVID-19 and 150 nurses in general wards. Quota sampling technique was used as sampling technique. Data analysis was performed through independent t-tests, chi-square tests and two-tailed tests. The p-value <0.05 was interpreted to statistically significant. Two hundred eighty-two participants responded and 142 belonged to the negative-pressure isolated ward. The overall response rate was 94%, and 94.7% in the negative isolated ward. The mean score ± standard deviation for nursing-related psychological stress at the current work site was 7.18 ± 1.58 points, and the mean score for physical workload was 7.65 ± 1.48. The need for MSRs was rated as 3.66 ± 0.86 out of 5. Overall, a positive attitude towards MSRs was confirmed, with no difference between ward groups. The overall ratings were 3.14 ± 1.15 for perceived availability and 3.26 ± 1.13 for perceived efficiency. Both nurse groups perceived that MSRs were most available and efficient for monitoring and measurements. Nurses in negative-pressure isolated wards perceived MSRs more positively than did nurses in general wards regarding setting of alarms (p = 0.003) and delivery of medical devices/materials (p = 0.013). Based on these results, functional development of MSRs associated with monitoring, measurements, setting of alarms and delivery should be prioritized. No patient or public contribution.

A multi-stage approach for desired part grasping under complex backgrounds in human-robot collaborative assembly

Human-robot collaborative assembly can leverage the unique capabilities of humans and robots to provide a flexible and efficient way for complex tasks. In this context, robots are expected to be endowed with the perception ability to collaborate with humans. For flexible processes, robotic grasping for a desired assembly part from variable multi-parts is essential but remains challenging, especially in situations of complex backgrounds including disordered placement, occlusion, similarity, and large-scale differences in size. To improve the perception and decision-making abilities of robots in collaborative assembly, firstly, a multi-stage approach integrating 2D part recognition and 6D pose estimation with perspective-focusing is proposed for desired part grasping under complex backgrounds. Secondly, to efficiently and accurately recognize a desired part from variable multi-parts under complex backgrounds, based on RGB data, an enhanced detector is adopted, which lays the foundation for the subsequent pose estimation. Thirdly, based on point cloud data, a pose estimation method based on perspective-focusing is proposed to enhance the success rate and efficiency of pose acquisition. Additionally, a modular autonomous robotic grasping system is designed. Finally, taking decelerator assembly as a case, the proposed approaches are comprehensively validated and compared on a real collaborative robot platform. Under complex backgrounds with occlusion, our approach shows stable performance with an average grasping success rate of 90.0 % and an average cycle time of 6.4175 s. Our work is expected to improve robots’ perception and decision-making capabilities while further reducing the cycle time of collaborative assembly tasks.

Consumer Attention to a Coffee Brewing Robot: An Eye‐Tracking Study

ABSTRACTDuring the COVID‐19, robots were increasingly utilized in the food service industry. This study compared perceptions of robot and human baristas by tracking participants' eye‐gaze behavior. Seventy participants viewed videos and images of both baristas, and their eye movements were tracked. Metrics such as entry time, dwell time, total fixation, hit ratio, revisits, revisitors, average fixation, first fixation, and fixation count in defined areas of interest (AOIs) were measured. Participants focused mostly on robot's body during coffee preparation and human barista's face. Attention stayed on the coffee with human baristas, while robot stimuli drew focus to the robot's hand and kettle. Participants fixated more on the robot itself, potentially due to its novelty in coffee preparation. This study can provide directions for the development of robots, including their appearance, so that they could be accepted familiarly by consumers and introduced seamlessly in the food industries.

Deep Learning-Driven Robot Arm Control Fusing Convolutional Visual Perception and Predictive Modeling for Motion Planning

The wide application of robotic technology in various industries from industrial automation to medical assistance is gradually changing our production and lifestyle, and has attracted widespread attention in many fields. However, existing robotic control systems often grapple with limited flexibility and poor adaptability to complex environments, particularly in highly dynamic operational contexts. Against this backdrop, the integration of deep learning technologies offers new possibilities in enhancing robotic perception and decision-making, especially in visual perception and motion planning. To address these challenges, we have introduced a novel robotic arm control network model, MPC-WGAN-Faster R-CNN, which combines Model Predictive Control concepts with Wasserstein Generative Adversarial Networks and Faster R-CNN visual recognition technology. This integration aims to improve the precision and adaptability of robotic arm operations in complex environments.

Human-Robot Intimacy: Acceptance of Robots as Intimate Companions.

Depictions of robots as romantic partners for humans are frequent in popular culture. As robots become part of human society, they will gradually assume the role of partners for humans whenever necessary, as assistants, collaborators, or companions. Companion robots are supposed to provide social contact to those who would not have it otherwise. These companion robots are usually not designed to fulfill one of the most important human needs: the one for romantic and intimate contact. Human-robot intimacy remains a vastly unexplored territory. In this article, we review the state-of-the-art research in intimate robotics. We discuss major issues limiting the acceptance of robots as intimate partners, the public perception of robots in intimate roles, and the possible influence of cross-cultural differences in these domains. We also discuss the possible negative effects human-robot intimacy may have on human-human contact. Most importantly, we propose a new term "intimate companion robots" to reduce the negative connotations of the other terms that have been used so far and improve the social perception of research in this domain. With this article, we provide an outlook on prospects for the development of intimate companion robots, considering the specific context of their use.

Minimal perception: enabling autonomy in resource-constrained robots.

The rapidly increasing capabilities of autonomous mobile robots promise to make them ubiquitous in the coming decade. These robots will continue to enhance efficiency and safety in novel applications such as disaster management, environmental monitoring, bridge inspection, and agricultural inspection. To operate autonomously without constant human intervention, even in remote or hazardous areas, robots must sense, process, and interpret environmental data using only onboard sensing and computation. This capability is made possible by advancements in perception algorithms, allowing these robots to rely primarily on their perception capabilities for navigation tasks. However, tiny robot autonomy is hindered mainly by sensors, memory, and computing due to size, area, weight, and power constraints. The bottleneck in these robots lies in the real-time perception in resource-constrained robots. To enable autonomy in robots of sizes that are less than 100 mm in body length, we draw inspiration from tiny organisms such as insects and hummingbirds, known for their sophisticated perception, navigation, and survival abilities despite their minimal sensor and neural system. This work aims to provide insights into designing a compact and efficient minimal perception framework for tiny autonomous robots from higher cognitive to lower sensor levels.