Autonomous Perception Research Articles

Autonomous intelligent additive manufacturing of continuous fiber-reinforced composites: data-enhanced knowledgebase and multi-sensor fusion

ABSTRACT Additive manufacturing (AM) are an emerging technique to generate complex structures of composite. However, the instability of the AM process leads to adverse manufacturing effects, including dimensional inaccuracies and poor mechanical performance in CFRP composites. Online monitoring and adaptive control based on autonomous cognition are suggested to ensure the accuracy of as-fabricated parts and enhance their quality upon manufacturing. In this study, a method of online monitoring and self-adaptive control based on multi-sensor fusion is proposed to predict and adjust the process parameters during AM of CFRP. The force sensor, visual camera, and thermal camera are employed to obtain the multiple signals upon manufacturing and then realize the autonomous perception, cognition, and decision features. Herein, the quantitative correlations between local defects and multi-sensing features are established to guide the decision-making of closed-loop adjustment. Besides, an empirical surrogate model between the misalignment of fiber bundles and input parameters is built to predict the proper parameters. Moreover, the temperature difference and contact force are selected as the controlled features, which are acquired using a thermal camera and a force sensor. The proposed system offers a novel framework for bolstering both the stability of the AM process and the quality of fabricated components.

K čemu je antropologizace historie?

This contribution aims to review the axiomatic premises of neurophysiology, psychoanalysis and deconstructionism (in their most extreme forms), and to discuss correspondences and differences in the concepts of humanness on which they are based. This discussion will reveal the high degree of similarity in these underlying concepts. Both sociobiology and psychoanalysis in effect negate human free will to a great extent with their explanations of behaviour as the result of a limited repertoire of basic drives or biochemically regulated patterns. Deconstructionism, on the other hand, while seeming to reject this approach, is in fact ultimatey also grounded in psychoanalytical axioms. On this basis, the value of further pursuing these epistemological paths appears questionable. If work is oriented around recognising the limits of the explainable, in the sense of a classic hermeneutic circle, then it would seem more useful to return to more modest goals, which can only be realised within the context of comprehensive historical and cultural contextualisation. The processes in which human patterns of interpretation and action emerge and are transformed can be reduced neither to "unconscious" drives nor to the autonomous perception of the body by a subject, and nor can they be interpreted as mere de-materialised language.

Firefighting Robot Extinguishment Decision‐Making Based on Visual Guidance: A Novel Attention and Scale U‐Net Model and Genetic Algorithm

ABSTRACTAt present, rescue firefighting relies mainly on firefighting robots, and robots with perception and decision‐making functions are the key elements for achieving intelligent firefighting. However, traditional firefighting robots often lack the ability for autonomous perception and decision‐making when extinguishing multiple fire sources, leading to low rescue efficiency and increased risk for rescue personnel, especially when making firefighting decisions in extreme fire scenes, which poses a challenge. To effectively handle firefighting tasks and ensure operational efficiency, a robot firefighting decision‐making method based on drone visual guidance is proposed. First, we introduce a novel Attention and Scale U‐Net (ASUNet) model to accurately capture crucial target information, including fire location and size, in a fire scene. The ASUNet model adopts an effective multiscale fusion strategy and attention mechanism to enhance the model's performance. Subsequently, based on the results of the ASUNet model, through pixel segmentation clustering and a genetic optimization algorithm, we obtain the robot's firefighting decision results, thereby guiding the robot to carry out firefighting operations systematically. Finally, through numerical experiments, it is verified that the proposed ASUNet model is superior and effective, as the model can perceive important information in a fire scene and extract it well. The use of improved genetic optimization can further accelerate algorithm convergence. To our knowledge, this study is the first to use drone‐based monocular vision guidance for firefighting decision‐making, providing significant engineering value.

Development Analysis of Key Autonomous Perception Applied to Path Planning and Autonomous Obstacle Avoidance of Land Wheeled Mobile Robots

Development Analysis of Key Autonomous Perception Applied to Path Planning and Autonomous Obstacle Avoidance of Land Wheeled Mobile Robots

Intelligent Soft Quasi‐Organism Equipped with Sensor‐Driven Integrated Tentacles

AbstractIntegration of soft electronic system components is critical to create a closed‐loop system that seamlessly integrates sensing, driving, processing, and autonomous control capabilities. However, the existence of these components, particularly sensors and actuators, in isolated or discrete forms complicates the endeavor to achieve seamless interface matching and in situ integration, no more than the autonomic system. Here, an intelligent soft quasi‐organism (SQO) equipped with sensor‐driven integrated tentacles is demonstrated, featuring real‐time state recognition and autonomous object recognition inspired by the sea anemones. By employing a heterogeneous mechanisms homotopic integration (HMHI) strategy, the tentacles of the SQO possess the unique ability to simultaneously perceive state changes and flexibly drive motions, utilizing the same material and structure. The sea anemone‐shaped SQO exhibits real‐time autonomous state identification capabilities through the integration of machine learning and customized circuits, attaining 100% recognition accuracy across sixteen states. With a neuromuscular system that facilitates active autonomous perception, the anemone‐shaped SQO can recognize and intelligently grasp static objects with an accuracy of 80.7%, surpassing that of a human hand (74.7%). The SQO provides a promising approach for realizing artificial neuromuscular systems, with great potential for applications in crucial areas such as intelligent soft robotics and in vivo therapy.

Coral-Inspired Terahertz-Infrared Bi-Stealth Electronic Skin.

The development of electronic skin with dual stealth functionality is crucial for enabling devices to operate effectively in dynamic electromagnetic environments, thereby facilitating intelligent electromagnetic protection for autonomous perception. However, achieving compatibility between terahertz (THz) and infrared (IR) stealth technologies remains largely unexplored due to their inherent contradictions. Herein, inspired by natural corals, a novel coral-like multi-scale composite foam (CMSF) was proposed that ingeniously reconciles these contradictions. The design capitalizes on the conductive network and heat insulation properties of the foam skeleton, the loss effects and low infrared emission of metal particles, and the infrared transparency of magneto-optical materials. This approach leads to the realization of a THz-IR bi-stealth electronic skin concept. The CMSF exhibits a maximum reflection loss of 84.8 dB in the terahertz band, while its infrared stealth capability ensures environmental adaptability under varying temperatures. Furthermore, the electronic skin exhibits exceptional sensitivity and reliability as a wearable device for perceiving environmental changes. This advanced material, combining multispectral stealth with sensing capabilities, holds immense potential for applications ranging from camouflage technology to smart wearables.

Coral‐Inspired Terahertz‐Infrared Bi‐Stealth Electronic Skin

AbstractThe development of electronic skin with dual stealth functionality is crucial for enabling devices to operate effectively in dynamic electromagnetic environments, thereby facilitating intelligent electromagnetic protection for autonomous perception. However, achieving compatibility between terahertz (THz) and infrared (IR) stealth technologies remains largely unexplored due to their inherent contradictions. Herein, inspired by natural corals, a novel coral‐like multi‐scale composite foam (CMSF) was proposed that ingeniously reconciles these contradictions. The design capitalizes on the conductive network and heat insulation properties of the foam skeleton, the loss effects and low infrared emission of metal particles, and the infrared transparency of magneto‐optical materials. This approach leads to the realization of a THz‐IR bi‐stealth electronic skin concept. The CMSF exhibits a maximum reflection loss of 84.8 dB in the terahertz band, while its infrared stealth capability ensures environmental adaptability under varying temperatures. Furthermore, the electronic skin exhibits exceptional sensitivity and reliability as a wearable device for perceiving environmental changes. This advanced material, combining multispectral stealth with sensing capabilities, holds immense potential for applications ranging from camouflage technology to smart wearables.

FPGA-base object tracking: integrating deep learning and sensor fusion with Kalman filter

This research presents an integrated approach for object detection and tracking in autonomous perception systems, combining deep learning techniques for object detection with sensor fusion and field programmable gate array (FPGA-based) hardware implementation of the Kalman filter. This approach is suitable for applications like autonomous vehicles, robotics, and augmented reality. The study explores the seamless integration of pre-trained deep learning models, sensor data from a depth camera, real-sense D435, and FPGA-based Kalman filtering to achieve robust and accurate 3D position and 2D size estimation of tracked objects while maintaining low latency. The object detection and feature extraction are implemented on a central processing unit (CPU), and the Kalman filter sensor fusion with universal asynchronous receiver transmitter (UART) communication is implemented on a Basys 3 FPGA board that performs 8 times faster compared to the software approach. The experimental result provides the hardware resource utilization of about 29% of look-up tables, 6% of lookup table RAMs (LUTRAM), 15% of Flip-flops, 32% of Block-RAM, 38% of DSP blocks operating at 100 MHz, and 230400 baud rates for the UART. The whole FPGA design executes at 2.1 milliseconds, the Kalman filter executes at 240 microseconds, and the UART at 1.86 milliseconds.

Review of Vision-Based Environmental Perception for Lower-Limb Exoskeleton Robots.

The exoskeleton robot is a wearable electromechanical device inspired by animal exoskeletons. It combines technologies such as sensing, control, information, and mobile computing, enhancing human physical abilities and assisting in rehabilitation training. In recent years, with the development of visual sensors and deep learning, the environmental perception of exoskeletons has drawn widespread attention in the industry. Environmental perception can provide exoskeletons with a certain level of autonomous perception and decision-making ability, enhance their stability and safety in complex environments, and improve the human-machine-environment interaction loop. This paper provides a review of environmental perception and its related technologies of lower-limb exoskeleton robots. First, we briefly introduce the visual sensors and control system. Second, we analyze and summarize the key technologies of environmental perception, including related datasets, detection of critical terrains, and environment-oriented adaptive gait planning. Finally, we analyze the current factors limiting the development of exoskeleton environmental perception and propose future directions.

A data and knowledge driven autonomous intelligent manufacturing system for intelligent factories

The non-ferrous metal industry is encountering several challenges, including production efficiency, manufacturing information fragmentation, and human health problems, which highlights the importance of implementing autonomous intelligent manufacturing systems (AIMS). Recently, the foundation model like GPT-4, has garnered attentions due to its exceptional capabilities and proficiency in diverse domains and tasks, facilitating the realization of AIMS. However, the existing foundation models can only address basic general-purpose tasks and are difficult to use for industrial applications. In this paper, we propose a data and knowledge driven AIMS with industrial-generative pretrained Transformer (Industrial-GPT) for intelligent factories. The paradigms and architecture of autonomous intelligent factories are firstly defined. Then, we explore the mechanism with knowledge graph, digital twin, and Industrial-GPT, including multi-level autonomous perception, cross layer and domain cognition, and event-driven collaborative decision-making. Finally, the detailed case study is based on the cooperation with a zinc smelting intelligent factory to achieve networked collaborative manufacturing, and explores the theory and realization mechanism of AIMS on a small scale. We explore the experimental analyses, evaluation mechanisms and platform applications of AIMS at the workshop level. We believe this will help to realize larger scale AIMS in the future.

A novel small object detection algorithm for UAVs based on YOLOv5

Due to the advances in deep learning, artificial intelligence is widely utilized in numerous areas. Technologies frontier, including computer vision, represented by object detection, have endowed unmanned aerial vehicles (UAVs) with autonomous perception, analysis, and decision-making capabilities. UAVs extensively used in numerous fields including photography, industry and agriculture, surveillance, disaster relief, and play an important role in real life. However, current object detection algorithms encountered challenges when it came to detecting small objects in images captured by UAVs. The small size of the objects, with high density, low resolution, and few features make it difficult for the algorithms to achieve high detection accuracy and are prone to miss and false detections especially when detecting small objects. For the case of enhancing the performance of UAV detection on small objects, a novel small object detection algorithm for UAVs adaptation based on YOLOv5s (UA-YOLOv5s) was proposed. (1) To achieve effective small-sized objects detection, a more accurate small object detection (MASOD) structure was adopted. (2) To boost the detection accuracy and generalization ability of the model, a multi-scale feature fusion (MSF) approach was proposed, which fused the feature information of the shallow layers of the backbone and the neck. (3) To enhance the model stability properties and feature extraction capability, a more efficient and stable convolution residual Squeeze-and-Excitation (CRS)module was introduced. Compared with the YOLOv5s, mAP@0.5 was achieved an impressive improvement of 7.2%. Compared with the YOLOv5l, mAP@0.5 increased by 1.0%, and GFLOPs decreased by 69.1%. Compared to the YOLOv3, mAP@0.5 decreased by 0.2% and GFLOPs by 78.5%. The study’s findings demonstrated that the proposed UA-YOLOv5s significantly enhanced the object detection performance of UAVs campared to the traditional algorithms.

LunarSim: Lunar Rover Simulator Focused on High Visual Fidelity and ROS 2 Integration for Advanced Computer Vision Algorithm Development

Autonomous lunar exploration is a complex task that requires the development of sophisticated algorithms to control the movement of lunar rovers in a challenging environment, based on visual feedback. To train and evaluate these algorithms, it is crucial to have access to both a simulation framework and data that accurately represent the conditions on the lunar surface, with the main focus on providing the visual fidelity necessary for computer vision algorithm development. In this paper, we present a lunar-orientated robotic simulation environment, developed using the Unity game engine, built on top of robot operating system 2 (ROS 2), which enables researchers to generate quality synthetic vision data and test their algorithms for autonomous perception and navigation of lunar rovers in a controlled environment. To demonstrate the versatility of the simulator, we present several use cases in which it is deployed on various efficient hardware platforms, including FPGA and Edge AI devices, to evaluate the performance of different vision-based algorithms for lunar exploration. In general, the simulation environment provides a valuable tool for researchers developing lunar rover systems.

Learning by doing: A dual-loop implementation architecture of deep active learning and human-machine collaboration for smart robot vision

To develop vision systems for autonomous robotic disassembly, this paper presents a dual-loop implementation architecture that enables a robot vision system to learn from human vision in disassembly tasks. The architecture leverages human visual knowledge through a collaborative scheme named ‘learning-by-doing’. In the dual-loop implementation architecture, a human-robot collaborative disassembly loop containing autonomous perception, human-robot interaction and autonomous execution processes is established to address perceptual challenges in disassembly tasks by introducing human operators wearing augmented reality (AR) glasses, while a deep active learning loop is designed to use human visual knowledge to develop robot vision through autonomous perception, human-robot interaction and model learning processes. Considering uncertainties in the conditions of products at the end of their service life, an objective ‘informativeness’ matrix integrating the label information and regional information is designed for autonomous perception, and AR technology is utilised to improve the operational accuracy and efficiency of the human-robot interaction process. By sharing the autonomous perception and human-robot interaction processes, the two loops are simultaneously executed. To validate the capability of the proposed architecture, a screw removal task was studied. The experiments demonstrated the capability to accomplish challenging perceptual tasks and develop the perceptual ability of robots accurately, stably, and efficiently in disassembly processes. The results highlight the potential of learning by doing in developing robot vision towards autonomous robotic disassembly through collaborative human-machine vision systems.

Multimodal fusion for sensorimotor control in steering angle prediction

Efficient reasoning about the spatial and temporal structure of the environment is crucial for perception in autonomous driving, particularly in an end-to-end approach. Although different sensor modalities are employed to capture the complex nature of the environment, they each have their limitations. For example, frame-based RGB cameras are susceptible to variations in illumination conditions. However, these limitations at the sensor level can be addressed by complementing them with sensor fusion techniques, enabling the learning of efficient feature representations for end-to-end autonomous perception. In this study, we address the end-to-end perception problem by fusing a frame-based RGB camera with an event camera to improve the learned representation for predicting lateral control. To achieve this, we propose a convolutional encoder–decoder architecture called DRFuser. DRFuser encodes the features from both sensor modalities and leverages self-attention to fuse the frame-based RGB and event camera features in the encoder part. The decoder component unrolls the learned features to predict lateral control, specifically in the form of a steering angle. We extensively evaluate the proposed method on three datasets: our collected Dataset, Davis Driving dataset, and the EventScape dataset for simulation. The results demonstrate the generalization capability of our method on both real-world and simulated datasets. We observe qualitative and quantitative improvements in the performance of the proposed method for predicting lateral control by incorporating the event camera in fusion with the frame-based RGB camera. Notably, our method outperforms state-of-the-art techniques on the Davis Driving Dataset, achieving a 5.6% improvement in the root mean square error (RMSE) score.

Cultural relic reconstruction device based on ORB-SLAM

ORB-SLAM, an advanced visual SLAM technology, has excellent positioning and mapping capabilities [1]. In the reconstruction of cultural relics, ORB-SLAM technology can achieve autonomous perception and modeling of the surrounding environment of cultural relics by fusing and analyzing the images of monocular depth cameras, thereby reconstructing the 3D model of cultural relics, providing new possibilities for the protection and inheritance of cultural relics. Point cloud images are a digital technology used for 3D modeling, which can save point data on object surfaces and their relationships as 3D point clouds [2]. In terms of cultural relic reconstruction, point cloud images can restore and reconstruct the shape, size, texture, and other features of cultural relics by processing the collected point data on the surface of cultural relics into a 3D model. The purpose of this device is to reconstruct cultural relics by generating point cloud images based on ORB-SLAM technology. Combined with the designed handheld pan tilt device, it can achieve multi-angle point cloud mapping of cultural relics scenes in complex terrain. This paper will introduce the handheld gimbal design of the device, the realization of point cloud mapping based on the ORB-SLAM3 algorithm [3], and the application of point cloud scenes. Through the research in this article, it is expected to provide a new and efficient method for cultural relic reconstruction in the field of cultural relic protection and make contributions to the protection and inheritance of cultural heritage.

Autonomous perception and adaptive standardization for few-shot learning

Identifying unseen classes with limited labeled data for reference is a challenging task, which is also known as few-shot learning. Generally, a knowledge-rich model is more robust than a knowledge-poor model when facing novel situations, and an intuitive way to enrich knowledge is to find additional training data, but this is not compatible with the principle of few-shot learning which aims to reduce reliance on big data. In contrast, improving the utilization of existing data is a more attractive option. In this paper, we propose a batch perception distillation approach, which improves the utilization of existing data by guiding individual classification with the intermixed information across a batch. In addition to data utilization, obtaining robust feature representation is also a concern. Specifically, the widely adopted metric-based few-shot classification approach classifies unseen testing classes by comparing the extracted features of different novel samples, which requires that the extracted features can accurately represent the class-related clues of the input images. In this paper, we propose a salience perception attention that enables the model to focus more easily on key clues in images, which helps to reduce the interference of irrelevant factors during classification. To overcome the distribution gap between the training classes and the unseen testing classes, we propose a weighted centering post-processing that standardizes the testing data according to the similarity between the training and testing classes. By combining the three proposed components, our method achieves superior performance on four widely used few-shot image classification datasets.

Robot trajectory planning for autonomous 3D reconstruction of cockpit in aircraft final assembly testing

The trend towards automation and intelligence in aircraft final assembly testing has led to a new demand for autonomous perception of unknown cockpit operation scenes in robotic collaborative airborne system testing. To address this demand, a robotic automated 3D reconstruction cell which enables to autonomously plan the robot end-camera's trajectory is developed for image acquisition and 3D modeling of the cockpit operation scene. A continuous viewpoint path planning algorithm is proposed that incorporates both 3D reconstruction quality and robot path quality into optimization process. Smoothness metrics for viewpoint position paths and orientation paths are introduced together for the first time in 3D reconstruction. To ensure safe and effective movement, two spatial constraints, Domain of View Admissible Position (DVAP) and Domain of View Admissible Orientation (DVAO), are implemented to account for robot reachability and collision avoidance. By using diffeomorphism mapping, the orientation path is transformed into 3D, consistent with the position path. Both orientation and position paths can be optimized in a unified framework to maximize the gain of reconstruction quality and path smoothness within DVAP and DVAO. The reconstruction cell is capable of automatic data acquisition and fine scene modeling, using the generated robot C-space trajectory. Simulation and physical scene experiments have confirmed the effectiveness of the proposed method to achieve high-precision 3D reconstruction while optimizing robot motion quality.

Individual autonomous safety intelligence technology for radioactive material transportation—Multi-sensor fusion early warning technology based on evidence theory

This paper briefly introduces the research project on individual autonomous security intelligence for radioactive materials using radar and microwave-infrared detectors to autonomously sense external threats during railroad transportation and achieve individual autonomous security intelligence. Due to the shortcomings of the single sensor with a false alarm, the project fuses the perception information of each sensor connected to the radar-infrared node to decide the level of danger warning and improve the level of individual autonomous security of radioactive materials. To further improve the accuracy and reduce the false alarm rate of individual autonomous hazard warnings, a reasonable basic probability assignment (BPA) is constructed for each sensor based on a fuzzy set and rough set according to the practical application, and the fusion of individual autonomous perception information is realized by combining Pearson correlation coefficient and D-S evidence theory. The experimental results show that the fused hazard warning accuracy is 91.72% and the false alarm rate is 8.28%, which indicates that individual autonomous safety intelligence can be effectively realized.

Can the Perception Data of Autonomous Vehicles Be Used to Replace Mobile Mapping Surveys?—A Case Study Surveying Roadside City Trees

The continuous flow of autonomous vehicle-based data could revolutionize current map updating procedures and allow completely new types of mapping applications. Therefore, in this article, we demonstrate the feasibility of using perception data of autonomous vehicles to replace traditionally conducted mobile mapping surveys with a case study focusing on updating a register of roadside city trees. In our experiment, we drove along a 1.3-km-long road in Helsinki to collect laser scanner data using our autonomous car platform ARVO, which is based on a Ford Mondeo hybrid passenger vehicle equipped with a Velodyne VLS-128 Alpha Prime scanner and other high-grade sensors for autonomous perception. For comparison, laser scanner data from the same region were also collected with a specially-planned high-grade mobile mapping laser scanning system. Based on our results, the diameter at breast height, one of the key parameters of city tree registers, could be estimated with a lower root-mean-square error from the perception data of the autonomous car than from the specially-planned mobile laser scanning survey, provided that time-based filtering was included in the post-processing of the autonomous perception data to mitigate distortions in the obtained point cloud. Therefore, appropriately performed post-processing of the autonomous perception data can be regarded as a viable option for keeping maps updated in road environments. However, point cloud-processing algorithms may need to be adapted for the post-processing of autonomous perception data due to the differences in the sensors and their arrangements compared to designated mobile mapping systems. We also emphasize that time-based filtering may be required in the post-processing of autonomous perception data due to point cloud distortions around objects seen at multiple times. This highlights the importance of saving the time stamp for each data point in the autonomous perception data or saving the temporal order of the data points.

Quantification of Uncertainty and Its Applications to Complex Domain for Autonomous Vehicles Perception System

Over the last decades,deep neural networks have been penetrated into all fields of science and the real world. As a result of the lack of quantifiable data and model uncertaint, deep learning is frequently brittle,illogical, and challenging to provide trustworthy assurance for autonomous vehicles(AV) perception.This hole is filled by the suggested approach to uncertainty quantification. Nevertheless, most of the previous studies focused on the methodology and there is still a lack of research on the application of AV. To our knowledge, this paper is the first time to review the application of uncertainty in the field of AV perception and localization. In the first place, this paper analyzes the sources of uncertainty in autonomous perception,including the uncertainty brought on by sensor internal and external factors as well as the sensor distortion caused on by complex scenes. In the second place,we propose an evaluation criterion and use the criterion to carry out a quantitative analysis of perception field of application for autonomous vehicles,and we discuss the mainstream datasets. Thirdly, we put forward a number of open issues and raise some future research directions, which is of guiding significance to readers who are beginning to enter this field.We believe that epistemic uncertainty is currently the dominant research direction and that there is still a long way to go in the study of aleatoric uncertainty.And our paper will be devoted to promoting the development of uncertainty research of AV perception.