Multi-sensor Fusion Technology Research Articles

A Robotics Experimental Design Method Based on PDCA: A Case Study of Wall-Following Robots.

There is a lack of research that proposes a complete and interoperable robotics experimental design method to improve students' learning outcomes. Therefore, this study proposes a student-oriented method based on the plan-do-check-act (PDCA) concept to design robotics experiments. The proposed method is based on our teaching experience and multiple practical experiences of allowing students to do hands-on experiments. It consists of eight steps, mainly including experimental goals, experimental activities, robot assembly, robot control, in-class evaluation criteria, and after-class report requirements. The after-class report requirements designed in the proposed method can help students improve their report-writing abilities. A wall-following robotics experiment designed using the PDCA method is proposed, and some students' learning outcomes and after-class reports in this experiment are presented to illustrate the effectiveness of the proposed method. This experiment also helps students to understand the fundamental application of multi-sensor fusion technology in designing an autonomous mobile robot. We can see that the proposed reference examples allow students to quickly assemble two-wheeled mobile robots with four different sensors and to design programs to control these assembled robots. In addition, the proposed in-class evaluation criteria stimulate students' creativity in assembling different wall-following robots or designing different programs to achieve this experiment. We present the learning outcomes of three stages of the wall-following robotics experiment. Three groups of 42, 37, and 44 students participated in the experiment in these three stages, respectively. The ratios of the time required for the robots designed by students to complete the wall-following experiment, less than that of the teaching example, are 3/42 = 7.14%, 26/37 = 70.27%, and 44/44 = 100%, respectively. From the comparison of learning outcomes in the three stages, it can be seen that the proposed PDCA-based design method can indeed improve students' learning outcomes and stimulate their active learning and creativity.

Research Progress on Quality Detection of Livestock and Poultry Meat Based on Machine Vision, Hyperspectral and Multi-Source Information Fusion Technologies

Presently, the traditional methods employed for detecting livestock and poultry meat predominantly involve sensory evaluation conducted by humans, chemical index detection, and microbial detection. While these methods demonstrate commendable accuracy in detection, their application becomes more challenging when applied to large-scale production by enterprises. Compared with traditional detection methods, machine vision and hyperspectral technology can realize real-time online detection of large throughput because of their advantages of high efficiency, accuracy, and non-contact measurement, so they have been widely concerned by researchers. Based on this, in order to further enhance the accuracy of online quality detection for livestock and poultry meat, this article presents a comprehensive overview of methods based on machine vision, hyperspectral, and multi-sensor information fusion technologies. This review encompasses an examination of the current research status and the latest advancements in these methodologies while also deliberating on potential future development trends. The ultimate objective is to provide pertinent information and serve as a valuable research resource for the non-destructive online quality detection of livestock and poultry meat.

Multi-Sensor Fusion Technology for 3D Object Detection in Autonomous Driving: A Review

Multi-Sensor Fusion Technology for 3D Object Detection in Autonomous Driving: A Review

Research on the application technology of lead acid batteries for mining electric vehicles

As a power source, ordinary explosion-proof large-capacity lead-acid batteries have been widely used in underground explosion-proof lead-acid battery scrapers and support trucks, but there are some problems, such as too long charging time, the need to manually add distilled water regularly, and the inability to realize intelligent management. In this paper, the mine special valve-regulated lead-carbon lead-acid battery with capacitance characteristics is applied to the explosion-proof heavy-duty electric drive car, which can shorten the charging time to about 1/5 of that of the ordinary lead-acid battery and reduce the labor maintenance cost. Through multi-sensor fusion technology, an online monitoring and intelligent expert management system for the explosion-proof battery is constructed, which can remotely monitor the real-time state of the battery. It can find out the causes of battery failure, give an alarm in advance, and prolong the service life of the battery pack, which is conducive to the further popularization and use of underground electric-driven trackless rubber tires.

Retracted: Application of Wireless Network Multisensor Fusion Technology in Sports Training.

[This retracts the article DOI: 10.1155/2022/9836697.].

MIFDELN: A multi-sensor information fusion deep ensemble learning network for diagnosing bearing faults in noisy scenarios

Owing to the harsh operating environment of rolling bearings, acquired vibration signals contain strong noise interference, which makes it challenging for conventional methods to effectively extract fault information directly from the original signal. Additionally, obtaining satisfactory diagnostic results using traditional methods based on a single sensor is often difficult because the complementary information between different sensors is ignored. To address these challenges, this paper proposes a multi-sensor information fusion deep ensemble learning network (MIFDELN) for diagnosing bearing faults. First, the collected multi-sensor signals are organically fused using the weighted fusion strategy based on the composite index, which not only avoids the selection of poor-quality sensor signals but also suppresses part of the strong noise interference in the original signal. Second, the cross-scale attention feature extraction module (CAFEM) is presented to automatically learn discriminative features from the fused signals and reduce the impact of useless feature information through cross-scale learning and attentional weight enhancement. Subsequently, a weighted topology learning module (WTLM) is introduced to further excavate the spatial structure features and strengthen the distinctness of the learned features. Finally, the softmax classifier is employed to finalize the fault identification. Two experiments were conducted to verify the effectiveness and superiority of the proposed approach. The results indicated that the proposed approach performs best in extracting discriminating features with the highest accuracy and greatest robustness compared with several representative multi-sensor fusion technologies.

An enhanced visualization image acquisition method for samples with poor conductivity under a conventional scanning electron microscope.

The low-vacuum and low-accelerating-voltage modes are the most simple and practical ways to directly analyze poorly conductive samples in conventional scanning electron microscopy (SEM). However, structural feature information may disappear or be obscured in these imaging modes, making it challenging to identify and analyze some local microstructures of poorly conductive samples. To overcome this challenge, an enhanced visualization image acquisition method for samples with poor conductivity is proposed based on the image registration and multi-sensor fusion technology. Experiments demonstrate that the proposed method can effectively obtain enhanced visualization images containing clearer terrain information than the SEM source images, thereby providing new references for measuring and analyzing microstructures.

AN INTELLIGENT MULTI-SENSOR SYSTEM FOR IDENTIFICATION AND ASSESSMENT OF THE TECHNICAL CONDITION OF ELECTRICAL EQUIPMENT USING WI-FI

The modern stage of industrial development is characterized by an increasing need for systems to monitor the condition and efficient functioning of electrical equipment. Breakdown or unplanned stoppage of an electromechanical system engine is accompanied by significant economic losses for enterprises. Therefore, the current stage of industrial development is characterized by an increasing need for systems for monitoring the condition and efficient functioning of electrical equipment.
 As a result, due to the rapid development of precision manufacturing technologies, many studies have been conducted in the field of multi-sensor measurement and data fusion technologies to improve monitoring capabilities in terms of measurement accuracy and information richness, thereby increasing production efficiency and accuracy. In a multi-sensor system, each sensor independently measures certain parameters. The system then uses an appropriate signal processing algorithm to combine all the independent measurements into a complete set of measurement results.
 An accurate assessment of the technical condition makes it possible to predict the term of the next repair of the equipment, increases the reliability and efficiency of its operation, reduces the damage from downtime of the equipment due to accidents, the costs of repair and restoration of operability, reduces the costs of maintenance and operation of the equipment.
 This article presents an overview of an intelligent multi-sensor system for monitoring the process and technological state of electrical equipment. Which in real time with the help of sensors makes it possible to obtain information from the environment and draw conclusions about the condition of the diagnosed equipment. The technology of an intelligent multi-sensor system for identification and assessment of the technical condition of electrical equipment has prospects for wide application in industries. Thanks to the most compact dimensions of the installation, we have the opportunity to take measurements in hard-to-reach places, without removing the equipment from work, which minimizes losses from downtime. Also, the use of this system allows detecting the development of a defect in the early stages of development, which prevents and significantly reduces the cost of repairing power equipment when it is out of service.

Computer technology of multisensor data fusion based on FWA–BP network

Abstract Due to the diversity and complexity of data information, traditional data fusion methods cannot effectively fuse multidimensional data, which affects the effective application of data. To achieve accurate and efficient fusion of multidimensional data, this experiment used back propagation (BP) neural network and fireworks algorithm (FWA) to establish the FWA–BP multidimensional data processing model, and a case study of PM2.5 concentration prediction was carried out by using the model. In the PM2.5 concentration prediction results, the trend between the FWA–BP prediction curve and the real curve was basically consistent, and the prediction deviation was less than 10. The average mean absolute error and root mean square error of FWA–BP network model in different samples were 3.7 and 4.3%, respectively. The correlation coefficient R value of FWA–BP network model was 0.963, which is higher than other network models. The results showed that FWA–BP network model could continuously optimize when predicting PM2.5 concentration, so as to avoid falling into local optimum prematurely. At the same time, the prediction accuracy is better with the improvement in the correlation coefficient between real and predicted value, which means, in computer technology of multisensor data fusion, this method can be applied better.

A novel fault classification feature extraction method for rolling bearing based on multi-sensor fusion technology and EB-1D-TP encoding algorithm

To improve the accuracy of bearing fault diagnosis in a multisensor monitoring environment, it is necessary to obtain more accurate and effective fault classification features for bearings. Accordingly, a bearing fault classification feature extraction method based on multisensor fusion technology and an enhanced binary one-dimensional ternary pattern (EB-1D-TP) algorithm were proposed in this study. First, an optimal equalization weighting algorithm was established to realize high-precision fusion of bearing signals by introducing an optimal equalization factor and determining the theoretical optimal equalization factor value. Second, an enhanced binary encoding method similar to balanced ternary encoding was developed, which increases the difference between the different fault features of the bearing. Finally, the new sequence obtained after encoding was used as the input to a support vector machine to classify and diagnose the faults of the rolling bearing. The experimental results show that the algorithm can significantly improve the accuracy and speed of rolling-bearing fault classification. Combining fusion-encoding features with other intelligent classification methods, the classification results were improved.

Research on the measurement method of gas-water two-phase flow based on dual-sensor system

Realizing the online accurate measurement of gas-water two-phase flow is of great significance for the natural gas industry. In this paper, a dual-sensor with the microwave resonant cavity sensor embedded in the throat of Venturi is proposed and both of the water volume fraction (WVF) and the separated flowrates of the two phases is attained by the method. To further improve the performance and the measurement accuracy of the sensor, a swirler is fixed inside the contraction section of the precession Venturi, which can regularize all flow patterns into spiral annular flow and reduce the influence caused by the nonuniform distribution of air and water. The microwave sensor response characteristics with WVF and Venturi over-reading (OR) characteristics have been researched through dynamic experiments, and the iteration method of OR model obtained by the precession Venturi and WVF model obtained by the microwave sensor is used to solve the gas and water flowrates respectively. The results show that gas phase flowrate relative error band of ±5% at a confidence probability of 86.08%, and Root Mean Square Error (RMSE) is 3.349%; while water flowrate relative error band of ±10% at a confidence probability of 87.70%, and RMSE is 9.680%, which suggests that the multi-sensor fusion technology provides a powerful solution for the measurement of separated flowrates of gas-water two-phase flow.

Novel cartographer using an OAK-D smart camera for indoor robots location and navigation

In recent years, service robots have been widely used in people’s daily life, and with the development of more and more intelligence, people put forward higher requirements for autonomous positioning and navigation functions of robots. Like outdoor navigation, indoor navigation also needs the support of navigation data. Although the indoor positioning and navigation scheme based on cameras, lidars and other sensors is gradually developing, due to the complexity of the indoor structure, manual production of indoor navigation data is time-consuming and laborious, and the efficiency is relatively low. In order to solve the problem of low productivity and improve the accuracy of robot automatic navigation, we added a new type of intelligent camera, called OpenCV AI kit or OAK-D, and proposed a method to automatically build data files that can be used for indoor navigation and location services using indoor 3D point cloud data. This intelligent camera performs neural reasoning on chips that do not use GPUs. It can also use stereo drills for depth estimation, and use 4K color camera images as input to run the neural network model. Python API can be called to realize real-time detection of indoor doors, windows and other static objects. The target detection technology uses an artificial intelligence camera, and the robot can well identify and accurately mark on the indoor map. In this paper, a high-performance indoor robot navigation system is developed, and multisensor fusion technology is designed. Environmental information is collected through artificial intelligent camera (OAK-D), laser lidar, and data fusion is carried out. In the experiment part of this paper,The static fusion map module is created based on the laser sensor information and the sensor information of the depth camera, the hierarchical dynamic cost map module is created in the real-time navigation, and the global positioning of the robot is realized by combining the word bag model and the laser point cloud matching. Then a software system is realized by integrating each module. The experiment proves that the system is practical and effective, and has practical value.

Digital Twin Simulation Tools, Spatial Cognition Algorithms, and Multi-Sensor Fusion Technology in Sustainable Urban Governance Networks

Relevant research has investigated how predictive modeling algorithms, deep-learning-based sensing technologies, and big urban data configure immersive hyperconnected virtual spaces in digital twin cities: digital twin modeling tools, monitoring and sensing technologies, and Internet-of-Things-based decision support systems articulate big-data-driven urban geopolitics. This systematic review aims to inspect the recently published literature on digital twin simulation tools, spatial cognition algorithms, and multi-sensor fusion technology in sustainable urban governance networks. We integrate research developing on how blockchain-based digital twins, smart infrastructure sensors, and real-time Internet of Things data assist urban computing technologies. The research problems are whether: data-driven smart sustainable urbanism requires visual recognition tools, monitoring and sensing technologies, and simulation-based digital twins; deep-learning-based sensing technologies, spatial cognition algorithms, and environment perception mechanisms configure digital twin cities; and digital twin simulation modeling, deep-learning-based sensing technologies, and urban data fusion optimize Internet-of-Things-based smart city environments. Our analyses particularly prove that virtual navigation tools, geospatial mapping technologies, and Internet of Things connected sensors enable smart urban governance. Digital twin simulation, data visualization tools, and ambient sound recognition software configure sustainable urban governance networks. Virtual simulation algorithms, deep learning neural network architectures, and cyber-physical cognitive systems articulate networked smart cities. Throughout January and March 2023, a quantitative literature review was carried out across the ProQuest, Scopus, and Web of Science databases, with search terms comprising “sustainable urban governance networks” + “digital twin simulation tools”, “spatial cognition algorithms”, and “multi-sensor fusion technology”. A Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) flow diagram was generated using a Shiny App. AXIS (Appraisal tool for Cross-Sectional Studies), Dedoose, MMAT (Mixed Methods Appraisal Tool), and the Systematic Review Data Repository (SRDR) were used to assess the quality of the identified scholarly sources. Dimensions and VOSviewer were employed for bibliometric mapping through spatial and data layout algorithms. The findings gathered from our analyses clarify that Internet-of-Things-based smart city environments integrate 3D virtual simulation technology, intelligent sensing devices, and digital twin modeling.

FDO-Calibr: visual-aided IMU calibration based on frequency-domain optimization

Nowadays, multi-sensor fusion technology is a fundamental prerequisite to achieve highly autonomous robots and robustness. Many studies have been conducted, such as visual–inertial odometry (VIO), integrated navigation, and LiDAR–inertial odometry. Typically, for VIO, gratifying results have been achieved, ascribing to the complementary sensing capabilities of inertial measurement units (IMUs) and cameras. However, this work mainly focuses on the fusion of visual and inertial data, while the IMU error is less considered, especially for low-cost or poorly calibrated microelectromechanical system (MEMS) IMU. Such errors may have a significant effect on the VIO performance. In this study, we compensated for the IMU using camera assistance. The key characteristic of the method is that we optimize the compensation parameters (scale factor) from coarse to fine by combining the time domain with the frequency domain. The proposed method is to use the time-domain and frequency-domain optimization to suppress large noise in the dynamic calibration process of the extremely low-cost sensor platform. The effectiveness of this method is validated through experiments and simulations. The minimal calibration error (0.46%) is commensurate with the advanced work. By feeding the compensated IMU into the VIO algorithm, the localization accuracy is improved by 9% to 15%. This method improves the performance in the VIO algorithm, which is equipped with the low-cost or poorly calibrated MEMS IMU and reduces the hardware and deployment costs of the system.

Metaverse Decentralized Governance and Networked Immersive Virtual Reality Systems, Machine Learning-based Image Recognition and Predictive Modeling Tools, and Cognitive Automation and Multisensor Fusion Technologies in Digital Hyper-Realistic Worlds

Metaverse Decentralized Governance and Networked Immersive Virtual Reality Systems, Machine Learning-based Image Recognition and Predictive Modeling Tools, and Cognitive Automation and Multisensor Fusion Technologies in Digital Hyper-Realistic Worlds

Remote Big Data Management and Visual Imagery Tools, Multisensor Fusion and Dynamic Routing Technologies, and 3D Space Mapping and Object Recognition Algorithms on Blockchain-based Metaverse Platforms

Remote Big Data Management and Visual Imagery Tools, Multisensor Fusion and Dynamic Routing Technologies, and 3D Space Mapping and Object Recognition Algorithms on Blockchain-based Metaverse Platforms

Survey on Image and Point-Cloud Fusion-Based Object Detection in Autonomous Vehicles

With the improvements in sensor performance (cameras, Lidars) and the application of deep learning in object detection, autonomous vehicles (AVs) are gradually becoming more notable. After 2019, AV has produced a wave of enthusiasm, and many papers on object detection were published, boasting both practicality and innovation. Due to hardware limitations, it is difficult to accomplish accurate and reliable environment perception using a single sensor. However, multi-sensor fusion technology provides an acceptable solution. Considering the AV cost and object detection accuracy, both the traditional and existing literature on object detection using image and point-cloud was reviewed in this paper. Additionally, for the fusion-based structure, the object detection method was categorized in this paper based on the image and point-cloud fusion types: early fusion, deep fusion, and late fusion. Moreover, a clear explanation of these categories was provided including both the advantages and limitations. Finally, the opportunities and challenges the environment perception may face in the future were assessed.

Dynamic Load Regulation of Robots with Multi-sensor Fusion

Multi-sensor fusion technology has been extensively deployed for the autonomous navigation and control of robots. Theoretically, by increasing the number of sensors, more effective data on the surrounding environment and obstacle information can be obtained. However, in practical applications, a massive number of sensors will not only increase the cost of using sensors in the system but also add to the difficulty and cost of data processing, leading to unnecessary resource consumption. To solve this problem, first, the robot navigation area is divided according to the importance of the navigation area and the characteristics of sensor parameters. The navigation non-core area adopts the single lidar sensor, while the core area adopts the multi-sensor fusion of “lidar +” to reduce the number of sensors used in the fusion and improve the performance of multi-sensor fusion after the division. Then, in consideration of the higher requirement for computing capacity of the device imposed by deep learning, the multi-sensor fusion algorithm is introduced into the dynamic load regulation mechanism to solve the problem from the algorithm level so that the outperformance of deep learning in feature extraction and other aspects can be utilized to reduce the performance requirements of the device and the overall power consumption of the system. Finally, by building the ROS experimental platform, it is verified that the robot can use autonomous navigation in complex and diversified home scenes.

Research on Waste Plastics Classification Method Based on Multi-Scale Feature Fusion.

Microplastic particles produced by non-degradable waste plastic bottles have a critical impact on the environment. Reasonable recycling is a premise that protects the environment and improves economic benefits. In this paper, a multi-scale feature fusion method for RGB and hyperspectral images based on Segmenting Objects by Locations (RHFF-SOLOv1) is proposed, which uses multi-sensor fusion technology to improve the accuracy of identifying transparent polyethylene terephthalate (PET) bottles, blue PET bottles, and transparent polypropylene (PP) bottles on a black conveyor belt. A line-scan camera and near-infrared (NIR) hyperspectral camera covering the spectral range from 935.9 nm to 1722.5 nm are used to obtain RGB and hyperspectral images synchronously. Moreover, we propose a hyperspectral feature band selection method that effectively reduces the dimensionality and selects the bands from 1087.6 nm to 1285.1 nm as the features of the hyperspectral image. The results show that the proposed fusion method improves the accuracy of plastic bottle classification compared with the SOLOv1 method, and the overall accuracy is 95.55%. Finally, compared with other space-spectral fusion methods, RHFF-SOLOv1 is superior to most of them and achieves the best (97.5%) accuracy in blue bottle classification.

Construction of Leisure Physical Education Teaching Model Based on Multisensor Fusion

Physical education is an important part of course teaching. Doing well in the teaching of this course can not only improve students’ quality, but also promote students’ healthy and all-round development. With the rapid development of China’s economy, leisure sports, as an important way of leisure life, are increasingly favored by people. However, the related leisure sports service talents and industrial management talents are in short supply, which has restricted the development of the leisure sports industry in breadth and depth. The demand in the leisure sports market for advanced applied talents has put forward the call of the times for the direction of running sports colleges and universities, which is also the significance of the study. To sum up, this paper builds a leisure physical education teaching model based on multisensor fusion. Firstly, it summarizes leisure sports and multisensor fusion technology and then introduces how to add multisensor technology to physical education teaching. Finally, some experiments are carried out using the data set published by the University of California. In the model training, the super parameters of the convolutional neural network with two to eight layers are tested. The experimental results verify the effectiveness of the model; it achieves an accuracy of 86% when the number of convolutional layers is 6. The connotation of sports leisure and entertainment and the content of talent training mode is clarified, which lays a theoretical foundation for the following research. Among them, the construction of a curriculum system is the core part of talent training measures, and it is also an important part of the construction of leisure sports specialty.