Multiple Sensor Information Research Articles

DRLSTM-DA: a novel aeroengine remaining useful life prediction method based on deep optimization of multiple sensors information

Abstract One of the crucial tasks in Remaining Useful Life (RUL) prediction is to effectively extract key sensor information from numerous sensor signals. In this article, a novel dual-attention enhanced deep residual LSTM (DRLSTM-DA) is developed to deeply optimize multidimensional sensor signals. First, two LSTM layers are designed to compress and reconstruct representative degradation information from input multidimensional time series data, to generate a new feature space. Second, a novel channel adaptive soft threshold module is designed to assign different weights according to the importance of different sensor information, and simultaneously eliminate the noise information in the signal. Thirdly, a temporal attention mechanism is designed to automatically highlight moments containing important decay information while suppressing unimportant moments. Finally, compared with other SOTA methods, our architecture achieves RMSE of 11.55, 13.74, 11.25, and 14.19 on four sub-datasets of the C-MAPSS dataset, with scores of 234.24, 465.49, 202.23, and 537.66, respectively. Meanwhile, in real aeroengine operation dataset, our architecture achieved the smallest RMSE (8.62). These results all validate the good predictive performance of our model.

An information fusion-based meta transfer learning method for few-shot fault diagnosis under varying operating conditions

In recent years, meta-learning has gained increasing attention in the field of fault diagnosis due to its advantages of handling small samples and exhibiting fast adaptation across different diagnostic tasks. However, the scenario of sharply varying operating conditions and the heavy computation burden still limit the effective application of meta-learning in the field of transfer fault diagnosis. To address the above challenges, a few-shot meta transfer diagnosis method is proposed based on information fusion-based model agnostic meta-learning (IFMAML). Firstly, the information enhancement method based on sparse principal component analysis is introduced to enhance the domain invariant features and reduce data redundancy. Subsequently, the information fusion strategy of multiple sensor data is proposed to form the red, green, and blue (RGB) channel information of images, which can enrich the diversity of domain invariant features and mine the spatial information of multiple sensors. Then, the IFMAML, with its enhanced potential for diagnostic performance and computational efficiency, is developed to address the challenging few-shot cross-domain transfer diagnosis under varying operating conditions. Finally, two case studies for gearbox fault diagnostics considering sharply varying speed conditions and unknown health conditions have been conducted to demonstrate the effectiveness and superiority of the proposed method. Experimental results have indicated that the proposed IFMAML method can achieve superior diagnostic accuracy and can be quickly adapted to new transfer diagnosis scenarios under varying operating conditions, when compared with several mainstream meta-learning methods.

Research on the Application of Recognition and Detection Technology in Automatic Driving

Object detection is the cornerstone of autonomous driving systems. In autonomous driving systems, vehicle detection is critical to maintaining traffic flow and avoiding collisions. The use of deep learning technology enables the system to skillfully and accurately identify vehicles on the road. Deep learning algorithms can provide autonomous vehicles with precise environmental awareness to enable informed driving decisions. Deep learning algorithms can achieve pedestrian detection. Such algorithms can improve the safety of vehicle operations. Through deep learning models, the system is able to identify and track pedestrians, ensuring appropriate responses in different traffic scenarios. The application of detection technology enables vehicles to promptly identify and interpret road signs and traffic lights in real-time. This recognition ability helps to comprehensively understand traffic conditions and improve the ability to adapt to different road conditions. This study provides an in-depth exploration of the importance and application of deep learning technology in the field of autonomous driving. Special attention is paid to the key role played by object detection methods. and propose future development directions. Deep learning algorithms can expand the application of learning technology to handle real-time traffic conditions and achieve rapid response. Achieve the fusion of multiple sensory information by improving the local processing capabilities of edge computing. This research aims to further improve the level of intelligence inherent in autonomous driving systems.

MM-Point: Multi-View Information-Enhanced Multi-Modal Self-Supervised 3D Point Cloud Understanding

In perception, multiple sensory information is integrated to map visual information from 2D views onto 3D objects, which is beneficial for understanding in 3D environments. But in terms of a single 2D view rendered from different angles, only limited partial information can be provided. The richness and value of Multi-view 2D information can provide superior self-supervised signals for 3D objects. In this paper, we propose a novel self-supervised point cloud representation learning method, MM-Point, which is driven by intra-modal and inter-modal similarity objectives. The core of MM-Point lies in the Multi-modal interaction and transmission between 3D objects and multiple 2D views at the same time. In order to more effectively simultaneously perform the consistent cross-modal objective of 2D multi-view information based on contrastive learning, we further propose Multi-MLP and Multi-level Augmentation strategies. Through carefully designed transformation strategies, we further learn Multi-level invariance in 2D Multi-views. MM-Point demonstrates state-of-the-art (SOTA) performance in various downstream tasks. For instance, it achieves a peak accuracy of 92.4% on the synthetic dataset ModelNet40, and a top accuracy of 87.8% on the real-world dataset ScanObjectNN, comparable to fully supervised methods. Additionally, we demonstrate its effectiveness in tasks such as few-shot classification, 3D part segmentation and 3D semantic segmentation.

High Precision Vehicle Location Method Based on Multi-sensor Fusion in the Urban Complex Environment

In this paper, a high-precision vehicle positioning method is proposed, which can achieve centimeter-level positioning accuracy in a complex environment. Based on the information of multiple sensors (GNSS, Lidar, and IMU), the multi-sensor fusion framework is used to fuse the positioning measurements from different sources with Kalman filter, which achieves high-precision positioning in the case of GNSS disconnection, the accuracy and robustness of the positioning system are significantly improved, and the effectiveness of the method is verified. The average RMS accuracy of 0.07m is realized, which meets the requirements of high-precision positioning for autonomous vehicles.

Combined navigation system for substation inspection robot based on DGPS/DR multi-sensor fusion

To realize the routine inspection work of the capacitor tower inspection operation robot on the capacitor tower in the substation, and to guarantee the accuracy and safety of the robot’s movement in the substation, a multi-sensor combination navigation system and the corresponding Kalman fusion filter are designed. Combined with the overall structure of the substation and the working environment of the inspection robot, the navigation and positioning requirements of the robot are analyzed and compared, and the multi-sensor combination positioning technology using a combination of DGPS, attitude sensors and velocity sensors is determined. The Kalman fusion filter based on the constant velocity model is used to fuse the positioning information of multiple sensors, and more accurate and reliable navigation information is obtained compared with the single-sensor positioning system. According to the working environment of the substation, an outdoor planar robot mobile navigation experiment is carried out to collect the positioning information of multiple sensors and carry out data fusion processing to verify the accuracy and robustness of multi-sensor combination navigation and positioning.

Interactive Prognosis Framework Between Deep Learning and a Stochastic Process Model for Remaining Useful Life Prediction.

Uncertainty quantification of the remaining useful life (RUL) for degraded systems under the big data era has been a hot topic in recent years. A general idea is to execute two separate steps: deep-learning-based health indicator (HI) construction and stochastic process-based degradation modeling. However, there exists a critical matching defect between the constructed HI and a degradation model, which seriously affects the RUL prediction accuracy. Toward this end, this article proposes an interactive prognosis framework between deep learning and a stochastic process model for the RUL prediction. First, we resort to stacked contractive autoencoders to fuse multiple sensor information of historical systems for constructing the HI in a typical unsupervised manner. Then, considering the nonlinear characteristic of the constructed HI, an exponential-like degradation model is introduced to construct its degradation evolving model, and theoretical expressions of the prediction results are derived under the concept of the first hitting time. Furthermore, we design an optimization objective function by integrating the HI construction and degradation modeling for the RUL prediction. To minimize the designed objective function of the proposed interactive prognosis framework, a gradient descent algorithm is employed to update the model parameters. Based on the well-trained interactive prognosis model, we can obtain the HI of a field system from stacked contractive autoencoders with sensor data and the probability density function (pdf) of the predicted RUL on the basis of the estimated parameters. Finally, the effectiveness and superiority of the proposed interactive prognosis method are verified by two case studies associated with turbofan engines.

Research on immersive interaction design based on visual and tactile feature analysis of visually impaired children

To propose improved and innovative visual-tactile interaction design for visually impaired children, multi-modal combination approaches have been applied, such as voice interaction, touch interaction, and multi-modal systems, what's more, aided cognitive approaches help them deepen their understanding of objects, improve their cognitive level, and increase their interest. Methods: To improve the amount of information in the visual-haptic interface by integrating multiple sensory information, based on the cognitive patterns of visually impaired children, a questionnaire was used to design a tactile-visual UI for the main content objects of visually impaired children when using the Internet, from which difficulties and problems in the design of visual-haptic for visually impaired children were found, and design improvements were proposed based on the principles and methods of accessible design. RESULTS: The personalized and humanized design activities enhance the confidence and improve the quality of life of the visually impaired children group and produce positive effects, improving the cognitive clarity of visually impaired children while increasing their level of understanding, imagination, learning interest and aesthetic experience. Conclusion: The physical and mental characteristics and visual and tactile characteristics of visually impaired children are analyzed, and the application of UI interaction design is based on these characteristics. The essence of interaction design is outlined through experiments, and it is found that with the development of the Internet, big data and artificial intelligence, visually impaired children have many difficulties in the use of the Internet, and through the improved practice of immersive interaction design, the humanized design approach is used to enhance visually impaired children's experience of using network interfaces. Through the improved practice of immersive interaction design, we improve the way of visually impaired children using the Internet, narrow the gap between them and normal children in the interaction, and give humanistic care.

A target spatial location method for fuze detonation point based on deep learning and sensor fusion

The spatial location of fuze detonation point is crucial for evaluating the working condition and improving the performance of fuze. Considering the observation safety, the non-contact long-distance accurate measurement technology is essential. In this paper, we propose a method that takes the sensor data of optics, spatial attitude and GPS as input and outputs the spatial position of the fuze detonation point. The proposed method consists of two steps. First, an object detection algorithm with post-processing algorithm is proposed to obtain rich information of the target. The algorithm achieves high-accuracy detection by introducing powerful backbone, attention mechanism, group convolution, and improved multi-scale feature fusion. Second, a Variational Auto-Encoder (VAE) algorithm model improved by dense connection structure and multiple source heterogeneous sensor information fusion structure is proposed as the position regression algorithm. It receives the status information of the observer camera and the output of the object detection algorithm, and then outputs the three-dimensional coordinates of the explosion point. Finally, method validation and performance analysis are realized through virtual scene simulation. Experiment results show the superiority of the proposed object detection algorithm over other typical algorithms on explosion flare detection, with its Average Precision (AP) of 0.889. The positioning error of the spatial location method is 0.896 m, while that of the binocular stereo vision method is 2.863 m. Therefore, the proposed target spatial location method is proved to be effective and accurate.

Autonomous Oropharyngeal-Swab Robot System for COVID-19 Pandemic

The outbreak of COVID-19 has led to the shortage of medical personnel and the increasing need for nucleic acid testing. Manual oropharyngeal sampling is susceptible to inconsistency caused by fatigue and close contact could also cause healthcare personnel exposure and cross infection. The innate deficiency calls for a safer and more consistent way to collect the oropharyngeal samples. Therefore a fully autonomous oropharyngeal-swab robot system is proposed in this paper. The system is installed in a negative pressure chamber and carrying out a standardized sampling process to minimize individual sampling differences. A hierarchical throat detection algorithm is presented and multiple modality sensory information are fused to safely and accurately localize the optimum sampling location. Also, a force/position hybrid control method is adopted to ensure both accurate sampling and subject comfort. The robot system described in this paper can safely and efficiently collect the oropharyngeal sample, providing a scalable solution for large-scale Polymerase Chain Reaction (PCR) Molecular sample collection for various respiratory diseases. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —During the COVID-19 pandemic, pre-diagnostic is essential for both prevention and treatment. Existing approaches, including nasal swab and oropharyngeal-swab, require extensive medical worker training and increase the chance of cross-infection. The robot system introduced in this paper can take oropharyngeal-swab samples from subjects with minimum human intervention, reducing medical worker exposure, alleviating the work pressure of medical staff, and speed up large quantity of sampling plan. The robot will first guide the subject into position with vocal commands, and automatically detect the optimum sampling location with a real-time machine learning algorithm. A dedicated control strategy aiming at minimizing discomfort and uniforming sample quantity is then applied to safely collect nucleic samples from the throat. Eventually, while the swab is being stored in the culture medium, a disinfection process is carried out simultaneously to prepare the robot for the next subject. Preliminary clinical trials show that our robot system can safely and accurately collect samples from subjects.

Grasping Pose Detection for Loose Stacked Object Based on Convolutional Neural Network With Multiple Self-Powered Sensors Information

There are a variety of objects, random postures and multiple objects stacked in a disorganized manner in unstructured home applications, which leads to the object grasping posture estimation and grasping planning based on machine vision become very complicated. This paper proposes a method of cluttering pose detection based on convolutional neural network with multiple self-powered sensors information. Firstly, a search strategy for candidate grasping poses based on the 3D point cloud is proposed, and the single-channel grasping image dataset representing grasping posture in this paper is established by using Bigbird dataset. Secondly, the ResNet is constructed to rank and filter the single channel captured images representing the captured bit pose. It is also compared with three mainstream classification networks, Inception V2, VGG-A and LetNet, and the perception analysis function and the execution planning function are developed under ROS. The effective grasping of the manipulator in the scene of scattered piles is realized based on the detection results of grasping position and combined with the information of multiple self-powered sensors, and the ResNet network is compared with other three classification networks. In a scattered and stacked environment of objects, the results of experiment show that the method based on ResNet network is superior to the other three networks, and the average success rate of grasping pose detection based on ResNet, InceptionV2, VGGA and LetNet networks is 90.67%, 82.67%, 86.67% and 87.33% respectively, which verifies the effectiveness and superiority of the deep learn-based grasping pose detection model proposed in this paper.

Knowledge Embedded Lightweight Vision Transformer for Machine Condition Monitoring

Machine condition monitoring methods are a powerful tool for the physical condition evaluation of high-end equipment. As one of the main manufacturing applications of machine condition monitoring, tool performance evaluation (TPE) suffers the limitation of model generalization ability across multiple operating conditions. Subsequently, designing a method that can meet the diversified industrial scenarios’ requirements to confirm the cutters’ machinability is of great practical significance. However, recent TPE studies indicate that the on-machine measurement processes are predominantly affected by the interference of ambient noise and signal aliasing of multi-source sensors. Recent advances in knowledge-embedding and artificial intelligence bring new vitality to the development of industrial condition-based maintenance applications. Considering the above-mentioned analysis, this paper proposes a new knowledge-embedded intelligent TPE method, where the prior knowledge of machining parameters is fused with features extracted from multiple sensor information. The proposed method gives an effective solution for TPE tasks under multiple working conditions.

Research on Application of Neural Network Model in Location Privacy Protection of Wireless Sensor Networks

Due to the characteristics of WSN (Wireless sensor network), attackers can easily eavesdrop on data packets transmitted between single or even multiple communication links, and separate sensitive data from multiple sensor information, which poses a great threat to location privacy. Therefore, it is very important to effectively protect the privacy of training sample data while using WSN. Neural network is an important research hotspot in AI field, and it is a model close to biological neural network in machine learning algorithm. In this paper, an application of neural network model in WSN location privacy protection is given. A CNN (Convolutional Neural Network) location privacy protection prediction protocol based solely on additive homomorphisms is proposed, which can effectively ensure that input features, model parameters, and intermediate values are not leaked during the prediction process. The experimental results show that the proposed method has good robustness and can effectively protect the private location of the source node.

Experimental Study of Influence Line–Based Damage Localization for Long-Span Cable Suspension Bridges

Influence line (IL)–based damage indices for long-span bridges are investigated in this study. Their effectiveness is experimentally validated for the first time through the scaled physical model of the Tsing Ma Bridge (TMB). First, the IL mechanism for damage detection and its corresponding damage indices are briefly introduced. Subsequently, the scaled TMB model instrumented with different types of sensors, including displacement sensors, strain gauges, and accelerometers, is introduced. The IL characteristics of different bridge components are compared. Two different damage cases with single- and double-damage locations at the bottom chord were tested. In the single-damage case, different ILs extracted from the nearby components were used for damage identification. These ILs can successfully locate damage visually. The strain IL (SIL) is more sensitive to local damage than deflection IL, but its detection performance degrades rapidly with the increasing distance between sensor and force locations. In the double-damage case, the SIL extracted from a single sensor cannot identify both damage cases because of the limited detectable range of each SIL; therefore, using multiple sensor information becomes necessary. For comparison, the modal parameters were also employed for damage detection. These experimental results validated the merits of the IL–based methods proposed for long-span bridges, indicating that IL–based damage indices are good indicators of local damage detection in long-span bridges. This finding contributes to the development of real-time techniques for damage localization in long-span bridges equipped with a structural health monitoring system.

Dealing with uncertainty in sensorimotor control in sports: A systematic review

Introduction&#x0D; In complex sensorimotor behavior, uncertainty arises from ambiguity in the sensory inputs of the environment (e.g. Kersten et al., 2004) as well as from noise in sensory detection and motor execution (e.g. Todorov &amp; Jordan, 2002). Moreover, the observable sensory inputs are delayed and there are typically multiple solutions to solve a motor task (e.g. Franklin &amp; Wolpert, 2011). In sports, where human capacity reaches its limit with these challenges, dealing with uncertainty is crucial. Thus, the question arises how humans are able to deal with uncertainty in sensorimotor control in naturalistic situations such as sports.&#x0D; Methods&#x0D; A systematic search for original articles was conducted in six scientific databases with the following terms: (uncertainty OR noise) AND (sensor* OR motor) AND control AND sport AND movement. After independently screening 4,309 articles and additional reference lists by two raters, 70 articles remained for the review.&#x0D; Results&#x0D; There is clear evidence that prior knowledge (or similar contextual information) and multiple sensory information affect perception, action and performance. Moreover, there is considerable evidence that prior knowledge and multiple sensory information get integrated according to their reliability in order to reduce uncertainty. Furthermore, it has been reported that gaze behavior is close to the optimal compared to an optimal Bayesian observer in an expected and unexpected uncertain environment. There is also considerable evidence that not all variables have to be controlled and rather uncertainty is accepted in variables that do not impair overall performance. Some findings show that inherent noise is taken into account when planning motor execution so that the potential costs and rewards of the movement outcome are optimized. Finally, some findings show that unforeseen perturbation can be absorbed by means of higher muscle stiffness resulting from muscular co-contraction.&#x0D; Discussion&#x0D; The strategies to deal with uncertainty mentioned in the results section can be summarized under the umbrella of Bayesian decision theory (Franklin &amp; Wolpert, 2011). Only a few alternative models were referenced that, however, were conceptually quite similar to Bayesian models. Therefore, Bayesian decision theory seems to offer a promising framework for future research on uncertainty in sensorimotor control in sport science.&#x0D; References&#x0D; Franklin, D. W., &amp; Wolpert, D. M. (2011). Computational mechanisms of sensorimotor control. Neuron, 72(3), 425-442. https://doi.org/10.1016/j.neuron.2011.10.006&#x0D; Kersten, D., Mamassian, P., &amp; Yuille, A. (2004). Object perception as Bayesian inference. Annual Review of Psychology, 55, 271-304. https://doi.org/10.1146/annurev.psych.55.090902.142005&#x0D; Todorov, E., &amp; Jordan, M. I. (2002). Optimal feedback control as a theory of motor coordination. Nature Neuroscience, 5(11), 1226-1235. https://doi.org/10.1038/nn963

Modeling and Parameter Identification of a 3D Measurement System Based on Redundant Laser Range Sensors for Industrial Robots.

The low absolute positioning accuracy of industrial robots is one of the bottlenecks preventing industrial robots from precision applications. Kinematic calibration is the main way to improve the absolute positioning accuracy of industrial robots, which greatly relies on three-dimensional (3D) measurement instruments, including laser trackers and pull rope mechanisms. These instruments are costly, and their required intervisibility space is large. In this paper, a precision 3D measurement instrument integrating multiple laser range sensors is designed, which fuses the information of multiple redundant laser range sensors to obtain the coordinates of a 3D position. An identification model of laser beam position and orientation parameters based on redundant distance information and standard spherical constraint is then developed to reduce the requirement for the assembly accuracy of laser range sensors. A hybrid identification algorithm of PSO-LM (particle swarm optimization Levenberg Marquardt) is designed to solve the high-order nonlinear problem of the identification model, where PSO is used for initial value identification, and LM is used for final value identification. Experiments of identification of position and orientation, verifications of the measuring accuracy, and the calibration of industrial robots are conducted, which show the effectiveness of the proposed 3D measurement instrument and identification methods. Moreover, the proposed instrument is small in size and can be used in narrow industrial sites.

Water-Modulated Biomimetic Hyper-Attribute-Gel Electronic Skin for Robotics and Skin-Attachable Wearables.

Electronic skin (e-skin), mimicking the physical-chemical and sensory properties of human skin, is promising to be applied as robotic skins and skin-attachable wearables with multisensory functionalities. To date, most e-skins are dedicated to sensory function development to mimic human skins in one or several aspects, yet advanced e-skin covering all the hyper-attributes (including both the sensory and physical-chemical properties) of human skins is seldom reported. Herein, a water-modulated biomimetic hyper-attribute-gel (Hygel) e-skin with reversible gel-solid transition is proposed, which exhibits all the desired skin-like physical-chemical properties (stretchability, self-healing, biocompatibility, biodegradability, weak acidity, antibacterial activities, flame retardance, and temperature adaptivity), sensory properties (pressure, temperature, humidity, strain, and contact), function reconfigurability, and evolvability. Then the Hygel e-skin is applied as an on-robot e-skin and skin-attached wearable to demonstrate its highly skin-like attributes in capturing multiple sensory information, reconfiguring desired functions, and excellent skin compatibility for real-time gesture recognition via deep learning. This Hygel e-skin may find more applications in advanced robotics and even skin-replaceable artificial skin.

Eggshell Cutting Task Strategy for Robot that Handles Drill Based on Multiple Information Feedback of Force-sensing, Auditory-sensing, and Hand-position

Autonomous robots are expected to perform minute tasks that require skill for humans. In order for robots to achieve autonomous tasks, multiple sensory information processing and feedback strategies are indispensable. This study focuses on eggshell cutting task strategy that supports the exposure of eggshell membranes by a robot that handles micro drill. We propose the task strategy that robot performs discrete and continuous cutting on eggshell and the failure detection when the eggshell membrane is damaged. Finally, we show the experimental results of the autonomous eggshell cutting tasks by robot and the human-assisted eggshell peeling and eggshell membrane exposure. Besides, we consider when to stop the cutting task so as not to break the eggshell membrane.

Vehicle assisted bridge damage assessment using probabilistic deep learning

Vehicle assisted monitoring has shown promising potential for the condition assessment of existing bridges in a road network, by removing practical complications faced in traditional Structural health monitoring (SHM) methods such as traffic interruption and dense deployment of sensors. However, the combination of different measurement sources during vehicle assisted monitoring has not yet been fully explored. This paper aims to evaluate the potential benefit of considering multiple measured responses from various sources, including fixed sensors on the bridge and on-board vehicle sensors. To this end, this paper proposes a Probabilistic Deep Neural Network, a stochastic data-driven framework for damage assessment. This framework enables the combination of vehicle and bridge responses to extract damage sensitive features for the classification of different damage states. In addition, the proposed method estimates the uncertainty of its predictions, providing an indication of the reliability of the result. The proposed method is validated using two numerical based case studies while considering realistic operational conditions, which include temperature oscillations, additional traffic, and measurement noise. The results from this study indicate that combining multiple sensor information results in lower uncertainties in damage detection and localisation. The results also suggest that the proposed method is robust in handling measurement noise and varying environmental conditions.

Sensor Fusion for Social Navigation on a Mobile Robot Based on Fast Marching Square and Gaussian Mixture Model.

Mobile robot navigation has been studied for a long time, and it is nowadays widely used in multiple applications. However, it is traditionally focused on two-dimensional geometric characteristics of the environments. There are situations in which robots need to share space with people, so additional aspects, such as social distancing, need to be considered. In this work, an approach for social navigation is presented. A multi-layer model of the environment containing geometric and topological characteristics is built based on the fusion of multiple sensor information. This is later used for navigating the environment considering social distancing from individuals and groups of people. The main novelty is combining fast marching square for path planning and navigation with Gaussian models to represent people. This combination allows to create a continuous representation of the environment from which smooth paths can be extracted and modified according to dynamically captured data. Results prove the practical application of the method on an assistive robot for navigating indoor scenarios, including a behavior for crossing narrow passages. People are efficiently detected and modeled to assure their comfort when robots are around.