External Sensors Research Articles

High-Precision Control of Aviation Photoelectric-Stabilized Platform Using Extended State Observer-Based Kalman Filter.

The accuracy of the line-of-sight of aviation photoelectric optoelectronic stabilization platforms is limited by two factors: external disturbance and sensor noise. An extended state observer (ESO) can effectively improve their anti-interference ability. However, due to the serious problem of gyroscope noise, further improvement of an ESO's disturbance suppression effect is limited. This article proposes a control structure that combines a Kalman filter (KF) and ESO, effectively improving upon the interference suppression ability of a traditional ESO under the influence of noise. Firstly, an ESO was used to observe the lumped disturbance of the system, and then, the observed disturbance was compensated for in the control loop. Secondly, based on the compensation servo control system, the state equation of the system was reconstructed using a Kalman filter. Finally, the reconstructed filtered state variables were iterated onto the universal state observer, achieving the observation of disturbances while filtering out sensor noise. Under the conditions of a laboratory flight simulation turntable, the line-of-sight stability accuracy level was improved under disturbance excitation. It can be seen that the combination of a Kalman filter and extended disturbance observer proposed in this project improves the ESO's anti-interference ability under the influence of noise.

Knowledge distillation of multi-scale dense prediction transformer for self-supervised depth estimation

Depth estimation is an inverse projection problem that estimates pixel-level distances from a single image. Although, supervised methods have shown promising results, it has intrinsic limitations in requiring ground truth depth from an external sensor. On the other hand, self-supervised depth estimation relieves the burden for collecting calibrated training data, while there is still a large performance gap between supervised and self-supervised methods. The objective of this study is to reduce the performance gap between the supervised and self-supervised approaches. The loss function of previous self-supervised methods is mainly based on a photometric error, which is indirectly computed from synthesized images using depth and pose estimates. In this paper, we argue that direct depth cue is more effective to train a depth estimation network. To obtain the direct depth cue, we employed a knowledge distillation technique, which is a teacher-student learning framework. The teacher network was trained in a self-supervised manner based on a photometric error, and its predictions were utilized to train a student network. We constructed a multi-scale dense prediction transformer with Monte Carlo dropout, and multi-scale distillation loss was proposed to train the student network based on the ensemble of stochastic estimates. Experiments were conducted on the KITTI and Make3D datasets, and our proposed method achieved the state-of-the-art accuracy in self-supervised depth estimation. Our code is publicly available at https://github.com/ji-min-song/KD-of-MS-DPT.

Research advanced in the visual SLAM methods under indoor environment

Simultaneous localization and mapping (SLAM) have been a hotspot study topic in the computer vision community these days, which aims to locate the agent’s position and perceive the surrounding environment to build a map. In Visual SLAM, the camera is the external sensor used, creating an environment map while locating itself. Compared with radar and other rangefinders, the visual sensor is usually tiny and has low power consumption, which has been the mainstream framework in SLAM research task. This paper starts from the system overview of visual SLAM, introduces the landmark achievements and research hotspots of visual SLAM, and discusses the key issues and three research methods based on visual SLAM in indoor environment. Meanwhile, the applications of visual SLAM in dynamic scenes and large indoor environments are listed. Finally, the trend of its future development is prospected.

Incipient fault detection and condition assessment in DFIGs based on external leakage flux sensing and modified multiscale poincare plots analysis

Abstract Although doubly fed induction generators (DFIG) are widely used, difficulties in early fault detection and severity assessment for inter-turn short-circuit (ITSC) faults are highly prominent. In this manuscript, a novel incipient fault detection and state assessment method based on external leakage flux sensing and modified multiscale poincare plots (MMSPP) is proposed. An external leakage flux sensor is placed on the axial-end position of the generator to monitor the presence and evolution of ITSC faults. Multiscale poincare mapping is a novel nonlinear tool that is further developed and modified using the normal cumulative distribution function and multiscale computing methods to capture the behavior evolution and changes in the generator’s external leakage flux signals. The healthy indicator is based on the analysis of elliptical orbit features extracted from MMSPP, established by support vector data description with parameter optimization. The effectiveness of the proposed method was implemented and verified under an experimental environment on a 100 kW DFIG platform to detect incipient inter-turn short circuits (mainly considering the two-turn ITSC) and evaluate the performance degradation (three- to eight-turn ITSC) with 0%, 50%, and 100% different load conditions. The experimental results showed the viability of the approach and fault indicator for incipient fault detection and condition assessment of the wind generator’s low inter-turn insulation faults and for relative quantification of ITSC fault severity.

A Soft Actor-Critic Approach for a Blind Walking Hexapod Robot with Obstacle Avoidance

This paper investigates a path planning approach for the walking and obstacle avoidance of a blind hexapod robot in various field conditions. Hexapod robots often perform field tasks in unstructured environments, and their external sensors are affected by weather and light. This paper proposes the use of internal sensors to sense the terrain and a slightly modified soft actor-critic algorithm to train the motion strategy. A hexapod robot is capable of walking smoothly on rough ground only using internal sensors that are not affected by weather factors, and the soft actor-critic approach is superior for overcoming high-dimensional issues for multi-degree-freedom robot motion in unstructured environments.The experiments showed that the hexapod robot not only traversed rugged terrain at a fixed speed but also possessed obstacle avoidance capabilities.

Investigating Optimal Smartphone Placement for Identifying Stairs Movement using Machine Learning

The identification of gait activities such as stair ascending and descending poses a significant challenge due to the proximity of data provided by the sensory pathway. Accurate identification of gait activities is crucial in conveying essential gait information to users for the recognition of human movement activities. However, gait patterns can vary significantly between individuals, making it challenging to develop a generalized algorithm for identifying incline surface gait activity. Factors such as walking speed, stride length, and body mechanics can all influence gait patterns, making it difficult to establish a consistent framework. Despite various research on gait event detection for level ground walking, the identification of gait activities on an inclined surface such as stairs, especially using smartphones as sensors, is currently lacking. The goal of this study is to investigate and develop a reliable and accurate method for detecting gait activities on an inclined surface such as stairs using smartphones as the sensing device. Specifically, this study focuses on investigating optimal placement of smartphones to extract tri- axis accelerometer data from the inertial sensors during stair movement. The inertial sensor data was collected from the smartphone at two different positions and two different orientations. The data was trained against 6 machine learning algorithms namely Decision Tree, Logistic Regression, Naive Bayes, Random Forest, Neural Networks and KNN. It was observed that, by using Decision Tree and Random Forest algorithm 100% accuracy was achieved, when smartphone was placed at the thigh during stair movement. Successful identification of stair movement activity by using a smartphone can significantly contribute to future research and could also prove useful to the wider community such as amputees and those with pathological gait. In addition, since smartphones are available to a wide group of people, a low-cost solution for gait activity identification can be realized, without requiring the use of external sensors and circuitry.

Capacitive self tension sensing properties of steel beam: electrode configuration and stress regime

A structural material is called self-sensing when it is able to reveal its own state without the need for any external sensors. Self-sensing for steel is valuable for the control and safety of steel structures such as buildings and bridges. The changes in the capacitance of steel in response to cyclic tensile stresses applied in low, medium and high stress regimes are measured by using a Inductnce-Capacitance-Resistance (LCR) meter. Coplanar and parallel plate electrode configuration is used for capacitance measurements. Aluminum foil is used as electrode. A steel beam of 100 mm in length, 30 mm in width and 2.5 mm in thickness is tensioned by holding it at both ends to produce direct tensile stresses in the material. The maximum stresses applied for low, medium and high stress regimes are 6.7 MPa, 33.3 MPa and 66.7 MPa. The capacitance value of the sample with coplanar and parallel plate electrode configurations measured without applying load are 203.42 pF and 196.00 pf, respectively. The fractional changes in capacitance are 0.059%, 0.192% and 0.275% when 6.7 MPa, 33.3 MPa and 66.7 MPa direct tensile stress is created in the steel beam. These values are 0.12%, 0.20% and 0.29% for parallel plate electrode configuration. Test results demonstrates that there is a relation between stress and fractional change in capacitance. In other words, measuring fractional change in capacitance gives information about the stress variations in the material. From the experimental results, parallel plate electrode configuration is found to be more effective in tensile stress self-sensing. In addition, the relationship between stress and fractional change in capacitance is more linear for both electrode configurations in the low stress regime. This paper aims to reveal the tensile stresses occurring in steel by means of capacitance-based sensing. Sensing capability in larger scale structures and factors effecting sensing sensitivity are to be addressed in future work.

Active Vibration Control Using Loudspeaker-Based Inertial Actuator with Integrated Piezoelectric Sensor

With the evolution of the aerospace industry, structures have become larger and more complex. These structures exhibit significant characteristics such as extensive flexibility, low natural frequencies, numerous modes, and minimal structural damping. Without implementing vibration control measures, the risk of premature structural fatigue failure becomes imminent. In present times, the installation of inertial actuators and control signal acquisition units typically requires independent setups, which can be cumbersome for practical engineering purposes. To address this issue, this study introduces a novel approach: an independent control unit combining a loudspeaker-based inertial actuator (LBIA) with an integrated piezoelectric ceramic sensor. This unit enables autonomous vibration control, offering the advantages of ease of use, low cost, and lightweight construction. Experimental verification was performed to assess the mechanical properties of the LBIA. Additionally, a mathematical model for the LBIA with an integrated piezoelectric ceramic sensor was developed, and its efficacy as a control unit for thin plate structure vibration control was experimentally validated, showing close agreement with numerical results. Furthermore, the LBIA’s benefits as an actuator for low-frequency mode control were verified through experiments using external sensors. To further enhance control effectiveness, a mathematical model of the strain differential feedback controller based on multi-bandpass filtering velocity improvement was established and validated through experiments on the clamp–clamp thin plate structure. The experimental results demonstrate that the designed LBIA effectively reduces vibration in low-frequency bands, achieving vibration energy suppression of up to 12.3 dB and 23.6 dB for the first and second modes, respectively. Moreover, the LBIA completely suppresses the vibration of the fourth mode. Additionally, the improved control algorithm, employing bandpass filtering, enhances the effectiveness of the LBIA-integrated sensor, enabling accurate multimodal damping control of the structure’s vibrations for specified modes.

Deep Learning of Sensor Data in Cybersecurity of Robotic Systems: Overview and Case Study Results

Recent technological advances have enabled the development of sophisticated robotic and sensor systems monitored and controlled by algorithms based on computational intelligence. The deeply intertwined and cooperating devices connected to the Internet and local networks, usually through wireless communication, are increasingly used in systems deployed among people in public spaces. The challenge is to ensure that physical and digital components work together securely, especially as the impact of cyberattacks is significantly increasing. The paper addresses cybersecurity issues of mobile service robots with distributed control architectures. The focus is on automatically detecting anomalous behaviors possibly caused by cyberattacks on onboard and external sensors measuring the robot and environmental parameters. We provide an overview of the methods and techniques for protecting robotic systems. Particular attention is paid to our technique for anomaly detection in a service robot’s operation based on sensor readings and deep recurrent neural networks, assuming that attacks result in the robot behaving inconsistently. The paper presents the architecture of two artificial neural networks, their parameters, and attributes based on which the potential attacks are identified. The solution was validated on the PAL Robotics TIAGo robot operating in the laboratory and replicating a home environment. The results confirm that the proposed system can effectively support the detection of computer threats affecting the sensors’ measurements and, consequently, the functioning of a service robotic system.

Experimental Assessment and Validation of the Hygrothermal Behaviour of an Innovative Light Steel Frame (LSF) Wall Incorporating a Monitoring System

Currently, the construction sector is witnessing a growing demand for lightweight solutions, which can be justified by the need to adopt high-performance solutions and the fact that the industry is struggling with a shortage of skilled labour. In this sense, this study focuses on a novel and flexible building wall system, constructed using an innovative extensible LSF profile. To enhance its functionality, a monitoring system comprising printed sensors was integrated into the wall. These sensors underwent a thorough verification process. To evaluate the hygrothermal performance of the complete LSF wall solution and validate the novel monitoring system, an extensive ageing test focused on heat/rain, freeze/thaw cycles was conducted on a large-scale wall prototype. Additionally, this research introduces a novel approach by simulating exceptional solar radiation conditions, surpassing the standard cycles outlined in EAD 040083-00-0404, for the first time in this kind of solution. The results cover the measurements taken inside the building system using the incorporated monitoring system. Additionally, supplementary external temperature and heat flow sensors were used to determine the thermal transmittance. Visual and thermography inspections were also carried out. The findings reveal no instances of failures or defects that could potentially impact the hygrothermal behaviour of the system. The hybrid LSF constructive solution leads to more stable temperatures on the inner surface. The presence of direct solar radiation can raise surface temperatures by up to 5 °C compared to surfaces not exposed to such radiation, even when a light-coloured surface is used. The monitoring system worked correctly. In conclusion, the innovative profile proved to be resistant to hygrothermal cycles and the monitoring system developed is efficient.

Application of External Torque Observer and Virtual Force Sensor for a 6-DOF Robot

Application of External Torque Observer and Virtual Force Sensor for a 6-DOF Robot

A Machine Learning Empowered Shape Memory Alloy Gripper With Displacement-Force-Stiffness Self-Sensing

This paper proposes a multiple self-sensing gripper (MssGripper) driven by the shape memory alloy (SMA) and empowered by machine learning algorithms. The MssGripper can identify objects without external sensors. A single SMA wire can drive the gripper for self-sensing accurately. This paper confirms the resistance of the SMA can reflect the phase transition and can be used for displacement and force prediction, as well as for object stiffness prediction. Through machine learning, a backpropagation neural network (BPNN) and long-short-term-memory (LSTM) are used to establish multiple self-sensing models for prediction. The robustness experimental results show that the self-sensing models based on LSTM have higher prediction accuracy. The average root-mean-square errors of displacement prediction and force prediction are 0.063 mm and 0.236 N, respectively, and the stiffness prediction error is less than 9.4%. Moreover, the accuracy of the classifier in stiffness identification is 97.2%. The MssGripper can accurately predict the displacement, force, and stiffness and identify objects such as springs, rubber bars and steel bars. The establishment of the models expands the novel idea of gripper sensing, which is beneficial to promoting miniaturization and compactness.

A Protocol for Reliable Real-Time Wireless Communication of Large Data Samples

In this paper, we present a middleware protocol, that transmits larger-than-a-frame data samples within application real-time and reliability requirements over a lossy wireless channel. The protocol efficiently deploys an intelligent retransmission control that exploits the extended deadline requirements of a sample for frame-level scheduling. The transmission of such samples is placed in the context of the necessary integration of such external sensor data into the decision-making process of an autonomous vehicle. Therefore, we provide parameterization and access rules for communication links in resource-managed scenarios and lay out how to integrate the protocol into the popular DDS middleware. The performance of the parameterized links to satisfy their reliability requirements is studied and compared to error protection protocols in the current software stack. We base the evaluation on an OMNeT++ simulation, whereby an established wireless error model is used. The use case is placed in the context of collaborative sensing in a valet parking environment, where external infrastructure sensor data augments the autonomous vehicle's data processing chain via a lossy wireless link. Thereby, the use case adheres to the underlying safety requirements. Results show that if the protocol's awareness of the application requirements is used for parameterization, guarantees for reliability can be provided that outperform existing solutions.

Self-Healing Control for Wastewater Treatment Process Based on Variable-Gain State Observer

This paper proposes a variable-gain state observer-based sliding mode self-healing control (VSO-based SMSHC) method for a wastewater treatment process (WWTP) with changeable external disturbances and sensor failures. To reconstruct the disturbances and unmeasured system states, a novel VSO is developed, in which the VSO gains are adjusted by following an automatic variable-gain mechanism for adapting to the variation of external disturbances. An adaptive compensation coefficient of failure factor is designed to counteract the failure effects on observation and tracking performance. By utilizing the cubic absolute-value Lyapunov stability criterion, it is shown that system stability and tracking performance of WWTP are guaranteed. Experimental studies are carried out on a standardized platform of WWTP, and the results on dissolved oxygen and nitrate nitrogen regulation indicate that the proposed control strategy can ensure excellent control performance and reduce both failure and disturbance impacts.

The Beginning of the AI-Enabled Preventative PAP Therapy Era: A First-in-Human Proof of Concept Interventional Study.

Positive Airway Pressure (PAP) therapy is the most common and efficacious treatment for Obstructive Sleep Apnea (OSA). However, it suffers from poor patient adherence due to discomfort and may not fully alleviate all adverse consequences of OSA. Identifying abnormal respiratory events before they have occurred may allow for improved management of PAP levels, leading to improved adherence and better patient outcomes. Our previous work has resulted in the successful development of a Machine-Learning (ML) algorithm for the prediction of future apneic events using existing airflow and air pressure sensors available internally to PAP devices. Although researchers have studied the use of ML for the prediction of apneas, research to date has focused primarily on using external polysomnography sensors that add to patient discomfort and has not investigated the use of internal-to-PAP sensors such as air pressure and airflow to predict and prevent respiratory events. We hypothesized that by using our predictive software, OSA events could be proactively prevented while maintaining patients' sleep quality. An intervention protocol was developed and applied to all patients to prevent OSA events. Although the protocol's cool-down period limited the number of prevention attempts, analysis of 11 participants revealed that our system improved many sleep parameters, which included a statistically significant 31.6% reduction in Apnea-Hypopnea Index, while maintaining sleep quality. Most importantly, our findings indicate the feasibility of unobtrusive identification and unique prevention of each respiratory event as well as paving the path to future truly personalized PAP therapy by further training of ML models on individual patients.

Early detection of tool wear in electromechanical broaching machines by monitoring main stroke servomotors

This paper aims to provide researchers and engineers with evidence that sensorless machine variable monitoring can achieve tool wear monitoring in broaching in real production environments, reducing production errors, enhancing product quality, and facilitating zero-defect manufacturing. Additionally, broaching plays a crucial role in improving the quality of manufacturing products and processes. These aspects are especially pertinent in aeronautical manufacturing, which serves as the experimental case in this study.The research presents findings that establish a correlation between the variables of a broaching machine’s servomotors and the condition of the broaching tools. The authors propose an effective method for measuring broaching tool wear without external sensors and provide a detailed explanation of the methodology, enabling reproducibility of similar results. The results stem from three trials conducted on an electromechanical vertical broaching machine, utilizing cemented carbide grade broaching tools to broach a superalloy Inconel 718 test piece. The machine data collected facilitated the training of a set of machine learning models, accurately estimating tool wear on the broaches. Each model demonstrates high predictive accuracy, with a coefficient of determination surpassing 0.9.

Research on dynamic parameter identification and collision detection method for cooperative robots

PurposeThe safety performance of cooperative robots is particularly important. This paper aims to study collision detection and response of cooperative robots, which meet the lightweight requirements of cooperative robots and help to ensure the safety of humans and robots.Design/methodology/approachThis paper proposes a collision detection, recognition and response method based on dynamic models. First, this paper identifies the dynamic model of the robot. Second, an external torque observer is established based on the model, and a dynamic threshold collision detection method is designed to reduce the interference of model uncertainty on collision detection. Finally, a collision position and direction estimation method is designed, and a robot collision response strategy is proposed to reduce the harm caused by collisions to humans.FindingsComparative experiments are conducted on static threshold and dynamic threshold collision detection, and the results showed that the static threshold only detected one collision while the dynamic threshold could detect all collisions. Conducting collision position and direction estimation and collision response experiments, and the results show that this method can determine the location and direction of collision occurrence, and enable the robot to achieve collision separation.Originality/valueThis paper designs a dynamic threshold collision detection method that does not require external sensors. Compared with static threshold collision detection methods, this method can significantly improve the sensitivity of collision detection. This paper also proposes a collision position direction estimation method and collision separation response strategy, which can enable robots to achieve post collision separation and improve the safety of cooperative robots.

Study on self-diagnosis/prediction technology for LIDAR sensor of autonomous vehicles

Along with the development of autonomous driving technology, the need for self-failure diagnosis and Remaining Useful Life (RUL) prediction technology for core parts for autonomous driving is increasing. In particular, the characteristics of the light detection and ranging (LIDAR) sensor exposed to the outside further increase the need to apply fault diagnosis and RUL prediction technology considering various environmental variables. In this study, based on the accelerated degradation test of LIDAR, the failure mode was analyzed. Through this, LIDAR failure due to thermal runaway, which is the first failure type in high temperature conditions, was identified, and whether there were major environmental data that could identify thermal runaway was identified. In the case of LIDAR's thermal runaway phenomenon, a study on an algorithm to identify the precursor symptoms of failure in an accidental failure situation is conducted. Afterwards, through the actual vehicle test process, various environmental variable information is analyzed for correlation with LIDAR internal sensor data, and the abnormal data for the temperature of the internal parts of the LIDAR is predicted through the external environmental sensor.

Implementation and displacement self-sensing of a 4-DOF piezoelectric micromanipulator

This paper reports the development and displacement self-sensing of a new four degrees-of-freedom (DOF) piezoelectric micromanipulator. In particular, each micromanipulator arm can move along the clamping and vertical directions, constructing with two degrees of freedom, and two micromanipulator arms exhibit four degrees of freedom. Thus, it improves operational flexibility. Meanwhile, the self-sensing method avoids additional external sensors when detecting output displacements. The piezoelectric micromanipulator is devised based on the transverse inverse piezoelectric effect of two sets of vertical spatial crossings. Static and dynamic characteristics are analyzed via finite element optimization of geometric parameters. Then, a displacement self-sensing method based on charge integration is proposed. This method measures the generated charge on the clamping arm surface from the piezoelectric wafer deformation under the driving voltage. A self-sensing circuit is designed to obtain the surface charge of the clamping arm. Then, several experiments are conducted to test the micromanipulator. The maximum displacements of the designed micromanipulator are about 66.29 μm and 63.09 μm along clamping and vertical directions. Moreover, output displacements, response time, and inherent frequencies obtained based on the self-sensing method agree well with those with laser sensors. In addition, an operation application for micro-shaft assembly is presented, demonstrating the grasping and docking operations of the micro-shaft in the clamping and vertical directions.

A Comprehensive Pattern Recognition Neural Network for Collision Classification Using Force Sensor Signals

In this paper, force sensor signals are classified using a pattern recognition neural network (PRNN). The signals are classified to show if there is a collision or not. In our previous work, the joints positions of a 2-DOF robot were used to estimate the external force sensor signal, which was attached at the robot end-effector, and the external joint torques of this robot based on a multilayer feedforward NN (MLFFNN). In the current work, the estimated force sensor signal and the external joints’ torques from the previous work are used as the inputs to the proposed designed PRNN, and its output is whether a collision is found or not. The designed PRNN is trained using a scaled conjugate gradient backpropagation algorithm and tested and validated using different data from the training one. The results prove that the PRNN is effective in classifying the force signals. Its effectiveness for classifying the collision cases is 92.8%, and for the non-collisions cases is 99.4%. Therefore, the overall efficiency is 99.2%. The same methodology and work are repeated using a PRNN trained using another algorithm, which is the Levenberg–Marquardt (PRNN-LM). The results using this structure prove that the PRNN-LM is also effective in classifying the force signals, and its overall effectiveness is 99.3%, which is slightly higher than the first PRNN. Finally, a comparison of the effectiveness of the proposed PRNN and PRNN-LM with other previous different classifiers is included. This comparison shows the effectiveness of the proposed PRNN and PRNN-LM.