Time Sequence Signals Research Articles

Gesture Recognition Based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory Network for a Wearable Wrist Sensor with Multi-Walled Carbon Nanotube/Cotton Fabric Material.

Flexible pressure sensors play a crucial role in detecting human motion and facilitating human-computer interaction. In this paper, a type of flexible pressure sensor unit with high sensitivity (2.242 kPa-1), fast response time (80 ms), and remarkable stability (1000 cycles) is proposed and fabricated by the multi-walled carbon nanotube (MWCNT)/cotton fabric (CF) material based on a dip-coating method. Six flexible pressure sensor units are integrated into a flexible wristband and made into a wearable and portable wrist sensor with favorable stability. Then, seven wrist gestures (Gesture Group #1), five letter gestures (Gesture Group #2), and eight sign language gestures (Gesture Group #3) are performed by wearing the wrist sensor, and the corresponding time sequence signals of the three gesture groups (#1, #2, and #3) from the wrist sensor are collected, respectively. To efficiently recognize different gestures from the three groups detected by the wrist sensor, a fusion network model combined with a convolutional neural network (CNN) and the bidirectional long short-term memory (BiLSTM) neural network, named CNN-BiLSTM, which has strong robustness and generalization ability, is constructed. The three types of Gesture Groups were recognized based on the CNN-BiLSTM model with accuracies of 99.40%, 95.00%, and 98.44%. Twenty gestures (merged by Group #1, #2, and #3) were recognized with an accuracy of 96.88% to validate the applicability of the wrist sensor based on this model for gesture recognition. The experimental results denote that the CNN-BiLSTM model has very efficient performance in recognizing different gestures collected from the flexible wrist sensor.

Entropy-Based Thunderstorm Imaging System With Real-Time Prediction and Early Warning

A combination of the sample entropy and the artificial intelligence (AI) would hopefully bring progress to the smart thunderstorm detection. In this paper, we establish an entropy-based thunderstorm point charge imaging system with the data clustering, wherein data are ultra-reliable and have low-latency three-dimensional atmospheric electric field (3DAEF) values. In particular, a high resolution 3DAEF sensor with the single-axis rotary vane is developed to measure the time sequence signal of the 3DAEF. The signal is first denoised, and then decomposed into multiple groups of branch data with the same number of samples. In order to break through the limitations of existing clustering rules, we propose a 3DAEF signal reconstruction method based on entropy intervals. By reconstructing the branch data, multiple time-scale data are formed. Finally, the curve fitting of the data is performed to realize the imaging after clustering the data. Experimental results suggest that the imaging error after the clustering is reduced by about 2.33%. Comparisons with radar charts show that the proposed system can effectively image the point charge moving paths.

Fault diagnosis of wind turbine pitch system based on LSTM with multi-channel attention mechanism

Pitch system can adjust the blade’s pitch angle, control the aerodynamic torque and capture the wind turbine’s aerodynamic power. It is critical guarantee for the turbine’s safety and stability. However, the turbine’s operating environment is harsh and the long-term component abrasion may cause the system failure. Only by the monitoring signals’ instantaneous value, it is difficult to discovery the system fault. Therefore, this paper fuses multiple time sequence signals and uses the multi-channel attention mechanism to improve the long short-time memory networks (MCA-LSTM) for pitch system fault diagnosis. Firstly, the data of SCADA is preprocessed to construct the time sequence ratio vectors and input into MCA-LSTM. Through the attention mechanism, the time sequences ratio vectors are integrated from different channels in MCA-LSTM. Then the MCA-LSTM establishes the logical relationship between the signals and the fault features. Finally, root-mean-square error and the residual fluctuation threshold are used to identify the abnormal states. The experiments utilized the SCADA data of wind farms. The results show that MCA-LSTM can detect the pitch bearing anomalies and hub faults, which are about ten hours earlier than the wind farm fault recording. So, MCA-LSTM can effectively predict pitch system faults and provides an important basis for the WT fault diagnosis.

Development of an HTS-SQUID Based Nondestructive Evaluation System for Boiler Tubes On-Site Inspection in Thermal Power Plant

In ultra-supercritical (USC) thermal power plants, Super304H (KA-SUS304J1HTB, ASME code: 2328) is widely used in the superheater and reheater tubes in boilers because of its excellent creep rupture strength under high-temperature and -pressure conditions. Thus, a degradation diagnosis of these tubes after long-term operation should be conducted using a quick and easy inspection. In this study, in order to discover the degradation of the boiler tubes for early diagnose before they rupture, a high-temperature superconductor (HTS)–superconducting quantum interference device (SQUID) with high sensitivity was used. A SQUID-based nondestructive evaluation system for the on-site inspection of boiler tubes in an USC thermal power plant was developed. Our system comprises a movable eddy current testing (ECT) probe and HTS-SQUID gradiometer. Five LED markers were set on a manually operated ECT probe for position detection using a stereo camera. Customized software was used for data acquisition through a lock-in amplifier and data visualization on a PC with a connected smart glass. Thus, the system can be used for quick inspection by time-sequence signals and detailed scanning through a two-dimensional contour map projected on the smart glass. Therefore, this study successfully introduced an HTS-SQUID-based nondestructive evaluation system for the on-site inspection of boiler tubes.

Development of the control system for electron cyclotron resonance heating on J-TEXT tokamak

A 105 GHz/500 kW/1 s electron cyclotron resonance heating (ECRH) system has been developed on J-TEXT tokamak since 2017. The core component of the ECRH system is a gyrotron manufactured by Gyrotron Complexes Ltd. (GYCOM Ltd.), which generates microwaves of a certain frequency and power. To guarantee safe and stable operation, it is necessary to design a specialized control system. The control system is expected to perform time sequence trigger, protection, signal monitoring, communication, and data acquisition. The hardware is built with real-time processors and data acquisition modules from National Instruments. The control program is realized by LabVIEW. Test results indicate that the control system can commit stable and safe operation of the gyrotron, which guarantees the integrated commissioning tests of the whole ECRH system and ECRH related physics experiments. Under the operation of this control system, the gyrotron can generate microwaves as expected, and the ECRH system is well protected when a fault takes place.

A Dense ResNet Model with RGB Input Mapping for Cross-Domain Mechanical Fault Diagnosis

In actual engineering applications, the mechanical machine is exposed to uncertain conditions such as noise interference and various loads. The commonly used fault diagnosis models suffer degradation in the prediction accuracy in such complex industrial environments where the available label samples are insufficient and the conditions are varied. To combat this challenge, a cross-domain mechanical fault diagnosis method based on the deep-learning networks is proposed. It utilizes small samples, i.e., 10% of the total, and operates on the time-series signal collected from the mechanical equipment. It provides a classification accuracy of more than 97% on the dataset from Case Western Reserve University (CWRU) under variable conditions and 97.56% with the noise interference of 0 dB. The one-dimensional vibration signal is first converted into an image through RGB mapping. Then, the derived RGB image is capable of the time dependent and spatial properties of the time sequence signal and can be directly used as the input of the deep-learning networks. The deep-learning networks model, i.e., the ResNet, is adopted for the fault feature extraction and additional dense connections are added among the residual blocks to supplement the insufficient labeled samples within the networks. Then, an RGB-DResNet is constructed, capable of retaining the robust features for the classification of the mechanical faults in different working conditions. Finally, through retraining the model by use of transfer learning, the derived RGB-TDResNet model gives a fine adaption to the feature distribution with a small amount of target domain information. The performance of the proposed fault diagnosis model was validated on the dataset from CWRU. The results show that it provides a high identification accuracy and strong robustness in variable operating conditions as well as the noise environment. It is a rather promising approach for dealing with the cross-domain tasks of mechanical fault diagnosis.

Smartwatch In-Air Signature Time Sequence Three-Dimensional Static Restoration Classification Based on Multiple Convolutional Neural Networks

In-air signatures are promising applications that have been investigated extensively in the past decades; an in-air signature involves gathering datasets through portable devices, such as smartwatches. During the signing process, individuals wear smartwatches on their wrists and sign their names in the air. The dataset we used in this study collected in-air signatures from 22 participants, resulting in a total of 440 smartwatch in-air signature signals. The dynamic time warping (DTW) algorithm was applied to verify the usability of the dataset. This paper analyzes and compares the performances of multiple convolutional neural networks (CNN) and the transformer using median-sized smartwatch in-air signatures. For the four CNN models, the in-air digital signature data were first transformed into visible three-dimensional static signatures. For the transformer, the nine-dimensional in-air signature signals were concatenated and downsampled to the desired length and then fed into the transformer for time sequence signal multi-classification. The performance of each model on the smartwatch in-air signature dataset was thoroughly tested with respect to 10 optimizers and different learning rates. The best testing performance score in our experiment was 99.8514% with ResNet by using the Adagrad optimizer under a 1×10−4 learning rate.

Myoelectric Pattern Recognition Using Gramian Angular Field and Convolutional Neural Networks for Muscle–Computer Interface

In the field of the muscle-computer interface, the most challenging task is extracting patterns from complex surface electromyography (sEMG) signals to improve the performance of myoelectric pattern recognition. To address this problem, a two-stage architecture, consisting of Gramian angular field (GAF)-based 2D representation and convolutional neural network (CNN)-based classification (GAF-CNN), is proposed. To explore discriminant channel features from sEMG signals, sEMG-GAF transformation is proposed for time sequence signal representation and feature modeling, in which the instantaneous values of multichannel sEMG signals are encoded in image form. A deep CNN model is introduced to extract high-level semantic features lying in image-form-based time sequence signals concerning instantaneous values for image classification. An insight analysis explains the rationale behind the advantages of the proposed method. Extensive experiments are conducted on benchmark publicly available sEMG datasets, i.e., NinaPro and CagpMyo, whose experimental results validate that the proposed GAF-CNN method is comparable to the state-of-the-art methods, as reported by previous work incorporating CNN models.

Hardness-and-Type Recognition of Different Objects Based on a Novel Porous Graphene Flexible Tactile Sensor Array.

Accurately recognizing the hardness and type of different objects by tactile sensors is of great significance in human-machine interaction. In this paper, a novel porous graphene flexible tactile sensor array with great performance is designed and fabricated, and it is mounted on a two-finger mechanical actuator. This is used to detect various tactile sequence features from different objects by slightly squeezing them by 2 mm. A Residual Network (ResNet) model, with excellent adaptivity and feature extraction ability, is constructed to realize the recognition of 4 hardness categories and 12 object types, based on the tactile time sequence signals collected by the novel sensor array; the average accuracies of hardness and type recognition are 100% and 99.7%, respectively. To further verify the classification ability of the ResNet model for the tactile feature information detected by the sensor array, the Multilayer Perceptron (MLP), LeNet, Multi-Channel Deep Convolutional Neural Network (MCDCNN), and ENCODER models are built based on the same dataset used for the ResNet model. The average recognition accuracies of the 4hardness categories, based on those four models, are 93.6%, 98.3%, 93.3%, and 98.1%. Meanwhile, the average recognition accuracies of the 12 object types, based on the four models, are 94.7%, 98.9%, 85.0%, and 96.4%. All of the results demonstrate that the novel porous graphene tactile sensor array has excellent perceptual performance and the ResNet model can very effectively and precisely complete the hardness and type recognition of objects for the flexible tactile sensor array.

Extraction of statistical features for type-2 fuzzy NILM with IoT enabled control in a smart home

Identification and monitoring of residential appliances are important facets for home energy management and essential for proper functioning of the connected devices. In this paper, residential appliances are identified using some pre-estimated statistical features which are derived from time sequence signal of electrical data. The extracted statistical features are then used to build a type-2 fuzzy system for proper identification of the household appliances. Internet-of-things (IoT) based load control utilizing power consumption data is also built for monitoring and control of the residential appliances remotely. The proposed non-intrusive load monitoring and identification technique has better identification accuracy than similar state-of-the-art techniques as observed from the comparison analysis. Furthermore, a laboratory scale prototype system is developed to validate the usefulness of the proposed load identification and monitoring technique.

Modeling and tagging of time sequence signals in the milling process based on an improved hidden semi-Markov model

Vibration signals are widely used in the field of tool wear, tool residual life prediction and health monitoring of mechanical equipment. However, the current data-driven research methods mostly rely on high-value and high-density labeled data to establish relevant models and algorithms. Therefore, it is of great significance to solve the problem of automatic tagging of data, realize automatic signal interception, and enhance the value density of manufacturing process data.The Hidden semi-Markov model (HSMM) can describe the real spatial statistical characteristics of random models through observable data. As HSMM does not need the real labels of the signal, it can reduce tagging work to improve the marking efficiency. In this paper, an improved HSMM was proposed to model and tag the spindle vibration signals in the milling process. First, the Mel frequency cepstral coefficients (MFCCs) were extracted as observation sequences from the collected spindle vibration signals, and the dimension of the original features was reduced by linear discriminant analysis (LDA). Subsequently, a signal automatic tagging model based on HSMM was developed, in which the state duration can be explicitly modeled. Finally, the evaluation of the proposed methodology was carried out in the laboratory and real industry machining. The experimental results confirmed the effectiveness and robustness of the proposed model.

Bearing Fault Diagnosis Based on Multiple Transformation Domain Fusion and Improved Residual Dense Networks

Automatic feature extraction is one of the most advantageous merits of deep neural network (DNN), meanwhile, it is an important part for intelligent bearing fault diagnosis. However, most of fault diagnosis methods based on DNN usually excavate the complex relations from original time sequence signals which only present the fault information in time domain. Convolutional Neural Network (CNN) has demonstrated powerful feature learning capabilities in bearing fault diagnosis and the deeper the diagnosis model is, the better the recognition performance is, which resulted in some problems. In order to enrich the fault information from different views and enhance the discrimination for features learned from diagnosis network, this paper proposed a bearing fault diagnosis method based on multi-domain information fusion and improved residual dense network. The original signal and its transformed signals composed the multi-channel input, which contained more comprehensive information and will benefit the deep learning. Then it designed a residual dense network and introduced the convolution attention mechanism which can discriminate the importance of features further improve the feature extraction capability and efficiency of diagnosis network. Finally, it achieved the fault classification, analyzed the effects of key parameters and compared with other diagnosis to verify the effectiveness by lots of experimental results.

Next-generation reservoir computing based on memristor array

As a kind of brain-inspired computing, reservoir computing (RC) has great potential applications in time sequence signal processing and chaotic dynamics system prediction due to its simple structure and few training parameters. Since in the RC randomly initialized network weights are used, it requires abundant data and calculation time for warm-up and parameter optimization. Recent research results show that an RC with linear activation nodes, combined with a feature vector, is mathematically equivalent to a nonlinear vector autoregression (NVAR) machine, which is named next-generation reservoir computing (NGRC). Although the NGRC can effectively alleviate the problems which traditional RC has, it still needs vast computing resources for multiplication operations. In the present work, a hardware implementation method of using computing-in memory paradigm for NGRC is proposed for the first time. We use memristor array to perform the matrix vector multiplication involved in the nonlinear vector autoregressive process for the improvement of the energy efficiency. The Lorenz63 time series prediction task is performed by simulation experiments with the memristor array, demonstrating the feasibility and robustness of this method, and the influence of the weight precision of the memristor devices on the prediction results is discussed. These results provide a promising way of implementing the hardware NGRC.

REACTOR NOISE ANALYSIS FOR A GRAPHITE-MODERATED AND -REFLECTED CORE IN KUCA

In graphite-reflected thermal reactors, even a detector placed far from fuel region may detect a certain degree of the correlation amplitude. This is because mean free path of neutrons in graphite is longer than that in water or polyethylene. The objective of this study is experimentally to confirm a high flexibility of neutron detector placement in graphite reflector for reactor noise analysis. The present reactor noise analysis was carried out in a graphite-moderated and -reflected thermal core in Kyoto University Critical Assembly (KUCA). BF3 proportional neutron counters (1” dia.) were placed in graphite reflector region, where the counters were separated by about 35cm and 30cm -thick graphite from the core, respectively. At a critical state and subcritical states, time-sequence signal data from these counters were acquired and analyzed by a fast Fourier transform (FFT) analyzer, to obtain power spectral density in frequency domain. The auto-power spectral density obtained from the counters far from the core contained a significant degree of correlated component. A least-squares fit of a familiar formula to the auto-power spectral density data was made to determine the prompt-neutron decay constant. The decay constant was 63.3±14.5 [1/s] in critical state. The decay constant determined from the cross-power spectral density and coherence function data between the two counters also had a consistent value. It is confirmed that reactor noise analysis is possible using a detector placed at about 35cm far from the core, as we expected.

Intelligent Rolling Bearing Fault Diagnosis via Vision ConvNet

Feature extraction from a time sequence signal without manual information is an important part for bearing intelligent diagnosis. With the merits of signal information and feature structure information excavation, Deep ConvNet is widely used in bearing fault diagnosis and analysis under complex working conditions. However, due to the complexity of the bearing operating environment in the actual operation process, the sensitive features show different scale distribution characteristics. Meanwhile, it is known that the convolution kernel of ConvNet is usually small, which mainly focuses on the small-scale details of state distribution characteristics while ignores the identification of the overall trend of characteristic distribution. Considering that the size of convolution kernel can sense information hidden in different scales, this paper designed a one-dimensional vision ConvNet (VCN), where the architecture is composed of multilayer small kernel network and single-layer large kernel network side by side. The multi-kernel structure improves the ability of network to detect fault characteristic frequency band. By analyzing the artificially generated data and experimental data, the setting method of large convolution kernel and stride is discussed. Compared with the traditional CNN, wide first-layer kernels (WDCNN) and multiscale kernel-based ResCNN (MK-ResCNN), this network improves the recognition accuracy with a better stable training process for rolling bearing fault classification.

Towards CSI-based diversity activity recognition via LSTM-CNN encoder-decoder neural network

Human activity recognition using WiFi signals is widespread for smart-environment sensing domain in recent years. Existing researches use learning-based methods to obtain several features of activity data and then recognize human activities. As we know, propagation characteristics of WiFi signals are different for individuals under different place conditions even in the same environment. In this paper, we focus on how to weaken the accuracy differences among individuals on activity recognition and improve the robustness in one indoor environment. Based on this, we design a novel deep learning model called LCED which consists of one LSTM-based Encoder, features image presentation, and one CNN-based Decoder to weaken the accuracy differences among individuals on activity recognition. We first use a low-pass filter to remove high-frequency noise data in time-sequence signal data and design variance-based window method to determine the start and the end of time-sequence signal data corresponding to an activity. After that, we utilize the proposed LCED model to learn informative features space of activity data and improve the accuracy of sixteen activities. Experimental results show that the average accuracy of sixteen activities is high 95% and the accuracy differences among individuals on activity recognition averagely decreases by 3%.

A Noniterative Convoluted Harmonic Beamforming Technique in Time Modulated Arrays

In this study, the synthesis of the time-modulated array (TMA) has been analyzed in terms of both the enhancement of the intended harmonic component and the steering capability of this harmonic. In order to accomplish the radiation pattern at desired harmonic frequencies, including high-order harmonics, the dual-phase variable pulse amplitude (DPVPA) time sequence signal is proposed. On the other hand, the periodic convolution operator, which based on a specific signal with only certain components in the frequency domain, demonstrates superior ability in completely suppressing the remaining unwanted harmonics. Moreover, without using any optimization technique, the pulse amplitude parameters are obtained through only the methodological analysis; therefore, the proposed method provides convenience in terms of reconfiguration. The harmonic-enhancement capability and harmonic steering flexibility of the proposed approach are shown through representative examples.

PSO–LSTM for short term forecast of heterogeneous time series electricity price signals

Electricity price forecasting plays an important role in the power system network, in order to promote the decision-making process for power generation and consumption. Long term forecasting is not viable as there is an uncertainty in the forecast due to increasing the integration of renewable sources with the existing grids. Since the behavior of the electricity price time sequence signal is highly non-linear and seasonal, deep neural network is the best model for learning the non-linear behavior within the data and for the purpose of forecasting. Hence this paper proposes an enhanced particle swarm optimization based long short-term memory (LSTM) neural network model, which is used to forecast the closing price of the Indian Energy Exchange. Particle swarm optimization technique is used to optimize the LSTM network input weights, which in turn minimize the forecast error with reduced architecture. This paper discusses the statistical analysis for input data selection and investigates the performance analysis for the optimal selection of layers with hidden units’ combination. Finally, the analysis deploys the best-suited configuration for forecasting the market clearing price with the least mean absolute percentage error.

EEG-based emotion classification based on Bidirectional Long Short-Term Memory Network

Abstract Emotion recognition can be achieved by speech recognition, the judgment of limb movements, analysis of Electrooculogram (EOG) or capturing of facial expressions. However, those types of emotion recognition methods cannot detect human emotion well, because humankind can use fake body movement and words to hide real emotions. In this paper, we proposed an EEG-based emotion classification method based on Bidirectional Long Short-Term Memory Network (BiLSTM). Electroencephalogram (EEG) signal can detect human emotion correctly because human represent their real emotions in their mind and cannot hide emotions there. Meanwhile, EEG is a time sequence signal which needs a model which can deal with this type of data. Therefore, we chose Long Short-term Memory Network to process the EEG signal. In particular, we used an improvement version of LSTM model BiLSTM to manage the signals. BiLSTM can processes input data from front to back and back to front. Meanwhile, BiLSTM can store important information and forget unnecessary information; therefore, this process increases the accuracy of the model. Our method classifies four discrete classifications (happy, sad, fear, and neutral) for emotion classification, which achieves competitive performance compared with other conventional emotion classification methods. The final experimental results show that we can achieve an accuracy of 84.21% for four emotional states classification by using our method.

Health Condition Evaluation of Servo Turret Based on Time Sequence Signal Feature Map

The health condition of servo turret depends machining quality of CNC lathe. Performance tests of servo turret are taken to evaluate the health condition and overall status during its service life. This paper selects the core signals during one tool change-cutting cycle from multichannel physical signals and describes the performance of tool change-cutting cycle by defining a series of feature matrixes. In order to clearly and visually express the health condition of different working stages and tool position, a multidimensional features map is designed according to the combination of different time sequence feature matrixes. The above method is applied on a servo turret test platform which is equipped to acquire abundant signals. The result shows that the feature map clearly defines the health condition with one easily interpreted picture instead of unarranged signals. The proposed method provides a new idea of feature selection and visual expression, which can be an instructive tool for health evaluation and fault isolation.