Time Sequence Data Research Articles

A bi-level mode decomposition framework for multi-step wind power forecasting using deep neural network

The proportion of wind energy in global energy structure is growing rapidly, promoting the development of wind power forecasting (WPF) technologies to solve the uncertainty and intermittence of wind power generation. However, the nonlinear and stochastic features of wind power time series restrain the accuracy of multi-step prediction performance. A multi-step WPF (MS-WPF) approach based on a time series bi-level empirical mode decomposition (BLEMD) method and BiLSTM neural network is proposed in this paper to improve the WPF accuracy of regional wind power generators. Since the uncertainty is always generated through coupled factors from both wind and weather-to-power conversion, the linearity feature is first introduced as an aspect apart from the frequency in the proposed approach to decompose the wind power time sequence data. The proposed BLEMD introduces Pearson product-moment correlation coefficient to evaluate the linearity of time series and a linearity-based decomposition algorithm is designed accordingly. To further enhance the precision and release computation burdens, a DL-based prediction strategy, including a BiLSTM network, a CNN-BiLSTM network, and a mean weight estimation method are implemented to predict the components separately. The proposed method only relies on local data, greatly reducing the data acquisition and computation cost. The precision of the proposed MS-WPF is verified by a 2.5 kW wind turbine with horizons from 5 s to 30 s, a 1.5 MW wind turbine with horizons from 10 min to 1 h, and a 51 MW wind farm with horizons from 1 h to 6 h. The comparative experimental results with other cutting-edge methods indicated that the proposed MS-WPF has superior prediction accuracy and stable performance for multi-step prediction.

RUL Prediction for Piezoelectric Vibration Sensors Based on Digital-Twin and LSTM Network

Piezoelectric vibration sensors (PVSs) are widely used in high-temperature environments, such as vibration measurements in aero-engines, because of their high accuracy, small size, and high temperature resistance. Accurate prediction of its RUL (Remaining Useful Life) is essential for applying and maintaining PVSs. Based on PVSs’ characteristics and main failure modes, this work combines the Digital-Twin (DT) and Long Short-Term Memory (LSTM) networks to predict the RUL of PVSs. In this framework, DT can provide rich data collection, analysis, and simulation capabilities, which have advantages in RUL prediction, and LSTM network has good results in predicting time sequence data. The proposed method exploits the advantages of those techniques in feature data collection, sample optimization, and RUL multiclassification. To verify the prediction of this method, a DT platform is established to conduct PVS degradation tests, which generates sample datasets, then the LSTM network is trained and validated. It has been proved that prediction accuracy is more than 99.7%, and training time is within 94 s. Based on this network, the RUL of PVSs is predicted using different test samples. The results show that the method performed well in prediction accuracy, sample data utilization, and compatibility.

Approach and Landing Energy Prediction Based on a Long Short-Term Memory Model

The statistical analysis of civil aircraft accidents reveals that the highest incidence of mishaps occurs during the approach and landing stages. Predominantly, these accidents are marked by abnormal energy states, leading to critical situations like stalling and heavy landings. Therefore, it is of great significance to accurately predict the aircraft energy state in the approach and landing stages to ensure a safe landing. In this study, a deep learning method based on time sequence data for the prediction of the aircraft approach and landing energy states is proposed. Firstly, by conducting an extensive overview of the existing literature, three characteristic parameters of altitude, velocity, and glide angle were selected as the indicators to characterize the energy state. Following this, a semi-physical simulation platform for a certain type of aircraft was developed. The approach and landing experiments were carried out with different throttle sizes and flap deflection under different wind speeds and wind directions. Then, a deep learning prediction model based on Long Short-Term Memory (LSTM) was established based on the experimental data to predict the energy state indicators during the approach and landing phases. Finally, the established LSTM model underwent rigorous training and testing under different strategies, and a comparative analysis was carried out. The results demonstrated that the proposed LSTM model exhibited high accuracy and a strong generalization ability in predicting energy states during the approach and landing phases. These results offer a theoretical basis for designing energy early warning systems and formulating the relevant flight control laws in the approach and landing stages.

Neural network method: withstanding noise for continuous-variable quantum key distribution with discrete modulation

Excess noise in continuous-variable quantum key distribution systems usually results in a loss of key rate, leading to fatal security breaches. This paper proposes a long short-term memory time-sequence neural network to predict the key rate of the system while counteracting the effects of excess noise. The proposed network model, which can be updated with historical data, predicts the key rate of the future moment for the input time-sequence data. To increase the key rate, we perform a postselection operation to combat excess noise. We demonstrate the asymptotic security of the protocol against collective attacks with the numerical simulations using the quadrature phase-shift keying protocol, where some parameters have been optimized to resist excess noise. It provides a potential solution for improving the security of quantum communication in practical applications.

Development of an Artificial Intelligence-Based Image Recognition System for Time-Sequence Analysis of Tracheal Intubation.

Total intubation time (TIT) is an objective indicator of tracheal intubation (TI) difficulties. However, large variations in TIT because of diverse initial and end targets make it difficult to compare studies. A video laryngoscope (VLS) can capture images during the TI process. By using artificial intelligence (AI) to detect airway structures, the start and end points can be freely selected, thus eliminating the inconsistencies. Further deconstructing the process and establishing time-sequence analysis may aid in gaining further understanding of the TI process. We developed a time-sequencing system for analyzing TI performed using a #3 Macintosh VLS. This system was established and validated on 30 easy TIs performed by specialists and validated using TI videos performed by a postgraduate-year (PGY) physician. Thirty easy intubation videos were selected from a cohort approved by our institutional review board (B-ER-107-088), and 6 targets were labeled: the lip, epiglottis, laryngopharynx, glottic opening, tube tip, and a black line on the endotracheal tube. We used 887 captured images to develop an AI model trained using You Only Look Once, Version 3 (YOLOv3). Seven cut points were selected for phase division. Seven experts selected the cut points. The expert cut points were used to validate the AI-identified cut points and time-sequence data. After the removal of the tube tip and laryngopharynx images, the durations between 5 identical cut points and sequentially identified the durations of 4 intubation phases, as well as TIT. The average and total losses approached 0 within 150 cycles of model training for target identification. The identification rate for all cut points was 92.4% (194 of 210), which increased to 99.4% (179 of 180) after the removal of the tube tip target. The 4 phase durations and TIT calculated by the AI model and those from the expert exhibited strong Pearson correlation (phase I, r = 0.914; phase II, r = 0.868; phase III, r = 0.964; and phase IV, r = 0.949; TIT, r = 0.99; all P < .001). Similar findings were obtained for the PGY's observations (r > 0.95; P < .01). YOLOv3 is a powerful tool for analyzing images recorded by VLS. By using AI to detect the airway structures, the start and end points can be freely selected, resolving the heterogeneity resulting from the inconsistencies in the TIT cut points across studies. Time-sequence analysis involving the deconstruction of VLS-recorded TI images into several phases should be conducted in further TI research.

The multi-strategy hybrid forecasting base on SSA-VMD-WST for complex system.

In view of the strong randomness and non-stationarity of complex system, this study suggests a hybrid multi-strategy prediction technique based on optimized hybrid denoising and deep learning. Firstly, the Sparrow search algorithm (SSA) is used to optimize Variational mode decomposition (VMD) which can decompose the original signal into several Intrinsic mode functions (IMF). Secondly, calculating the Pearson correlation coefficient (PCC) between each IMF component and the original signal, the subsequences with low correlation are eliminated, and the remaining subsequence are denoised by Wavelet soft threshold (WST) method to obtain effective signals. Thirdly, on the basis of the above data noise reduction and reconstruction, our proposal combines Convolutional neural network (CNN) and Bidirectional short-term memory (BiLSTM) model, which is used to analyze the evolution trend of real time sequence data. Finally, we applied the CNN-BiLSTM-SSA-VMD-WST to predict the real time sequence data together with the other methods in order to prove it's effectiveness. The results show that SNR and CC of the SSA-VMD-WST are the largest (the values are 20.2383 and 0.9342). The performance of the CNN-BiLSTM-SSA-VMD-WST are the best, MAE and RMSE are the smallest (which are 0.150 and 0.188), the goodness of fit R2 is the highest(its value is 0.9364). In contrast with other methods, CNN-BiLSTM-SSA-VMD-WST method is more suitable for denoising and prediction of real time series data than the traditional and singular deep learning methods. The proposed method may provide a reliable way for related prediction in various industries.

Time-tired compaction: An elastic compaction scheme for LSM-tree based time-series database

Time-series DBMSs based on the LSM-tree have been widely applied in numerous scenarios ranging from daily life to industrial production. Compared to the traditional key–value data, the time-series data workload has significant features of writing and querying in chronological order. While simultaneously, such features bring new challenges to efficient queries, especially data compaction. Namely, an effective time-series data compaction algorithm is crucial for efficient storage and query of massive time-series data. However, the current data compaction method based on traditional LSM-tree cannot solve the problems such as high delay and inaccurate range of time sequence data query. Therefore, we propose a novel compaction algorithm, Time-Tiered Compaction, to customize for time-series scenarios. Time-Tiered Compaction leverages the characteristics of time-series workloads to estimate query loads to select optimum SSTables for merging during every compaction process, reducing unnecessary expenses. Time-Tiered Compaction is implemented on Apache IoTDB to evaluate algorithm performance using TPC-xIoT. Results show that the presented Time-Tiered Compaction significantly reduces (30%) the latency of the range queries with only a slight increase (5%) in point queries, compared with traditional strategies.

Forecasting System For Increasing Crime At The Binjai City Police Station With The Application Of The Website-Based Exponential Smoothing Method

Binjai City is the closest city to the city of Medan, which is the heart of North Sumatra. As one of the closest cities to the city gate of Medan, there are many activities that require a lot of activities. In Binjai City there is a Binjai City Police Office which is located Jl. Sultan Hasanuddin No.1, Binjai City, North Sumatra. Based on crime data at the Binjai City Police Station, it shows that crime that occurs every day is constantly increasing. This is because several factors influence it, namely internal factors within oneself such as having a realistic mindset and so on while external factors such as economic level factors or low education levels that have an impact on the difficulty of finding jobs, uneven population density, very minimal salaries, urgent needs, supportive environmental situations, social inequality trigger envy and resentment, environmental associations that require costs and so on. When people are faced with such a situation, the thing that will come to their mind is how to get money to meet the needs of their families at all costs. So that it affects one's mentality and actions to commit criminal acts.Exponential smoothing method is a procedure that continuously improves forecasting with the average (smoothing = smoothing) past values of a time sequence data with. The Exponential smoothing method is a development of the Moving Averages method.

Diagnosis of patellofemoral osteoarthritis using enhanced sequential deep learning techniques

The surface electromyography (sEMG) signal is a complex interference pattern resulting from the electrical activity of contracting muscles, which directly correlates with muscle activity and exercise level. Patellofemoral osteoarthritis (PF OA) refers to the softening and destruction of the articular cartilage of the knee cap, which is important to diagnose in the early phase. The aim of this study is to develop a predictive model, called the Enhanced Sequencing Deep Learning (PESDL) Algorithm, to detect PF OA from the sEMG signal. The algorithm consists of four main modules: (i) Data Acquisition Module (DAM), (ii) Signal Preprocessing Module (SPM), (iii) Data Augmentation and Concatenation Module (DACM), and (iv) Lower Limb Classification (LLC). The sEMG signals from five core muscles of the lower limb were acquired during stepping stairs up and down, and machine learning was used to obtain the muscle activation data signals. The acquired data was preprocessed, augmented, concatenated, and shuffled. Three feedforward deep networks (RNN, LSTM, and GRU) were used to classify the lower limb sEMG time-sequence data, with the GRU network showing better performance than the other two. The model's performance was evaluated using seven performance measures and 10-fold cross-validation. The maximum values of feedforward deep networks (RNN), LSTM, and GRU for the five muscles (RF, BF, VM, ST, and FX) using seven performance measures were as follows: RNN (0.961, 0.154, 0.943, 0.244, 1.0, 1.0, 0.899), LSTM (0.967, 0.147, 0.945, 0.217, 0.997, 1.0, 0.927), and GRU (0.976, 0.111, 0.949, 0.212, 0.996, 0.998, 0.938), respectively, listed in the order of accuracy, loss, validation accuracy, validation loss, recall, precision, and F1-score. Comparisons with other state-of-the-art models using the same datasets demonstrated the effective performance of the predictive model. The results suggest that RNN models, particularly GRU and LSTM, can be effective for detecting PF OA, and the specific RNN architecture used can have a significant impact on performance.

Identification and Calibration Method of Deviation of Main Transformer Online Monitoring Date Groups

Dissolved gas in transformer oil (DGA) online monitoring data can reflect equipment insulation performance, which provides an important basis for transformer condition assessment. The accuracy of online monitoring data directly affects the correctness of transformer condition assessment, and the health condition of the online monitoring data device also affects the effectiveness and reasonableness of online monitoring data. With the accumulation of operation time, the DGA online monitoring device faces many problems, resulting in the overall deviation of this indicator data from the actual value, and the transformer online monitoring device cannot play its monitoring effect. To solve these problems, this paper firstly linearizes the data of the DGA online monitoring device by segmentation, extracts the characteristics of the line segment curve, constructs a segmented correlation mining model, and mines the correlation of different indicators of DGA. Then, according to the change of correlation strength and weakness between online monitoring data in time sequence, the data deviation is tracked. Finally, a multi‐indicator back propagation (BP) neural network algorithm optimized by the genetic simulated annealing algorithm is constructed to calibrate the deviated data. The analysis shows that the method can identify the population deviation of online monitoring data and calibrate the deviated data effectively. © 2023 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

Multi-modal deep learning for automated assembly of periapical radiographs

ObjectivesPeriapical radiographs are oftentimes taken in series to display all teeth present in the oral cavity. Our aim was to automatically assemble such a series of periapical radiographs into an anatomically correct status using a multi-modal deep learning model. Methods4,707 periapical images from 387 patients (on average, 12 images per patient) were used. Radiographs were labeled according to their field of view and the dataset split into a training, validation, and test set, stratified by patient. In addition to the radiograph the timestamp of image generation was extracted and abstracted as follows: A matrix, containing the normalized timestamps of all images of a patient was constructed, representing the order in which images were taken, providing temporal context information to the deep learning model. Using the image data together with the time sequence data a multi-modal deep learning model consisting of two residual convolutional neural networks (ResNet-152 for image data, ResNet-50 for time data) was trained. Additionally, two uni-modal models were trained on image data and time data, respectively. A custom scoring technique was used to measure model performance. ResultsMulti-modal deep learning outperformed both uni-modal image-based learning (p<0.001) and time-based learning (p<0.05). The multi-modal deep learning model predicted tooth labels with an F1-score, sensitivity and precision of 0.79, respectively, and an accuracy of 0.99. 37 out of 77 patient datasets were fully correctly assembled by multi-modal learning; in the remaining ones, usually only one image was incorrectly labeled. ConclusionsMulti-modal modeling allowed automated assembly of periapical radiographs and outperformed both uni-modal models. Dental machine learning models can benefit from additional data modalities. Clinical significanceLike humans, deep learning models may profit from multiple data sources for decision-making. We demonstrate how multi-modal learning can assist assembling periapical radiographs into an anatomically correct status. Multi-modal learning should be considered for more complex tasks, as clinically a wealth of data is usually available and could be leveraged.

Chaotic Time Series Prediction of Multi‐Dimensional Nonlinear System Based on Bidirectional LSTM Model

AbstractThe current work proposes a hybrid data‐driven model—Convolutional bidirectional long–short term memory (CNN‐BLSTM) for predicting chaotic behavior of three‐coupled Duffing oscillator nonlinear system, in which the CNN is for efficiently extracting the more robust and informative representations of chaotic sequences while the BLSTM is for holding the long‐term dependencies combining the past and future contexts. Different from traditional analytical and numerical approaches, the proposed prediction model features the benefit of focusing on the measured data solely without extensive professional domain knowledge. Additionally, three more recurrent neural network (RNN) models, including simple RNNs, stack LSTMs, and BLSTM, are built and comparisons of generalization performances to the CNN‐BLSTM are conducted. From the findings so far, the CNN‐BLSTM is able to learn the pattern of chaotic time sequence data with less training time and apply the acquired knowledge to the unseen dataset with lower errors. Moreover, the current work decently demonstrates that the proposed model outperforms other three models in terms of stability at different noise levels from two evaluation criteria. The CNN‐BLSTM provides useful guidance for the consideration of predicting multi‐dimensional nonlinear chaotic behavior.

Research on IGBT aging prediction method based on adaptive VMD decomposition and GRU-AT model

The insulated gate bipolar transistor (IGBT) is widely used in the power electronic system, but its aging state is difficult to predict in advance due to the complicated failure mechanism, which will affect the performance of the equipment, and even cause serious disaster. Therefore, we propose a combined prediction model based on adaptive VMD decomposition (AVMD) and attention-based gated recurrent unit (GRU-AT) to achieve accurate prediction of the aging state of IGBT. For the nonlinear characteristics of the IGBT failure characteristic parameters, AVMD with the key parameters optimized by improved sparrow search algorithm (ISSA) was used to disaggregate the character sequence to a series of finite wide subcomponents, which overcome the interference of the irregularity of aging time sequence data to the prediction accuracy. Secondly, the attention mechanism is introduced to capture the temporal feature relationship between the current moment output and the historical degraded data, further improving the generalization ability of the model. Finally, the GRU-AT network is used to model each wide modal sub-component independently, and final prediction results of IGBT aging parameters are obtained by superimposing the prediction results of each mode. The experimental results indicate that the AVMD-GRU-AT model designed in this article has superior prediction performance in both over-the-top single-step prediction and multi-step prediction of aging state of IGBT.

Development of the control system for electron cyclotron resonance heating on J-TEXT tokamak

A 105 GHz/500 kW/1 s electron cyclotron resonance heating (ECRH) system has been developed on J-TEXT tokamak since 2017. The core component of the ECRH system is a gyrotron manufactured by Gyrotron Complexes Ltd. (GYCOM Ltd.), which generates microwaves of a certain frequency and power. To guarantee safe and stable operation, it is necessary to design a specialized control system. The control system is expected to perform time sequence trigger, protection, signal monitoring, communication, and data acquisition. The hardware is built with real-time processors and data acquisition modules from National Instruments. The control program is realized by LabVIEW. Test results indicate that the control system can commit stable and safe operation of the gyrotron, which guarantees the integrated commissioning tests of the whole ECRH system and ECRH related physics experiments. Under the operation of this control system, the gyrotron can generate microwaves as expected, and the ECRH system is well protected when a fault takes place.

Pseudo-Label-Vector-Guided Parallel Attention Network for Remaining Useful Life Prediction

Prognostic health management (PHM) has become important in many industries as a critical technology to increase machine stability and operational efficiency. Recently, various methods using deep learning to estimate the remaining useful life (RUL) as a core task of PHM have been proposed. However, the existing attention methods do not explicitly capture the correlation between temporal and spatial time series, reducing the RUL prediction accuracy. This article proposes a novel RUL prediction algorithm using a spatiotemporal attention mechanism based on the pseudo-label vectors to solve this problem. The proposed attention network uses the pseudo-label vector learned in the intermediate prediction process as a query vector to focus on time sequence data related to the RUL. Therefore, compared with conventional attention models that extract correlations for all the sequences, the proposed model captures features directly related to RUL with less computational cost. Experiments have been performed on two widely used datasets, and the experimental results show that the proposed approach outperforms the state of the art for root-mean-square error, with averages 4.27 and 3039 in the NASA Commercial Modular Aero-Propulsion System Simulation dataset and the IEEE PHM 2012 Prognostic challenge dataset, respectively. In addition, the analysis in the experiment reveals that the proposed model has better interpretability than the existing models by obtaining the correlation between time-series data and the RUL through the attention score in terms of time and features.

A hybrid model for metro passengers flow prediction

In this paper, a novel ensemble learning model named EWT-EnsemLSTM-SSA, which assembles long short-term memory (LSTM), support vector regression (SVR), and sparrow search algorithm (SSA), is a proposed to deal with long term metro passenger flow volume prediction, which is an essential content of traffic flow prediction problems. Firstly, the empirical wavelet transform (EWT) method is introduced to decompose the original dataset into five wavelet time-sequence data for further prediction. Then, a cluster of LSTMs with diverse hidden layers and neuron counts are employed to explore and exploit the implicit information of the EWT-decomposed signals. Next, the output of LSTMs is aggregated into a nonlinear regression method SVR. Lastly, SSA is utilized to optimize the SVR automatically. The proposed EWT-EnsemLSTM-SSA model is applied in three case studies, using the data collected from the passengers’ amount in the Minneapolis, America metro, divided into one hour in one day. Experiment results, which compare the proposed EnsemLSTM-SSA model with five conventional time series forecasting models, show that the proposed model can achieve a better performance than the traditional prediction algorithms.

A brute force tuning of training length for concept drift

We present a brute-force approach to analyze the concept drift behind time sequence data. This approach, named SELECT, searches for the optimal length of training data to minimize error metrics. In other words, SELECT searches for the start point of a new concept from the input sequence. Unlike many related methods, SELECT does not require a pre-specified error threshold to detect drift. In addition, the visual analysis obtained from SELECT enables us to understand how significant a drift has occurred. We test SELECT on two real-world datasets, stock price and COVID-19 infection data. The experimental results show that SELECT can improve the model performance of both datasets. In addition, the visual analysis shows the points of significant drifts, e.g., Lehman’s collapse in stock price data and the spread of variants in COVID-19 data. These results show the effectiveness of the brute-force approach in analyzing concept drift.

Brute force concept drift detection

We present a brute-force approach to detect concept drift behind time sequence data. This approach, named Select-Starţ searches for start points of concept drift to minimize error. In other words, Select-Start searches for the start points of new concepts from the input sequence. Unlike many related works, Select-Start does not require a pre-specified error threshold to detect drift. This paper compares Select-Start with previous representative methods and clarifies its characteristics. The experimental results show that Select-Start is accurate for concept drift problems where the threshold changes slowly. However, existing methods are better at analyzing concept drift problems where the model behind data changes rapidly.

The Impact of a Single Discretionary Lane Change on Surrounding Traffic: An Analytic Investigation

As one of the most frequently occurred driving behaviors, the lane-changing’s impact on surrounding traffic is important. From the aspect of macro-traffic flows, its impact on traffic efficiency and safety has been investigated. However, as a non-essential lane-changing type, a single discretionary lane-changing’s impact on surrounding traffic is still difficult to be quantified by real-word traffic data. To solve this problem, this paper first proposes a trajectory data-based analytic method to calculate response time of the lane-changing vehicle and its following vehicles, then develops a time sequence data processing method based on feature points to analyze the change pattern of the variable minimum space, and then a lane-changing’s temporal and spatial on surrounding traffic can be captured by the decreasing-increasing pattern of minimum space. Besides, a high-precision trajectory data set, called Zen-traffic, is employed to demonstrate the effectiveness of the proposed methods and reach conclusions. Given the condition that the speed of a discretionary lane-changing vehicle is between 6 to 20 m/s, i.e., the traffic condition at which lane changes more likely occur, the results of the paper include (1) on average, 4 or 5 vehicles are usually impacted by a discretionary lane change; (2) the average impact time is 12 to 13 seconds and it has little relationship with the distance between the lane-changing vehicle and the following vehicle on the target/initial lane; (3) the lateral movement direction (i.e., changing to left or to right) of a lane change has a significant impact on the number of the vehicles affected.

A Remaining Useful Life Prediction Method for Lithium-ion Battery Based on Temporal Transformer Network

The remaining useful life prediction is significant for Lithium-ion batteries to ensure safety and reliability. Due to the advantages of handling time sequence data, recurrent neural network based methods have achieved impressive performance on RUL prediction. However, most of these methods develop the RUL prediction model without considering the operating time of the battery, which is an important factor on capacity degradation. Therefore, this paper proposed a Temporal Transformer Network (TTN) for RUL of Lithium-ion batteries. The proposed method combines the self-attention mechanism of the Transformer Network with Denoising Autoencoder to implement the noise of raw data. More importantly, the proposed method designs a temporal encoding layer to introduce the operating time to the input of RUL prediction model. The performance of the proposed method is evaluated on two frequently used battery datasets. The proposed method achieves the best result compared with other frequently used RUL prediction methods.