One-dimensional Time Series Research Articles

Comparing End-to-End Machine Learning Methods for Spectra Classification

In scientific research, spectroscopy and diffraction experimental techniques are widely used and produce huge amounts of spectral data. Learning patterns from spectra is critical during these experiments. This provides immediate feedback on the actual status of the experiment (e.g., time-resolved status of the sample), which helps guide the experiment. The two major spectral changes what we aim to capture are either the change in intensity distribution (e.g., drop or appearance) of peaks at certain locations, or the shift of those on the spectrum. This study aims to develop deep learning (DL) classification frameworks for one-dimensional (1D) spectral time series. In this work, we deal with the spectra classification problem from two different perspectives, one is a general two-dimensional (2D) space segmentation problem, and the other is a common 1D time series classification problem. We focused on the two proposed classification models under these two settings, the namely the end-to-end binned Fully Connected Neural Network (FCNN) with the automatically capturing weighting factors model and the convolutional SCT attention model. Under the setting of 1D time series classification, several other end-to-end structures based on FCNN, Convolutional Neural Network (CNN), ResNets, Long Short-Term Memory (LSTM), and Transformer were explored. Finally, we evaluated and compared the performance of these classification models based on the High Energy Density (HED) spectra dataset from multiple perspectives, and further performed the feature importance analysis to explore their interpretability. The results show that all the applied models can achieve 100% classification confidence, but the models applied under the 1D time series classification setting are superior. Among them, Transformer-based methods consume the least training time (0.449 s). Our proposed convolutional Spatial-Channel-Temporal (SCT) attention model uses 1.269 s, but its self-attention mechanism performed across spatial, channel, and temporal dimensions can suppress indistinguishable features better than others, and selectively focus on obvious features with high separability.

GW-DC: A Deep Clustering Model Leveraging Two-Dimensional Image Transformation and Enhancement

Traditional time-series clustering methods usually perform poorly on high-dimensional data. However, image clustering using deep learning methods can complete image annotation and searches in large image databases well. Therefore, this study aimed to propose a deep clustering model named GW_DC to convert one-dimensional time-series into two-dimensional images and improve cluster performance for algorithm users. The proposed GW_DC consisted of three processing stages: the image conversion stage, image enhancement stage, and image clustering stage. In the image conversion stage, the time series were converted into four kinds of two-dimensional images by different algorithms, including grayscale images, recurrence plot images, Markov transition field images, and Gramian Angular Difference Field images; this last one was considered to be the best by comparison. In the image enhancement stage, the signal components of two-dimensional images were extracted and processed by wavelet transform to denoise and enhance texture features. Meanwhile, a deep clustering network, combining convolutional neural networks with K-Means, was designed for well-learning characteristics and clustering according to the aforementioned enhanced images. Finally, six UCR datasets were adopted to assess the performance of models. The results showed that the proposed GW_DC model provided better results.

Application of the NARX neural network for predicting a one-dimensional time series

Time series data analysis and forecasting tool for studying the data on the use of network traffic is very important to provide acceptable and good quality network services, including network monitoring, resource management, and threat detection. More and more, the behavior of network traffic is described by the theory of deterministic chaos. The traffic of a modern network has a complex structure, an uneven rate of packet arrival for service by network devices. Predicting network traffic is still an important task, as forecast data provide the necessary information to solve the problem of managing network flows. Numerous studies of actually measured data confirm that they are nonstationary and their structure is multicomponent. This paper presents modeling using Nonlinear Autoregression Exogenous (NARX) algorithm for predicting network traffic datasets. NARX is one of the models that can be used to demonstrate non-linear systems, especially in modeling time series datasets. In other words, they called the categories of dynamic feedback networks covering several layers of the network. An artificial neural network (ANN) was developed, trained and tested using the LM learning algorithm (Levenberg-Macwardt). The initial data for the prediction is the actual measured network traffic of the packet rate. As a result of the study of the initial data, the best value of the smallest mean-square error MSE (Mean Squared Error) was obtained with the epoch value equal to 18. As for the regression R, its output ANN values in relation to the target for training, validation and testing were 0.97743. 0.9638 and 0.94907, respectively, with an overall regression value of 0.97134, which ensures that all datasets match exactly. Experimental results (MSE, R) have proven the method's ability to accurately estimate and predict network traffic

Simulation of future dissolved oxygen distribution in pond culture based on sliding window-temporal convolutional network and trend surface analysis

Dissolved oxygen (DO) is a key ecological factor to measure the quality of water in the aquaculture. As the pond water body is affected by the breeding environment, the spatial distribution of DO shows a certain law in the entire pond. Therefore, to simulate the distribution of DO in aquaculture waters and grasp the temporal and spatial variation of DO is the key to achieving precise regulation of DO. For this purpose, this paper proposed a method for simulating the temporal and spatial distribution of DO in pond culture based on a sliding window-temporal convolutional network together with trend surface analysis (SW-TCN-TSA). This paper first utilized SW to construct DO data sets with different prediction durations, and then used the improved TCN model to realize one-dimensional time series prediction for DO at single monitoring point. Based on the prediction results of DO, a TSA method was performed on the predicted values of DO at the extreme moments of all discrete monitoring points, so as to realize the simulation of the temporal and spatial distribution of DO in the pond. Experimental results show that the SW-TCN model has better prediction performance for one-dimensional time series prediction of DO. Compared with traditional deep networks, such as CNN, GRU, LSTM, CNN-GRU and CNN-LSTM, the values of evaluation indicators (MSE, MAE and RMSE) have been greatly improved. In the process of trend surface fitting, all fitting R2 of DO at different water depths are higher than 0.9, indicating that the TSA can accurately reflect the temporal and spatial distribution of DO. This method can provide a basis for the prediction and early warning of DO in the three-dimensional space of the pond and has high practicability in aquaculture.

Spatial components dependence for bidimensional time-constant AR(1) model with alpha -stable noise and triangular coefficients matrix

In this paper, we examine the bidimensional time-constant autoregressive model of order 1 with alpha -stable noise. We focus on the case of the triangular coefficients matrix for which one of the spatial components of the model simplifies to the one-dimensional autoregressive time series. We study the asymptotic behaviour of the cross-codifference and the cross-covariation applied to describe the dependence in time between the spatial components of the model. As a result, we formulate the theorem about the asymptotic relation between both measures, which is consistent with the result that is correct for the case of the non-triangular coefficients matrix.

Semi-supervised graph convolutional network and its application in intelligent fault diagnosis of rotating machinery

Aiming at the difficulty of mechanical fault diagnosis with small samples, an intelligent fault diagnosis method for rotating machinery is proposed based on semi-supervised graph convolutional network (SSGCN). SSGCN has a good application in analyzing the naturally formed graph data, but it has not been researched for the complex mechanical vibration data. Besides, SSGCN is only applicable to graph data, but the collected vibration signals are one-dimensional time series. To well reflect the local geometry property between all vibration samples, we construct all vibration samples into an undirected and weighted k-nearest neighbor graph. The detailed parameter analysis is also carried out for SSGCN. Experimental results indicate that our proposed method can adaptively extract the available fault features from the raw vibration signals. Even if the label ratio is only 0.05 for each gear or bearing condition, our proposed method can still obtain an average accuracy of more than 98%.

스펙트로그램 등고선 신경망의 타당성 검증 - 변압기의 돌입전류에 적용

In this study, in order to effectively analyze the state of the devices using one-dimensional time-series data generated from industrial devices, a new learning method was designed based on Short-Time Fourier Transform (STFT), and its validity was verified. The proposed learning method is a method of extracting contour lines with a preset threshold before applying a 2D spectrogram image to a deep neural network. As such, the contour transformation of the spectrogram image of the frequency distribution is to extract the characteristics of the frequency, so it will be effective in improving the performance of learning. In order to verify the validity of the proposed contour neural network, Learning was conducted as a pre-study, using the time series values of the chaotic system with nonlinear high-dimensional characteristics as a data set, and it was confirmed that the learning rate rapidly increased at a specific contour line.

A CNN-based FBG demodulation method adopting the GAF-assisted ascending dimension of complicated signal

We introduce the convolutional neural network (CNN) to extract the effective information of some complex signals in the fiber sensing, and verify the feasibility in the specific process of fiber Bragg grating (FBG) demodulation. In the experiments, in order to eliminate the cross interference in the energized coil, classical double grating configuration is adopted to distinguish magnetic field and temperature. At the same time, the introduction of thermal induced chirp (TIC) phenomenon produces more wavelength components to achieve more precise demodulation. The signal received by oscilloscope presents complex waveform, and the traditional analytical solution cannot demodulate it precisely, while the deep learning capabilities of neural networks can be leveraged to solve this problem. The one-dimensional (1D) time-series signal is transformed into a two-dimensional (2D) image by using the Gramian angle field (GAF) method. On this basis, the data are augmented according to the characteristics of signal noise. Five classical neural network structures are used to process the data. The root mean square errors (RMSEs) of the demodulation results can achieve 0.3 °C and 17.1 Gs respectively, which are relatively high precision in terms of intensity demodulation. This method can be extended to demodulate 1D frequency domain signals and wavelength domain signals, rendering a demodulation scheme to process some complex signals in optical fiber sensing effectively.

Ordpy: A Python package for data analysis with permutation entropy and ordinal network methods.

Since Bandt and Pompe's seminal work, permutation entropy has been used in several applications and is now an essential tool for time series analysis. Beyond becoming a popular and successful technique, permutation entropy inspired a framework for mapping time series into symbolic sequences that triggered the development of many other tools, including an approach for creating networks from time series known as ordinal networks. Despite increasing popularity, the computational development of these methods is fragmented, and there were still no efforts focusing on creating a unified software package. Here, we present ordpy (http://github.com/arthurpessa/ordpy), a simple and open-source Python module that implements permutation entropy and several of the principal methods related to Bandt and Pompe's framework to analyze time series and two-dimensional data. In particular, ordpy implements permutation entropy, Tsallis and Rényi permutation entropies, complexity-entropy plane, complexity-entropy curves, missing ordinal patterns, ordinal networks, and missing ordinal transitions for one-dimensional (time series) and two-dimensional (images) data as well as their multiscale generalizations. We review some theoretical aspects of these tools and illustrate the use of ordpy by replicating several literature results.

Classification analysis of analytical and probabilistic methods for predicting the state of complex systems

The paper considers a class of analytical forecasting methods, a class of probabilistic methods, and methods of artificial neural networks for solving the problem of predicting the state of complex systems on board an aircraft, such as: mathematical simulation, operator method, one-dimensional time series forecasting method, potential function method, zone method, and generalized point method.

Comparison of Forecasting Models for Real-Time Monitoring of Water Quality Parameters Based on Hybrid Deep Learning Neural Networks

Accurate real-time water quality prediction is of great significance for local environmental managers to deal with upcoming events and emergencies to develop best management practices. In this study, the performances in real-time water quality forecasting based on different deep learning (DL) models with different input data pre-processing methods were compared. There were three popular DL models concerned, including the convolutional neural network (CNN), long short-term memory neural network (LSTM), and hybrid CNN–LSTM. Two types of input data were applied, including the original one-dimensional time series and the two-dimensional grey image based on the complete ensemble empirical mode decomposition algorithm with adaptive noise (CEEMDAN) decomposition. Each type of input data was used in each DL model to forecast the real-time monitoring water quality parameters of dissolved oxygen (DO) and total nitrogen (TN). The results showed that (1) the performances of CNN–LSTM were superior to the standalone model CNN and LSTM; (2) the models used CEEMDAN-based input data performed much better than the models used the original input data, while the improvements for non-periodic parameter TN were much greater than that for periodic parameter DO; and (3) the model accuracies gradually decreased with the increase of prediction steps, while the original input data decayed faster than the CEEMDAN-based input data and the non-periodic parameter TN decayed faster than the periodic parameter DO. Overall, the input data preprocessed by the CEEMDAN method could effectively improve the forecasting performances of deep learning models, and this improvement was especially significant for non-periodic parameters of TN.

Sports Injury Rehabilitation Intervention Algorithm Based on Visual Analysis Technology

Sports injuries of high-level athletes restrict the improvement of sports performance. Under this premise, an efficient and accurate sports injury assessment method is needed to detect potential sports injuries and conduct injury prevention training. Therefore, this paper proposes a novel sports injury prediction algorithm based on visual analysis technology. The proposed algorithm first takes the time-frequency of sensed data as the convolutional neural network (CNN) input. The one-dimensional time series collected by the sensor is converted into two-dimensional images using the Gram angle domain algorithm. The one-dimensional sensed data provides a new perspective and provides a basis for better use of convolutional neural networks and computer vision technology. Second, combining the residual network’s structure and advantages and hole convolution and multihole convolution kernel residual module is proposed. It improves the model’s ability to extract features at different scales while effectively controlling the parameter scale. Based on these modules, a single-sensor-based athlete action recognition algorithm is proposed. Several comparative experiments have been conducted on a public data set containing only acceleration sensors to verify the proposed algorithm’s effectiveness.

Rotating Machinery Remaining Useful Life Prediction Scheme Using Deep-Learning-Based Health Indicator and a New RVM

Remaining useful life (RUL) prediction plays a significant role in developing the condition-based maintenance and improving the reliability and safety of machines. This paper proposes a remaining useful life prediction scheme combining deep-learning-based health indicator and a new relevance vector machine. First, both one-dimensional time-series information and two-dimensional time-frequency maps are input into a hybrid deep-learning structure network consisting of convolutional neural network (CNN) and long short-term memory network (LSTM) to construct health indicator (HI). Then, the prediction results and confidence interval are calculated by a new RVM enhanced by a polynomial regression model. The proposed method is verified by the public PRONOSTIA bearing datasets. Experimental results demonstrate the effectiveness of the proposed method in improving the prediction accuracy and analyzing the prediction uncertainty.

A Deep Neural Network Model for Speaker Identification

Speaker identification is a classification task which aims to identify a subject from a given time-series sequential data. Since the speech signal is a continuous one-dimensional time series, most of the current research methods are based on convolutional neural network (CNN) or recurrent neural network (RNN). Indeed, these methods perform well in many tasks, but there is no attempt to combine these two network models to study the speaker identification task. Due to the spectrogram that a speech signal contains, the spatial features of voiceprint (which corresponds to the voice spectrum) and CNN are effective for spatial feature extraction (which corresponds to modeling spectral correlations in acoustic features). At the same time, the speech signal is in a time series, and deep RNN can better represent long utterances than shallow networks. Considering the advantage of gated recurrent unit (GRU) (compared with traditional RNN) in the segmentation of sequence data, we decide to use stacked GRU layers in our model for frame-level feature extraction. In this paper, we propose a deep neural network (DNN) model based on a two-dimensional convolutional neural network (2-D CNN) and gated recurrent unit (GRU) for speaker identification. In the network model design, the convolutional layer is used for voiceprint feature extraction and reduces dimensionality in both the time and frequency domains, allowing for faster GRU layer computation. In addition, the stacked GRU recurrent network layers can learn a speaker’s acoustic features. During this research, we tried to use various neural network structures, including 2-D CNN, deep RNN, and deep LSTM. The above network models were evaluated on the Aishell-1 speech dataset. The experimental results showed that our proposed DNN model, which we call deep GRU, achieved a high recognition accuracy of 98.96%. At the same time, the results also demonstrate the effectiveness of the proposed deep GRU network model versus other models for speaker identification. Through further optimization, this method could be applied to other research similar to the study of speaker identification.

Complexity of hydrologic basins: A chaotic dynamics perspective

The objective of this study is to examine the dynamic complexity of hydrologic basins using techniques of phase space reconstruction. These non-parametric, model-free techniques which have their basis in the theory of chaotic dynamical systems are advantageous in resurrecting the features of multivariable dynamics solely from one-dimensional timeseries. Dynamic complexity is characterized by two measures directly interpretable as number of active degrees of freedom (dimensionality) and strength of nonlinearity. The analysis is conducted on daily streamflow observations of 408 basins from the Model Parameter Estimation Project (MOPEX) dataset spanning a wide range of climatic, topographic and land surface properties. The results show that the two complexity measures vary considerably across basins; however, their distributions are positively skewed, indicating that most basins exhibit low dimensionality and moderate nonlinearity. By examining how dynamic complexity relate to a set of 15 basin properties, it was found that dynamics dimensionality is primarily related to basin size; precisely larger basins exhibit lower dimensional dynamics. On the other hand, strength of nonlinearity is found to be regulated by the extent of vegetation in land cover. Specifically, basins with limited vegetation cover tend to have more linear dynamics. Furthermore, some observations regarding the presence of chaotic dynamics and the relationship between dynamic complexity and accuracy of prediction are presented. The results of this study have implications and potential relevance to catchment similarity and classification frameworks as well as model selection and extrapolation of model parameters to ungauged basins.

Time Series Prediction Method Based on E-CRBM

To solve the problems of delayed prediction results and large prediction errors in one-dimensional time series prediction, a time series prediction method based on Error-Continuous Restricted Boltzmann Machines (E-CRBM) is proposed in this paper. This method constructs a deep conversion prediction framework, which is composed of two E-CRBMs and a neural network (NN). Firstly, the E-CRBM models of the original input sequence and the target prediction sequence are trained, respectively, to extract the time features of the two sequences. Then the NN model is used to connect and transform the time features. Secondly, the feature sequence H1 is extracted from the original input sequence of test data through E-CRBM1, which is used as input of NN to obtain feature transformation sequence H2. Finally, the target prediction sequence is obtained by reverse reconstruction of feature transformation sequence H2 through E-CRBM2. The E-CRBM in this paper introduces the residual sequence of NN feature transformation in the hidden layer of CRBM, which increases the robustness of CRBM and improves the overall prediction accuracy. The classical time series data (sunspot time series) and the actual operation data of reciprocating compressor are selected in the experiment. Compared with the traditional time series prediction method, the results verify the effectiveness of the proposed method in single-step prediction and multi-step prediction.

Intelligent Fault Diagnosis of Aeroengine Sensors Using Improved Pattern Gradient Spectrum Entropy

Timely and effective fault diagnosis of sensors is crucial to enhance the working efficiency and reliability of the aeroengine. A new intelligent fault diagnosis scheme combining improved pattern gradient spectrum entropy (IPGSE) and convolutional neural network (CNN) is proposed in this paper, aiming at the problem of poor fault diagnosis effect and real-time performance when CNN directly processes one-dimensional time series signals of aeroengine. Firstly, raw fault signals are converted into spectral entropy images by introducing pattern gradient spectral entropy (PGSE), which is used as the input of CNN, because of the great advantage of CNN in processing images and the simple and rapid calculation of the modal gradient spectral entropy. The simulation results prove that IPGSE has more stable distinguishing characteristics. Then, we improved PGSE to use particle swarm optimization algorithm to adaptively optimize the influencing parameters (scale factor λ ), so that the obtained spectral entropy graph can better match the CNN. Finally, CNN mode is proposed to classify the spectral entropy diagram. The method is validated with datasets containing different fault types. The experimental results show that this method can be easily applied to the online automatic fault diagnosis of aeroengine control system sensors.

SpecView: Malware Spectrum Visualization Framework With Singular Spectrum Transformation

With the rapid development of automation tools including polymorphic and metamorphic engines, generic packers, and genetic programming, many variants of malware have emerged, which pose a significant threat to the Internet security. To effectively detect malware variants, researchers have developed visualization-based approaches that can visualize malware adaptations for in-depth malware analysis. However, most existing visualization approaches rely on the binary image of a malware sample, which fail to provide an effective texture feature representation and thus often result in low efficiency in coping with challenging malware samples. In this paper, we propose SpecView , a malware spectrum visualization framework with singular spectrum transformation. SpecView converts malware binary code into one-dimensional time series spectrum data, and leverages the singular spectrum transformation method to obtain the structural changes preserved in the time series spectrum data. Then, we utilize the particle swarm optimization algorithm to optimize the singular spectrum transformation performance in SpecView. We apply SpecView in the task of malware classification. Extensive experimental results show that SpecView is effective and efficient in malware classification on the Malimg, Malheur, Drebin, and PRAGuard Malgenome Class Encryption datasets, with classification accuracy exceeding 99%, and it can effectively identify malware variants that use evasive techniques such as packer and encryption obfuscation. The proposed method outperforms the state-of-the-art methods on all datasets and the classification accuracy reaches 100% for 5 malware families packed by the UPX packer on the Malimg dataset, as well as 9 malware families that use Class Encryption obfuscation techniques on the PRAGuard Malgenome Class Encryption datasets.

Многомерный алгоритм кумулятивных сумм для обнаружения изменений характеристик временных рядов

The article presents a cumulative sum algorithm intended to detect a sudden step-like change in the probabilistic characteristics of a monitored time series when such a change (“disorder”) is associated with a simultaneous change in both the location characteristics and the dispersion characteristics of the corresponding distribution functions. In the general case of a multidimensional time series, the disorder is associated with a jump in the values of the mathematical expectation vector (the vector of means) and covariance matrix entries. To solve this problem, it is proposed to use a preliminary linear transformation of the time series values, as a result of which the covariance matrix is transformed to the unity form before disordering and to the diagonal form after disordering. The change in the vector of means is analyzed, and the main relations describing the considered detection algorithm are derived. It is noted that by using the above-mentioned linear transformation it is possible to simplify the obtaining of the reference data necessary for synthesizing the monitoring algorithm with the predetermined properties. As an example, a particular case of a one-dimensional time series and a disorder in the form of a simultaneous change in the mean and variance is considered. For this case, reference data obtained by applying the simulation method are given, using which it is possible to find the monitoring algorithm triggering threshold and estimate the average delay time of detecting the specified disorder from the given interval between false alarms. This study is a logical continuation and further development of the approach to construction of multidimensional algorithms for detecting disorders [1].

Frame Deletion Detection Based on Optical Flow Orientation Variation

Frame deletion detection is an arrestive field for video forensics in recent years. This paper presents a frame deletion detection algorithm based on optical flow orientation variation. Optical flow field has been proved to be effective for the detection of the frame deletion operation. However, current optical-flow based methods have relatively poor detection performance for real-world videos. Our research aims to develop a new forensic method to detect frame deletion for real-world videos. The proposed algorithm firstly develops an analytical model for flow orientation variation between adjacent optical flow fields, and exhibits a novel forensic clue for frame deletion detection, under an ideal but reasonable assumption. Accordingly, an effective descriptor, namely pseudo flow orientation variation (PFOV), is created to approximate the flow orientation variation, as a discriminative description feature to capture the frame deletion trace. The Frobenius norm and a smooth function are also introduced to quantify each descriptor to form an one-dimensional time series, and further statistics technique is adopted to detect the anomalous points in the time series caused by frame deletion. In addition, to improve the descriptor performance, the Robust Principal Component Analysis (RPCA) is applied to extract moving objects from video sequences and compute the proposed descriptor on them. The tests are made on 324 real-world videos, and experiment results demonstrate that the true positive rate can reach 90.12%, meanwhile the false alarm rate is 7.71%, which indicates the superiority of the proposed method in the forensic algorithms for real-world video.