Processing Time Series Data Research Articles

Industrial Data-Driven Processing Framework Combining Process Knowledge for Improved Decision-Making—Part 2: Framework Application Considering Activity-Based Costing Concepts

Operating time series data collected and stored in historian must be managed to extract their full potential. Part 1 of this paper proposed a structured way (a sophisticated approach) to process industrial data; this first part explains in detail the data processing framework used as the basis for the costing analysis present in the second part of this series. The framework considers the analysis scope definition, data management steps, and operating regimes detection and identification. The added value of this proposed framework is demonstrated in Part 2 via the use of cost accounting for operational problem-solving (debottlenecking), i.e., its practicality is validated via its application alongside a cost analysis on the brownstock washing department of a kraft pulp mill. The traditional debottlenecking approach assumes a single operating condition considering that operating regimes allow for a much more sophisticated debottlenecking study of the washing department. With the use of operations-driven cost modeling (contingent on activity-based costing concepts) and processed time series data corresponding to steady-state operation, incremental profit can be assigned to each operating regime in order to identify the most cost-efficient one. The overall objective of this two-part series is to convert processed industrial steady-state data and cost information into knowledge that can be used to optimize the washing department of a chemical pulp mill. More specifically, different operating regimes are assessed, and the most suitable operating strategy is defined. The application of activity-based costing on a large amount of historically processed industrial data led to the improvement in the operation. The identified optimal way to operate (pulp throughput, pulp conductivity, defoamer and bleaching chemical quantity, etc.) led to a profit of CAD 49 M per year. Lastly, a contribution analysis of the regimes based on PCA highlighted how the process was operated when the preferred performances happened.

A rapid analysis framework for seismic response prediction and running safety assessment of train-bridge coupled systems

With high-speed railway lines increasing gradually, the running safety assessment (RSA) of train-bridge coupled (TBC) systems has become an indispensable part of railway seismic design. However, the modeling of TBC systems is difficult and computationally complex. Therefore, it is difficult to conduct real-time assessment and response analysis rapidly. With this in mind, this paper integrates a surrogate framework based on deep learning methods and can help rapid-diagnose the seismic response and RSA. Meanwhile, real-time information feedback is performed to determine the safe speed of the train. To this end, a convolutional neural network-long short-term memory (CNN-LSTM) model with excellent time-series data processing capability is newly used for the seismic response analysis of the TBC system. Furthermore, response data obtained are processed and input to an interpretable adaptive network-based fuzzy inference system (ANFIS). The ANFIS derives a series of fuzzy rules while conducting the RSA, and these rules can help non-specialists understand the assessment process. Finally, parameter inversion is performed by the ANFIS to determine a safe threshold of train speed. According to the tests, the proposed rapid analysis framework can be applied to seismic response analysis and RSA with high efficiency and accuracy in multiple engineering scenarios. The framework may be helpful for railway design and make it possible to realize real-time data analysis for TBC systems.

Redox-based ion-gating reservoir consisting of (104) oriented LiCoO2 film, assisted by physical masking

Reservoir computing (RC) is a machine learning framework suitable for processing time series data, and is a computationally inexpensive and fast learning model. A physical reservoir is a hardware implementation of RC using a physical system, which is expected to become the social infrastructure of a data society that needs to process vast amounts of information. Ion-gating reservoirs (IGR) are compact and suitable for integration with various physical reservoirs, but the prediction accuracy and operating speed of redox-IGRs using WO3 as the channel are not sufficient due to irreversible Li+ trapping in the WO3 matrix during operation. Here, in order to enhance the computation performance of redox-IGRs, we developed a redox-based IGR using a (104) oriented LiCoO2 thin film with high electronic and ionic conductivity as a trap-free channel material. The subject IGR utilizes resistance change that is due to a redox reaction (LiCoO2 ⟺ Li1−xCoO2 + xLi+ + xe−) with the insertion and desertion of Li+. The prediction error in the subject IGR was reduced by 72% and the operation speed was increased by 4 times compared to the previously reported WO3, which changes are due to the nonlinear and reversible electrical response of LiCoO2 and the high dimensionality enhanced by a newly developed physical masking technique. This study has demonstrated the possibility of developing high-performance IGRs by utilizing materials with stronger nonlinearity and by increasing output dimensionality.

A flow-rate-aware data-driven model of vanadium redox flow battery based on gated recurrent unit neural network

The vanadium redox flow battery (VRB) system involves complex multi-physical and multi-timescale interactions, where the electrolyte flow rate plays a pivotal role in both static and dynamic performance. Traditionally, fixed flow rates have been employed for operational convenience. However, in today’s highly dynamic energy market environment, adjusting flow rates based on operating conditions can provide significant advantages for improving VRB energy conversion efficiency and cost-effectiveness. Unfortunately, incorporating the electrolyte flow rate into conventional multi-physical models is overly complex for VRB management and control systems, as real-time operations demand low-computational and low-complexity models for onboard functionalities. This paper introduces a novel data-driven approach that integrates flow rates into VRB modeling, enhancing data processing capabilities and prediction accuracy of VRB behaviors. The proposed model adopts a gated recurrent unit (GRU) neural network as its fundamental framework, exhibiting exceptional proficiency in capturing VRB’s nonlinear voltage segments. The GRU network structure is carefully designed to optimize the predictive ability of the model, with flow rate considered as a crucial input parameter to account for its influence on VRB behavior. Model refinement involves analyzing well-designed simulation results obtained during VRB operations under various flow rates. Laboratory experiments were also designed and conducted, covering different conditions of currents and flow rates to validate the proposed data-driven modeling method. Comparative analyses were performed against several state-of-the-art algorithms, including equivalent circuit models and other data-driven models, demonstrating the superiority of the proposed GRU-based VRB model considering flow rates. Thanks to the GRU’s outstanding capability in processing time series data, the proposed model delivers impressively accurate terminal voltage predictions with a low error margin of no more than 0.023 V (1.3%) under wide operating ranges. These results indicate the efficacy and robustness of the proposed approach, highlighting the novelty and significance of accounting for flow rates in accurate VRB modeling for management and control system design.

Reservoir concatenation and the spectrum distribution of concatenated reservoir state matrices

Reservoir computing, one of the state-of-the-art machine learning architectures, processes time-series data generated by dynamical systems. Nevertheless, we have realized that reservoir computing with the conventional single-reservoir structure suffers from capacity saturation. This leads to performance stagnation in practice. Therefore, we propose an extended reservoir computing architecture called reservoir concatenation to further delay such stagnation. Not only do we provide training error analysis and test error comparison of reservoir concatenation, but we also propose a crucial measure, which is the trace associated with a reservoir state matrix, that explains the level of responsiveness to reservoir concatenation. Two reservoir dynamics are compared in detail, one by using the echo state network and the other by using a synchronization model called an explosive Kuramoto model. The distinct eigenvalue distributions of the reservoir state matrices from the two models are well reflected in the trace values that are shown to account for the different reservoir capacity behaviors, determining the different levels of responsiveness.

Enhancing Data Engineering Frameworks for Scalable Real-Time Marketing Solutions

This report discusses improvements to the data engineering frameworks for cost-efficient real-time marketing environment, with the applications in mobile advertising and smart city analytics. The proposed framework combines the processing of time-series data with map-reduce (M-R) systems in order to compensate for inefficiencies in the execution of temporal queries and the processing of real-time data streams. It enables effective behavioral targeting in mobile marketing and helps integrate various domains’ real-time data for smart city utilization. Specific examples that illustrate further refinements in the algorithms are showcased through case study and experimental evidence where gains in throughput, data reliability, and system effectiveness are quantified in real-time Marketing and Smart City applications.

Stock Price Prediction based on LSTM and XGBoost Combination Model

In recent years, many machine learning and deep learning algorithms have been applied to stock prediction, providing a reference basis for stock trading, and LSTM neural network and XGBoost algorithm are two typical representatives, each with advantages and disadvantages in prediction. In view of this, we propose a combination model based on LSTM and XGBoost, which combines the advantages of LSTM in processing time series data and the ability of XGBoost to evaluate the importance of features. The combination model first selects feature variables with high importance through XGBoost, performs data dimensionality reduction, and then uses LSTM to make predictions. In order to verify the feasibility of the combination model, we built XGBoost, LSTM and LSTM-XGBoost models, and carried out experiments on three data sets of China Eastern Airlines, China Merchants Bank and Kweichow Moutai respectively. Finally, we concluded that the proposed LSTM-XGBoost model has good feasibility and universality in stock price prediction by comparing the accuracy of the predicted images and their performance in RMSE, RMAE, and MAPE indicators.

Spatio-Temporal Alignment and Track-To-Velocity Module for Tropical Cyclone Forecast

The prediction of a tropical cyclone’s trajectory is crucial for ensuring marine safety and promoting economic growth. Previous approaches to this task have been broadly categorized as either numerical or statistical methods, with the former being computationally expensive. Among the latter, multilayer perceptron (MLP)-based methods have been found to be simple but lacking in time series capabilities, while recurrent neural network (RNN)-based methods excel at processing time series data but do not integrate external information. Recent works have attempted to enhance prediction performance by simultaneously utilizing both time series and meteorological field data through feature fusion. However, these approaches have relatively simplistic methods for data fusion and do not fully explore the correlations between different modalities. To address these limitations, we propose a systematic solution called TC-TrajGRU for predicting tropical cyclone tracks. Our approach improves upon existing methods in two main ways. Firstly, we introduce a Spatial Alignment Feature Fusion (SAFF) module to address feature misalignment issues in different dimensions. Secondly, our Track-to-Velocity (T2V) module leverages time series differences to integrate external information. Our experiments demonstrate that our approach yields highly accurate predictions comparable to the official optimal forecast for a 12 h period.

Characterization of Information-Transmitting Materials Produced in Ionic Liquid-based Neuromorphic Electrochemical Devices for Physical Reservoir Computing.

Device implementation of reservoir computing, which is expected to enable high-performance data processing in simple neural networks at a low computational cost, is an important technology to accelerate the use of artificial intelligence in the real-world edge computing domain. Here, we propose an ionic liquid-based physical reservoir device (IL-PRD), in which copper cations dissolved in an IL induce diverse electrochemical current responses. The origin of the electrochemical current from the IL-PRD was investigated spectroscopically in detail. After operating the device under various operating conditions, X-ray photoelectron spectroscopy of the IL-PRD revealed that electrochemical reactions involving Cu, Cu2O, Cu(OH)2, CuSx, and H2O occur at the Pt electrode/IL interface. These products are considered information transmission materials in IL-PRD similar to neurotransmitters in biological neurons. By introducing the Faradaic current components due to the electrochemical reactions of these materials into the output signal of IL-PRD, we succeeded in improving the time-series data processing performance of the nonlinear autoregressive moving average task. In addition, the information processing efficiency in machine learning to classify electrocardiogram signal waveforms was successfully improved by using the output current from IL-PRD. Optimizing the electrochemical reaction products of IL-PRD is expected to advance data processing technology in society.

A Transformer-Based Channel Estimation Method for OTFS Systems.

Orthogonal time frequency space (OTFS) is a novel modulation scheme that enables reliable communication in high-mobility environments. In this paper, we propose a Transformer-based channel estimation method for OTFS systems. Initially, the threshold method is utilized to obtain preliminary channel estimation results. To further enhance the channel estimation, we leverage the inherent temporal correlation between channels, and a new method of channel response prediction is performed. To enhance the accuracy of the preliminary results, we utilize a specialized Transformer neural network designed for processing time series data for refinement. The simulation results demonstrate that our proposed scheme outperforms the threshold method and other deep learning (DL) methods in terms of normalized mean squared error and bit error rate. Additionally, the temporal complexity and spatial complexity of different DL models are compared. The results indicate that our proposed algorithm achieves superior accuracy while maintaining an acceptable computational complexity.

Analysis of spectrum sensing using deep learning algorithms: CNNs and RNNs

Spectrum sensing is a critical component of cognitive radio systems, enabling the detection and utilization of underutilized frequency bands. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs), in particular, have showed promise in recent years for enhancing the precision and effectiveness of spectrum sensing. This paper presents an overview of spectrum sensing using CNNs and RNNs and their performance in cognitive radio systems. Furthermore, the paper delves into the spectrum sensing performance of RNNs, particularly for processing time-series data. RNNs are capable of capturing temporal dependencies in sequential data, which is essential for spectrum sensing tasks where signals evolve over time. Further, the parakeets such as probability of detection (Pd), Probability of false alarm (PFA), Bite Error rate (BER) and Power spectral density (PSD) are analysed for spectrum sensing algorithms. RNNs and CNNs, two examples of deep learning methods (DLM), performed better than more traditional approaches.

Forecasting vault cash with an extreme value long short-term memory network

Effective cash management is key in banking operations and has implications for cost control, customer service, and risk management. As transactions become more diverse, manual forecasting methods have become inadequate for accurate vault cash forecasting, which involves extensive data analysis. To address this challenge, the banking industry has adopted FinTech tools based on big data and deep learning for various client services. These methods are generally accurate but perform poorly in cases with extreme events, for which data are scarce. In this study, we propose a time series prediction model with long short-term memory and an attention mechanism that effectively predicts the presence of extreme values. We applied extreme value theory to define the extreme value loss for extreme situations and use a sliding window to process time series data. The enhanced extreme value loss function in our model yields improved prediction accuracy for time series data.We evaluated the proposed model against previous methods in evaluation experiments on data from three branches of a commercial bank in Taiwan, where the vault cash data of each exhibited extreme values. The proposed model was highly accurate: it had a lower mean absolute percentage error and higher trend accuracy than competing methods on a majority of time series, and it was also more accurate in predicting extreme values in time series data.

RNNtcs: A test case selection method for Recurrent Neural Networks

Recurrent Neural Network (RNN) is a typical feedback neural network, which is particularly effective in processing time-series data tasks such as image description, text generation and classification, etc. However, its safety and quality issues have aroused strong concern in academia and industry. One of the most commonly used methods to ensure RNN-driven software quality is testing and optimizing RNN. Its core idea is to introduce human efforts to label test cases so that many labeled test cases can be used to trigger faults in RNNs. But this brings a huge overhead. Therefore, it is of great importance to study the test case selection method for RNN. Current research on test case selection for RNN has achieved some results from the stateful view of RNN, but there are still some problems, such as a low bug detection rate, poor inclusiveness, and lack of diversity.In this paper, according to the structural characteristics of RNN, we propose a method of RNN test case selection, RNNtcs, which combines clustering and uncertainty, in order to select test cases that can touch RNN bugs as much as possible, thus reducing the cost of labeling. In order to evaluate RNNtcs, we conduct experiments on popular image datasets, text datasets and RNN models. The experimental results show that RNNtcs is superior to the state-of-the-art approach DeepState in bug detection effectiveness, bug detection inclusiveness and bug detection diversity. The test cases selected by RNNtcs can effectively improve the robustness of the RNN model.

Leveraging Google Earth Engine (GEE) for determining land use and land cover changes around Tasik Chini Malaysia.

This study explores the capability of Google Earth Engine in determining land use and land cover changes around Tasik Chini Malaysia which is one of the tourism areas severely affected by landscape changes. Two Landsat satellite composite data spanning ten years of difference and Machine Learning Approach algorithm namely Random Forest (RF) and Support Vector machine (SVM) are used to create landuse land cover changes (LULCC) map of the area. GEE is capable of processing time series data as well as performing temporal aggregation. In our case median metrics is used in creating many different alternatives of image composites for creating the LULC map with ease but accurate result. It is an excellent alternative for geospatial and big data analysts for both advance and novice users in processing long term EO dataset especially in dealing with many imageries. The best classification accuracy with the highest Overall Accuracy (OA) is by using Random Forest classifier with 81.58% for the year 2010 and 83.59% for 2020. The Kappa coefficient of both years are 0.75 and 0.78. It is found using this technique, Tasik Chini lost about 6600 hectares of forest area and an increase of bareland and develop area especially around the Tasik Chini lake due to the reported increase of mining activities for the past few years.

Robust optimisation of electricity‐heating integrated energy system based on data‐driven PV interval estimation

AbstractShort‐term interval estimation can effectively and precisely quantify the uncertainties of renewable energy, accurately represent the range of fluctuations of uncertain variables in robust optimisation of electricity‐heating integrated energy system (EHIES) and it is getting crucial for reliable and flexible operation of renewable dominated new energy systems. The authors present a multivariate data‐driven short‐term PV power interval prediction model that consists of multiple layers, including one‐dimensional convolutional layer, ultra‐lightweight subspace attention mechanism (ULSAM), bidirectional long and short‐term memory (BiLSTM), quantile regression (QR) and kernel density estimation (KDE). The one‐dimensional convolutional layer and ULSAM can extract sequential features and highlight key information from the data; the BiLSTM processes time series data in both directions and conveys historical information; the QR and KDE models generate interval prediction with a given confidence level. Based on the proposed interval estimation, a refined PV uncertainty set can be established and adopted by robust optimal scheduling of EHIES utilising min‐max‐min algorithm. The simulation results have demonstrated the estimation accuracy and adaptability to various weather scenarios.

Radar Timing Range–Doppler Spectral Target Detection Based on Attention ConvLSTM in Traffic Scenes

With the development of autonomous driving and the emergence of various intelligent traffic scenarios, object detection technology based on deep learning is more and more widely applied to real traffic scenarios. Commonly used detection devices include LiDAR and cameras. Since the implementation of traffic scene target detection technology requires mass production, the advantages of millimeter-wave radar have emerged, such as low cost and no interference from the external environment. The performance of LiDAR and cameras is greatly reduced due to their sensitivity to light, which affects target detection at night and in bad weather. However, millimeter-wave radar can overcome the influence of these harsh environments and has a great auxiliary effect on safe driving on the road. In this work, we propose a deep-learning-based object detection method considering the radar range–Doppler spectrum in traffic scenarios. The algorithm uses YOLOv8 as the basic architecture, makes full use of the time series characteristics of range–Doppler spectrum data in traffic scenarios, introduces the ConvLSTM network, and exerts the ability to process time series data. In order to improve the model’s ability to detect small objects, an efficient and lightweight Efficient Channel Attention (ECA) module is introduced. Through extensive experiments, our model shows better performance on two publicly available radar datasets, CARRADA and RADDet, compared to other state-of-the-art methods. Compared with other mainstream methods that can only achieve 30–60% mAP performance when the IOU is 0.3, our model can achieve 74.51% and 75.62% on the RADDet and CARRADA datasets, respectively, and has better robustness and generalization ability.

Short‐term wind power prediction based on the combination of firefly optimization and LSTM

AbstractWith the development of social resources, people's consumption of energy is huge, so renewable energy, such as wind energy, has been widely concerned and developed. Although there has been sufficient development of wind power generation, its output has some problems such as uncertainty, which leads to insufficient utilization of wind energy resources and uneven power output quality level, which brings great challenges to the grid connection. To solve this problem, a short‐term wind power prediction model combining firefly algorithm and long term memory network is proposed. The main motivation of the research is to improve the accuracy of wind power prediction and thus improve the utilization of wind energy resources. Compared with the existing methods, the innovation of FA‐LSTM model lies in the integration of the two algorithms, making full use of the advantages of FA in global search optimization and LSTM in time series data processing, and improving the accuracy and stability of prediction. During the experiment, we used different wind farm data to train and test the model. The results show that the FA‐LSTM model can improve the optimal fitness by more than 50% compared with other algorithms, and the iterative prediction error is smaller. Standard root mean square error (RMSE) and mean absolute error (MAE) were used to evaluate the model. The accuracy of RMSE and MAE reached over 97% and 98% respectively. When the test data is highly volatile, the data accuracy of FA‐LSTM model reaches 92% and 94%, and the FA‐LSTM model drops to the stable value faster. FA‐LSTM model has the best fitting degree with the true value curve, and the fitting degree reaches more than 90%. Comparing the actual power and predicted power of different units, the actual power of Unit 1 is 34.875, and the predicted power obtained by FA‐LSTM model is 34.935, with an error of only 0.06. The key finding of this study is that the prediction model combining FA and LSTM has high accuracy and stability in wind power prediction, and can effectively deal with the uncertainty and volatility of wind energy resource utilization. FA‐LSTM model provides an effective solution for wind power prediction, which is helpful to improve the utilization rate of wind energy resources.

Digital Architecture for Monitoring and Operational Analytics of Multi-Vector Microgrids Utilizing Cloud Computing, Advanced Virtualization Techniques, and Data Analytics Methods

Microgrids are considered a viable solution for achieving net-zero targets and increasing renewable energy integration. However, there is a lack of conceptual work focusing on practical data analytics deployment schemes and case-specific insights. This paper presents a scalable and flexible physical and digital architecture for extracting data-driven insights from microgrids, with a real-world microgrid utilized as a test-bed. The proposed architecture includes edge monitoring and intelligence, data-processing mechanisms, and edge–cloud communication. Cloud-hosted data analytics have been developed in AWS, considering market arrangements between the microgrid and the utility. The analysis involves time-series data processing, followed by the exploration of statistical relationships utilizing cloud-hosted tools. Insights from one year of operation highlight the potential for significant operational cost reduction through the real-time optimization and control of microgrid assets. By addressing the real-world applicability, end-to-end architectures, and extraction of case-specific insights, this work contributes to advancing microgrid design, operation, and adoption.

Water Quality Prediction Based on Machine Learning and Comprehensive Weighting Methods.

In the context of escalating global environmental concerns, the importance of preserving water resources and upholding ecological equilibrium has become increasingly apparent. As a result, the monitoring and prediction of water quality have emerged as vital tasks in achieving these objectives. However, ensuring the accuracy and dependability of water quality prediction has proven to be a challenging endeavor. To address this issue, this study proposes a comprehensive weight-based approach that combines entropy weighting with the Pearson correlation coefficient to select crucial features in water quality prediction. This approach effectively considers both feature correlation and information content, avoiding excessive reliance on a single criterion for feature selection. Through the utilization of this comprehensive approach, a comprehensive evaluation of the contribution and importance of the features was achieved, thereby minimizing subjective bias and uncertainty. By striking a balance among various factors, features with stronger correlation and greater information content can be selected, leading to improved accuracy and robustness in the feature-selection process. Furthermore, this study explored several machine learning models for water quality prediction, including Support Vector Machines (SVMs), Multilayer Perceptron (MLP), Random Forest (RF), XGBoost, and Long Short-Term Memory (LSTM). SVM exhibited commendable performance in predicting Dissolved Oxygen (DO), showcasing excellent generalization capabilities and high prediction accuracy. MLP demonstrated its strength in nonlinear modeling and performed well in predicting multiple water quality parameters. Conversely, the RF and XGBoost models exhibited relatively inferior performance in water quality prediction. In contrast, the LSTM model, a recurrent neural network specialized in processing time series data, demonstrated exceptional abilities in water quality prediction. It effectively captured the dynamic patterns present in time series data, offering stable and accurate predictions for various water quality parameters.

Graph Neural Rough Differential Equations for Traffic Forecasting

Traffic forecasting is one of the most popular spatio-temporal tasks in the field of machine learning. A prevalent approach in the field is to combine graph convolutional networks and recurrent neural networks for the spatio-temporal processing. There has been fierce competition and many novel methods have been proposed. In this article, we present the method of spatio-temporal graph neural rough differential equation (STG-NRDE). Neural rough differential equations (NRDEs) are a breakthrough concept for processing time-series data. Their main concept is to use the log-signature transform to convert a time-series sample into a relatively shorter series of feature vectors. We extend the concept and design two NRDEs: one for the temporal processing and the other for the spatial processing. After that, we combine them into a single framework. We conduct experiments with 6 benchmark datasets and 27 baselines. STG-NRDE shows the best accuracy in all cases, outperforming all those 27 baselines by non-trivial margins.