Dynamic Time Series Research Articles

Interpretable extreme wind speed prediction with concept bottleneck models

Concept-bottleneck models (CBMs) are a new paradigm to construct interpretable classifiers. The CBM architecture can be regarded as a neural network with a single hidden layer whose neurons are replaced by binary classifiers. Each of these binary classifiers implements a concept, that is, it attempts to answer a meaningful yes/no question: the presence or absence of a concept in the observation presented to the network. The output layer is usually implemented as a linear stage that combines the concepts into final decisions. This paper presents an application of the CBM paradigm to a problem of Extreme Wind Speed prediction, such that the forecasting properties of the system can be interpreted in terms of different concepts considered for this problem. A significant contribution of this paper is the proposal of an automated concept generation method, with controlled sparsity, in which the concepts are automatically extracted from the branches of decision trees fitted with informative features that capture the dynamics of the wind speed time series. The final forecasting system is able to predict extreme wind speed values in wind farms with good accuracy and interpretable results, as experiments over real wind speed data from a wind farm in Spain show.

Large sampling intervals for learning and predicting chaotic systems with reservoir computing

Reservoir computing (RC) is an efficient artificial neural network for model-free prediction and analysis of dynamical systems time series. As a data-based method, the capacity of RC is strongly affected by the time sampling interval of training data. In this paper, taking Lorenz system as an example, we explore the influence of this sampling interval on the performance of RC in predicting chaotic sequences. When the sampling interval increases, the prediction capacity of RC is first enhanced then weakened, presenting a bell-shaped curve. By slightly revising the calculation method of the output matrix, the prediction performance of RC with small sampling interval can be improved. Furthermore, RC can learn and reproduce the state of chaotic system with a large time interval, which is almost five times larger than that of the classic fourth-order Runge–Kutta method. Our results show the capacity of RC in the applications where the time sampling intervals are constrained and laid the foundation for building a fast algorithm with larger time iteration steps.

A Photovoltaic Prediction Model with Integrated Attention Mechanism

Solar energy has become a promising renewable energy source, offering significant opportunities for photovoltaic (PV) systems. Accurate and reliable PV generation forecasts are crucial for efficient grid integration and optimized system planning. However, the complexity of environmental factors, including seasonal and daily patterns, as well as social behaviors and user habits, presents significant challenges. Traditional prediction models often struggle with capturing the complex nonlinear dynamics in multivariate time series, leading to low prediction accuracy. To address this issue, this paper proposes a new PV power prediction method that considers factors such as light, air pressure, wind direction, and social behavior, assigning different weights to them to accurately extract nonlinear feature relationships. The framework integrates long short-term memory (LSTM) and gated recurrent units (GRU) to capture local time features, while bidirectional LSTM (BiLSTM) and an attention mechanism extract global spatiotemporal relationships, effectively capturing key features related to historical output. This improves the accuracy of multi-step predictions. To verify the feasibility of the method for multivariate time series, we conducted experiments using PV power prediction as a scenario and compared the results with LSTM, CNN, BiLSTM, CNN-LSTM and GRU models. The experimental results show that the proposed method outperforms these models, with a mean absolute error (MAE) of 12.133, root mean square error (RMSE) of 14.234, mean absolute percentage error (MAPE) of 2.1%, and a coefficient of determination (R2) of 0.895. These results indicate the effectiveness and potential of the method in PV prediction tasks.

BRIDGING DATA GAPS: A FEDERATED LEARNING APPROACH TO HEAT EMISSION PREDICTION IN LASER POWDER BED FUSION

Abstract Deep learning has impacted defect prediction in Additive Manufacturing (AM), which is important to ensure process stability and part quality. However, its success depends on extensive training, requiring large, homogenous datasets–remaining a challenge for the AM industry, particularly for small- and medium-sized enterprises (SMEs). The unique and varied characteristics of AM parts, along with the limited resources of SMEs, hampers data collection, posing difficulties in the independent training of deep learning models. Addressing these concerns requires enabling knowledge sharing from the similarities in the physics of the AM process and defect formation mechanisms while carefully handling privacy concerns. Federated learning (FL) offers a solution to allow collaborative model training across multiple entities without sharing local data. This paper introduces an FL framework to predict section-wise heat emission during Laser Powder Bed Fusion (LPBF), a vital process signature. It incorporates a customized Long Short-Term Memory (LSTM) model for each client, capturing the dynamic AM process's time series properties without sharing sensitive information. Three advanced FL algorithms are integrated–FedAvg, FedProx, and FedAvgM–to aggregate model weights rather than raw datasets. Experiments demonstrate that the FL framework ensures convergence and maintains prediction performance comparable to individually trained models. This work demonstrates the potential of FL-enabled AM modeling and prediction where SMEs can improve their product quality without compromising data privacy.

Constrained hidden Markov models reveal further Hsp90 protein states

Time series of conformational dynamics in proteins are usually evaluated with hidden Markov models (HMMs). This approach works well if the number of states and their connectivity is known. However, for the multi-domain protein Hsp90, a standard HMM analysis with optimization of the BIC (Bayesian information criterion) cannot explain long-lived states well. Therefore, here we employ constrained HMMs, which neglect transitions between states by including assumptions. Gradually tuning a model with justified and focused changes allows us to improve its effectiveness and the score of the BIC. This became possible by analyzing time traces with several thousand observable transitions and, therefore, superb statistics. In this scheme, we also monitor the residences in the states reconstructed by the model, aiming to find exponentially distributed dwell times. We show how introducing new states can achieve these statistics but also point out limitations, e.g. for substantial similarity of two states connected to a common neighbor. One of the states displays the lowest free energy and could be the idle open ‘waiting state’, in which Hsp90 waits for the binding of nucleotides, cochaperones, or clients.

Research into Prediction Method for Pressure Pulsations in a Centrifugal Pump Based on Variational Mode Decomposition-Particle Swarm Optimization and Hybrid Deep Learning Models.

Centrifugal pump pressure pulsation contains various signals in different frequency domains, which interact and superimpose on each other, resulting in characteristics such as intermittency, non-stationarity, and complexity. Computational Fluid Dynamics (CFD) and traditional time series models are unable to handle nonlinear and non-smooth problems, resulting in low accuracy in the prediction of pressure fluctuations. Therefore, this study proposes a new method for predicting pressure fluctuations. The pressure pulsation signals at the inlet of the centrifugal pump are processed using Variational Mode Decomposition-Particle Swarm Optimization (VMD-PSO), and the signal is predicted by Convolutional Neural Networks-Long Short-Term Memory (CNN-LSTM) model. The results indicate that the proposed prediction model combining VMD-PSO with four neural networks outperforms the single neural network prediction model in terms of prediction accuracy. Relatively high accuracy is achieved by the VMD-PSO-CNN-LSTM model for multiple forward prediction steps, particularly for a forward prediction step of 1 (Pre = 1), with a root mean square error of 0.03145 and an average absolute percentage error of 1.007%. This study provides a scientific basis for the intelligent operation of centrifugal pumps.

A Novel ANN-ARMA Scheme Enhanced by Metaheuristic Algorithms for Dynamical Systems and Time Series Modeling and Identification

A Novel ANN-ARMA Scheme Enhanced by Metaheuristic Algorithms for Dynamical Systems and Time Series Modeling and Identification

Predictive analysis of stochastic stock pattern utilizing fractional order dynamics and heteroscedastic with a radial neural network framework

Modelling of high-dimension chaotic, noisy, and non-stationary time series of complex fractal dynamics is a big challenge. In this research work, a novel approach of Leverage Convolution LC ARFIMA– GARCH model is presented for sequential learning of irregular, fractal dynamic patterns of stochastic time series dynamics. Different classical methods including the auto-regressive approach indecently are unable to capture fragile patterns and consequently convert some weak signals into random errors by creating white noise patterns. The Convolution Leverage provides an additional degree of freedom for critical noisy points and asymmetrical distribution dynamics to regulate the frequency of imbalance and skewed observation. The designed transformation in the ARFIMA model preserves the loss of information by shifting the skewed data distribution toward a normal pattern. The pattern of population parameters in the proposed leverage paradigm provides out of box approach to track and extract additional information on population variance patterns in the form of an additional degree of freedom to stabilize imbalance signals. The model can provide reliable performance for long-range dependence, particularly for mean reversion chaotic phase variation at critical non-differentiable points. The performance of the dynamic model is verified on chaotic real data of the Ireland Stock Market (ISEQ). The result statistics confirmed the optimized outcome with the addition of the GARCH heteroscedastic multimodal and radial basis neural network. The novel technique can help to address inherited challenges in the imbalance learning and extreme events of the time series modeling by monitoring the chaotic trajectory of fractal physical phenomena, particularly in, finance healthcare, climate, and intelligent sustainability.

TodyNet: Temporal dynamic graph neural network for multivariate time series classification

Multivariate time series classification (MTSC) is a crucial data mining task that can be effectively tackled using prevalent deep learning technology. However, current methods often overlook hidden dependencies across dimensions and struggle to capture dynamic time series features adequately, leading to poor accuracy. To tackle this challenge, we propose TodyNet, a novel dynamic temporal graph neural network. TodyNet extracts latent spatio-temporal dependencies without predefined graph structures, facilitating information flow among isolated but implicitly interdependent variables. It captures associations between time slots through dynamic graphs, boosting classification performance. Furthermore, to address the limitation that the hierarchical representations of graphs are challenging to learn with graph neural networks (GNNs), we introduce a temporal graph pooling layer for obtaining a global graph-level representation. The dynamic graph, graph information propagation, and temporal convolution are jointly learned within an end-to-end framework. Experimental results on 26 UEA benchmark datasets indicate that compared to current state-of-the-art deep learning methods in the MTSC task, the proposed Todynet achieved a performance improvement of 5.5%.

Data-driven forecasting of FOWT dynamics and load time series using lidar inflow measurements

This study focuses on forecasting the fairlead tension and floater dynamics time series of floating offshore wind turbines (FOWTs) using a data-driven approach that incorporates onboard sensor measurements and lidar inflow data. Sensors on FOWTs can provide data on turbine dynamics, such as rotational and translational movements, and load metrics like mooring line loads. However, these sensors are limited to current state measurements and do not provide future signal projections. In this research, we investigate a data-driven forecasting methodology using a simulated dataset. This dataset encompasses FOWT responses to diverse environmental conditions and the associated lidar measurements. Utilizing a Long Short-Term Memory (LSTM) sequence-to-sequence model, this study forecasts the time series of fairlead tension, surge, and pitch for forecasting horizons of 20, 40, and 60 seconds, considering lidar ranges from 100m to 500m. The performance of these forecasting models is benchmarked against a simple persistence model. The results indicate that incorporating lidar inflow measurements significantly improves the forecasts of fairlead tensions and platform motions. The enhancement for pitch motion forecasts is observed across all forecasting horizons. For fairlead tension and surge motion, the enhancement is observed for the longer horizons of 40s and 60s. These findings underscore the value of lidar data in accurate forecasting and emphasize the need to account for the interplay between lidar range, wind speed, and forecasting horizon to achieve optimal forecast accuracy.

Government Size and Corruption in Turkey

Corruption has been one of the challenging topics in public economics for years, prompting extensive discussions from various perspectives. In this study, we investigate the causal relationship between government size and corruption in Turkey, where the anticorruption performance has been getting worse over time until recently. By utilizing a long-span news-based corruption index spanning approximately three-quarters of a century, we conclude that government size serves as the long-term driving force behind corruption in Turkey from 1950 to 2019. This conclusion is robustly reached through a battery of dynamic time series analyses.

CP factor model for dynamic tensors

Abstract Observations in various applications are frequently represented as a time series of multidimensional arrays, called tensor time series, preserving the inherent multidimensional structure. In this paper, we present a factor model approach, in a form similar to tensor CANDECOMP/PARAFAC (CP) decomposition, to the analysis of high-dimensional dynamic tensor time series. As the loading vectors are uniquely defined but not necessarily orthogonal, it is significantly different from the existing tensor factor models based on Tucker-type tensor decomposition. The model structure allows for a set of uncorrelated one-dimensional latent dynamic factor processes, making it much more convenient to study the underlying dynamics of the time series. A new high-order projection estimator is proposed for such a factor model, utilizing the special structure and the idea of the higher order orthogonal iteration procedures commonly used in Tucker-type tensor factor model and general tensor CP decomposition procedures. Theoretical investigation provides statistical error bounds for the proposed methods, which shows the significant advantage of utilizing the special model structure. Simulation study is conducted to further demonstrate the finite sample properties of the estimators. Real data application is used to illustrate the model and its interpretations.

Tunnel indirect damage identification algorithm based on service train dynamic response time series

In this study, a tunnel indirect damage identification algorithm based on a service train dynamic response time series was proposed. First, the principles and processes of the proposed algorithm are introduced. The instantaneous frequency and spectral entropy of the trend and seasonal terms of the train dynamic response decomposed by Seasonal and Trend decomposition using Loess are calculated, respectively, to further extract the time-frequency features of the signal. The calculated sequence is input into the Bi-directional Long Short-Term Memory network, and the type of damage case is the output. Second, the feasibility of the algorithm is verified using a model test. Finally, the key links in the algorithm are discussed. The results indicate that the proposed algorithm can identify damage cases with a small number of training and test sets, and the accuracy of the test set can reach 83.33%. Through time-frequency feature extraction and normalization processing, the length of the network input sequence is reduced, the training speed is increased, and damage identification can be realized without a deep network.

BANKING SECTOR DEVELOPMENT AND ECONOMIC GROWTH DEVELOPING COUNTRIES: A BOOTSTRAP PANEL GRANGER CAUSALITY ANALYSIS

The purpose of this paper is to revisit the Granger causal relationship between banking sector development and economic growth for forty developing countries in the period 1970-2012. We devised two indices to comprehensively assess the development of the banking sector and employed a panel bootstrapped approach for Granger causality testing. This method adequately addresses concerns related to cross-sectional dependence and heterogeneity. Our empirical findings offer modest backing for the supply-leading, demand-following, and complementarity hypotheses. Moreover, our analysis suggests a causal relationship between banking sector development and economic growth across twenty-five countries. This paper investigates the relationship between banking sector development and economic growth in developing countries using a Bootstrap Panel Granger Causality Analysis. The banking sector plays a crucial role in fostering economic development by mobilizing savings, allocating capital efficiently, and channeling investments into productive sectors. However, the precise nature of the causal linkages between banking sector development and economic growth remains subject to empirical scrutiny, particularly in the context of developing economies characterized by diverse institutional settings and economic structures. Employing a panel data approach, this study examines the causal dynamics between banking sector development and economic growth across a sample of developing countries. By leveraging the richness of cross-country variation and time-series dynamics, we aim to provide a nuanced understanding of the symbiotic relationship between banking sector development and economic growth. Our discoveries illuminate the causal connection between banking sector advancement and economic growth in developing nations, presenting significant insights for policymakers, economists, and scholars. Through informing policy discussions and promoting data-driven decision-making, this research contributes to the crucial goals of sustainable development and inclusive growth in developing economies.

The performance analysis of deep learning algorithms for modelling and forecasting the particulate matter (PM10) in the eastern part of Turkey

The main purpose of this study is to select the most reliable nonlinear computational model to predict the particulate matter (PM10) concentrations. Time series data of three years PM10 concentrations were used as input variable. For the prediction, three different types of dynamic nonlinear autoregressive models were built and compared. These models are the Levenberg-Marquardt algorithm, the Bayesian Regulization algorithm, and the Scaled Conjugate Gradient algorithm. For each of these algorythms, various settings were adopted with the subsequent statistical analysis. To analyse the model performance, we used mean prediction error, root mean square error, and correlation coefficient. The lowest root mean square errors were found for the Levenberg-Marquardt algorithm with 15 neurons, for the Bayesian Regularization and for the Scaled Conjugate Gradient algorithm with 20 neurons in hidden layer. In our study, we focused on the long-term forecast of stationary dynamic time series data and on the large amount of data, which is presented as a scientific novelty. Additionally, we determined the main model parameters that most improve quality in terms of training and network capacity. Therefore, the derived forecasting model can be used as a priori for air quality management and regulations aimed on the reducing of pollutant level.

Long Short-Term Memory Parameter Optimization Based on Improved Sparrow Search Algorithm for Molten Iron Quality Prediction

Blast furnace (BF) ironmaking is a key process in iron and steel production. Because BF ironmaking is a dynamic time series process, it is more appropriate to use a recurrent neural network for modeling. The long short-term memory (LSTM) network is commonly used to model time series data. However, its model performance and generalization ability heavily depend on the parameter configuration. Therefore, it is necessary to study parameter optimization for the LSTM model. The sparrow search algorithm (SSA) holds advantages over traditional optimization algorithms in several aspects, such as no need for prior knowledge, fewer parameters, fast convergence, and high scalability. However, the algorithm still faces some challenges, such as the tendency to become trapped in the local optimum and the imbalance between global search ability and local search ability. Therefore, on the basis of SSA, this study examined the Levy flight strategy, sine search strategy, and step size factor adjustment strategy to improve it. This algorithm, improved by three strategies, is called the improved sparrow search algorithm (ISSA). Then, the ISSA-LSTM model was established. Furthermore, considering the limitations of SSA in dealing with multi-objective problems, the fast non-dominated sorting genetic algorithm (NSGAII) was introduced, and the ISSA-NSGAII model was established. Finally, experimental validation was performed using real blast furnace operation data, which demonstrated the proposed algorithm’s superiority in parameter optimization for the LSTM model and prediction for real industrial data.

Pulse Train Fx-LMS Algorithm for Drive File Identification

A novel time-domain algorithm is proposed in this paper for the iterative estimation of drive files. A drive file is a synchronized batch of dynamic time series commands that are simultaneously sent to one or more actuators in a test rig that is designed for service environment replication (SER). When drive file commands are input to an SER test rig, the response of the article under test is similar to what was measured in a service environment. The proposed Pulse Train Filtered-X Least Mean Square (PT-Fx-LMS) algorithm is based on methods developed for active noise and vibration control (ANVC). A time-domain PT-Fx-LMS algorithm is shown through several simulation studies to rapidly converge to a dynamic solution in a small number of iterations for a one degree-of-freedom nonlinear suspension. The PT-Fx-LMS algorithm is also shown to enable targeted iteration over isolated time slices within the data set, which challenges conventional frequency-domain techniques.

DPDGAD: A Dual-Process Dynamic Graph-based Anomaly Detection for multivariate time series analysis in cyber-physical systems

With the rapid development of cyber–physical systems, an increasing amount of data is stored in the form of multivariate time series. Detecting anomalies within multivariate time series has become a crucial means to ensure the proper functioning of cyber–physical systems. Traditional methods for anomaly detection in multivariate time series often categorize features into temporal and spatial features based on their representative dimensions. However, these approaches tend to overlook the unique characteristics of cyber–physical systems reflected in the multivariate time series. This hinders their ability to perform well in cases with complex multivariate time series data. To overcome this challenge, we introduce a novel approach that classifies the spatiotemporal features of multivariate time series data into steady features (features that remain constant under external variations) and dynamic features (features that respond to external changes), aligning with the nature of cyber–physical systems. Addressing this challenge involves two main aspects: (1) determining how to classify steady and dynamic features in multivariate time series data and (2) leveraging these features for anomaly detection.To address these challenges, we propose a Dual-Process Dynamic Graph-based time series Anomaly Detection (DPDGAD) method. DPDGAD enhances the generalizability of anomaly detection in complex spatiotemporal environments and improves the interpretability of the detected anomalies. We validate the performance of DPDGAD through six real-world datasets generated by cyber–physical systems, achieving F1 scores of 0.7529 and 0.9009 on WADI and SWaT, respectively.

AI-Driven Network Security: Innovations in Dynamic Threat Adaptation and Time Series Analysis for Proactive Cyber Defense

AI-Driven Network Security: Innovations in Dynamic Threat Adaptation and Time Series Analysis for Proactive Cyber Defense

The Role of Smart City in Achieving Sustainable Development: Google Trends Analysis and Exponential Time Series Smoothing Models

The concept of sustainable smart urban development is an important element of the modern discourse on the future development of cities, regions, and the world as a whole, a key component of a strategy aimed at introducing innovations, improving the life quality of citizens, ensuring inclusiveness and sustainability of local development. It integrates the concept of Smart City as the application of advanced innovative technologies to solve urban problems and the concept of Sustainable Development as a balancing of socio-economic and environmental interests to achieve the Sustainable Development Goals. While discussing the problems and prospects introducing any social models or strategies, it is important to understand the level of public interest in this issue, the level of citizens’ awareness, the readiness to receive new information, and the readiness for change. The article is focused on the analysis of retrospective dynamics (from 01.01.2010 to 01.12.2023) and economic and mathematical modelling of forecast trends in changing public interest in the concepts of “Smart City” and “Sustainable Development” (identified using the Google Trends tool). The analysis was carried out for the world as a whole and for Ukraine specifically. The study identifies and describes the evolution of changes in public interest in these topics, identifies the time points of a sharp surge in interest, and conducts a comparative analysis in the content (trends for these two search queries) and spatial context (trends in Ukraine compared to global trends). The countries of the world and regions of Ukraine where user interest in the relevant topics dominates are ranked, and the countries of the world and regions of Ukraine where user interest in the relevant topics dominates are ranked, the TOP-10 are identified. The factors whose influence is relevant to the public interest dynamics are identified (consumer trends, media activity, dynamics of the introduction of relevant concepts, cultural, linguistic, and historical contexts, the general level of education, digital inclusion and consciousness of the population of different countries, etc.) Using the exponential smoothing method, models were built to predict the dynamics of time series – the number of queries from Ukrainian Internet users regarding the concepts of “Smart City” and “Sustainable Development”. Additive forecasting models, which differ in the trend component, are obtained.