Processing Time Series Data Research Articles

Physical reservoir computing using vertically aligned graphene/diamond photomemristors

Reservoir computing is one of the most promising machine learning architectures and could allow highly efficient, high-speed processing of time-series data. Physical reservoir computing based on various physical phenomena that exhibit complicated dynamics has been widely investigated in recent years. The present work demonstrates vertically aligned graphene/diamond junctions (photomemristors) could be employed for physical reservoir computing involving image recognition of single digits. Exceptional image recognition performance of 92% was obtained due to their complex photoconducting behaviors. This work is expected to assist in the realization of novel visual information processing systems using photomemristors that mimic human brain functions.

Biological neurons act as generalization filters in reservoir computing

Reservoir computing is a machine learning paradigm that transforms the transient dynamics of high-dimensional nonlinear systems for processing time-series data. Although the paradigm was initially proposed to model information processing in the mammalian cortex, it remains unclear how the nonrandom network architecture, such as the modular architecture, in the cortex integrates with the biophysics of living neurons to characterize the function of biological neuronal networks (BNNs). Here, we used optogenetics and calcium imaging to record the multicellular responses of cultured BNNs and employed the reservoir computing framework to decode their computational capabilities. Micropatterned substrates were used to embed the modular architecture in the BNNs. We first show that the dynamics of modular BNNs in response to static inputs can be classified with a linear decoder and that the modularity of the BNNs positively correlates with the classification accuracy. We then used a timer task to verify that BNNs possess a short-term memory of several 100 ms and finally show that this property can be exploited for spoken digit classification. Interestingly, BNN-based reservoirs allow categorical learning, wherein a network trained on one dataset can be used to classify separate datasets of the same category. Such classification was not possible when the inputs were directly decoded by a linear decoder, suggesting that BNNs act as a generalization filter to improve reservoir computing performance. Our findings pave the way toward a mechanistic understanding of information representation within BNNs and build future expectations toward the realization of physical reservoir computing systems based on BNNs.

Effect of Islamicity Performance Index on Profitability of Sharia Commercial Banks

Dramatic development of sharia economic has changed the existing structure of banking industry and created the so called sharia banking. Sharia Commercial Banks as the implementer of sharia banking feel necessary to maintain and improve performance and profitability. The objective of this research is to understand and analyze the effect of performance of Sharia Commercial Banks on profitability. Research population includes all Sharia Commercial Banks in the registry of Financial Services Authority in period 2020. Sampling technique is purposive sampling. Research sample with criteria is 6 Sharia Commercial Banks. Panel data regression is used as data analysis method because this research is processing cross-section data and time series data. The results of research show that Profit Sharing Ratio and Islamic Income Ratio have negative effect but not significant on profitability of Sharia Commercial Banks, Zakat Performance Ratio has negative and significant effect on profitability of Sharia Commercial Banks, and Equitable Distribution Ratio has positive and significant effect on profitability of Sharia Commercial Banks.

Air Quality Index Prediction Based on a Long Short-Term Memory Artificial Neural Network Model

<p>Air pollution has become one of the important challenges restricting the sustainable development of cities. Therefore, it is of great significance to achieve accurate prediction of Air Quality Index (AQI). Long Short Term Memory (LSTM) is a deep learning method suitable for learning time series data. Considering its superiority in processing time series data, this study established an LSTM forecasting model suitable for air quality index forecasting. First, we focus on optimizing the feature metrics of the model input through Information Gain (IG). Second, the prediction results of the LSTM model are compared with other machine learning models. At the same time the time step aspect of the LSTM model is used with selective experiments to ensure that model validation works properly. The results show that compared with other machine learning models, the LSTM model constructed in this paper is more suitable for the prediction of air quality index.</p> <p> </p>

Novel aeroengine fault diagnosis method based on feature amplification

Data-driven fault diagnosis methods have high requirements for data samples. The ideal state is that the input samples have good separability. However, because of the harsh working environment and complex operating conditions, the collected data are highly nonlinear inseparability. The existing methods often cannot achieve a good classification effect. To solve this problem, two feature amplification methods that make full use of the useful information from the aeroengine service data are proposed. One is a high-dimensional mapping method based on explicit mapping of kernel function to amplify features. The other is an experiential method to amplify features. Both methods map the input samples to high-dimensional space to make them more separable in high-dimensional space and retain the useful information of raw data. Each feature after feature amplification still covers strong independent information that can describe fault feature information from different dimensions. Then, four deep learning algorithms that have a good effect on processing time-series data are selected as the classifier. The aeroengine service dataset and the bearing vibration dataset from Case Western University are used to verify the effectiveness of the two feature amplification methods. The experimental results show that the fault diagnosis accuracy can be improved if sample features after high-dimensional mapping possess good orthogonality, otherwise, the accuracy will be reduced.

Fault Diagnosis of Nuclear Power Plant Based on Sparrow Search Algorithm Optimized CNN-LSTM Neural Network

Nuclear power is a type of clean and green energy; however, there is a risk of radioactive material leakage when accidents occur. When radioactive material leaks from nuclear power plants, it has a great impact on the environment and personnel safety. In order to enhance the safety of nuclear power plants and support the operator’s decisions under accidental circumstances, this paper proposes a fault diagnosis method for nuclear power plants based on the sparrow search algorithm (SSA) optimized by the CNN-LSTM network. Firstly, the convolutional neural network (CNN) was used to extract features from the data before they were then combined with the long short-term memory (LSTM) neural network to process time series data and form a CNN-LSTM model. Some of the parameters in the LSTM neural network need to be manually tuned based on experience, and the settings of these parameters have a great impact on the overall model results. Therefore, this paper selected the sparrow search algorithm with a strong search capability and fast convergence to automatically search for the hand-tuned parameters in the CNN-LSTM model, and finally obtain the SSA-CNN-LSTM model. This model can classify the types of accidents that occur in nuclear power plants to reduce the nuclear safety hazards caused by human error. The experimental data are from a personal computer transient analyzer (PCTRAN). The results show that the classification accuracy of the SSA-CNN-LSTM model for the nuclear power plant fault classification problem is as high as 98.24%, which is 4.80% and 3.14% higher compared with the LSTM neural network and CNN-LSTM model, respectively. The superiority of the sparrow search algorithm for optimizing model parameters and the feasibility and accuracy of the SSA-CNN-LSTM model for nuclear power plant fault diagnosis were verified.

DC–DC​ circuit fault diagnosis based on GWO optimization of 1DCNN-GRU network hyperparameters

Aiming at the problem that traditional machine learning methods rely on manual feature extraction in DC–DC circuit soft faults, it is important to effectively obtain the fault characteristics of DC–DC circuit soft faults. In this study, combining the advantages of one-dimensional convolutional neural network (1DCNN) and gated logic unit (GRU), a deep learning model for fault identification of time series signals is proposed to realize soft fault diagnosis of DC–DC​ circuits. In this study, a 1DCNN-GRU network model is constructed, and 1DCNN can directly perform automatic feature extraction on the data, while the GRU makes up for the shortcomings of CNN in processing time series data, thereby ensuring the comprehensiveness of the extracted features. For the hyperparameter problem in the network model, the powerful optimization ability of the gray wolf optimization algorithm is used to automatically search for the best hyperparameters in the 1DCNN-GRU network, and then the optimized 1DCNN-GRU network model is used for comprehensive feature learning. In order to meet the needs of deep learning for large data samples, the overlapping sampling method is used to enrich the data sample set. Experimental results show that the proposed method achieves 99.62% accuracy in DC–DC circuit fault diagnosis, and still maintains good robustness in noisy environment.

Feature selection in wind speed forecasting systems based on meta-heuristic optimization.

Technology for anticipating wind speed can improve the safety and stability of power networks with heavy wind penetration. Due to the unpredictability and instability of the wind, it is challenging to accurately forecast wind power and speed. Several approaches have been developed to improve this accuracy based on processing time series data. This work proposes a method for predicting wind speed with high accuracy based on a novel weighted ensemble model. The weight values in the proposed model are optimized using an adaptive dynamic grey wolf-dipper throated optimization (ADGWDTO) algorithm. The original GWO algorithm is redesigned to emulate the dynamic group-based cooperative to address the difficulty of establishing the balance between exploration and exploitation. Quick bowing movements and a white breast, which distinguish the dipper throated birds hunting method, are employed to improve the proposed algorithm exploration capability. The proposed ADGWDTO algorithm optimizes the hyperparameters of the multi-layer perceptron (MLP), K-nearest regressor (KNR), and Long Short-Term Memory (LSTM) regression models. A dataset from Kaggle entitled Global Energy Forecasting Competition 2012 is employed to assess the proposed algorithm. The findings confirm that the proposed ADGWDTO algorithm outperforms the literature's state-of-the-art wind speed forecasting algorithms. The proposed binary ADGWDTO algorithm achieved average fitness of 0.9209 with a standard deviation fitness of 0.7432 for feature selection, and the proposed weighted optimized ensemble model (Ensemble using ADGWDTO) achieved a root mean square error of 0.0035 compared to state-of-the-art algorithms. The proposed algorithm's stability and robustness are confirmed by statistical analysis of several tests, such as one-way analysis of variance (ANOVA) and Wilcoxon's rank-sum.

Multi-head self-attention mechanism enabled individualized hemoglobin prediction and treatment recommendation systems in anemia management for hemodialysis patients

Anemia is a critical complication in hemodialysis patients, but the response to erythropoietin-stimulating agents (ESA) treatment varies from patient to patient and is not linear across different time points. The aim of this study was to develop deep learning algorithms for individualized anemia management. We retrospectively collected 36,677 data points from 623 hemodialysis patients, including clinical data, laboratory values, hemoglobin levels, and previous ESA doses. To reduce the computational complexity associated with recurrent neural networks (RNN) in processing time-series data, we developed neural networks based on multi-head self-attention mechanisms in an efficient and effective hemoglobin prediction model. Our proposed model achieved a more accurate hemoglobin prediction than the state-of-the-art RNN model, as shown by the smaller mean absolute error (MAE) of hemoglobin (0.451 vs. 0.593 g/dL, p = 0.014). In ESA (including darbepoetin and epoetin) dose recommendation, the simulation results by our model revealed a higher rate of achieved hemoglobin targets (physician prescription vs. model: 86.3 % vs. 92.7 %, p < 0.001), a lower rate of hemoglobin levels below 10 g/dL (13.7 % vs. 7.3 %, p < 0.001) and smaller change in hemoglobin levels (0.6 g/dL vs. 0.4 g/dL, p < 0.001) in all patients. Our model holds great potential for individualized anemia management as a computerized clinical decision support system for hemodialysis patients. Further external validation with other datasets and prospective clinical utility studies are warranted.

A hybrid attention and time series network for enterprise sales forecasting under digital management and edge computing

Enterprises have both new opportunities and new challenges as a result of the rapid advancements in information technology that have accompanied the age of economic globalization. With the growth of internet of Things devices, data sizes have significantly increased. Further, the traditional cloud platform has been enriched with edge computing so that the huge data can be processed where it is collected. Therefore, businesses must adapt to new size requirements and rising standards for technical content. Forecasting corporate sales has emerged as a hot topic in the field of digital management. To successfully direct the future production and existence of enterprises, time series forecasting is of utmost importance and value. This is because it makes use of already-existing data to get the best predicting result. This work proposes a combination of enterprise sales forecasting from the perspective of digital management and neural networks, and proposes a network HATT-CNN-BiLSTM model for enterprise sales forecasting. First, this work combines multi-scale CNN (MSCNN) with improved BiLSTM (IBiLSTM) model. The MSCNN is utilized to extract spatial features with different scale, and it is often impossible to effectively explore the rules of time series features, and the processing of time series data is the strength of the LSTM network. Moreover, the IBiLSTM model can explore time series features in both directions, and therefore more useful information can be obtained. The MSCNN-IBiLSTM model, which is composed of MSCNN and IBiLSTM, can take advantage of strengths and avoid weaknesses, and give full play to the roles of the two models in different fields. Second, this work proposes a hybrid attention mechanism that combines self-attention, channel attention, and spatial attention. It enhances features extracted by MSCNN-IBiLSTM through a hybrid attention to build HATT-MSCNN-IBiLSTM network, which can extract more discriminative features. Third, this work conducts comprehensive and systematic experiments on HATT- MSCNN-IBiLSTM to verify feasibility of the proposed method. The proposed model is implemented over an edge computing platform that increases the model training speed and improve the response time.

Application of Deep Learning Method to Capital Assets Pricing

The key problem of financial assets allocation is the price of assets. Assets pricing is the core content of Modern Finance, and revealing the law of assets pricing is always a hot spot of finance research. In recent years, deep learning technology has been applied in the research process of assets pricing and achieved good effect. This paper introduced the theory and characteristics of deep learning, started from extracting and utilizing nonlinear information, effectively processing time series data, and intellectual prediction model, and explored the application of deep learning method in the capital assets pricing. Meanwhile, this paper explores the applicability and limitations of deep learning methods and discusses possible future research trends in learning-based asset pricing.

A Method of Noise Reduction for Radio Communication Signal Based on RaGAN

Radio signals are polluted by noise in the process of channel transmission, which will lead to signal distortion. Noise reduction of radio signals is an effective means to eliminate the impact of noise. Using deep learning (DL) to denoise signals can reduce the dependence on artificial domain knowledge, while traditional signal-processing-based denoising methods often require knowledge of the artificial domain. Aiming at the problem of noise reduction of radio communication signals, a radio communication signal denoising method based on the relativistic average generative adversarial networks (RaGAN) is proposed in this paper. This method combines the bidirectional long short-term memory (Bi-LSTM) model, which is good at processing time-series data with RaGAN, and uses the weighted loss function to construct a noise reduction model suitable for radio communication signals, which realizes the end-to-end denoising of radio signals. The experimental results show that, compared with the existing methods, the proposed algorithm has significantly improved the noise reduction effect. In the case of a low signal-to-noise ratio (SNR), the signal modulation recognition accuracy is improved by about 10% after noise reduction.

Brain-Inspired Spike Echo State Network Dynamics for Aero-Engine Intelligent Fault Prediction

Aero-engine fault prediction aims to accurately predict the development trend of the future state of aero-engines, so as to diagnose faults in advance. Traditional aero-engine parameter prediction methods mainly use the nonlinear mapping relationship of time series data but generally ignore the adequate spatio-temporal features contained in aero-engine data. To this end, we propose a brain-inspired spike echo state network (Spike-ESN) model for aero-engine intelligent fault prediction, which is used to effectively capture the evolution process of aero-engine time series data in the framework of spatio-temporal dynamics. In the proposed approach, we design a spike input layer based on Poisson distribution inspired by the spike neural encoding mechanism of biological neurons, which can extract the useful temporal characteristics in aero-engine sequence data. Then, the temporal characteristics are input into a spike reservoir through the current calculation method of spike accumulation in neurons, which projects the data into a high-dimensional sparse space. In addition, we use the ridge regression method to read out the internal state of the spike reservoir. Finally, the experimental results of aero-engine states prediction demonstrate the superiority and potential of the proposed method.

Speech Separation Algorithm Using Gated Recurrent Network Based on Microphone Array

Speech separation is an active research topic that plays an important role in numerous applications, such as speaker recognition, hearing prosthesis, and autonomous robots. Many algorithms have been put forward to improve separation performance. However, speech separation in reverberant noisy environment is still a challenging task. To address this, a novel speech separation algorithm using gate recurrent unit (GRU) network based on microphone array has been proposed in this paper. The main aim of the proposed algorithm is to improve the separation performance and reduce the computational cost. The proposed algorithm extracts the sub-band steered response power-phase transform (SRP-PHAT) weighted by gammatone filter as the speech separation feature due to its discriminative and robust spatial position information. Since the GRU network has the advantage of processing time series data with faster training speed and fewer training parameters, the GRU model is adopted to process the separation features of several sequential frames in the same sub-band to estimate the ideal Ratio Masking (IRM). The proposed algorithm decomposes the mixture signals into time-frequency (TF) units using gammatone filter bank in the frequency domain, and the target speech is reconstructed in the frequency domain by masking the mixture signal according to the estimated IRM. The operations of decomposing the mixture signal and reconstructing the target signal are completed in the frequency domain which can reduce the total computational cost. Experimental results demonstrate that the proposed algorithm realizes omnidirectional speech separation in noisy and reverberant environments, provides good performance in terms of speech quality and intelligibility, and has the generalization capacity to reverberate.

Information-based Preprocessing of PLC Data for Automatic Behavior Modeling

Cyber-physical systems (CPS) offer immense optimization potential for manufacturing processes through the availability of multivariate time series data of actors and sensors. Based on automated analysis software, the deployment of adaptive and responsive measures is possible for time series data. Due to the complex and dynamic nature of modern manufacturing, analysis and modeling often cannot be entirely automated. Even machine- or deep learning approaches often depend on a priori expert knowledge and labelling. In this paper, an information-based data preprocessing approach is proposed. By applying statistical methods including variance and correlation analysis, an approximation of the sampling rate in event-based systems and the utilization of spectral analysis, knowledge about the underlying manufacturing processes can be gained prior to modeling. The paper presents, how statistical analysis enables the pruning of a dataset's least important features and how the sampling rate approximation approach sets the base for further data analysis and modeling. The data's underlying periodicity, originating from the cyclic nature of an automated manufacturing process, will be detected by utilizing the fast Fourier transform. This information-based preprocessing method will then be validated for process time series data of cyber-physical systems’ programmable logic controllers (PLC).

An optimised multi-channel neural network model based on CLDNN for automatic modulation recognition

To achieve high accuracy blind modulation identification of wireless communication, a novel multi-channel deep learning framework based on the Convolutional Long Short-Term Memory Fully Connected Deep Neural Network (MC-CLDNN) is proposed. To make network training more efficient, we use the Gated Recurrent Unit (GRU) sequence model as the substructure. Furthermore, the skip connection is added to alleviate the problem of gradient disappearance in the network training and reduce the negative effect of pooling layer processes time series data on the subsequent sequence model. We test the feasibility of the model based on two open-source data sets RadioML2016.10a and RadioML2016.10b. The simulation results show that the proposed model can identify most modulation modes efficiently under the influence of various factors such as Additive White Gaussian Noise (AWGN), multipath fading, frequency offset. In the Signal-to-Noise Ratio (SNR) range of 0-18 dB, the overall recognition accuracy of the MC-CLDNN can reach 93% and the area under the Receiver Operating Characteristic (ROC) curve accounts for more than 99%. Therefore, the model has the characteristics of high recognition accuracy and strong generalisation ability. Its comprehensive performance is better than most of the existing deep learning models.

Interpretation of Explainable AI Methods as Identification of Local Linearized Models

Artificial intelligence (AI) models are increasingly ubiquitous in daily life, their unparalleled predictive and decision-making capabilities are being utilized for applications of all magnitudes, ranging from minor decisions to those with significant impacts on individuals and society. However, many of these models feature a multitude of redundant parameters, which render them incomprehensible to human understanding. The lack of transparency raises concerns about the reliability and fairness of the decisions made by AI models motivating a new field of research called eXplainable AI (XAI), which aims to elucidate complex AI model outcomes and develop tools to enable human understanding. Given the increasing impact of machine learning in the fields of data-driven estimation and control, it becomes crucial to integrate XAI tools with control theory to better comprehend the decisions made by AI models in estimation and control. In this article, we propose the use of an XAI method called Local Interpretable Model-Agnostic Explanations (LIME) to explain the mechanisms behind a black-box estimation algorithm processing time-series data. Moreover, we demonstrate that LIME can be used to identify a local linearized model that approximates the complex machine learning algorithm. We show that the identified local linearized model can shed light on the dynamic of the model that generated the training data.

Edge-of-chaos learning achieved by ion-electron-coupled dynamics in an ion-gating reservoir.

Physical reservoir computing has recently been attracting attention for its ability to substantially reduce the computational resources required to process time series data. However, the physical reservoirs that have been reported to date have had insufficient computational capacity, and most of them have a large volume, which makes their practical application difficult. Here, we describe the development of a Li+ electrolyte-based ion-gating reservoir (IGR), with ion-electron-coupled dynamics, for use in high-performance physical reservoir computing. A variety of synaptic responses were obtained in response to past experience, which were stored as transient charge density patterns in an electric double layer, at the Li+ electrolyte/diamond interface. Performance for a second-order nonlinear dynamical equation task is one order of magnitude higher than memristor-based reservoirs. The edge-of-chaos state of the IGR enabled the best computational capacity. The IGR described here opens the way for high-performance and integrated neural network devices.

Development of a Low-Cost Remote Fault Detection System for a Photovoltaic Power Plant

The main purpose of this study is to develop a cost-effective photovoltaic module array (PVMA) fault detection system. The system installs a fault detection module in the junction box at the back of each photovoltaic module (PVM), connects it to the Wi-Fi on the case site, and transfers data and information to the server for data analysis. The PVM outputs the voltage to the isolated DC-DC power module as the power supply of the overall circuits. The fault detection module sends data to the message queuing telemetry transport (MQTT) broker through the protocol in MQTT. As for the server host end, the visualization IoT development tool Node-RED is set up to display complex data sent by the fault detection module by means of data visualization. At the same time, it is linked to the open source time series database (OSTSDB) to process time series data. Finally, the results of the PVM are instantly sent to the maintenance personnel through an exception alert notification in order to inform the maintenance personnel to quickly rule out the failure. As the developed fault detection system hardware is more cost-effective than existing detection systems, the cost-effectiveness is favorable for mass production with market competitiveness.

Research on PM2.5 Pollution Prediction Method in Hefei City Based on CNN-LSTM Hybrid Model

The air quality data has the characteristics of large scale and high time sequence. In the deep learning model, CNN has good feature extraction and expression ability through convolution operation. LSTM has strong time series data processing ability, which effectively solves the long-term dependence problem in the time series data that CNN cannot process. However, a single feature extraction or time series processing method will affect the accuracy of the model. This paper proposes a PM2.5 pollution prediction method based on the CNN-LSTM hybrid model, which can fuse the time series and nonlinear features of the data. By selecting the air quality and meteorological data of Hefei City in Anhui Province from 2018 to 2021 for prediction research, the experimental results show that the CNN-LSTM hybrid model is better than the LSTM model in terms of RMSE, MAE, and R2 evaluation indexes, which reflects higher prediction accuracy. It can not only be used for real-time prediction of PM2.5 pollution, but also for the prediction of other pollutants in the atmosphere. It also provides a reference for the application of CNN-LSTM model in solving other time series big data.