Temporal Convolutional Network Model Research Articles

Accurate and fast calibration for FBG demodulation based on deep learning and ensemble learning

Accurate and rapid demodulation plays a crucial role in fiber Bragg grating (FBG) sensing systems. The Fabry-Perot (F-P) filter is a dependable demodulation technique with excellent accuracy. However, the F-P filter suffers from demodulation error drift in a temperature-changing environment. The non-repeatable wavelength scanning affects the accuracy and stability of demodulation. To solve the problem, an accurate and rapid calibration approach is proposed for FBG demodulation. First, the temporal convolutional network (TCN) is utilized to extract the hidden information and long-term temporal relationships in the input features including temperature, a temperature changing rate, and shift of reference grating. Second, a state-of-the-art light gradient boosting machine (LightGBM) capable of forecasting demodulation error is adopted for rapid forecasting. To demonstrate the effectiveness of the proposed approach, the traditional TCN model and the composited-residual-block TCN (CTCN) model are both discussed, and the temperature-drift experiments are designed and conducted in two temperature-changing environments. The experimental results show that, compared to the widely applied long short-term memory (LSTM) model, the TCN-LightGBM model achieves higher prediction accuracy and reduces computation time by at least 59.68%. The proposed approach is an affordable and effective alternative to the existing hardware-based calibration techniques.

Vehicle Crashworthiness Performance Prediction Through Fusion of Multiple Data Sources

Abstract This study aims to improve the prediction accuracy of the computer-aided engineering (CAE) model for crashworthiness performance evaluation at speeds beyond those defined by current regulations and public domain testing protocols. One way of achieving this is by integrating data from a few physical crash tests with the CAE data using machine learning models. In this study, two scenarios are investigated: (1) improving CAE model prediction accuracy using test data of a vehicle type that is the same as that of the CAE model; (2) improving CAE model prediction accuracy using test data from two different types of vehicles (e.g., two different sizes of SUVs). In the first scenario, a novel approach is proposed in the displacement domain (deceleration versus displacement) to enable data fusion to help recover the unmodeled physics in the CAE model. A nonlinear spring-mass model is used to simulate rigid-barrier vehicle frontal impact. A Gaussian process regression (GPR) model is then applied in conjunction with a Gaussian mixture model to capture the model bias of the nonlinear spring constant under a dynamic analysis scheme. In the second scenario, we propose a time-domain method (deceleration versus time) based on temporal convolutional network (TCN) and transfer learning. An initial TCN model is first trained by fusing CAE data with physical test data of the first vehicle type based on data augmentation. This data-augmented TCN model is then fine-tuned through transfer learning using CAE and test data of the second vehicle type. It leverages the domain-invariant representations of the two types of vehicles to enhance the CAE model prediction accuracy of the second vehicle type. Case studies are used to validate the proposed approaches and to demonstrate their efficacy in improving the prediction accuracy of the CAE models.

Holistic cold-start management in serverless computing cloud with deep learning for time series

Serverless computing is the most innovative and affordable cloud service model that improves three strategic business objectives: flexibility, efficiency, and innovation. Compared to monolithic applications, serverless computing applications scale better as the applications are further broken down into small building blocks of functions. Resources are assigned per function and can scale up as function requests increase. Hence, serverless computing has been increasingly employed in high-impact areas such as banking and health care. An issue affecting serverless computing’s performance is cold-start — the delays in assigning resources for function execution. Such delays can violate the Service Level Agreements especially those of time-sensitive applications. Current cold-start management policies operate in silos with narrow focuses on function, application, runtime, or infrastructure levels. Such policies do not incorporate higher-level requirements like those in the Service Level Agreements. Additionally, operating in silos does not follow the principle of continuous feedback loops among management policies at different system levels, which is essential for AI-driven self-adaptive future computing systems. Therefore, this paper proposes a novel 2-prong cold-start management policy that allows feedback loops with other higher management policies and orchestrates lower-level cold-start optimization policies. The main policy driver is the temporal convolutional network (TCN) model that can predict function instance arrivals from 5 to 15 min into the future. Evaluation strategies were proposed for both the TCN model and the management policy. Evaluation results show that the TCN model performs reliably across two trace datasets from leading serverless computing providers. Notably, the proposed cold-start policy evaluation plan allows organizations to realistically evaluate their cold-start management in ways that have never been done before.

A Graded Assessment System for Parkinson’s Upper-Limb Bradykinesia Based on a Temporal Convolutional Network Model

A Graded Assessment System for Parkinson’s Upper-Limb Bradykinesia Based on a Temporal Convolutional Network Model

Enhancing Accuracy of Groundwater Level Forecasting with Minimal Computational Complexity Using Temporal Convolutional Network

Multiscale forecasting of groundwater levels (GWLs) is essential for ensuring the sustainable management of groundwater resources, particularly considering the potential impacts of climate change. Such forecasting requires a model that is not only accurate in predicting GWLs but also computationally efficient, ensuring its suitability for practical applications. In this study, a temporal convolutional network (TCN) is implemented to forecast GWLs for 17 monitoring wells possessing diverse hydrogeological characteristics, located across South Korea. Using deep learning, the influence of meteorological variables (i.e., temperature, precipitation) on the forecasted GWLs was investigated by dividing the input features into three categories. Additionally, the models were developed for three forecast intervals (at 1-, 3-, and 6-month lead times) using each category input. When compared with state-of-the-art models, that is, long short-term memory (LSTM) and artificial neural network (ANN), the TCN model showed superior performance and required much less computational complexity. On average, the TCN model outperformed the LSTM model by 24%, 21%, and 25%, and the ANN model by 24%, 37%, and 47%, respectively, for 1-, 3-, and 6-month lead times. Based on these results, the proposed TCN model can be used for real-time GWL forecasting in hydrological applications.

A stacked ensemble method based on TCN and convolutional bi-directional GRU with multiple time windows for remaining useful life estimation

With the widespread popularity of sensors, time-series data of engine degradation processes have been widely applied for remaining useful life (RUL) prediction. As a result, the large-dimensional, large-scale, and multi-state data characteristics of degraded data make accurate prediction very challenging. To overcome this issue, this paper proposes a stacked integration method based on temporal convolutional network (TCN) and convolutional bi-directional gate recurrent unit (CNN-Bi-GRU) with multiple time windows for RUL prediction, which has smaller ensemble dimensions and stronger reliability and adaptability. In the proposed model, the TCN model can well overcome the limitations of large amounts of data which leads to the problem of difficulty in learning temporal relationships; CNN-Bi-GRU model is used to extract important features to solve the problem of high-dimensional data. In addition, the multi-time window method is used to enhance the adaptability of the method and increase the information obtained by the model. Compared with the popular prediction methods, the prediction results of the proposed ensemble model have been improved by about 20 % on average on data sets. In addition, on the N-CMAPSS data set with larger data volume and more diverse states, the proposed model also achieves an average improvement of about 10% over comparison methods. It shows that the proposed method enhances the reliability and applicability of prediction.

Fraud-Free Green Finance: Using Deep Learning to Preserve the Integrity of Financial Statements for Enhanced Capital Market Sustainability

In the era of green finance and sustainable energy markets, preserving the integrity of financial statements is paramount. This paper introduces an innovative approach to financial statement fraud detection through the application of deep learning techniques. We employ a Temporal Convolutional Network (TCN) to analyze stock market data and identify deceptive practices. Our study meticulously reviews related work, highlighting the critical intersection between financial statement fraud detection and the principles of green energy finance. The Materials and Methods section outlines our data sources, variable selection, and the TCN model's architecture, while the Results and Discussion section presents comprehensive evaluations and comparisons against traditional baselines. Our findings demonstrate the exceptional accuracy and reliability of the TCN model in detecting financial statement fraud, underscoring its potential to instill transparency in energy markets. In conclusion, our research contributes significantly to the promotion of financial integrity and sustainability, offering a powerful tool for investors and stakeholders committed to responsible and ethical investments within the framework of green finance.

Metaheuristic evolutionary deep learning model based on temporal convolutional network, improved aquila optimizer and random forest for rainfall-runoff simulation and multi-step runoff prediction

Accurate and reliable runoff prediction is of great significance to water resources management, disaster monitoring and rational development and utilization of water resources. In this paper, a metaheuristic evolutionary deep learning model based on Temporal Convolutional Network (TCN), Improved Aquila Optimizer (IAO) and Random Forest (RF) is proposed for rainfall-runoff simulation and multi-step runoff prediction. In this study, the influence of various input variables on the prediction accuracy is discussed. First of all, in order to avoid the dimensional disaster problems and reduce the calculation time, RF is used to calculate the correlation between the input variables and the prediction object, and the data with high correlation is selected as the final inputs. Then, the filtered data are sent to the TCN model, and the parameters of the TCN model are optimized using the IAO algorithm, and the final prediction results are obtained. In this study, the rainfall and runoff data of five stations in the middle reaches of Jinsha River, China were selected, and the runoff of Panzhihua station was simulated and predicted by establishing multiple models. By analyzing and comparing the predictive results of several models, it shows that the models and improvements proposed in this study are effective.

A unified modeling framework for lane change intention recognition and vehicle status prediction

Accurately detecting and predicting Lane Change (LC) processes of human-driven vehicles can help autonomous vehicles better understand their surrounding environment, recognize potential safety hazards, and improve traffic safety. This paper focuses on LC processes, first developing a Temporal Convolutional Network (TCN) with an attention mechanism (ATM) model to recognize LC intention. Then, considering the intrinsic relationship among output variables, the Multi-Task Learning (MTL) framework is employed to simultaneously predict multiple LC vehicle status indicators. Furthermore, a unified modeling framework for LC intention recognition and driving status prediction (LC-IR-SP) is developed. The results indicate that the classification accuracy of LC intention was improved from 95.83% to 98.20% when incorporating the ATM into the TCN model. For LC vehicle status prediction issues, Pearson's correlation coefficient indicates that metrics extracted from LC processes show stronger correlation than those extracted from Lane-keeping processes. Consequently, three multi-tasking learning models are constructed based on the MTL framework. The results indicate that the MTL with Long Short-Term Memory (MTL-LSTM) model outperforms the MTL with TCN (MTL-TCN) and MTL with TCN-ATM (MTL-TCN-ATM) models. Compared to the corresponding single-task model, the MTL-LSTM model demonstrates an average decrease of 26.04% in MAE and 25.19% in RMSE. The LC-IR-SP model developed holds great potential in enhancing autonomous vehicles' perception and prediction capabilities, such as identifying LC behaviors, calculating real-time traffic conflict indices, and improving vehicle control strategies.

GTFP: Network Fault Prediction Based on Graph and Time Series

With the explosion of 5G network scale, the network structure becomes increasingly complex. During the operation of the network devices, the probability of anomalies or faults increases accordingly. Network faults may lead to the disappearance of important information and cause unpredictable losses. The prediction of network faults can enhance the quality of network services and reduce economic loss. In this paper, we propose the concept of 4D features and use the BERT algorithm to extract semantic features, the graph neural network algorithm to extract network topology information, and the Temporal Convolutional Network (TCN) algorithm to extract time series. Based on this, we propose Fault Prediction based on GraphSage and TCN (GTFP), an end-to-end solution of network fault alarm prediction, which is based on GraphSage and TCN (GTCN), a hybrid algorithm of a graph neural network and the TCN model. Our solution takes the historical alarm data as input. First, we filter out the alarm noises irrelevant to the faults through data cleaning. Then, we employ feature engineering to extract the valid alarm features, including the statistical features of the network alarm information, the semantic features of the alarm texts, the sequential features of the alarms and the network topology features of the nodes where the alarms are located. Finally, we use GTCN to predict future fault alarms based on the extracted features. Experiments on the alarm data of a real service system show that GTFP performs better than the state-of-the-art algorithms of fault alarm prediction.

A Non-Intrusive Identification Approach for Residential Photovoltaic Systems Using Transient Features and TCN with Attention Mechanisms

In order to reduce the negative impact of the large-scale grid connection of residential photovoltaic (PV) equipment on the distribution network, it is of great significance to realize the real-time accurate identification of the grid connection state and its switching of residential PV equipment from the distribution network side. This paper introduces a non-intrusive method for identifying residential PV systems using transient features, leveraging the temporal convolutional network (TCN) model with attention mechanisms. Firstly, the discrimination and redundancy of transient features for residential PV devices are measured using a feature selection method based on the semi-Fisher score and maximal information coefficient (MIC). This enables the construction of a subset of identification features that best characterize the PV devices. Subsequently, a sliding window two-sided cumulative sum (CUSUM) event detection algorithm, incorporating a time threshold, is proposed for the real-time capturing of PV state switching and grid connection behavioral events. This algorithm effectively filters out disturbances caused by the on/off cycles of low-power residential devices and captures the transient time windows of PV behaviors accurately. On this basis, a TCN model with attention mechanisms is proposed to match the discerned event features by assigning varying weights to different types of characteristics, thereby facilitating the precise recognition of a PV grid connection and state-switching events. Finally, the proposed method is validated on a custom-designed non-intrusive experimental platform, demonstrating its precision and real-time efficiency in practical applications.

UAV instantaneous power consumption prediction using LR‐TCN with simple moving average

SummaryThe usage of multi‐rotor unmanned aerial vehicles (UAVs) has rapidly increased in a variety of industries recently, which has caused a quick rise in the quantity of research works on the topic. Remaining battery life prediction, anomaly detection and instantaneous power consumption prediction are among the topics that attract the most attention of researchers. This article presents the development and utilization of a modified temporal convolutional network (TCN) model, a commonly employed approach for anomaly detection and instantaneous power consumption prediction in unmanned aerial vehicles (UAVs). The first modification to the TCN model was to use the Leaky ReLu activation function in place of the rectified linear unit (ReLu) activation function from the original TCN model. In the next step, instantaneous power consumption prediction for UAVs was performed by using the data obtained from both sensors and simple moving average (SMA) algorithm. As a result of the tests performed with the created simulation setup, it has been clearly shown that the proposed method gives better results compared to other deep learning models used for comparison with the lowest RMSE of 0.0496.

State of health estimation of lithium-ion batteries based on modified flower pollination algorithm-temporal convolutional network

Lithium-ion batteries (LIBs) need to maintain high energy efficiency and power level in several application scenario. Accurate state of health (SOH) forecast is essential for designing a safe and reliable battery management systems (BMS). Temporal convolutional network (TCN) is a prevailing deep learning method for estimating the SOH of lithium-ion batteries. However, the hyperparameters in the network are usually difficult to predefine, which poses a challenge for the SOH estimation accuracy in real-world. To solve this problem, this paper proposes a data-driven estimation approach, where the TCN is combined with the modified flower pollination algorithm (MFPA) to determine the network topology. After hyperparameter optimization, the external sensor raw data and identified ohmic resistances trajectories in the equivalent circuits model (ECM) are both input to the TCN model to estimate SOH of LIBs. In contrast to prior approaches for feature extraction, this method is not only conductive to improve SOH estimation accuracy, but also can reduce on-board estimation computing burden. We carry out experiments on the same type of cells from NASA public data resources. The experimental results systematically validate the superiority of the proposed method, which covers high estimation accuracy, great robustness to varied training set and satisfied universality to different batteries.

Robust Malware identification via deep temporal convolutional network with symmetric cross entropy learning

AbstractRecent developments in the field of Internet of things (IoT) have aroused growing attention to the security of smart devices. Specifically, there is an increasing number of malicious software (Malware) on IoT systems. Nowadays, researchers have made many efforts concerning supervised machine learning methods to identify malicious attacks. High‐quality labels are of great importance for supervised machine learning, but noises widely exist due to the non‐deterministic production environment. Therefore, learning from noisy labels is significant for machine learning‐enabled Malware identification. In this study, motivated by the symmetric cross entropy with satisfactory noise robustness, the authors propose a robust Malware identification method using temporal convolutional network (TCN). Moreover, word embedding techniques are generally utilised to understand the contextual relationship between the input operation code (opcode) and application programming interface function names. Here, considering the numerous unlabelled samples in real‐world intelligent environments, the authors pre‐train the TCN model on an unlabelled set using a word embedding method, that is, Word2Vec. In the experiments, the proposed method is compared with several traditional statistical methods and more recent neural networks on a synthetic Malware dataset and a real‐world dataset. The performance comparisons demonstrate the better performance and noise robustness of their proposed method, especially that the proposed method can yield the best identification accuracy of 98.75% in real‐world scenarios.

Central Aortic Blood Pressure Waveform Estimation with a Temporal Convolutional Network.

A novel temporal convolutional network (TCN) model is utilized to reconstruct the central aortic blood pressure (aBP) waveform from the radial blood pressure waveform. The method does not need manual feature extraction as traditional transfer function approaches. The data acquired by the SphygmoCor CVMS device in 1,032 participants as a measured database and a public database of 4,374 virtual healthy subjects were used to compare the accuracy and computational cost of the TCN model with the published convolutional neural network and bi-directional long short-term memory (CNN-BiLSTM) model. The TCN model was compared with CNN-BiLSTM in the root mean square error (RMSE). The TCN model generally outperformed the existing CNN-BiLSTM model in terms of accuracy and computational cost. For the measured and public databases, the RMSE of the waveform using the TCN model was 0.55 ± 0.40 mmHg and 0.84 ± 0.29 mmHg, respectively. The training time of the TCN model was 9.63 min and 25.51 min for the entire training set; the average test time was around 1.79 ms and 8.58 ms per test pulse signal from the measured and public databases, respectively. The TCN model is accurate and fast for processing long input signals, and provides a novel method for measuring the aBP waveform. This method may contribute to the early monitoring and prevention of cardiovascular disease.

Short-term Load Forecasting with Deep Learning Techniques

Short-term load forecasting is essential to the modern power system, which is the basis to guarantee the equilibrium of power supply and demand. It not only ensures the secure and stable operation of power grid, but also provides an important basis for power load dispatch, which can effectively reduce the economic loss caused by energy surplus or energy shortage in a short term. In the previous research work, traditional linear regression models, typical machine learning models and other models have been explored, but there are still problems of low accuracy or single influence factors. In this paper, a short-term forecasting model based on Temporal Convolutional Network (TCN) is proposed. Historical power load data, weather conditions, date information and their combinations are separately used to forecast the power load in different prediction models. Compared with other models, the evaluation metric of TCN model shows that its prediction accuracy is higher, and it is more suitable for short-term load forecasting.

Hybrid solar radiation forecasting model with temporal convolutional network using data decomposition and improved artificial ecosystem-based optimization algorithm

Solar energy is highly economical and widespread in new energy applications, and analyzing solar radiation information is an important part of solar photovoltaic power applications. However, because of its data complexity and difficulty to measure, solar radiation data needs to be predicted. Temporal Convolutional Network (TCN) model is used to extract features and Artificial Ecosystem-based Optimization (AEO) algorithm is used to optimize the parameters of TCN. Out of consideration for the phenomenon of strong fluctuations and complex features of solar radiation data, the optimal variational mode decomposition (OVMD) method is incorporated into the model. First, the signal decomposition is performed on the original data to obtain several subsequences, and then aggregated by fuzzy entropy to reduce the number of sequences, after which the data are fed into the TCN model and the model parameters are optimized using the improved AEO algorithm, and finally the results of the model prediction are the output. Four months of solar radiation data are selected for testing, it is finally concluded that the OVMD-IAEO-TCN model can be used for solar radiation prediction with higher accuracy and reliability than the other nine comparison models.

TCN-based M-ary mobile spread spectrum underwater acoustic communication

A deep learning-based direct sequence spread spectrum (DSSS) underwater acoustic (UWA) communication system was proposed in this study to improve the performance of the conventional system under Doppler effects and low signal-to-noise ratio (SNR). The Temporal Convolutional Network (TCN) model was used as the receiver of the system. Simulation results showed that the TCN-based M-ary DSSS UWA communication system outperformed the conventional system under Doppler effects and complex shallow water acoustic channel. A transfertraining algorithm was used for fine-tuning of the model based on sea real-time data to achieve optimal performance. Sea experimental results showed that the fine-tuned TCN model improved the communication effect under Doppler background in the complex dynamic environment of shallow water.

Prediction of melt pool width and layer height for Laser Directed Energy Deposition enabled by physics-driven temporal convolutional network

Laser Directed Energy Deposition (L-DED) is a momentous metal additive manufacturing technology. Owing to high flexibility characteristic, it has been progressively adopted by high-value manufacturing industries. For the technology, one of the fundamental research challenges is how to accurately predict the melt pool size to ensure high-quality L-DED processes. To tackle the challenge, a novel physics-driven temporal convolutional network (TCN) approach is presented. In this research, the high prediction accuracy for L-DED is achievable via the following innovations: (i) a TCN model is designed as the core of the approach to leveraging the distinctive characteristics of the TCN model to address the temporal nature of the L-DED process during heat accumulation and incremental deposition; (ii) the physical models of the peak temperature, Marangoni effect and liquid jets affecting the melt pool formulization during the L-DED process are specified to strengthen the prediction accuracy of the approach. Experiments for manufacturing thin-walled parts using L-DED were conducted for approach validation and analyses. On average, the mean absolute percentage errors (MAPEs) of predicting the melt pool width and the layer height of a melt pool attained by this approach are 3.421% and 4.643%, respectively. The experiments demonstrate that the approach is competent to support the L-DED process in producing good-quality thin-walled parts.

Temporal convolutional network for a Fast DNA mutation detection in breast cancer data.

Early detection of breast cancer can be achieved through mutation detection in DNA sequences, which can be acquired through patient blood samples. Mutation detection can be performed using alignment and machine learning techniques. However, alignment techniques require reference sequences, and machine learning techniques still cannot predict index mutation and require supporting tools. Therefore, in this research, a Temporal Convolutional Network (TCN) model was proposed to detect the type and index mutation faster and without reference sequences and supporting tools. The architecture of the proposed TCN model is specifically designed for sequential labeling tasks on DNA sequence data. This allows for the detection of the mutation type of each nucleotide in the sequence, and if the nucleotide has a mutation, the index mutation can be obtained. The proposed model also uses 2-mers and 3-mers mapping techniques to improve detection performance. Based on the tests that have been carried out, the proposed TCN model can achieve the highest F1-score of 0.9443 for COSMIC dataset and 0.9629 for RSCM dataset, Additionally, the proposed TCN model can detect index mutation six times faster than BiLSTM model. Furthermore, the proposed model can detect type and index mutations based on the patient's DNA sequence, without the need for reference sequences or other additional tools.