Input Time Series Research Articles

Quantifying the Effect of ICME Removal and Observation Age for in Situ Solar Wind Data Assimilation

AbstractAccurate space weather forecasting requires advanced knowledge of the solar wind conditions in near‐Earth space. Data assimilation (DA) combines model output and observations to find an optimum estimation of reality and has led to large advances in terrestrial weather forecasting. It is now being applied to space weather forecasting. Here, we use solar wind DA with in‐situ observations to reconstruct solar wind speed in the ecliptic plane between 30 solar radii and Earth's orbit. This is used to provide solar wind speed hindcasts. Here, we assimilate observations from the Solar Terrestrial Relations Observatory and the near‐Earth data set, OMNI. Analysis of two periods of time, one in solar minimum and one in solar maximum, reveals that assimilating observations from multiple spacecraft provides a more accurate forecast than using any one spacecraft individually. The age of the observations also has a significant impact on forecast error, whereby the mean absolute error (MAE) sharply increases by up to 23% when the forecast lead time first exceeds the corotation time associated with the longitudinal separation between the observing spacecraft and the forecast location. It was also found that removing coronal mass ejections from the DA input and verification time series reduces the forecast MAE by up to 10% as it removes false streams from the forecast time series. This work highlights the importance of an L5 space weather monitoring mission for near‐Earth solar wind forecasting and suggests that an additional mission to L4 would further improve future solar wind DA forecasting capabilities.

Memory-based Transformer with shorter window and longer horizon for multivariate time series forecasting

Multivariate time series forecasting is an important problem that spans many fields. One challenge of this problem is the complex and non-linear interdependence between time steps and different variables. Recent studies have shown that Transformer has potential in capturing long-term dependencies. However, in the field of time series forecasting, Transformer still has some problems to solve, such as prediction fragmentation and insensitivity to data scale. In addition, traditional forecasting models often require a large amount of input data to support the training of the model when predicting long-term data. However, it is hard to provide sufficient time series input data due to equipment damage or weather situation. To solve these limitations, a memory-based Transformer with shorter window and longer horizon is proposed, called SWLHT. It uses the memory mechanism to make the model no longer only rely on a single input, but can combine the previous forecast results to assist in capturing long-term dependencies, thereby avoiding the requirement of excessively long input sequence. Furthermore, the memory mechanism can alleviate the prediction fragmentation to some extent. The experimental results and comparison of baselines on several real-world multivariate time series datasets have verified the effectiveness of the proposed model.

Forecasting tunnel boring machine penetration rate using LSTM deep neural network optimized by grey wolf optimization algorithm

Achieving an accurate and reliable estimation of tunnel boring machine (TBM) performance can diminish the hazards related to extreme capital costs and planning tunnel construction. Here, a hybrid long short-term memory (LSTM) model enhanced by grey wolf optimization (GWO) is developed for predicting TBM-penetration rate (TBM-PR). 1125 datasets were considered including six input parameters. To vanish overfitting, the dropout technique was used. The effect of input time series length on the model performance was studied. The TBM-PR results of the LSTM-GWO model were compared to some other machine learning (ML) models such as LSTM. The results were evaluated using root mean square error (RMSE), mean absolute percentage error (MAPE), and correlation coefficient (R2). Finally, the LSTM-GWO model produced the most accurate results (test: R2 = 0.9795; RMSE = 0.004; MAPE = 0.009 %). The mutual information test revealed that input parameters of rock fracture class and uniaxial compressive strength have the most and least impact on the TBM-PR, respectively.

AnomalyKiTS: Anomaly Detection Toolkit for Time Series

This demo paper presents a design and implementation of a system AnomalyKiTS for detecting anomalies from time series data for the purpose of offering a broad range of algorithms to the end user, with special focus on unsupervised/semi-supervised learning. Given an input time series, AnomalyKiTS provides four categories of model building capabilities followed by an enrichment module that helps to label anomaly. AnomalyKiTS also supports a wide range of execution engines to meet the diverse need of anomaly workloads such as Serveless for CPU intensive work, GPU for deep-learning model training, etc.

Training Robust Deep Models for Time-Series Domain: Novel Algorithms and Theoretical Analysis

Despite the success of deep neural networks (DNNs) for real-world applications over time-series data such as mobile health, little is known about how to train robust DNNs for time-series domain due to its unique characteristics compared to images and text data. In this paper, we fill this gap by proposing a novel algorithmic framework referred as RObust Training for Time-Series (RO-TS) to create robust deep models for time-series classification tasks. Specifically, we formulate a min-max optimization problem over the model parameters by explicitly reasoning about the robustness criteria in terms of additive perturbations to time-series inputs measured by the global alignment kernel (GAK) based distance. We also show the generality and advantages of our formulation using the summation structure over time-series alignments by relating both GAK and dynamic time warping (DTW). This problem is an instance of a family of compositional min-max optimization problems, which are challenging and open with unclear theoretical guarantee. We propose a principled stochastic compositional alternating gradient descent ascent (SCAGDA) algorithm for this family of optimization problems. Unlike traditional methods for time-series that require approximate computation of distance measures, SCAGDA approximates the GAK based distance on-the-fly using a moving average approach. We theoretically analyze the convergence rate of SCAGDA and provide strong theoretical support for the estimation of GAK based distance. Our experiments on real-world benchmarks demonstrate that RO-TS creates more robust deep models when compared to adversarial training using prior methods that rely on data augmentation or new definitions of loss functions. We also demonstrate the importance of GAK for time-series data over the Euclidean distance.

Transformer‐based time series prediction of the maximum power point for solar photovoltaic cells

AbstractThis paper proposes an improved deep learning‐based maximum power point tracking (MPPT) in solar photovoltaic cells considering various time series‐based environmental inputs. Generally, artificial neural network‐based MPPT algorithms use basic neural network architectures and inputs which do not represent the ambient conditions in a comprehensive manner. In this article, the ambient conditions of a location are represented through a comprehensive set of environmental features. Furthermore, the inclusion of time‐based features in the input data is considered to model cyclic patterns temporally within the atmospheric conditions leading to robust modeling of the MPPT algorithm. A transformer‐based deep learning architecture is trained as a time series prediction model using multidimensional time series input features. The model is trained on a dataset containing typical meteorological‐year data points of ambient weather conditions from 50 locations. The attention mechanism in the transformer modules allows the model to learn temporal patterns in the data efficiently. The proposed model achieves a 0.47% mean average percentage error of prediction on non‐zero operating voltage points in a test dataset consisting of data collected over a period of 200 consecutive hours; resulting in the average power efficiency of 99.54% and peak power efficiency of 99.98%. The proposed model is validated through real‐time simulations. The proposed model performs power point tracking in a robust, dynamic, and nonlatent manner, over a wide range of atmospheric conditions.

Prediction of Battery SOH by CNN-BiLSTM Network Fused with Attention Mechanism

During the use and management of lead–acid batteries, it is very important to carry out prediction and study of the state of the health (SOH) of the battery. To this end, this paper proposes a SOH prediction method for lead–acid batteries based on the CNN-BiLSTM-Attention model. The model utilizes the convolutional neural network (CNN) to carry out feature extraction and data dimension reduction in the input factors of model, and then these factors are used as the input of the bidirectional long short-term memory network (BiLSTM). The BiLSTM is used to learn the temporal correlation information in the local features of input time series bidirectionally. The attention mechanism is introduced to assign more attention to key features in the input sequence with more significant influence on the output result by assigning weights to important features, and finally, multi-step prediction of the battery SOH is realized. Compared with the prediction results of battery SOH using other neural network methods, the method proposed in this study can provide higher prediction accuracy and achieve accurate multi-step prediction of battery SOH. Measured results show that most of the multi-step prediction errors of the proposed method are controlled within 3%.

Recurrent autoencoder model for unsupervised seismic facies analysis

Machine learning-based automatic seismic facies analysis has increased significantly over the past few decades. The key is to select the most representative features (such as the commonly used poststack amplitude data or the induced seismic attributes) as the input of the different machine learning algorithms. As an advanced branch of machine learning, deep learning can be used to extract the discriminatively deep features from seismic data similar to those used in image classification. In this study, we havedeveloped an unsupervised seismic facies analysis method by using a recurrent autoencoder model. First, we have constructed and trained an autoencoder architecture combined with long short-term memory-based recurrent operation. Its main aim is to learn the deep discriminative features by taking the windowed poststack seismic data as the input time series data. This type of unsupervised learning takes advantage of the no labeling requirement of seismic data. In addition, the recurrent operation is beneficial in delineating the time-sequential characteristics of seismic data. Second, we have taken the learned features as the input of simple K-means clustering and analyzed the corresponding seismic facies. In other words, the clustering is executed in the learned feature space (learned feature-based clustering). Real data results have demonstrated that our method reveals more details than the original amplitude-based K-means clustering, depending on the cluster calibration. In particular, according to the known natural gamma-ray logs and lithological descriptions of five wells, different amounts of sandstone and mudstone deposit are more accurately discriminated, which is substantially informative in reservoir prediction and hydrocarbon exploration. Furthermore, the estimated average silhouette scores have quantitatively shown the effectiveness of our method.

Early identification of crop types using Sentinel-2 satellite images and an incremental multi-feature ensemble method (Case study: Shahriar, Iran)

Thematic crop-type maps provide helpful information for the decision-makers to ensure society’s food security, control market prices, impose new export or import restrictions, and effective disaster management. This article aims to identify the cultivated products (wheat, barley, alfalfa, and rapeseed) in Shahriar’s farmlands before cropping season ends using time-series of Sentinel-2 satellite images and a novel multi-feature ensemble classifier. The proposed classifier consisted of 3 steps: 1-ensemble generation, 2- ensemble pruning, 3- ensemble integration. First, different spectral indices were derived from the input time-series of Sentinel-2 images, and we trained two popular machine learning classifiers, Random Forest (RF) and Support Vector Machine (SVM), for each index. In the pruning step, Q- statistics analysis selected the best candidate classifiers to be ensembled. The final decision was made using weighted average integration of candidate classifiers based on overall accuracy. We used this classifier in an incremental process that determines the shortest time series enabling early identification of different crops. The results indicate the superiority of the proposed method in early crop-type identification as it achieved at least 9.56% and 4.01% higher overall accuracy than the best base classifier and the best classifier trained by all indices, respectively. Ensemble pruning also resulted in an increase of at least 2.58% in overall accuracy. In addition, the proposed classifier could identify different agricultural products more than two months before the end of the cropping season with an overall accuracy of 80.12%.

Incorporating Snowmelt into Daily Estimates of Recharge Using a State-Space Model of Infiltration.

A state-space model (SSM) of infiltration estimates daily groundwater recharge using time-series of groundwater-level altitude and meteorological inputs (liquid precipitation, snowmelt, and evapotranspiration). The model includes diffuse and preferential flow through the unsaturated zone, where preferential flow is a function of liquid precipitation and snowmelt rates and a threshold rate, above which there is direct recharge to the water table. Model parameters are estimated over seasonal periods and the SSM is coupled with the Kalman Filter (KF) to assimilate recent observations (hydraulic head) and meteorological inputs into recharge estimates. The approach can take advantage of real-time hydrologic and meteorological data to deliver real-time recharge estimates. The model is demonstrated on daily observations from two bedrock wells in carbonate aquifers of northwestern New York (USA) between 2013 and 2018. Meteorological inputs for liquid precipitation and snowmelt are compiled from SNODAS (2021). Results for recharge during winter and spring seasons show preferential flow events to the water table from liquid precipitation, snowmelt, or a combination of the two. Recharge estimates summed annually are consistent with previous estimates of recharge reported from groundwater flow and surface-process models. Results from the SSM and KF point to errors in meteorological inputs, such as the snowmelt rate, that are not compatible with hydraulic head observations. Whereas liquid and solid precipitation are measured at discrete stations and extrapolated to 1-km2 grid cells, snowmelt is a meteorological modeled outcome that may not represent conditions in the vicinity of monitoring well locations.

District heating load prediction algorithm based on bidirectional long short-term memory network model

Heating load prediction based on machine learning algorithms has received increasing attention, especially the Long Short Term Memory (LSTM) network, have been shown to have a superior performance in predicting the heat load consumption. However, most of the current research reports on load prediction models using LSTM models are focused on the unidirectional (Uni-LSTM) network. In this paper, a bidirectional (Bi-LSTM) network for heat load prediction is proposed to make full use of the model hyperparameters to obtain the optimal model and to fully compare with the Uni-LSTM model, and the Bi-LSTM model can improve the prediction accuracy of heat load in a district heating system by using both past and future weather information. In addition, the two types of models are set up with different depth-stacked layers, and for each of the proposed models, a hyperparametric optimization tool has been used to obtain the best model. The results indicate that the increase in depth-stacked LSTM layers has no significant improvement in the prediction accuracy. The input time series length reflects the inertia influence duration of the district heating system, and the optimal model can be obtained for different settings of input time series length. The best optimally configured models were compared, and the single-layer Bi-LSTM model outperformed the single-layer Uni-LSTM model by 19.56%, 16.43%, 14.16%, and 20.69% in terms of the Root Mean Square Error (RMSE), Mean Average Percentage Error (MAPE), Mean Absolute Error (MAE), and the coefficient of variation of the RMSE (CV-RMSE), respectively.

Domain-Agnostic Outlier Ranking Algorithms—A Configurable Pipeline for Facilitating Outlier Detection in Scientific Datasets

Automatic detection of outliers is universally needed when working with scientific datasets, e.g., for cleaning datasets or flagging novel samples to guide instrument acquisition or scientific analysis. We present Domain-agnostic Outlier Ranking Algorithms (DORA), a configurable pipeline that facilitates application and evaluation of outlier detection methods in a variety of domains. DORA allows users to configure experiments by specifying the location of their dataset(s), the input data type, feature extraction methods, and which algorithms should be applied. DORA supports image, raster, time series, or feature vector input data types and outlier detection methods that include Isolation Forest, DEMUD, PCA, RX detector, Local RX, negative sampling, and probabilistic autoencoder. Each algorithm assigns an outlier score to each data sample. DORA provides results interpretation modules to help users process the results, including sorting samples by outlier score, evaluating the fraction of known outliers in n selections, clustering groups of similar outliers together, and web visualization. We demonstrated how DORA facilitates application, evaluation, and interpretation of outlier detection methods by performing experiments for three real-world datasets from Earth science, planetary science, and astrophysics, as well as one benchmark dataset (MNIST/Fashion-MNIST). We found that no single algorithm performed best across all datasets, underscoring the need for a tool that enables comparison of multiple algorithms.

Long-term multi-step ahead forecasting of root zone soil moisture in different climates: Novel ensemble-based complementary data-intelligent paradigms

The root zone soil moisture (RZSM) is essential for monitoring and forecasting agricultural, hydrological, and meteorological systems. Accordingly, researchers are determined to improve robust machine learning (ML) models to increase the accuracy of the RZSM predictions. This paper designed new complementary forecasting paradigms hybridizing Empirical Wavelet Transform (EWT) and two modern ensemble-based ML models, namely, extreme gradient boosting (XGBoost) and categorical boosting (CatBoost), to forecast long-term multi-step ahead daily RZSM in very cold and very warm semi-arid regions of Iran. For this purpose, the required datasets consisting of soil properties and meteorological information were extracted from the satellite datasets during 2005–2020 for Ardabil and Minab sites. Afterward, the significant lags of RZSM time series and optimal influence candidate inputs were sought based on the partial autocorrelation function (PACF) and mutual information techniques, respectively. Selected lagged components of RZSM time series were decomposed using EWT into different sub-sequences and consequently concatenated with candidate inputs to feed the ensemble ML models to forecast one-, three-, and seven-day-ahead RZSM at each case study. The performance of EWT-CatBoost and EWT-XGBoost and their counterpart standalone approaches was firstly evaluated in forecasting one-, three-, and seven-day-ahead RZSM using satellite data in this study and their accuracy were compared with a standalone kernel ridge regression (KRR) and complementary EWT-KRR models based on several statistical metrics (e.g., correlation coefficient (R), root mean square error (RMSE), Nash–Sutcliffe model efficiency coefficient (NSE)) and diagnostic analysis. The outcomes of testing phase in Ardabil site ascertained that the EWT-CatBoost (for RZSM(t + 1), R= 0.9979, RMSE= 0.0019, and NSE= 0.9985; for RZSM(t + 3), R= 0.9934, RMSE= 0.0035, and NSE= 0.9885; for RZSM(t + 7), R= 0.9489, RMSE= 0.0109, and NSE= 0.8634) outperformed the other models. On the other hand, the EWT-XGBoost model according to its best results (for RZSM(t + 1), R= 0.9911, RMSE= 0.0064, and NSE= 0.9805; for RZSM(t + 3), R= 0.9807, RMSE= 0.0092, and NSE= 0.9589; for RZSM(t + 7), R= 0.9680, RMSE= 0.0120, and NSE= 0.9309) yielded the most promising accuracy in forecasting multi-step ahead daily RZSM followed by the EWT-CatBoost, and EWT-KRR, respectively.

Backdoor Attacks Against Transfer Learning With Pre-Trained Deep Learning Models

Transfer learning provides an effective solution for feasibly and fast customize accurate \textit{Student} models, by transferring the learned knowledge of pre-trained \textit{Teacher} models over large datasets via fine-tuning. Many pre-trained Teacher models used in transfer learning are publicly available and maintained by public platforms, increasing their vulnerability to backdoor attacks. In this paper, we demonstrate a backdoor threat to transfer learning tasks on both image and time-series data leveraging the knowledge of publicly accessible Teacher models, aimed at defeating three commonly-adopted defenses: \textit{pruning-based}, \textit{retraining-based} and \textit{input pre-processing-based defenses}. Specifically, (A) ranking-based selection mechanism to speed up the backdoor trigger generation and perturbation process while defeating \textit{pruning-based} and/or \textit{retraining-based defenses}. (B) autoencoder-powered trigger generation is proposed to produce a robust trigger that can defeat the \textit{input pre-processing-based defense}, while guaranteeing that selected neuron(s) can be significantly activated. (C) defense-aware retraining to generate the manipulated model using reverse-engineered model inputs. We launch effective misclassification attacks on Student models over real-world images, brain Magnetic Resonance Imaging (MRI) data and Electrocardiography (ECG) learning systems. The experiments reveal that our enhanced attack can maintain the $98.4\%$ and $97.2\%$ classification accuracy as the genuine model on clean image and time series inputs respectively while improving $27.9\%-100\%$ and $27.1\%-56.1\%$ attack success rate on trojaned image and time series inputs respectively in the presence of pruning-based and/or retraining-based defenses.

A Novel Soft Sensor Modeling Approach Based on Difference-LSTM for Complex Industrial Process

The main purpose of soft sensor modeling is to capture the dynamic nonlinear features between the easy-to-measure auxiliary variables and the difficult-to-measure key variables. However, in complex industrial process, it is a challenging work due to the too complicated relationships among the process variables and the base measurement problems. Recently, long short-term memory (LSTM) network shows powerful long-term feature extraction capabilities in complex industrial processes. LSTM focuses on the relationship between the input time series and the output. However, what we concern with are the impact of changes in secondary variables over time on the being detected key variables. In this article, a novel soft sensor modeling approach called difference long short-term memory network is proposed for key variables prediction in complex industrial process. In the method, dynamic information of the inputs is introduced to build a new network unit. Thus, the dynamic temporal features in difference variable and nonlinear features in sequential data are merged to improve the prediction performance. Effectiveness and superiority of the method are validated through detection of the particle size index for a grinding-classification process by comparing to other popular methods.

Deep Learning based Load Forecasting with Decomposition and Feature Selection Techniques

The forecasting of short term electricity load plays a vital role in power system. It is essential for the power system's reliable, secure, and cost-effective functioning. This paper contributes significantly for enhancing the accuracy of short term electricity load forecasting. It presents a hybrid forecasting model called Gated Recurrent Unit with Ensemble Empirical Mode Decomposition and Boruta feature selection (EBGRU). It is a hybrid model that addresses the non-stationary, non-linearity and noisy issues of the time series input by using Ensemble Empirical Mode Decomposition (EEMD). It also addresses overfitting and curse of dimensionality issues of load forecasting by identifying the pertinent features using Boruta wrapper feature selection. It effectively handles the uncertainty and temporal dependency characteristics of load and forecasts the future load using deep learning based Gated Recurrent Unit (GRU). The proposed EBGRU model is experimented by using European and Australian Electricity load datasets. The temperature has high correlation with load demand. In this study, both load and temperature features are considered for the accurate short term load forecasting. The experimental outcome demonstrates that the proposed EBGRU model outperforms other deep learning models such as RNN, LSTM, GRU, RNN with EEMD and Boruta (EBRNN) and LSTM with EEMD and Boruta (EBLSTM).

Machine-learning approach for predicting the occurrence and timing of mid-winter ice breakups on canadian rivers

The increasingly common occurrence of Mid-Winter Breakups (MWBs) in Canadian rivers, consisting of the early breakup of ice cover outside of the typical spring season, is a cause for concern. This study applied various data-driven modelling techniques to predict MWBs occurrence and timing with sufficient lead times on a national scale using a new Canadian River Ice Database (CRID) coupled with National Resources Canada gridded climate data. A two-level machine learning model structure was developed, with the first level model predicting MWB occurrence within a given period and the second level model predicting the timing of an MWB occurrence within that period. Machine learning techniques that can handle class imbalance were employed to address many of the issues inherent in rare event forecasting, including the implementation of data preprocessing, the selection of algorithms and performance metrics suited to rare events. Multiple configurations of both model levels, including variations on time series arrangement and input variables, were tested to select the optimal model structure. The best performing configuration, focussing on a biweekly time period, attained overall accuracies of 80.1% and 77.6% for the first and second level models respectively on the 452 MWBs in the CRID. In addition, probabilistic prediction results were analyzed to evaluate model uncertainty and robustness. This new modelling framework provides the first tool capable of predicting MWBs on a national scale, with easily extendable methodology to locations that have not yet experienced MWBs and can form the basis of vital decision-making support to affected communities.

Winter Wheat Yield Estimation Based on Optimal Weighted Vegetation Index and BHT-ARIMA Model

This study aims to use remote sensing (RS) time-series data to explore the intrinsic relationship between crop growth and yield formation at different fertility stages and construct a high-precision winter wheat yield estimation model applicable to short time-series RS data. Sentinel-2 images were acquired in this study at six key phenological stages (rejuvenation stage, rising stage, jointing stage, heading stage, filling stage, filling-maturity stage) of winter wheat growth, and various vegetation indexes (VIs) at different fertility stages were calculated. Based on the characteristics of yield data continuity, the RReliefF algorithm was introduced to filter the optimal vegetation index combinations suitable for the yield estimation of winter wheat for all fertility stages. The Absolutely Objective Improved Analytic Hierarchy Process (AOIAHP) was innovatively proposed to determine the proportional contribution of crop growth to yield formation in six different phenological stages. The selected VIs consisting of MTCI(RE2), EVI, REP, MTCI(RE1), RECI(RE1), NDVI(RE1), NDVI(RE3), NDVI(RE2), NDVI, and MSAVI were then fused with the weights of different fertility periods to obtain time-series weighted data. For the characteristics of short time length and a small number of sequences of RS time-series data in yield estimation, this study applied the multiplexed delayed embedding transformation (MDT) technique to realize the data augmentation of the original short time series. Tucker decomposition was performed on the block Hankel tensor (BHT) obtained after MDT enhancement, and the core tensor was extracted while preserving the intrinsic connection of the time-series data. Finally, the resulting multidimensional core tensor was trained with the Autoregressive Integrated Moving Average (ARIMA) model to obtain the BHT-ARIMA model for wheat yield estimation. Compared to the performance of the BHT-ARIMA model with unweighted time-series data as input, the weighted time-series input significantly improves yield estimation accuracy. The coefficients of determination (R2) were improved from 0.325 to 0.583. The root mean square error (RMSE) decreased from 492.990 to 323.637 kg/ha, the mean absolute error (MAE) dropped from 350.625 to 255.954, and the mean absolute percentage error (MAPE) decreased from 4.332% to 3.186%. Besides, BHT-ARMA and BHT-CNN models were also used to compare with BHT-ARIMA. The results indicated that the BHT-ARIMA model still had the best yield prediction accuracy. The proposed method of this study will provide fast and accurate guidance for crop yield estimation and will be of great value for the processing and application of time-series RS data.

A Fusion Transformer for Multivariable Time Series Forecasting: The Mooney Viscosity Prediction Case.

Multivariable time series forecasting is an important topic of machine learning, and it frequently involves a complex mix of inputs, including static covariates and exogenous time series input. A targeted investigation of this input data is critical for improving prediction performance. In this paper, we propose the fusion transformer (FusFormer), a transformer-based model for forecasting time series data, whose framework fuses various computation modules for time series input and static covariates. To be more precise, the model calculation consists of two parallel stages. First, it employs a temporal encoder–decoder framework for extracting dynamic temporal features from time series data input, which analyzes and integrates the relative position information of sequence elements into the attention mechanism. Simultaneously, the static covariates are fed to the static enrichment module, which is inspired by gated linear units, to suppress irrelevant information and control the extent of nonlinear processing. Finally, the prediction results are calculated by fusing the outputs of the above two stages. Using Mooney viscosity forecasting as a case study, we demonstrate considerable forecasting performance improvements over existing methodologies and verify the effectiveness of each component of FusFormer via ablation analysis, and an interpretability use case is conducted to visualize temporal patterns of time series. The experimental results prove that FusFormer can achieve accurate Mooney viscosity prediction and improve the efficiency of the tire production process.

Siamese Time Series and Difference Networks for Performance Monitoring in the Froth Flotation Process

Accurate and in-time performance monitoring plays a great role in industrial processes. However, since the labeled performances are usually measured by some special devices with a relatively long-time interval, the current deep neural networks for the performance monitoring always treat them only as an output. But actually, in the industrial process like froth flotation, labeled performance at the previous moment could also offer valuable information for performance monitoring at the current moment, especially under unstable working conditions. Therefore, we propose a Siamese time series and difference network (STS-D net), which integrates input features at different time steps and labeled performance at the previous moment effectively. In this article, the proposed STS-D net includes two sub-networks. One is the Siamese time series network, which aims to extract effective and uniform feature representations for the input time series at the current and previous moments; the other is the difference network, which integrates the feature representations of the two input time series with labeled performance at the previous moment to predict the performance at the current moment in an incremental way. Effectiveness of the proposed STS-D net is validated in a real-world froth flotation process.