Sequential Prediction Model Research Articles

Physics-informed neural networks for efficient aeroacoustic computational based on two time sequence prediction models

Physics-informed neural networks (PINNs) have been proven powerful for solving partial differential equations (PDEs). However, conventional PINNs fail handling time/space variables in the training data distinctly and overlook the temporal relationships among data points, which is critical for accurately modeling dynamic systems. To address this issue, this work proposes two novel PINN models based on the transformer and the convolutional long short-term memory (ConvLSTM) architectures, respectively. An encoder-decoder neural network structure is adopted to enhance handling of time-space variables and a high-order finite-difference method is used to compute partial derivatives. These computations are implemented using the convolution within the PyTorch framework that incorporates both physical and mathematical information into the neural networks. The efficacy of the proposed methods has been evaluated for the wave equation and the Reynolds-Averaged Navier-Stokes equations. Results suggest that the transformer-based PINN outperforms both the conventional and ConvLSTM-based PINN in terms of accuracy, efficiency and generalizability.

FluPMT: Prediction of Predominant Strains of Influenza A Viruses via Multi-Task Learning.

Seasonal influenza vaccines play a crucial role in saving numerous lives annually. However, the constant evolution of the influenza A virus necessitates frequent vaccine updates to ensure its ongoing effectiveness. The decision to develop a new vaccine strain is generally based on the assessment of the current predominant strains. Nevertheless, the process of vaccine production and distribution is very time-consuming, leaving a window for the emergence of new variants that could decrease vaccine effectiveness, so predictions of influenza A virus evolution can inform vaccine evaluation and selection. Hence, we present FluPMT, a novel sequence prediction model that applies an encoder-decoder architecture to predict the hemagglutinin (HA) protein sequence of the upcoming season's predominant strain by capturing the patterns of evolution of influenza A viruses. Specifically, we employ time series to model the evolution of influenza A viruses, and utilize attention mechanisms to explore dependencies among residues of sequences. Additionally, antigenic distance prediction based on graph network representation learning is incorporated into the sequence prediction as an auxiliary task through a multi-task learning framework. Experimental results on two influenza datasets highlight the exceptional predictive performance of FluPMT, offering valuable insights into virus evolutionary dynamics, as well as vaccine evaluation and production.

A blended framework for audio spoof detection with sequential models and bags of auditory bites

An automated speaker verification system uses the process of speech recognition to verify the identity of a user and block illicit access. Logical access attacks are efforts to obtain access to a system by tampering with its algorithms or data, or by circumventing security mechanisms. DeepFake attacks are a form of logical access threats that employs artificial intelligence to produce highly realistic audio clips of human voice, that may be used to circumvent vocal authentication systems. This paper presents a framework for the detection of Logical Access and DeepFake audio spoofings by integrating audio file components and time-frequency representation spectrograms into a lower-dimensional space using sequential prediction models. Bidirectional-LSTM trained on the bonafide class generates significant one-dimensional features for both classes. The feature set is then standardized to a fixed set using a novel Bags of Auditory Bites (BoAB) feature standardizing algorithm. The Extreme Learning Machine maps the feature space to predictions that differentiate between genuine and spoofed speeches. The framework is evaluated using the ASVspoof 2021 dataset, a comprehensive collection of audio recordings designed for evaluating the strength of speaker verification systems against spoofing attacks. It achieves favorable results on synthesized DeepFake attacks with an Equal Error Rate (EER) of 1.18% in the most optimal setting. Logical Access attacks were more challenging to detect at an EER of 12.22%. Compared to the state-of-the-arts in the ASVspoof2021 dataset, the proposed method notably improves EER for DeepFake attacks by an improvement rate of 95.16%.

Predictive Maintenance Based on Identity Resolution and Transformers in IIoT

The burgeoning development of next-generation technologies, especially the Industrial Internet of Things (IIoT), has heightened interest in predictive maintenance (PdM). Accurate failure forecasting and prompt responses to downtime are essential for improving the industrial efficiency. Traditional PdM methods often suffer from high false alarm rates and inefficiencies in complex environments. This paper introduces a predictive maintenance framework using identity resolution and a transformer model. Devices receive unique IDs via distributed identifiers (DIDs), followed by a state awareness model to assess device health from sensor signals. A sequence prediction model forecasts future signal sequences, which are then used with the state awareness model to determine future health statuses. Combining these predictions with unique IDs allows for the rapid identification of facilities needing maintenance. Experimental results show superior performance, with 99% accuracy for the state awareness model and a mean absolute error (MAE) of 0.062 for the sequence prediction model, underscoring the effectiveness of the framework.

A systematic method of long-sequence prediction of natural gas supply in IES based on spatio-temporal causal network of multi-energy

Natural gas plays an important role in the energy peak-shaving process of Integrated Energy Systems (IES). Accurate prediction of natural gas supply over longer time scales is one of the most important conditions to ensure the stability and security of IES energy supply. However, as the sequence length of the predicted time series increases, it becomes more difficult to extract valid dependencies from the data. At the same time, there are complex spatio-temporal coupling characteristics of various energy resources in the IES, which makes the construction of long sequence prediction models for natural gas supply very difficult. To solve this problem, we propose a systematic method for long-sequence prediction with natural gas supply in IES based on spatio-temporal causal network of multi-energy. First, the proposed approach uses a causal analysis algorithm to represent the complex coupled spatio-temporal relationships of multiple energy resources in an IES. Then, the natural gas time-series are decomposed through the Variational Modal Decomposition (VMD), to reveal its inherent trend and cyclical characteristics. Finally, the long-sequence prediction model is proposed by fusing the attention mechanism into Graph Convolutional Neural Network (GCN). Attentional mechanisms are introduced to obtain important long-term dependencies in the long sequence prediction process. We use a integrated energy dataset from Spain to test the validity and superiority of the method. The results show that the proposed model in this paper improves the Root Mean Square Error (RMSE) by at least 9.5% and reduces the Mean Absolute Error (MAE) by at least 14.5% compared to several baseline models.

Enhanced Air Quality Prediction Using a Coupled DVMD Informer-CNN-LSTM Model Optimized with Dung Beetle Algorithm.

Accurate prediction of air quality is crucial for assessing the state of the atmospheric environment, especially considering the nonlinearity, volatility, and abrupt changes in air quality data. This paper introduces an air quality index (AQI) prediction model based on the Dung Beetle Algorithm (DBO) aimed at overcoming limitations in traditional prediction models, such as inadequate access to data features, challenges in parameter setting, and accuracy constraints. The proposed model optimizes the parameters of Variational Mode Decomposition (VMD) and integrates the Informer adaptive sequential prediction model with the Convolutional Neural Network-Long Short Term Memory (CNN-LSTM). Initially, the correlation coefficient method is utilized to identify key impact features from multivariate weather and meteorological data. Subsequently, penalty factors and the number of variational modes in the VMD are optimized using DBO. The optimized parameters are utilized to develop a variationally constrained model to decompose the air quality sequence. The data are categorized based on approximate entropy, and high-frequency data are fed into the Informer model, while low-frequency data are fed into the CNN-LSTM model. The predicted values of the subsystems are then combined and reconstructed to obtain the AQI prediction results. Evaluation using actual monitoring data from Beijing demonstrates that the proposed coupling prediction model of the air quality index in this paper is superior to other parameter optimization models. The Mean Absolute Error (MAE) decreases by 13.59%, the Root-Mean-Square Error (RMSE) decreases by 7.04%, and the R-square (R2) increases by 1.39%. This model surpasses 11 other models in terms of lower error rates and enhances prediction accuracy. Compared with the mainstream swarm intelligence optimization algorithm, DBO, as an optimization algorithm, demonstrates higher computational efficiency and is closer to the actual value. The proposed coupling model provides a new method for air quality index prediction.

A novel method for settlement imputation and monitoring of earth-rockfill dams subjected to large-scale missing data

Various missing values inevitably exist in the settlement monitoring data of prolonged operation of earth-rockfill dams, which delays or even interrupts the dam structure analytical procedure. This study aims to develop a reliable method for addressing the missing data problem and monitoring earth-rockfill dam settlement behavior. First, an imputation model, support vector regression based on the finite element method and time input (FEMTSVR), is proposed and offers superior performance when capturing complex nonlinear mappings from environmental variables to settlement on small sample data. Herein, an improved particle swarm optimization algorithm (IPSO) is developed, realizing the nonlinear adjustment of weights and parameter reduction during hyperparameter optimization. Then, this study establishes a sequential prediction model based on gate recurrent unit (GRU) networks to monitor dam behavior on the imputed and complete dataset. Eventually, the proposed method is evaluated using real-world earth-rockfill dam monitoring data with the help of statistical indicators, demonstrating its efficiency in monitoring settlement data subjected to large-scale missingness. This study provides a robust database for dam structural health monitoring, while also providing a promising framework for the safety assessment of other civil or hydraulic engineering based on raw monitoring data.

A sequence to sequence prediction model for remaining useful life of lithium-ion batteries with Bayesian optimisation process visualization

Accurate battery remaining useful life (RUL) prediction plays an important role in ensuring reliable operation of electric vehicles. In this paper, a hybrid model based on Bayesian optimization of deep convolutional neural network and long short-term memory neural network (BO-DCNN-LSTM) is proposed for battery RUL prediction. Feature extraction of raw charging characteristic curves is performed by the multilayer CNN and preliminary capacity prediction is performed by the multilayer LSTM. The model performance is explored with different training, validation and testing strategies and different prediction starting points. Validation using NASA battery aging data shows that the mean absolute error (MAE) and root mean square error (RMSE) of the RUL prediction are 0.0139 Ah and 0.0195 Ah, respectively, when the prediction starting point is the 50th cycle. In addition, this paper visualizes the process of how the Bayesian optimization (BO) algorithm searches for the global optimal combinations in the high-dimensional hyperparameter space and discusses the impact of these hyperparameters on the prediction, filling the gap in this part of the research.

Formal Logic Enabled Personalized Federated Learning through Property Inference

Recent advancements in federated learning (FL) have greatly facilitated the development of decentralized collaborative applications, particularly in the domain of Artificial Intelligence of Things (AIoT). However, a critical aspect missing from the current research landscape is the ability to enable data-driven client models with symbolic reasoning capabilities. Specifically, the inherent heterogeneity of participating client devices poses a significant challenge, as each client exhibits unique logic reasoning properties. Failing to consider these device-specific specifications can result in critical properties being missed in the client predictions, leading to suboptimal performance. In this work, we propose a new training paradigm that leverages temporal logic reasoning to address this issue. Our approach involves enhancing the training process by incorporating mechanically generated logic expressions for each FL client. Additionally, we introduce the concept of aggregation clusters and develop a partitioning algorithm to effectively group clients based on the alignment of their temporal reasoning properties. We evaluate the proposed method on two tasks: a real-world traffic volume prediction task consisting of sensory data from fifteen states and a smart city multi-task prediction utilizing synthetic data. The evaluation results exhibit clear improvements, with performance accuracy improved by up to 54% across all sequential prediction models.

A cascading model for nudging employees towards energy-efficient behaviour in tertiary buildings.

Energy-related occupant behaviour in the built environment is considered crucial when aiming towards Energy Efficiency (EE), especially given the notion that people are most often unaware and disengaged regarding the impacts of energy-consuming habits. In order to affect such energy-related behaviour, various approaches have been employed, being the most common the provision of recommendations towards more energy-efficient actions. In this work, the authors extend prior research findings in an effort to automatically identify the optimal Persuasion Strategy (PS), out of ten pre-selected by experts, tailored to a user (i.e., the context to trigger a message, allocate a task or providing cues to enact an action). This process aims to successfully influence the employees' decisions about EE in tertiary buildings. The framework presented in this study utilizes cultural traits and socio-economic information. It is based on one of the largest survey datasets on this subject, comprising responses from 743 users collected through an online survey in four countries across Europe (Spain, Greece, Austria and the UK). The resulting framework was designed as a cascade of sequential data-driven prediction models. The first step employs a particular case of matrix factorisation to rank the ten PP in terms of preference for each user, followed by a random forest regression model that uses these rankings as a filtering step to compute scores for each PP and conclude with the best selection for each user. An ex-post assessment of the individual steps and the combined ensemble revealed increased accuracy over baseline non-personalised methods. Furthermore, the analysis also sheds light on important user characteristics to take into account for future interventions related to EE and the most effective persuasion strategies to adopt based on user data. Discussion and implications of the reported results are provided in the text regarding the flourishing field of personalisation to motivate pro-environmental behaviour change in tertiary buildings.

High-Risk Sequence Prediction Model in DNA Storage: The LQSF Method.

Traditional DNA storage technologies rely on passive filtering methods for error correction during synthesis and sequencing, which result in redundancy and inadequate error correction. Addressing this, the Low Quality Sequence Filter (LQSF) was introduced, an innovative method employing deep learning models to predict high-risk sequences. The LQSF approach leverages a classification model trained on error-prone sequences, enabling efficient pre-sequencing filtration of low-quality sequences and reducing time and resources in subsequent stages. Analysis has demonstrated a clear distinction between high and low-quality sequences, confirming the efficacy of the LQSF method. Extensive training and testing were conducted across various neural networks and test sets. The results showed all models achieving an AUC value above 0.91 on ROC curves and over 0.95 on PR curves across different datasets. Notably, models such as Alexnet, VGG16, and VGG19 achieved a perfect AUC of 1.0 on the Original dataset, highlighting their precision in classification. Further validation using Illumina sequencing data substantiated a strong correlation between model scores and sequence error-proneness, emphasizing the model's applicability. The LQSF method marks a significant advancement in DNA storage technology, introducing active sequence filtering at the encoding stage. This pioneering approach holds substantial promise for future DNA storage research and applications.

DSPM: Dual sequence prediction model for efficient energy management in micro-grid

Power generation and consumption predictions are fundamental for smart grid operations, addressing challenges posed by renewable energy variability and irregular consumer demand. This study introduces the Dual Sequence Predictive Model (DSPM) based on Spatiotemporal CNN (STCNN) architecture, offering a holistic solution for forecasting Electricity Generation (EG) and Electricity Consumption (EC) collectively. Leveraging STCNN, the DSPM efficiently extracts spatial and temporal information, improving prediction accuracy and processing speed. A 1D spatial attention module is added to capture crucial spatial dependencies in historical data. Incorporating shared historical weather information streamlines learning process and reduces model complexity. Extensive benchmark evaluations demonstrate the DSPM superior performance, achieving the lowest error rates compared to baseline models. The DSPM also employs the Kullback–Leibler Divergence (KLD) algorithm for power generation and consumption matching, ensuring efficient power distribution within smart and microgrids. Furthermore, the DSPM is evaluated alongside Single Sequence Prediction (SSP) models, providing a comprehensive analysis of its capabilities and comparisons with state-of-the-art models. The SSP model achieved an average RMSE reduction of 17.3% for solar EG data and 15.86% for IHEPC residential EC data, while DSPM reduced RMSE by 7.57% over PowerGrid dataset compared to baseline models.

A comprehensive CMR analysis improves risk stratification in left ventricular noncompaction

Abstract Background Left ventricular noncompaction (LVNC) is a heterogeneous and controversial entity. Risk assessment by cardiovascular magnetic resonance (CMR) remains poorly defined. Purpose To develop a CMR prediction model of major adverse cardiovascular events (MACE) in LVNC. Methods We conducted a retrospective longitudinal multicentre cohort study of LVNC patients with core-lab CMR analysis. The endpoint was a composite of MACE: heart failure, ventricular arrhythmias, systemic embolisms and all-cause death. The effect of variables on the endpoint was analysed by Cox regression and variables were dichotomized according to their association with the outcome. Three sequential prediction models were developed: a "traditional model" based on LV ejection fraction (LVEF); a "clinical model" based on daily-practice CMR variables (e.g.: LVEF, right ventricular (RV) EF, and late gadolinium enhancement (LGE)); and a "research model" including advanced parameters (e.g.: global longitudinal strain (GLS) and hemodynamic forces (HDF)). The best prediction models were chosen based on C-statistic and Akaike information criterion (AIC). A risk score was developed based on the third model, and according to proportional hazard ratios. Results A total of 348 patients were included, age was 46 (19) years and 150 (43%) were female. LVEF was 48 (14) % and 32 (10%) had LGE. During a median follow-up of 3.9 (1.8-6.1) years, 54 (16%) patients developed MACE: 27 (8%) heart failure, 32 (9%) ventricular arrhythmias, 2 (1%) systemic embolisms and 6 (2%) deaths. Figure 1 shows the association of the main CMR variables with the endpoint. The "traditional model" (dichotomic LVEF with cut-off point at 40%) had an AUC of 0.71 (95% CI: 0.64-0.78). The best "clinical model" was a combination of LVEF, RVEF (cut-off point 35%), left atrium (LA) volume index (cut-off point 53 mL/m2) and percentage of LGE mass (LGE%, cut-off point 5%), with an AUC of 0.77 (95% CI: 0.70-0.85; p=0.006 compared to the first model). The best "research model" included LVEF, RVEF, LGE%, LA GLS (two cut-off points 17% and 28%), and lateral-septal HDF (cut-off point 3%), with an AUC of 0.85 (95% CI: 0.80-0.90; p=0.007 compared to the second model) (Figure 2a). Subsequently, a risk score was developed: 1 point was assigned to LVEF<40%, RVEF<35%, LGE% >5% and LA-GLS <28%, 2 points to LA-GLS <17% and 4 points to HDF<3%. The score was accurate at identifying both low- (0-4 points), intermediate- (5-7 points) and high-risk (8-9 points) patients (Figure 2b). Conclusions A comprehensive CMR evaluation in LVNC allows for precise risk stratification. Our proposed prediction model could be used to individualise patients’ management, which might improve long-term outcomes.CMR characteristics according to MACEPrediction models of MACE and risk score

Research on time sequence prediction of supersonic cascade flow field based on compressed sensing artificial neural network

The aviation engine that achieves efficient energy conversion is the core power equipment for low-carbon navigation, and the thermal components represented by the combustion chamber greatly affect the stable state of the compressor cascade flow field. Accurate, comprehensive and promptly monitoring of the compressor cascade flow state is related to the safe operation of the engine. A flow field time sequence prediction framework named compressed convolutional gate recurrent unit (CC-GRU) was proposed in this study to predict future supersonic cascade flow parameters based on the flow field at previous moments. CC-GRU embedded convolution into gate recurrent unit (GRU) to deal with the complex spatial-temporal behavior of supersonic cascade flow field. Firstly, unsteady numerical simulations driven by linear changes in back pressure were carried out, and the dataset for model training and validation was obtained to verify the feasibility of time sequence prediction of the cascade flow field. The verification results indicate that the framework can comprehensively predict the flow field with high back pressure in the future according to the flow field with low back pressure in the past period, and further validated the effectiveness of the proposed time sequence prediction model in ground wind tunnel experiments. The CC-GRU model can accurately capture the fine shock wave structure and flow separation zone, with a relative error of less than 10% in predicting the pressure field, and is mainly concentrated within the shock wave structure. Therefore, this research provides new research perspective and technical support for comprehensive and promptly condition monitoring of supersonic cascade flow field.

Personalized event prediction for Electronic Health Records

Clinical event sequences consist of hundreds of clinical events that represent records of patient care in time. Developing accurate predictive models of such sequences is of a great importance for supporting a variety of models for interpreting/classifying the current patient condition, or predicting adverse clinical events and outcomes, all aimed to improve patient care. One important challenge of learning predictive models of clinical sequences is their patient-specific variability. Based on underlying clinical conditions, each patient’s sequence may consist of different sets of clinical events (observations, lab results, medications, procedures). Hence, simple population-wide models learned from event sequences for many different patients may not accurately predict patient-specific dynamics of event sequences and their differences. To address the problem, we propose and investigate multiple new event sequence prediction models and methods that let us better adjust the prediction for individual patients and their specific conditions. The methods developed in this work pursue refinement of population-wide models to subpopulations, self-adaptation, and a meta-level model switching that is able to adaptively select the model with the best chance to support the immediate prediction. We analyze and test the performance of these models on clinical event sequences of patients in MIMIC-III database.

Integrating the traffic science with representation learning for city-wide network congestion prediction

Recent studies on traffic congestion prediction have paved a promising path towards the reduction of potential economic and environmental loss. However, at the city-wide scale, current approaches face substantial hurdles, such as being unable to support the multiple sensors modalities, insufficient congestion fluctuation and propagation modeling, and weak generalization to heterogeneous traffic network structures. To address these pitfalls, this paper investigates how to integrate the missing urban science domain priors into a general sequential prediction model, and proposes the customized Traffic-informed Transformer (TinT). To prevent receptive field bias, a novel mixture of long and short range information routing mechanism is proposed with the traffic-informed tokenization. To capture the unbalanced traffic flow propagation, an original anisotropic graph aggregation is developed to differentiate the traffic fluctuation based on orientations. Extensive results demonstrated TinT’s outstanding performance over other twelve state-of-the-art models and its broad applicability to multiple data modalities in six well-known cities throughout the world. We released our implementations at: https://github.com/VITA-Group/TinT.

Time Series Analysis Based on Informer Algorithms: A Survey

Long series time forecasting has become a popular research direction in recent years, due to the ability to predict weather changes, traffic conditions and so on. This paper provides a comprehensive discussion of long series time forecasting techniques and their applications, using the Informer algorithm model as a framework. Specifically, we examine sequential time prediction models published in the last two years, including the tightly coupled convolutional transformer (TCCT) algorithm, Autoformer algorithm, FEDformer algorithm, Pyraformer algorithm, and Triformer algorithm. Researchers have made significant improvements to the attention mechanism and Informer algorithm model architecture in these different neural network models, resulting in recent approaches such as wavelet enhancement structure, auto-correlation mechanism, and depth decomposition architecture. In addition to the above, attention algorithms and many models show potential and possibility in mechanical vibration prediction. In recent state-of-the-art studies, researchers have used the Informer algorithm model as an experimental control, and it can be seen that the algorithm model itself has research value. The informer algorithm model performs relatively well on various data sets and has become a more typical algorithm model for time series forecasting, and its model value is worthy of in-depth exploration and research. This paper discusses the structures and innovations of five representative models, including Informer, and reviews the performance of different neural network structures. The advantages and disadvantages of each model are discussed and compared, and finally, the future research direction of long series time forecasting is discussed.

LASSO and attention-TCN: a concurrent method for indoor particulate matter prediction.

Long time exposure to indoor air pollution environments can increase the risk of cardiovascular and respiratory system damage. Most previous studies focus on outdoor air quality, while few studies on indoor air quality. Current neural network-based methods for indoor air quality prediction ignore the optimization of input variables, process input features serially, and still suffer from loss of information during model training, which may lead to the problems of memory-intensive, time-consuming and low-precision. We present a novel concurrent indoor PM prediction model based on the fusion model of Least Absolute Shrinkage and Selection Operator (LASSO) and an Attention Temporal Convolutional Network (ATCN), together called LATCN. First, a LASSO regression algorithm is used to select features from PM1, PM2.5, PM10 and PM (>10) datasets and environmental factors to optimize the inputs for indoor PM prediction model. Then an Attention Mechanism (AM) is applied to reduce the redundant temporal information to extract key features in inputs. Finally, a TCN is used to forecast indoor particulate concentration in parallel with inputting the extracted features, and it reduces information loss by residual connections. The results show that the main environmental factors affecting indoor PM concentration are the indoor heat index, indoor wind chill, wet bulb temperature and relative humidity. Comparing with Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) approaches, LATCN systematically reduced the prediction error rate (19.7% ~ 28.1% for the NAE, and 16.4% ~ 21.5% for the RMSE) and improved the model running speed (30.4% ~ 81.2%) over these classical sequence prediction models. Our study can inform the active prevention of indoor air pollution, and provides a theoretical basis for indoor environmental standards, while laying the foundations for developing novel air pollution prevention equipment in the future.

NOx emission prediction of coal-fired power units under uncertain classification of operating conditions

Coal-fired power units are widely involved in peak shaving in the future renewable energy grid, leading to frequent changes in operating conditions, which makes it more difficult to predict NOx emissions. Aiming to address this challenge, a NOx emission prediction method based on a deep learning algorithm and uncertainty classification is proposed. Firstly, in order to better capture the intrinsic information in the NOx time series data, the deep learning network is used to construct the original NOx sequence prediction model. Secondly, a prediction error classification model based on a support vector machine is proposed to consider the uncertain working conditions and prediction errors of thermal power units. Then, the original sequence prediction results are corrected at different levels to obtain the final prediction results. The simulation experiments were compared with the operation data of a 660 MW subcritical boiler in the thermal power unit. Compared with several traditional models, the mean absolute percentage error, root mean square error, mean absolute error, and determination coefficient of the proposed model are 2.60 %, 11.09 mg/m3, and 8.27 mg/m3 and 0.86, respectively, which has higher prediction accuracy.

Shaping energy cost management in process industries through clustering and soft sensors

With the ever-increasing growth of energy demand and costs, process monitoring of operational costs is of great importance for process industries. In this light, both financial budget management and local operational optimization supposed to be guaranteed properly. To achieve this goal, a support vector machine recursive feature elimination (SVM-RFE) method together with clustering algorithm was developed to extract features while serving as importance measurements of each input variable for the sequential prediction model construction. Then, the four variants of autoregressive and moving average (ARMA), i.e., ARMA with exogenous input (ARMAX) based on recursive least squares algorithm (RLS), ARMAX based on recursive extended least squares algorithm (RELS), nonlinear auto-regressive neural network (NARNN) and nonlinear auto-regressive neural network with exogenous input (NARXNN), were applied, respectively, to predict the costs incurred in the daily production for process industries. The methods were validated in the Benchmark Simulation Model No.2-P (BSM2-P) and a practical data set about steel industry energy consumption from an open access database (University of California, Irvine (UCI)), respectively. The nonlinear model, NARXNN, was validated to achieve better performance in terms of mean square error (MSE) and correlation coefficient (R), when used for multi-step prediction of the aforementioned datasets with strong nonlinear and coupled characteristics.