Series Neural Network Research Articles

Comparing classic time series models and state-of-the-art time series neural networks for forecasting as-fired liquor properties

Abstract The properties of as-fired black liquor dictate kraft recovery boiler operation. If these properties could be forecasted, operations could be adjusted to optimize boiler performance. Here, we compare the performances of classic time series models and two state-of-the-art time series neural networks for forecasting as-fired liquor heating value, viscosity, and boiling point rise at a Canadian mill. Additionally, we show that, like classic time series models, autoregressive neural networks can be regarded as functions of unknown disturbances, which is useful in comparing model complexities. Our results show that classic time series models can accurately forecast as-fired liquor properties and that classic time series models perform comparably to state-of-the-art time series neural networks. We suspect this is due to the high autocorrelation of mill data that results from frequent measurements relative to long residence times. This autocorrelation is suspected to attenuate the cross-correlations between upstream disturbances and as-fired liquor properties. As a result, neural networks, which are useful for accommodating non-linear cross-correlations and dynamics, struggle to outperform classic time series models and may not always be appropriate for forecasting chemical process parameters.

Comparative analysis of time series neural network methods for three-way catalyst modeling

Relative Oxygen Level of the Three-Way Catalyst is an important parameter that affects the conversion efficiency of pollutants. ROL is a time-varying hidden state variable that is difficult to directly observe in practice. Therefore, it is common to use a method of clearing oxygen storage to simplify control in vehicles. However, this method negates the positive effects of ROL on pollutant treatment. ROL can be indirectly observed through modeling methods. Chemical modeling methods involve extensive computational requirements that cannot meet the demands of practical control. In contrast, time-series neural networks offer computational speed advantages when dealing with similar problems. Therefore, the ROL observation models using both NARX and LSTM neural networks are developed and compared in this study. The results indicate that the NARX neural network exhibits higher precision with a smaller number of neurons and time steps. The LSTM neural network demonstrates greater stability when dealing with data error fluctuations. In practical applications, the ROL model can monitor the TWC operating status and assist in the development of intelligent pollutant aftertreatment control strategies.

News-driven stock market index prediction based on trellis network and sentiment attention mechanism

Applying deep learning, especially time series neural network, to stock market prediction, has become one of the important applications in the quantitative finance field. However, due to the multi-correlation and volatility of the stock market, how to timely and accurately predict it has become a challenging issue. In order to cope with this challenge, a news-driven stock market index prediction model based on TrellisNet and a sentiment attention mechanism (SA-TrellisNet) is proposed. A sentiment analysis model based on CNN and LSTM is presented to obtain the sentiment index of massive news crawled from authoritative financial websites. Furthermore, a sentiment attention mechanism is designed for data fusion of stock data and news sentiment index as the input of the simple and efficient TrellisNet network for model training and prediction. The performance of our model is systematically evaluated using seven major international stock market indices including S&P500, NYSE, DJI, NASDAQ, FTSE 100, Nikkei 225 and SSE, and comparative experiments demonstrate that SA-TrellisNet is competitive to the other state-of-the-art methods in predicting stock market indices.

Predicting the Posture of High-Rise Building Machines Based on Multivariate Time Series Neural Network Models.

High-rise building machines (HBMs) play a critical role in the successful construction of super-high skyscrapers, providing essential support and ensuring safety. The HBM's climbing system relies on a jacking mechanism consisting of several independent jacking cylinders. A reliable control system is imperative to maintain the smooth posture of the construction steel platform (SP) under the action of the jacking mechanism. Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Temporal Convolutional Network (TCN) are three multivariate time series (MTS) neural network models that are used in this study to predict the posture of HBMs. The models take pressure and stroke measurements from the jacking cylinders as inputs, and their outputs determine the levelness of the SP and the posture of the HBM at various climbing stages. The development and training of these neural networks are based on historical on-site data, with the predictions subjected to thorough comparative analysis. The proposed LSTM and GRU prediction models have similar performances in the prediction process of HBM posture, with medians R2 of 0.903 and 0.871, respectively. However, the median MAE of the GRU prediction model is more petite at 0.4, which exhibits stronger robustness. Additionally, sensitivity analysis showed that the change in the levelness of the position of the SP portion of the HBM exhibited high sensitivity to the stroke and pressure of the jacking cylinder, which clarified the position of the cylinder for adjusting the posture of the HBM. The results show that the MTS neural network-based prediction model can change the HBM posture and improve work stability by adjusting the jacking cylinder pressure value of the HBM.

การพยากรณ์ยอดขายและความต้องการสินค้าในอุตสาหกรรมผลิตอุปกรณ์ไฟฟ้า กรณีศึกษา บริษัทผลิตอุปกรณ์ไฟฟ้าแรงสูง

The purpose of this study is to find the product demand forecasting model from the artificial neural network method in a case study of a high-voltage equipment manufacturing company. And reduce fines for late delivery of goods by using the Matlab program. The forecasting model is established by using time series neural network method, and the nonlinear autoregressive neural network (NARX) function model is established by combining with external input data. Then the forecast deviation is the average deviation percentage error (MAPE) uses the least dynamics model. By using data that collects product sales from 2019 – 2022 total 48 set of data are divided into 2 data sets used in the study. by the first dataset is learning 36 datasets and the second dataset is learning 12 datasets. The model of this study was defined with the number of hidden layer nodes as 2, 6 and 10. The time delay was 2 and 3. Forecasting models are combined into 6 models. The best model result of each item from NARX forecast through the learning process Levenberg-Marquardt (trainlm), including models 1, 4, 5, 4, 3, 3, and 2 sequential The results show the least average absolute percentage error at 8.32, 5.85, 2.61, 3.39, 2.91, 2.19 and 7.76 and the results of the forecasting model were analyzed to reduce fines from selling all 7 items Able to reduce penalties for late delivery of 4 items showing the percentage of difference in cost reduction equal to 0.28% 0.24% 0.32% and 0.22%. It can be concluded that the prediction of the neural network inverted neural network model It can be used to forecast product sales and product demand as well. And reduce the cost of late delivery fines for better results.

Strain Gauge Neural Network-Based Estimation as an Alternative for Force and Torque Sensor Measurements in Robot Manipulators

When a robotic manipulator interacts with its environment, the end-effector forces need to be measured to assess if a task has been completed successfully and for safety reasons. Traditionally, these forces are either measured directly by a 6-dimensional (6D) force–torque sensor (mounted on a robot’s wrist) or by estimation methods based on observers, which require knowledge of the robot’s exact model. Contrary to this, the proposed approach is based on using an array of low-cost 1-dimensional (1D) strain gauge sensors mounted beneath the robot’s base in conjunction with time series neural networks, to estimate both the end-effector 3-dimensional (3D) interaction forces as well as robot joint torques. The method does not require knowledge of robot dynamics. For comparison reasons, the same approach was used but with 6D force sensor measurements mounted beneath the robot’s base. The trained networks showed reasonably good performance, using the long-short term memory (LSTM) architecture, with a root mean squared error (RMSE) of 1.945 N (vs. 2.004 N; 6D force–torque sensor-based) for end-effector force estimation and 3.006 Nm (vs. 3.043 Nm; 6D force–torque sensor-based) for robot joint torque estimation. The obtained results for an array of 1D strain gauges were comparable with those obtained with a robot’s built-in sensor, demonstrating the validity of the proposed approach.

Predicting global energy demand for the next decade: A time-series model using nonlinear autoregressive neural networks

Energy demand forecasting has been an indispensable research target for academics, which has led to creative solutions for energy utilities in terms of power system design, control, and planning. The usefulness of energy demand forecasting is confined to the power engineering industry but globally exceeds such outcomes to contribute to the environment and health sectors. Despite the large number of research projects published on this topic, the challenge of energy demand forecasting still exists, especially with the developments in modeling concepts via artificial intelligence, which motivates more attractive solutions for the variables involved in energy demand forecasting. Mathematical correlation or extrapolation-like methods cannot be effective in all situations; however, when a time series neural network is presented, most statistical, empirical, and theoretical problems can be easily handled. This paper presents a simple and easy-to-understand method for the next decade of energy demand forecasting based on a nonlinear autoregressive (NAR) neural network. From its time series past values, NAR structurally is an optimal predictor for a future variable. A publicly available data set for global energy consumption has been used to construct the network model with sufficiently accurate results. The evidence has appeared in precisely following the exponential trend of energy consumption as well as the regressions for training, testing, and validation, which ensures the model's robustness and avoids getting involved in overfitting. The proposed model concepts and results can be easily used in undergraduate engineering education, training graduates, and future research.

A novel Fourier series-neural network hybrid approach for the boundary supporting stiffness identification of a flexible beam

A novel Fourier series-neural network hybrid approach for the boundary supporting stiffness identification of a flexible beam

A new approach based on current controlled hybrid power compensator for power quality improvement using time series neural network

In this paper, a current controlled-hybrid power compensator (CC-HPC) is presented to reduce the effect of input current harmonics on battery chargers. Passive filters have significant power loss and degrade system frequency due to excessive harmonic attenuation. The proposed system integrates the Higher Order Sliding Mode Controller (HOSMC) with a generalized form of p–q power theory and a Time Series – Artificial Neural Network (TS-ANN) is used to produce compensating reference current for a three-phase system and generates DC link inductor current. Switching pulses to Current Controlled-Active Power Compensator (CC-APC) switches are generated using a reference compensated signal. The development of CC-HPC and its control approach helps to reduce the overall harmonic distortion of the supply current used in battery chargers are the main contributions of the proposed system. HOSMC is a robust and adaptable controller that tracks reference current without causing chattering is the significant advantage of the proposed method. The control algorithm is designed in MATLAB/SIMULINK software for various load conditions and the experimental setup has been developed for rectified fed RC load using TS-ANN. The filtering process of CC-HPC can maintain the harmonic distortion of supply current within the IEEE 519-2014 standard.

Provide Bitcoin's daily optimal trading strategy based on LSTM

The purpose of this report is to develop a model that gives the best daily trading strategy based only on the price data before the day. We want to provide traders with strategies that will enable them to maximize their returns.we first utilize the LSTM time series neural network model using the first 956 data sets are used for training , and the subsequent 300 test sets are used to evaluate the model accuracy, calculate the prediction error and word comparison plots, and assess the difference between the prediction performance of the model and the actual value,This method has a better credibility. Finally, based on this model, we construct the optimal trading strategy model with the maximum value of total assets held as the objective function, add constraints, and finally arrive at an investment value of $1000.

Simultaneous Quantitative Analysis of Salmeterol and Fluticasone in Inhalation Spray Using HPLC and a Fast Spectrophotometric Technique Combined with a Time Series Neural Network and Multivariate Calibration Methods.

Chromatographic methods have been used for the simultaneous determination of salmeterol (SMT) and fluticasone (FLU). The methods take a lot of analysis time, need sample pre-treatment and large amounts of solvents, and are costly. The aim of this paper was to propose a simple, quick, and low-cost method for the determination of SMT and FLU using a time series neural network and multivariate calibration methods, including partial least-squares (PLS) and principal component regression (PCR). The simultaneous spectrophotometric determination of SMT and FLU in binary mixtures and anti-asthma spray was performed by applying multivariate calibration methods and an intelligent approach. The coefficient of determination (R2) of the time series neural network was obtained: 1 and 0.9997 for SMT and FLU, respectively. The mean recovery of PLS was found to be 99.29% and 99.84% for SMT and FLU, respectively. Also, the mean recovery related to the PCR method was 102.05% and 103.72% for SMT and FLU, respectively. The inhalation spray was assessed using HPLC and the results were compared with chemometrics methods via an analysis of variance (ANOVA) test. Intelligent and multivariate calibration methods were proposed. Simultaneous spectrophotometric determination of salmeterol and fluticasone was studied in an anti-asthma spray. HPLC as a reference method was performed and compared with chemometrics methods. Rapid, simple, low-cost, and accurate are the benefits of the proposed approaches.

Data-driven multi-joint waveguide bending sensor based on time series neural network.

Due to the bulky interrogation devices, traditional fiber optic sensing system is mainly connected by wire or equipped only for large facilities. However, the advancement in neural network algorithms and flexible materials has broadened its application scenarios to bionics. In this paper, a multi-joint waveguide bending sensor based on color dyed filters is designed to detect bending angles, directions and positions. The sensors are fabricated by casting method using soft silicone rubber. Besides, required optical properties of sensor materials are characterized to better understand principles of the sensor design. Time series neural networks are utilized to predict bending position and angle quantitatively. The results confirm that the waveguide sensor demodulated by the data-driven neural network algorithm performs well and can be used for engineering applications.

Neural network-based monitoring of critical parameters in drinking water for enhanced quality assurance

Every nation has its own approved parameters for water quality standards provided by the World Health Organization (WHO). These parameters undergo variations based on geography. The Sultanate of Oman has listed prime important parameters for unbottled drinking water according to Oman Standards. Currently, the utilization of advanced Internet of Things (IoT) technology involves numerous internal measurements through multiple sensors. These sensors continuously monitor water quality indicators such as pH levels, turbidity, and microbial content, ensuring that the drinking water meets the stringent standards set by the Sultanate of Oman. This integration of IoT technology enhances real-time data collection and analysis, facilitating prompt responses to any deviations from the prescribed water quality parameters. However, determining the specific purpose of water usage is not yet defined. Traditionally, water quality is typically examined only for physiochemical properties. However, given that water is a colorless liquid, its suitability for drinking cannot be solely determined based on these properties. By employing Time Series Neural Networks, we can identify the current parameters of water and display or transmit information wirelessly as a report. This report would include a comparison between the actual value and the measured value using recent sensors and instrumentation. Additionally, the Time Series Neural Networks allow for real-time monitoring, enabling swift detection of anomalies or fluctuations in water parameters, contributing to proactive decision-making in water management. The seamless integration of advanced technology not only enhances the accuracy of data analysis but also facilitates timely responses to potential environmental changes or emerging issues.

A Novel Outlier Detection Method of Long-Term Dam Monitoring Data Based on SSA-NAR

Outlier generally exists in dam monitoring data which may seriously affect the accuracy of dam safety evaluation results. Aiming at the outlier detection of dam monitoring data, a novel dynamic detection method of dam outlier data based on SSA-NAR is proposed. This combined method does not depend on the effect quantity and influence quantity relationship of traditional dam safety theory and only uses the time series of effect quantity to mine the variation, which can avoid the impact of missing or abnormal of the influence quantity. The Nonlinear Autoregression (NAR) is a classical time series neural network widely used in engineering field. However, the prediction accuracy of NAR is greatly affected by the selection of model parameters, the Sparrow Search Algorithm (SSA) which is a novel model parameter solution method and can be combined with NAR to derive the optimal parameters of NAR prediction model. The outlier is identified through the analysis of the residual distribution between the predicted data and the measured data. The case study shows that when the original data does not contain outliers, the prediction accuracy of the model is high. When the outlier is included, the proposed model has good robustness which the outlier has little influence on the prediction effect. It can effectively detect the outlier in the original dam monitoring data and provide a reliable data basis for dam safety evaluation.

Study on utilization of machine learning techniques for geological CO2 sequestration simulations

In Geological Carbon Sequestration (GCS), CO2 is injected and stored in geological formations. Conducting these studies through experiments could be highly impractical both economically and scientifically. Employing conventional numerical techniques was proven to be computationally intensive and slow with complex geometrical models. In this work, a study is conducted to integrate traditional numerical simulation results with machine learning techniques to forecast the futuristic trend of output parameters. Primary simulations are performed using multiphase flow modelling for the entire geological time scale. Then the time series neural network is used by considering CO2 sequestration parameter values as input and target data to forecast the trend of output parameters during the post-injection time. Both recurrent neural network models have shown a reasonable forecast of the output variables. Among the different training algorithms used, the Levenberg-Marquardt (LM) algorithm has given a good prediction for the output variable; the results are validated with the RMSE and R-squared values for obtained values. The R2 and RMSE values of the NAR model for structural trapping are 0.9801 and 0.0515, respectively, and for residual trapping, they are 0.9805 and 0.0506, respectively. This work will provide the initial understanding of integrated machine learning techniques in the GCS analysis for a heterogeneous reservoir model.

The Forecast of the Number of Police Cases Based on Time Series and Convolutional Neural Network Model

The Forecast of the Number of Police Cases Based on Time Series and Convolutional Neural Network Model

DMPCONV: DECOUPLING MULTI-BRANCH POINTWISE CONVOLUTIONS FOR LIGHT-WEIGHT REMOTE SENSING SCENE CLASSIFICATION

Abstract. The use of multi-branch architectures in off-the-shelf light-weight residual series neural networks can significantly improve their performance in remote sensing scene classification tasks. However, such architectures come at the expense of an increased number of parameters and calculations. In this paper, we propose the Decoupling Multi-branch Pointwise Convolutions (DMPConv), which works without a corresponding increase in parameters and calculations during inferencing, and at the same time, can maintain the same performance improvement ability as the multi-branch architectures. DMPConv can be decoupled into two states, the training-time DMPConv and the inferencing-time DMPConv. The training-time DMPConv enhances the expressivity of the network by using weighted multi-branch 1×1 convolutions. After training, we use structural reconstruction to convert the training-time DMPConv to the inferencing-time DMPConv, which has the same form as vanilla 1×1 convolution, so as to realize the inferencing-free. Extensive experiments were conducted on multiple remote sensing scene classification benchmarks, including Aerial Image data set and NWPU-RESISC45 data set to demonstrate the superiority of DMPConv.

Research on the Feasibility of Applying GRU and Attention Mechanism Combined with Technical Indicators in Stock Trading Strategies

The vigorous development of Time Series Neural Network in recent years has brought many potential possibilities to the application of financial technology. This research proposes a stock trend prediction model that combines Gate Recurrent Unit and Attention mechanism. In the proposed framework, the model takes the daily opening price, closing price, highest price, lowest price and trading volume of stocks as input, and uses technical indicator transition prediction as a label to predict the possible rise and fall probability of future trading days. The research results show that the proposed model and labels designed by this research can effectively predict important stock price fluctuations and can be effectively applied to financial commodity trading strategies.

An approach for predicting multiple-type overflow vulnerabilities based on combination features and a time series neural network algorithm

Overflow vulnerability is a common and dangerous software vulnerability that can lead to information theft, resource control, system collapse and other hazards. However, recent studies on predicting software overflow vulnerability have failed to specifically analyze factors and features that can lead to each type of overflow vulnerability and have only focused on binary classification problems rather than multiclassification problems, which are inefficient and time-consuming. Therefore, this paper proposes a multiple-type overflow vulnerability prediction method based on a combination of features and a time series neural network algorithm. First, by analyzing software overflow vulnerability features, a method is proposed to extract the internal vulnerability features of program source code. Then, an IFS set of internal vulnerability features of software overflow vulnerability is constructed. Second, an EFS set of external vulnerability features of software overflow vulnerability is extracted using a source code static analysis tool. A software overflow vulnerability feature library is constructed based on the IFS set and the EFS set. Finally, a multiple-type overflow vulnerability prediction method is constructed based on a time series bidirectional recurrent neural network after the symbol transformation and vector transformation of software overflow vulnerability features. Experiments show that the proposed method offers a higher precision, accuracy, recall rate, and F1 value. Moreover, this method can accurately detect the overflow vulnerability in actual software vulnerability predictions.

Multi-Timescale Load Forecast of Large Power Customers Based on Online Data Recovery and Time Series Neural Networks

This paper presents a multi-timescale receding horizon framework for the load forecast of large power customers. The future load pattern of individual users could be very difficult to predict because of its chronological and high volatile properties. Also, the sampling of nonaggregated load data may suffer from severe information missing issues. To address these challenges, we first develop an online singular value thresholding (SVT) algorithm, which utilizes the approximate low-rank property of load data matrices to efficiently recover the missing information. Then, a combinatorial deep learning method is developed, which applies the multi-layer perception (MLP) neural network and the long short-term memory (LSTM) neural network with gated recurrent unit (GRU) to deal with the short-term and ultra-short-term load forecast, respectively. Specifically, an early stopping strategy is designed and implemented to avoid the over-fitting of model training. Moreover, the receding time window is imposed to dynamically update the data recovery and load forecast outcomes, which supports the online computing on a Spark platform. Numerical experiments on real-world load data from North China confirms the effectiveness of the proposed methodology, which can support more complex applications in embedded systems and cyber physical systems.