Time Series Neural Network Research Articles

Monitoring of plasma processing chamber using ion energy analyser and time series neural network

A new technique to monitor plasma processes is presented. An autocorrelated neural time series (A-NTS) network was used to model ion energy distribution (IED). The ion energy data were collected during a deposition of silicon nitride films in a SiH4–NH3 inductively coupled plasma. A backpropagation neural network was used to build IED model. Prediction performance of A-NTS model was evaluated as a function of training tolerance as well as a detection sensitivity. The A-NTS models demonstrated detection sensitivities high enough to detect plasma faults. Maximum sensitivity of A-NTS models obtained was more than 55% for all fault cases. Optimised A-NTS models yielded the prediction errors of 1·41 and 2·18% for 80 and 60% IED respectively. The presented technique can be applied to monitor any kinds of plasma faults and is expected effective particularly to those faults sensitive to ion bombardment variations.

An adaptive network-based fuzzy inference system for short-term natural gas demand estimation: Uncertain and complex environments

Accurate short-term natural gas (NG) demand estimation and forecasting is vital for policy and decision-making process in energy sector. Moreover, conventional methods may not provide accurate results. This paper presents an adaptive network-based fuzzy inference system (ANFIS) for estimation of NG demand. Standard input variables are used which are day of the week, demand of the same day in previous year, demand of a day before and demand of 2 days before. The proposed ANFIS approach is equipped with pre-processing and post-processing concepts. Moreover, input data are pre-processed (scaled) and finally output data are post-processed (returned to its original scale). The superiority and applicability of the ANFIS approach is shown for Iranian NG consumption from 22/12/2007 to 30/6/2008. Results show that ANFIS provides more accurate results than artificial neural network (ANN) and conventional time series approach. The results of this study provide policy makers with an appropriate tool to make more accurate predictions on future short-term NG demand. This is because the proposed approach is capable of handling non-linearity, complexity as well as uncertainty that may exist in actual data sets due to erratic responses and measurement errors.

Next Day Price Forecasting in Deregulated Market by Combination of Artificial Neural Network and ARIMA Time Series Models

Electricity price forecasting is becoming increasingly relevant to power producers and consumers in the new competitive electric power markets, when planning bidding strategies in order to maximize their benefits and utilities, respectively. This paper proposed a method to predict hourly electricity prices for next-day electricity markets by combination methodology of ARIMA and ANN models. The proposed method is examined on the Australian National Electricity Market (NEM), New South Wales regional in year 2006. Comparison of forecasting performance with the proposed ARIMA, ANN and combination (ARIMA-ANN) models are presented. Empirical results indicate that an ARIMA-ANN model can improve the price forecasting accuracy.

Electricity Market Forecasting using Artificial Neural Network Models Optimized by Grid Computing

This paper reports a grid computing approach to parallel-process a neural network time-series model for forecasting electricity market prices. The grid computing of the neural network model not only processes several times faster than a single iterative process but also provides chances of improving forecasting accuracy. A grid-computing environment implemented in a university computing laboratory improves utilization rate of otherwise underused computing resources. Results of numerical tests using the real market data by more than twenty grid-connected PCs are presented.

A combined model predictive control and time series forecasting framework for production-inventory systems

Model Predictive Control (MPC) has been previously applied to supply chain problems with promising results; however most systems that have been proposed so far possess no information on future demand. The incorporation of a forecasting methodology in an MPC framework can promote the efficiency of control actions by providing insight in the future. In this paper this possibility is explored, by proposing a complete management framework for production-inventory systems that is based on MPC and on a neural network time series forecasting model. The proposed framework is tested on industrial data in order to assess the efficiency of the method and the impact of forecast accuracy on the overall control performance. To this end, the proposed method is compared with several alternative forecasting approaches that are implemented on the same industrial dataset. The results show that the proposed scheme can improve significantly the performance of the production-inventory system, due to the fact that more accurate predictions are provided to the formulation of the MPC optimization problem that is solved in real time.

Scalability of the genetic hybrid algorithm on a parallel supercomputer

PurposeTo demonstrate the scalability of the genetic hybrid algorithm (GHA) in monitoring a local neural network algorithm for difficult non‐linear/chaotic time series problems.Design/methodology/approachGHA is a general‐purpose algorithm, spanning several areas of mathematical problem solving. If needed, GHA invokes an accelerator function at key stages of the solution process, providing it with the current population of solution vectors in the argument list of the function. The user has control over the computational stage (generation of a new population, crossover, mutation etc) and can modify the population of solution vectors, e.g. by invoking special purpose algorithms through the accelerator channel. If needed, the steps of GHA can be partly or completely superseded by the special purpose mathematical/artificial intelligence‐based algorithm. The system can be used as a package for classical mathematical programming with the genetic sub‐block deactivated. On the other hand, the algorithm can be turned into a machinery for stochastic analysis (e.g. for Monte Carlo simulation, time series modelling or neural networks), where the mathematical programming and genetic computing facilities are deactivated or appropropriately adjusted. Finally, pure evolutionary computation may be activated for studying genetic phenomena. GHA contains a flexible generic multi‐computer framework based on MPI, allowing implementations of a wide range of parallel models.FindingsThe results indicate that GHA is scalable, yet due to the inherent stochasticity of neural networks and the genetic algorithm, the scalability evidence put forth in this paper is only indicative. The scalability of GHA follows from maximal node intelligence allowing minimal internodal communication in problems with independent computational blocks.Originality/valueThe paper shows that GHA can be effectively run on both sequential and parallel platforms. The multicomputer layout is based on maximizing the intelligence of the nodes – all nodes are provided with the same program and the available computational support libraries – and minimizing internodal communication, hence GHA does not limit the size of the mesh in problems with independent computational tasks.

Land cover classification using moderate resolution imaging spectrometer-enhanced vegetation index time-series data and self-organizing map neural network in Inner Mongolia, China

The Moderate Resolution Imaging Spectroradiometer (MODIS) data offers a unique combination of spectral, temporal, and spatial resolution in comparison to other global sensors. The MODIS Enhanced Vegetation Index (EVI) product has several advantages, which make it suitable for regional land cover mapping. This paper investigates the application of MODIS EVI time-series data for mapping temperate arid and semi-arid land cover at a moderate resolution (500 m), for regional land-cover/land-use monitoring purposes. A 16-day composite EVI time-series data for 2003 (22 March 2003 - 30 September 2003) was adopted for the study. A land cover map was generated for the Inner Mongolia Autonomous Region using 7 tiles of MODIS EVI time-series data and Self-Organizing Map (SOM) neural network classification. Land-use GIS data, Landsat TM/ETM, and ASTER data were employed as reference data. The results show that the overall accuracy of land cover classification is about 84% with a Kappa coefficient of 0.8170. These results demonstrate that the SOM neural network model could work well for the multi-temporal MODIS EVI data, and suggest a potential of using MODIS EVI time-series remote sensing data to monitor desertification in Inner Mongolia with limited ancillary data and little labor-input in comparison with using high-spatial resolution remote sensing data.

Neural network time series prediction of environmental variables in a small upland headwater in NE Scotland

AbstractShort‐term prediction of environmental variables such as stream flow rate is useful to members of the general public and environmental scientists alike, providing the ability to predict environmental disasters or scientifically interesting events. Here, a neural‐network based method is presented, which is capable of providing advance flood warnings or the prediction of high stream flow events for research purposes in a small upland headwater in NE Scotland. This method relies on training from past time series data acquired in the field, and provides the ability to predict a range of hydrological and meteorological variables up to 24 h ahead using feedback of predicted values at time t as new inputs for the next time step t + 1. The system is rapid and effective, relies on standard neural network training methods, and has the potential to be implemented in a web‐based monitoring and prediction package. The model design could be implemented at any study site where time series data has been gathered, and is sufficiently flexible to accept whatever data is available. Copyright © 2007 John Wiley & Sons, Ltd.

Hybrid neural network models for hydrologic time series forecasting

The need for increased accuracies in time series forecasting has motivated the researchers to develop innovative models. In this paper, a new hybrid time series neural network model is proposed that is capable of exploiting the strengths of traditional time series approaches and artificial neural networks (ANNs). The proposed approach consists of an overall modelling framework, which is a combination of the conventional and ANN techniques. The steps involved in the time series analysis, e.g. de-trending and de-seasonalisation, can be carried out before gradually presenting the modified time series data to the ANN. The proposed hybrid approach for time series forecasting is tested using the monthly streamflow data at Colorado River at Lees Ferry, USA. Specifically, results from four time series models of auto-regressive (AR) type and four ANN models are presented. The results obtained in this study suggest that the approach of combining the strengths of the conventional and ANN techniques provides a robust modelling framework capable of capturing the non-linear nature of the complex time series and thus producing more accurate forecasts. Although the proposed hybrid neural network models are applied in hydrology in this study, they have tremendous scope for application in a wide range of areas for achieving increased accuracies in time series forecasting.

Time Series Forecasting of Cyanobacteria Blooms in the Crestuma Reservoir (Douro River, Portugal) Using Artificial Neural Networks

In this work, time series neural networks were used to predict the occurrence of toxic cyanobacterial blooms in Crestuma Reservoir, which is an important potable water supply for the Porto region, located in the north of Portugal. These models can potentially be used to provide water treatment plant operators with an early warning for developing cyanobacteria blooms. Physical, chemical, and biological parameters were collected at Crestuma Reservoir from 1999 to 2002. The data set was then divided into three independent time series, each with a fortnightly periodicity. One training series was used to "teach" the neural networks to predict results. Another series was used to verify the results, and to avoid over-fitting of the data. An additional independently collected data series was then used to test the efficacy of the model for predicting the abundance of cyanobacteria. All of the models tested in this study incorporated a prediction time (look-ahead parameter) equal to the sampling interval (two weeks). Various lag periods, from 2 to 52 weeks, were also investigated. The best model produced in this study provided the following correlations between the target and forecast values in the training, verification, and validation series: 1.000 (P = 0.000), 0.802 (P = 0.000), and 0.773 (P = 0.001), respectively. By applying this model to the three-year data set, we were able to predict fluctuations in cyanobacteria abundance in the Crestuma Reservoir, with a high level of precision. By incorporating a lag-period of eight weeks, we were able to detect secondary fluctuations in cyanobacterial abundance over the annual cycle.

Neural-Network-Based Sensor Fusion of Optical Emission and Mass Spectroscopy Data for Real-Time Fault Detection in Reactive Ion Etching

To achieve timely and accurate fault detection in reactive ion etching, neural networks (NNs) have been applied for the fusion of data generated by two in-situ sensors: optical emission spectroscopy (OES) and residual gas analysis (RGA). While etching is performed, OES and RGA data are simultaneously collected in real time. Several pre-determined, statistically significant wavelengths (for OES data) and atomic masses (for RGA signals) are monitored. These data are subsequently used for training NN-based time series models of process behavior. Such models, referred to herein as time series NNs (TSNNs), are realized using multilayered perceptron NNs. Results indicate that the TSNNs not only predict process parameters of interest, but also efficiently perform as sensor fusion of the in-situ sensor data.

Linear models, smooth transition autoregressions, and neural networks for forecasting macroeconomic time series: A re-examination

In this paper, we examine the forecast accuracy of linear autoregressive, smooth transition autoregressive (STAR), and neural network (NN) time series models for 47 monthly macroeconomic variables of the G7 economies. Unlike previous studies that typically consider multiple but fixed model specifications, we use a single but dynamic specification for each model class. The point forecast results indicate that the STAR model generally outperforms linear autoregressive models. It also improves upon several fixed STAR models, demonstrating that careful specification of nonlinear time series models is of crucial importance. The results for neural network models are mixed in the sense that at long forecast horizons, an NN model obtained using Bayesian regularization produces more accurate forecasts than a corresponding model specified using the specific-to-general approach. Reasons for this outcome are discussed.

Neural network time series forecasting of finite-element mesh adaptation

Basic learning algorithms and the neural network model are applied to the problem of mesh adaptation for the finite-element method for solving time-dependent partial differential equations. Time series prediction via the neural network methodology is used to predict the areas of “interest” in order to obtain an effective mesh refinement at the appropriate times. This allows for increased numerical accuracy with the same computational resources as compared with more “traditional” methods.

Neural Network-Based Real-Time Malfunction Diagnosis of Reactive Ion Etching Using In Situ Metrology Data

To mitigate capital equipment investments and enhance product quality, semiconductor manufactures are turning to advanced process control (APC) methods. With the objective of facilitating APC, this paper investigates a methodology for real-time malfunction diagnosis of reactive ion etching (RIE) employing two types of in situ metrology: optical emission spectroscopy (OES) and residual gas analysis (RGA). Based on metrology data, time series neural networks (TSNNs) are trained to generate evidential belief for potential malfunctions in real time, and Dempster-Shafer (D-S) theory is adopted for evidential reasoning. Successful malfunction diagnosis is achieved, with only a single missed alarm and a single false alarm occurring out of 21 test runs when both sensors are used in tandem. From the results, we conclude that the OES and RGA sensors, in conjunction with the TSNN models, can be effectively used for RIE monitoring and diagnosis. Furthermore, D-S theory is shown to be an appropriate inference methodology.

An empirical investigation of bias and variance in time series forecasting: modeling considerations and error evaluation

Bias and variance play an important role in understanding the fundamental issue of learning and generalization in neural network modeling. Several studies on bias and variance effects have been published in classification and regression related research of neural networks. However, little research has been done in this area for time-series modeling and forecasting. We consider modeling issues related to understanding error components given the common practices associated with neural-network time-series forecasting. We point out the key difference between classification and time-series problems in consideration of the bias-plus-variance decomposition. A Monte Carlo study on the role of bias and variance in neural networks time-series forecasting is conducted. We find that both input lag structure and hidden nodes are important in contributing to the overall forecasting performance. The results also suggest that overspecification of input nodes in neural network modeling does not impact the model bias, but has significant effect on the model variance. Methods such as neural ensembles that focus on reducing the model variance, therefore, can be valuable and effective in time-series forecasting modeling.

Application of neural networks in forecasting engine systems reliability

This paper presents a comparative study of the predictive performances of neural network time series models for forecasting failures and reliability in engine systems. Traditionally, failure data analysis requires specifications of parametric failure distributions and justifications of certain assumptions, which are at times difficult to validate. On the other hand, the time series modeling technique using neural networks provides a promising alternative. Neural network modeling via feed-forward multilayer perceptron (MLP) suffers from local minima problems and long computation time. The radial basis function (RBF) neural network architecture is found to be a viable alternative due to its shorter training time. Illustrative examples using reliability testing and field data showed that the proposed model results in comparable or better predictive performance than traditional MLP model and the linear benchmark based on Box–Jenkins autoregressive-integrated-moving average (ARIMA) models. The effects of input window size and hidden layer nodes are further investigated. Appropriate design topologies can be determined via sensitivity analysis.

Prediction of the hourly ionospheric parameter ƒ0F2 using a novel nonlinear interpolation technique to cope with missing data points

A technique for neural network time series prediction using radial basis functions, where the input data contain a significant proportion of missing points, is developed. This technique is intended to model the data while simultaneously providing a means of minimizing the impact upon the model of the missing points that are typical of geophysical time series. The two issues are inextricably entwined because missing data points have a significant impact upon the performance of data‐derived models in terms of prediction availability and accuracy. The core of the technique is a nonlinear interpolation scheme that assigns values to gaps in the input time series. Each missing point is interpolated such that the error introduced into any specific predictive function is minimized. This interpolative technique has a general application in any instance where the effects of interpolation upon a given analysis process need to be minimized or a complete time series needs to be constructed from incomplete data. The technique has been applied to the prediction of ƒ0F2 from Slough, United Kingdom. The resultant model prediction root‐mean‐square (RMS) error is shown to be 2.3% better than using recurrence interpolation (in terms of overall model accuracy rather than relative to each other), 3.8% better than using persistence interpolation, and 34.3% better than not using any interpolation. Utilizing the interpolation algorithm lowers the RMS error by 26% when incomplete data, in addition to complete data, are used as an input to both the interpolated and the uninterpolated models.

An investigation of model selection criteria for neural network time series forecasting

Artificial neural networks (ANNs) have received more and more attention in time series forecasting in recent years. One major disadvantage of neural networks is that there is no formal systematic model building approach. In this paper, we expose problems of the commonly used information-based in-sample model selection criteria in selecting neural networks for financial time series forecasting. Specifically, Akaike’s information criterion (AIC) and Bayesian information criterion (BIC) as well as several extensions have been examined through three real time series of Standard and Poor’s 500 index (S&P 500 index), exchange rate, and interest rate. In addition, the relationship between in-sample model fitting and out-of-sample forecasting performance with commonly used performance measures is also studied. Results indicate that the in-sample model selection criteria we investigated are not able to provide a reliable guide to out-of-sample performance and there is no apparent connection between in-sample model fit and out-of-sample forecasting performance.

Modeling exchange rates: smooth transitions, neural networks, and linear models

The goal of this paper is to test and model nonlinearities in several monthly exchange rates time series. We apply two different nonlinear alternatives, namely: the artificial neural-network time series model estimated with Bayesian regularization; and a flexible smooth transition specification, called the neuro-coefficient smooth transition autoregression. The linearity test rejects the null hypothesis of linearity in 10 out of 14 series. We compare, using different measures, the forecasting performance of the nonlinear specifications with the linear autoregression and the random walk models.

Use of neural network models to predict industrial bioreactor effluent quality.

Engineered bioreactors are useful tools for degrading wastes from crude oil refining facilities. One such bioreactor forms part of the wastewater remediation process used at a refinery in the San Francisco Bay Area. The flow rate and chemical concentrations of the waste vary, and it is necessary to be able to predict the efficiency of the reactor degradation process for this varied input. The complex biological, physical, and chemical processes of the reactor make deterministic modeling unsuitable. Therefore, predictive modeling for this system was performed using a neural network model. A predictive, time-series neural network model requires a complete data set. Often, in the case of a large industrial facility, data are missing. Various techniques can be used to reconstruct missing data, but comparisons of techniques have not been performed for large-scale remediation processes. In this manuscript, four techniques are used for reconstructing missing data to examine which ones provide superior predictive capabilities. It was found that the interpolated and moving average values methods provided the best predictions. The mean and median replacement methods, commonly used in neural network modeling, provided much poorer predictions. Another goal of this study is to determine which water quality parameters are more accurately predicted than others. In this study, pH was the most accurately predicted, while ammonia and total phenolics concentrations were the least accurately predicted.