Standard Artificial Neural Network Research Articles

Extended artificial neural networks: Incorporation of a priori chemical knowledge enables use of ion selective electrodes for in-situ measurement of ions at environmentally relevant levels

A novel artificial neural network (ANN) architecture is proposed which explicitly incorporates a priori system knowledge, i.e., relationships between output signals, while preserving the unconstrained non-linear function estimator characteristics of the traditional ANN. A method is provided for architecture layout, disabling training on a subset of neurons, and encoding system knowledge into the neuron structure. The novel architecture is applied to raw readings from a chemical sensor multi-probe (electric tongue), comprised of off-the-shelf ion selective electrodes (ISEs), to estimate individual ion concentrations in solutions at environmentally relevant concentrations and containing environmentally representative ion mixtures. Conductivity measurements and the concept of charge balance are incorporated into the ANN structure, resulting in (1) removal of estimation bias typically seen with use of ISEs in mixtures of unknown composition and (2) improvement of signal estimation by an order of magnitude or more for both major and minor constituents relative to use of ISEs as stand-alone sensors and error reduction by 30-50% relative to use of standard ANN models. This method is suggested as an alternative to parameterization of traditional models (e.g., Nikolsky-Eisenman), for which parameters are strongly dependent on both analyte concentration and temperature, and to standard ANN models which have no mechanism for incorporation of system knowledge. Network architecture and weighting are presented for the base case where the dot product can be used to relate ion concentrations to both conductivity and charge balance as well as for an extension to log-normalized data where the model can no longer be represented in this manner. While parameterization in this case study is analyte-dependent, the architecture is generalizable, allowing application of this method to other environmental problems for which mathematical constraints can be explicitly stated.

Weather Adaptive Traffic Prediction Using Neurowavelet Models

Climate change is a prevalent issue facing the world today. Unexpected increase in rainfall intensity and events is one of the major signatures of climate change. Rainfall influences traffic conditions and, in turn, traffic volume in urban arterials. For improved traffic management under adverse weather conditions, it is important to develop a traffic prediction algorithm considering the effect of rainfall. This inclusion is not intuitive as the effect is not immediate, and the influence of rainfall on traffic volume is often unrecognizable in a direct correlation analysis between the two time-series data sets; it can only be observed at certain frequency levels. Accordingly, it is useful to employ a multiresolution prediction framework to develop a weather adaptive traffic forecasting algorithm. Discrete wavelet transform (DWT) is a well-known multiresolution data analysis methodology. However, DWT imparts time variance in the transformed signal and makes it unsuitable for further time-series analysis. Therefore, the stationary form of DWT known as stationary wavelet transform (SWT) has been used in this paper to develop a neurowavelet prediction algorithm to forecast hourly traffic flow considering the effect of rainfall. The proposed prediction algorithm has been evaluated at two urban arterial locations in Dublin, Ireland. This paper shows that the rainfall data successfully augments the traffic flow data as an exogenous variable in periods of inclement weather, resulting in accurate predictions of future traffic flow at the two chosen locations. The forecasts from the neurowavelet model outperform the forecasts from the standard artificial neural network (ANN) model.

Wavelet neural network model for river flow time series

A new hybrid model that combines wavelets and an artificial neural network (ANN) called the wavelet neural network (WNN) model is proposed and applied for time series modelling of river flow. Time series of daily river flow of the Malaprabha River basin (Karnataka state, India) were analysed by the WNN model. The observed time series are decomposed into sub-series using a discrete wavelet transform and then an appropriate sub-series is used as input to the neural network for forecasting hydrological variables. The hybrid model (WNN) was compared with the standard ANN and auto-regressive (AR) models. The WNN model was able to provide a good fit with the observed data, especially the peak values during testing. The benchmark results from WNN model applications show that the hybrid model produces better results than the ANN and AR models in estimating hydrograph properties.

Logic minimization and rule extraction for identification of functional sites in molecular sequences

BackgroundLogic minimization is the application of algebraic axioms to a binary dataset with the purpose of reducing the number of digital variables and/or rules needed to express it. Although logic minimization techniques have been applied to bioinformatics datasets before, they have not been used in classification and rule discovery problems. In this paper, we propose a method based on logic minimization to extract predictive rules for two bioinformatics problems involving the identification of functional sites in molecular sequences: transcription factor binding sites (TFBS) in DNA and O-glycosylation sites in proteins. TFBS are important in various developmental processes and glycosylation is a posttranslational modification critical to protein functions.MethodsIn the present study, we first transformed the original biological dataset into a suitable binary form. Logic minimization was then applied to generate sets of simple rules to describe the transformed dataset. These rules were used to predict TFBS and O-glycosylation sites. The TFBS dataset is obtained from the TRANSFAC database, while the glycosylation dataset was compiled using information from OGLYCBASE and the Swiss-Prot Database.We performed the same predictions using two standard classification techniques, Artificial Neural Networks (ANN) and Support Vector Machines (SVM), and used their sensitivities and positive predictive values as benchmarks for the performance of our proposed algorithm. SVM were also used to reduce the number of variables included in the logic minimization approach.ResultsFor both TFBS and O-glycosylation sites, the prediction performance of the proposed logic minimization method was generally comparable and, in some cases, superior to the standard ANN and SVM classification methods with the advantage of providing intelligible rules to describe the datasets. In TFBS prediction, logic minimization produced a very small set of simple rules. In glycosylation site prediction, the rules produced were also interpretable and the most popular rules generated appeared to correlate well with recently reported hydrophilic/hydrophobic enhancement values of amino acids around possible O-glycosylation sites. Experiments with Self-Organizing Neural Networks corroborate the practical worth of the logic minimization method for these case studies.ConclusionsThe proposed logic minimization algorithm provides sets of rules that can be used to predict TFBS and O-glycosylation sites with sensitivity and positive predictive value comparable to those from ANN and SVM. Moreover, the logic minimization method has the additional capability of generating interpretable rules that allow biological scientists to correlate the predictions with other experimental results and to form new hypotheses for further investigation. Additional experiments with alternative rule-extraction techniques demonstrate that the logic minimization method is able to produce accurate rules from datasets with large numbers of variables and limited numbers of positive examples.

Pricing and Hedging Short Sterling Options Using Neural Networks

This paper compares the performance of artificial neural networks (ANNs) with that of the modified Black model in both pricing and hedging Short Sterling options. Using high frequency data, standard and hybrid ANNs are trained to generate option prices. The hybrid ANN is significantly superior to both the modified Black model and the standard ANN in pricing call and put options. Hedge ratios for hedging Short Sterling options positions using Short Sterling futures are produced using the standard and hybrid ANN pricing models, the modified Black model, and also standard and hybrid ANNs trained directly on the hedge ratios. The performance of hedge ratios from ANNs directly trained on actual hedge ratios is significantly superior to those based on a pricing model, and to the modified Black model.

PRICING AND HEDGING SHORT STERLING OPTIONS USING NEURAL NETWORKS

SUMMARYThis paper compares the performance of artificial neural networks (ANNs) with that of the modified Black model in both pricing and hedging short sterling options. Using high‐frequency data, standard and hybrid ANNs are trained to generate option prices. The hybrid ANN is significantly superior to both the modified Black model and the standard ANN in pricing call and put options. Hedge ratios for hedging short sterling options positions using short sterling futures are produced using the standard and hybrid ANN pricing models, the modified Black model, and also standard and hybrid ANNs trained directly on the hedge ratios. The performance of hedge ratios from ANNs directly trained on actual hedge ratios is significantly superior to those based on a pricing model, and to the modified Black model. Copyright © 2012 John Wiley & Sons, Ltd.

Hybrid soft computing systems for reservoir PVT properties prediction

In reservoir engineering, the knowledge of Pressure–Volume–Temperature (PVT) properties is of great importance for many uses, such as well test analyses, reserve estimation, material balance calculations, inflow performance calculations, fluid flow in porous media and the evaluation of new formations for the potential development and enhancement oil recovery projects. The determination of these properties is a complex problem because laboratory-measured properties of rock samples (“cores”) are only available from limited and isolated well locations and/or intervals. Several correlation models have been developed to relate these properties to other measures which are relatively abundant. These models include empirical correlations, statistical regression and artificial neural networks (ANNs). In this paper, a comprehensive study is conducted on the prediction of the bubble point pressure and oil formation volume factor using two hybrid of soft computing techniques; a genetically optimised neural network and a genetically enhanced subtractive clustering for the parameter identification of an adaptive neuro-fuzzy inference system. Simulation experiments are provided, showing the performance of the proposed techniques as compared with commonly used regression correlations, including standard artificial neural networks.

ERANN: An Algorithm to Extract Symbolic Rules from Trained Artificial Neural Networks

This paper presents an algorithm to extract symbolic rules from trained artificial neural networks (ANNs), called ERANN. In many applications, it is desirable to extract knowledge from ANNs for the users to gain a better understanding of how the networks solve the problems. Although ANN usually achieves high classification accuracy, the obtained results sometimes may be incomprehensible, because the knowledge embedded within them is distributed over the activation functions and the connection weights. This problem can be solved by extracting rules from trained ANNs. To do so, a rule extraction algorithm has been proposed in this paper to extract symbolic rules from trained ANNs. A standard three-layer feedforward ANN with four-phase training is the basis of the proposed algorithm. Extensive experimental studies on a set of benchmark classification problems, including breast cancer, iris, diabetes, wine, season, golfplaying, and lenses classification, demonstrates the applicability of the proposed method. Extracted rules are comparable with other methods in terms of number of rules, average number of conditions for a rule, and the rules accuracy. The proposed method achieved accuracy values 96.28%, 98.67%, 76.56%, 91.01%, 100%, 100%, and 100% for the above problems, respectively. It has been seen that these results are one of the best results comparing with results obtained from related previous studies.

Higher-Order Approximations for Testing Neglected Nonlinearity

We illustrate the need to use higher-order (specifically sixth-order) expansions in order to properly determine the asymptotic distribution of a standard artificial neural network test for neglected nonlinearity. The test statistic is a quasi-likelihood ratio (QLR) statistic designed to test whether the mean square prediction error improves by including an additional hidden unit with an activation function violating the no-zero condition in Cho, Ishida, and White (2011). This statistic is also shown to be asymptotically equivalent under the null to the Lagrange multiplier (LM) statistic of Luukkonen, Saikkonen, and Teräsvirta (1988) and Teräsvirta (1994). In addition, we compare the power properties of our QLR test to one satisfying the no-zero condition and find that the latter is not consistent for detecting a DGP with neglected nonlinearity violating an analogous no-zero condition, whereas our QLR test is consistent.

Topological constraints and robustness in liquid state machines

The Liquid State Machine (LSM) is a method of computing with temporal neurons, which can be used amongst other things for classifying intrinsically temporal data directly unlike standard artificial neural networks. It has also been put forward as a natural model of certain kinds of brain functions. There are two results in this paper: (1) We show that the Liquid State Machines as normally defined cannot serve as a natural model for brain function. This is because they are very vulnerable to failures in parts of the model. This result is in contrast to work by Maass et al. which showed that these models are robust to noise in the input data. (2) We show that specifying certain kinds of topological constraints (such as “small world assumption”), which have been claimed are reasonably plausible biologically, can restore robustness in this sense to LSMs.

Time Series Modeling of River Flow Using Wavelet Neural Networks

A new hybrid model which combines wavelets and Artificial Neural Network (ANN) called wavelet neural network (WNN) model was proposed in the current study and applied for time series modeling of river flow. The time series of daily river flow of the Malaprabha River basin (Karnataka state, India) were analyzed by the WNN model. The observed time series are decomposed into sub-series using discrete wavelet transform and then appropriate sub-series is used as inputs to the neural network for forecasting hydrological variables. The hybrid model (WNN) was compared with the standard ANN and AR models. The WNN model was able to provide a good fit with the observed data, especially the peak values during the testing period. The benchmark results from WNN model applications showed that the hybrid model produced better results in estimating the hydrograph properties than the latter models (ANN and AR).

Comparison of Artificial Neural Network and Multiple Regression Analysis Techniques in Predicting the Mechanical Properties of A3 56 Alloy

The mechanical properties of aluminium alloy castings, such as EL%, YS and UTS, are controlled by the casting and heat treatment variables, alloy's composition, and melt treatment. Despite the abundance of literature data, the large number of the controlling parameters has made it difficult to predict and model the mechanical properties by the conventional techniques. Another obstacle encountered when making such a prediction is the complex kinetics and interactions that exist among the many variables. The goal of this study was to develop Artificial Neural Network (ANN) and Multiple Regression models to predict the mechanical properties of A356 alloy from the processing variables. Several standard multi-layer ANN models were developed and trained using data from the literature and experimental results. A series of nonlinear regression models were also developed and the results were compared with the predictions made by the ANN models. Due to the complexity of A356's solidification behaviour, the nonlinear regression produced results that were not as accurate as those produced by the ANN model. Unlike the nonlinear regression analysis, ANN can simplify the modelling process by eliminating the need to define a function. The results indicate that ANN is a suitable modelling technique for predicting mechanical properties of castings based on the alloy chemistry and processing variables.

An Annual Midterm Energy Forecasting Model Using Fuzzy Logic

The objective of this paper is to present a new fuzzy logic method for midterm energy forecasting. The proposed method properly transforms the input variables to differences or relative differences, in order to predict energy values not included in the training set and to use a minimal number of patterns. The input variables, the number of the triangular membership functions and their base widths are simultaneously selected by an optimization process. The standard deviation is calculated analytically by mathematical expressions based on the membership functions. Results from an extensive application of the method to the Greek power system and for different categories of customers are compared to those obtained from the application of standard regression methods and artificial neural networks (ANN).

Forecasting of ozone episode days by cost-sensitive neural network methods

Forecasting the occurrence of ozone episode days can be regarded as an imbalanced dataset classification problem. Since the standard artificial neural network (ANN) methods cannot make accurate predictions of such a problem, two cost-sensitive ANN methods, cost-penalty and moving threshold, were used in this study. The models classify each day as episode or non-episode according to the standard of daily maximum 8 h O 3 concentration. The ozone measurements from six monitoring stations in Taiwan were used for model training and performance evaluation. Two different input datasets, regional and single-site, were generated from raw air quality and meteorological observations. According to the numerical experiments, the predictions based on the regional dataset are much better than those obtained from the single-site dataset. Two cost-sensitive ANN methods were evaluated by receiver operating characteristic (ROC) curves. It was found that the results obtained by the two approaches are similar. If the misclassification costs are known, the cost-sensitive method can minimise the total costs. If the misclassification costs are unknown, the cost-sensitive ANN can obtain a better forecast than the standard ANN method when an appropriate cost ratio is used. For clean areas where episode days are very rare, the forecasts are poor for all methods.

New application of intelligent agents in sporadic amyotrophic lateral sclerosis identifies unexpected specific genetic background

BackgroundFew genetic factors predisposing to the sporadic form of amyotrophic lateral sclerosis (ALS) have been identified, but the pathology itself seems to be a true multifactorial disease in which complex interactions between environmental and genetic susceptibility factors take place. The purpose of this study was to approach genetic data with an innovative statistical method such as artificial neural networks to identify a possible genetic background predisposing to the disease. A DNA multiarray panel was applied to genotype more than 60 polymorphisms within 35 genes selected from pathways of lipid and homocysteine metabolism, regulation of blood pressure, coagulation, inflammation, cellular adhesion and matrix integrity, in 54 sporadic ALS patients and 208 controls. Advanced intelligent systems based on novel coupling of artificial neural networks and evolutionary algorithms have been applied. The results obtained have been compared with those derived from the use of standard neural networks and classical statistical analysisResultsAdvanced intelligent systems based on novel coupling of artificial neural networks and evolutionary algorithms have been applied. The results obtained have been compared with those derived from the use of standard neural networks and classical statistical analysis. An unexpected discovery of a strong genetic background in sporadic ALS using a DNA multiarray panel and analytical processing of the data with advanced artificial neural networks was found. The predictive accuracy obtained with Linear Discriminant Analysis and Standard Artificial Neural Networks ranged from 70% to 79% (average 75.31%) and from 69.1 to 86.2% (average 76.6%) respectively. The corresponding value obtained with Advanced Intelligent Systems reached an average of 96.0% (range 94.4 to 97.6%). This latter approach allowed the identification of seven genetic variants essential to differentiate cases from controls: apolipoprotein E arg158cys; hepatic lipase -480 C/T; endothelial nitric oxide synthase 690 C/T and glu298asp; vitamin K-dependent coagulation factor seven arg353glu, glycoprotein Ia/IIa 873 G/A and E-selectin ser128arg.ConclusionThis study provides an alternative and reliable method to approach complex diseases. Indeed, the application of a novel artificial intelligence-based method offers a new insight into genetic markers of sporadic ALS pointing out the existence of a strong genetic background.

Statistical models of KSE100 index using hybrid financial systems

This paper utilizes hybrid financial systems (HFSs) to model Karachi Stock Exchange index data, KSE100, for short-term prediction. These HFSs developed for this purpose are combination of artificial neural networks (ANN) and ARIMA or autoregressive conditional heteroskedasticity/generalized autoregressive conditional heteroskedasticity (ARCH/GARCH) models. We compared ANN with ARIMA and ARCH/GARCH on the basis of forecast mean square error (FMSE), ANN gave better forecasting performance and out played ARIMA and ARCH/GARCH models. While comparing the performance of HFSs of ANN ARIMA and ANN ARCH/GARCH with ANN model, it is found that the HFS ANN ARCH/GARCH is superior to standard ANN and HFS ANN ARIMA in forecasting KSE100 index.

Artificial neural networks in the recognition of the presence of thyroid disease in patients with atrophic body gastritis

To investigate the role of artificial neural networks in predicting the presence of thyroid disease in atrophic body gastritis patients. A dataset of 29 input variables of 253 atrophic body gastritis patients was applied to artificial neural networks (ANNs) using a data optimisation procedure (standard ANNs, T&T-IS protocol, TWIST protocol). The target variable was the presence of thyroid disease. Standard ANNs obtained a mean accuracy of 64.4% with a sensitivity of 69% and a specificity of 59.8% in recognizing atrophic body gastritis patients with thyroid disease. The optimization procedures (T&T-IS and TWIST protocol) improved the performance of the recognition task yielding a mean accuracy, sensitivity and specificity of 74.7% and 75.8%, 78.8% and 81.8%, and 70.5% and 69.9%, respectively. The increase of sensitivity of the TWIST protocol was statistically significant compared to T&T-IS. This study suggests that artificial neural networks may be taken into consideration as a potential clinical decision-support tool for identifying ABG patients at risk for harbouring an unknown thyroid disease and thus requiring diagnostic work-up of their thyroid status.

An automated cervical pre-cancerous diagnostic system

This paper proposes to develop an automated diagnostic system for cervical pre-cancerous. METHODS AND DATA SAMPLES: The proposed automated diagnostic system consists of two parts; an automatic feature extraction and an intelligent diagnostic. In the automatic feature extraction, the system automatically extracts four cervical cells features (i.e. nucleus size, nucleus grey level, cytoplasm size and cytoplasm grey level). A new features extraction algorithm called region-growing-based features extraction (RGBFE) is proposed to extract the cervical cells features. The extracted features will then be fed as input data to the intelligent diagnostic part. A new artificial neural network (ANN) architecture called hierarchical hybrid multilayered perceptron (H(2)MLP) network is proposed to predict the cervical pre-cancerous stage into three classes, namely normal, low grade intra-epithelial squamous lesion (LSIL) and high grade intra-epithelial squamous lesion (HSIL). We empirically assess the capability of the proposed diagnostic system using 550 reported cases (211 normal cases, 143 LSIL cases and 196 HSIL cases). For evaluation of the automatic feature extraction performance, correlation test approach was used to determine the capability of the RGBFE algorithm as compared to manual extraction by cytotechnologist. The manual extraction of size was recorded in micrometer while the automatic extraction of size was recorded in number of pixels. Region color was recorded in mean of grey level value for both manual and automatic extraction. The results show that the estimated size and mean of grey level have strong linear relationship (correlation test more than 0.8) with those extracted manually by cytotechnologist. Hence, the size of nucleus, size of cytoplasm and grey level of cytoplasm created very strong linear relationship with correlation test more than 0.95 (approaching one). For the intelligent diagnostic, the performance of the H(2)MLP network was compared with three standard ANNs (i.e. multilayered perceptron (MLP), radial basis function (RBF) and hybrid multilayered perceptron (HMLP)). The performance was done based on accuracy, sensitivity, specificity, false negative and false positive. The H(2)MLP network performed the best diagnostic performance as compared to other ANNs. It was able to achieve 97.50% accuracy, 100% specificity and 96.67% sensitivity. The false negative and false positive were 1.33% and 3.00%, respectively. This project has successfully developed an automatic diagnostic system for cervical pre-cancerous. This study has also successfully proposed one image processing technique namely the RGBFE algorithm for automatic feature extraction process and a new ANN architecture namely the H(2)MLP network for better diagnostic performance.

Bayesian neural network for rainfall‐runoff modeling

In this paper, a Bayesian learning approach is introduced to train a multilayer feed‐forward network for daily river flow and reservoir inflow simulation in a cold region river basin in Canada. In Bayesian approach, uncertainty about the relationship between inputs and outputs is initially taken care of by an assumed prior distribution of parameters (weights and biases). This prior distribution is updated to posterior distribution using a likelihood function following Bayes' theorem while data are observed. This posterior distribution is called the objective function of a network in the Bayesian learning approach. The objective function is maximized using a suitable optimization technique. Once the network is trained, the predictive distribution of the network outputs is obtained by integrating over the posterior distribution of weights. In this study, Gaussian prior distribution and a Gaussian noise model are used in defining posterior distribution. The network has been optimized using a scaled conjugate gradient technique. Posterior distribution of weights is approximated to Gaussian during prediction. Prediction performance of the Bayesian neural network (BNN) is compared with the results obtained from a standard artificial neural network (ANN) model and a widely used conceptual rainfall‐runoff model, namely, HBV‐96. The BNN model outperformed the conceptual model and slightly outperformed the standard ANN model in simulating mean, peak, and low river flows and reservoir inflows. The significant contribution of the Bayesian method over the conventional ANN approach, among others, is the uncertainty estimation of the outputs in the form of confidence intervals which are particularly needed in practical water resources applications. Prediction confidence limits (or intervals) indicate the extent to which one can rely on predictions for decision making. It is shown that the BNN can provide reliable streamflow and reservoir inflow forecasts without a loss in model prediction accuracy as compared to standard ANN and conceptual model HBV. Another significant advantage of BNN approach is that the overfitting and underfitting problems are automatically taken care of by the Bayesian learning algorithm, which conversely remain serious problems with conventional ANN learning algorithm.

ARTIFICIAL INTELLIGENCE HELPS PREDICT THE SOURCE, SEVERITY, NEED FOR URGENT ENDOSCOPY AND DISPOSITION IN PATIENTS WITH ACUTE GASTROINTESTINAL BLEEDING

Purpose: Develop a simple model to reliably predict: a) source, b) need for urgent endoscopy, and c) disposition in patients with acute GI bleed. Methods: Modern machine learning methods, such as artificial neural networks (ANN) and support vector machines (SVM) with learning capabilities analogous to human learning have been utilized to predict clinical outcomes. Training these models allows classification functions that generalize for all possible inputs, which can then be utilized to predict output for any given input. Relevant clinical data was collected on 117 patients representing acute upper and lower GIB. Endoscopic data was utilized to confirm the source and to ascertain if the patient would have benefitted from an urgent endoscopy. Criteria for an urgent endoscopy were: a) active & fresh bleeding, b) findings of high risk stigmata on upper endoscopy, and c) history of cirrhosis. Both endoscopic and clinical data were utilized to ascertain disposition. Performance of ANN was compared to clinicians. CLINICAL DATA Presentation Hematemesis/Coffee Grounds Hematochezia/Melena Demographic Age Comorbidities CVD/COPD Risk of Stress Ulcer Cirrhosis ASA/NSAID use Prior history of GIB Clinical Exam BP, HR, Orthostasis NG Lavage Laboratory Data Drop of Hct Platelet count Creatinine, BUN/Cr ratio Results: We utilized the SVMTorch package and Matlab coding of a standard ANN with backpropagation to implement the classifiers. Extensive evaluation through multi-fold cross-validation was not performed due to a limited dataset. 78 randomly selected patients were used for training, and 39 patients weere used for testing. Table 2 summarizes the results for each prediction variable and for each classifier. Both models yielded similar performance; the SVM model was slightly better.Table. MISCLASSIFICATIONSConclusions: Although clinical tools may not replace experience and clinical acumen, they may play an important role to help guide therapy, standardize clinical care, improve outcomes, optimize healthcare costs, and prevent adverse complications. The application of such tools in patients with GIB needs further development and validation in prospective randomized studies. If successful, this application may serve as a model for use of artificial intelligence in a variety of conditions.