NN Approach Research Articles

A New Adaptive Elastic Net Method for Cluster Analysis

Clustering is inherently a highly challenging research problem. The elastic net algorithm is designed to solve the traveling salesman problem initially, now is verified to be an efficient tool for data clustering in n-dimensional space. In this paper, by introducing a nearest neighbor learning method and a local search preferred strategy, we proposed a new Self-Organizing NN approach, called the Adaptive Clustering Elastic Net (ACEN) to solve the cluster analysis problems. ACEN consists of the adaptive clustering elastic net phase and a local search preferred phase. The first phase is used to find a cyclic permutation of the points as to minimize the total distances of the adjacent points, and adopts the Euclidean distance as the criteria to assign each point. The local search preferred phase aims to minimize the total dissimilarity within each clusters. Simulations were made on a large number of homogeneous and nonhomogeneous artificial clusters in n dimensions and a set of publicly standard problems available from UCI. Simulation results show that compared with classical partitional clustering methods, ACEN can provide better clustering solutions and do more efficiently.

NN approach to state-feedback control of high-order non-linear time-delay systems with time-varying control coefficient

This paper considers the adaptive state-feedback control problem for a class of high-order non-linear systems with unknown control coefficient and time delays. By applying the neural network approximation method and the Nussbaum function approach, the restrictions on non-linear functions and the conditions on the time-varying control coefficient are largely relaxed. In addition, an adaptive neural network state-feedback controller with only one adaptive parameter is successfully constructed by introducing proper Lyapunov–Krasovskii functionals and using the backstepping technique. The proposed scheme guarantees the closed-loop system to be semi-globally uniformly ultimately bounded. Finally, a simulation example demonstrates the effectiveness of the controller.

<inline-formula><tex-math>$k$</tex-math><alternatives> <inline-graphic xlink:type="simple" xlink:href="alharthi-ieq1-2460735.gif"/></alternatives></inline-formula>NNVWC: An Efficient <inline-formula><tex-math>$k$</tex-math><alternatives><inline-graphic xlink:type="simple" xlink:href="alharthi-ieq2-2460735.gif"/> </alternatives></inline-formula>-Nearest Neighbors Approach Based on Various-Widths Clustering

The $k$ -nearest neighbor approach ( $k$ -NN) has been extensively used as a powerful non-parametric technique in many scientific and engineering applications. However, this approach incurs a large computational cost. Hence, this issue has become an active research field. In this work, a novel $k$ -NN approach based on various-widths clustering, named $k$ NNVWC, to efficiently find $k$ -NNs for a query object from a given data set, is presented. $k$ NNVWC does clustering using various widths, where a data set is clustered with a global width first and each produced cluster that meets the predefined criteria is recursively clustered with its own local width that suits its distribution. This reduces the clustering time, in addition to balancing the number of produced clusters and their respective sizes. Maximum efficiency is achieved by using triangle inequality to prune unlikely clusters. Experimental results demonstrate that $k$ NNVWC performs well in finding $k$ -NNs for query objects compared to a number of $k$ -NN search algorithms, especially for a data set with high dimensions, various distributions and large size.

A Proposal for Local $k$ Values for $k$ -Nearest Neighbor Rule.

The k -nearest neighbor ( k -NN) classifier is one of the most widely used methods of classification due to several interesting features, including good generalization and easy implementation. Although simple, it is usually able to match and even outperform more sophisticated and complex methods. One of the problems with this approach is fixing the appropriate value of k . Although a good value might be obtained using cross validation, it is unlikely that the same value could be optimal for the whole space spanned by the training set. It is evident that different regions of the feature space would require different values of k due to the different distributions of prototypes. The situation of a query instance in the center of a class is very different from the situation of a query instance near the boundary between two classes. In this brief, we present a simple yet powerful approach to setting a local value of k . We associate a potentially different k to every prototype and obtain the best value of k by optimizing a criterion consisting of the local and global effects of the different k values in the neighborhood of the prototype. The proposed method has a fast training stage and the same complexity as the standard k -NN approach at the testing stage. The experiments show that this simple approach can significantly outperform the standard k -NN rule for both standard and class-imbalanced problems in a large set of different problems.

Evolutionary Neural Network model for West Texas Intermediate crude oil price prediction

This paper proposes an alternative approach based on a genetic algorithm and neural network (GA–NN) for the prediction of the West Texas Intermediate (WTI) crude oil price. Comparative simulation results suggested that the proposed GA–NN approach is better than the baseline algorithms in terms of prediction accuracy and computational efficiency. Mann–Whitney test results indicated that the WTI crude oil price predicted by the proposed GA–NN and the observed price are statistically equal. Further comparison of the proposed GA–NN with previous studies indicated performance improvement over existing results. The proposed model can be useful in the formulation of policies related to international crude oil price estimations, development plans and industrial production.

OD Matrices Network Estimation from Link Counts by Neural Networks

This paper attempts to deal with traffic Origin Destination (OD) matrix estimation starting from the measurements of flow on road network links. It proposes a different approach from published articles to date, by applying multilayer feed-forward neural networks. Since the relationship between link flow and the related OD matrix is continuous, it is possible to use the well known approximation property of Neural Network models. The method is proposed for a real-time correction of the OD matrix. Two application scenarios were developed: a trial network and an actual rural network were both simulated by a micro-simulator that assigns known OD matrices. A Principal Component Analysis (PCA) technique was used to reduce the amount of variables and to achieve improved significance for input data. The estimated error was relatively low and, as opposed to analytical approaches, the processing time was almost in real time, making this approach suitable for applications in dynamic traffic management. Comparisons with results obtained by an OD estimation commercial program show better performance in the NN approach both as regards error and computing time.

الذکاء الاصطناعی وتحلیل السلاسل الزمنیة

هذا البحث يعتبر امتدادا لبحث Alshawadfi (2003) والذي قدم فيه طريقة جديدة ومؤثرة للتنبؤ بنماذج ARMA, باستخدام أحد طرق الذکاء الاصطناعي وهي طريقة الشبکات العصبية, کما قارن بين الطريقة المقترحة وطريقة بوکس-جينکنز.ويتضمن هذا البحث هدفين: أولهما: تعميم طريقة Alshawadfi للتنبؤ بالسلاسل الزمنية المولدة من نماذج ARMAX باستخدام الشبکات العصبية الاصطناعية (ANN), ولتحقيق هذا الهدف تم توليد32000 عينة بأحجام مختلفة (25-40-60-100-150) من نماذج ARMAX بمعلمات مختلفة تستخدم لتدريب الشبکة,بعد ذلک تم مقارنة البيانات المولدة بالبيانات الحقيقية لقياس دقة التنبؤات. الهدف الثاني: مقارنة أداء الطريقة المقترحة للشبکات العصبية الاصطناعية مع أداء طريقة بوکس و جينکنز لتوضيح أي منهما أفضل للتنبؤ بنماذج ARMAX وذلک من خلال قياس دقة التنبؤات لکل من الطريقتين باستخدام ثلاث طرق هي متوسط مربعات الخطأ Mean of Squared Error (MSE), متوسط الانحرافات المطلقة Mean Absolute Deviation (MAD), نسبة الحالات التي تتحقق فيها أخطاء مطلقة أقل في الطريقتين The Percentage of cases of Minimum Absolute error (MAEP). ولانجاز عملية تدريب الشبکة واختبارها وحساب تنبؤات طريقتي بوکس-جينکنز والشبکات العصبية مع المقارنة بين الطريقتين تم استخدام حزم برامج ماتلاب MATLAB. وتم تصميم برنامج کمبيوتر مناسب ضمن حزم برامج ماتلاب MATLAB (أنظر الملحق المرفق مع البحث ) لتوليد البيانات و لانجاز عملية تدريب الشبکة واختبارها وحساب تنبؤات طريقتي بوکس-جينکنز والشبکات العصبية مع المقارنة بينهما. وقد أظهرت نتائج البحث وجود قدرة عالية للطريقة المقترحة للتنبؤ لنماذج ARMAX باستخدام أحد طرق الذکاء الاصطناعي - وهي طريقة الشبکات العصبية - على التنبؤ بالقيم المستقبلية للسلسلة الزمنية المعطاة وذلک بطريقة آلية, حيث أظهرت النتائج من دراسة 32000 عينة والتي تم توليدها أن متوسط مربعات أخطاء التنبؤMSE خاصة في حالة العينات الصغيرة, ومتوسط القيمة المطلقة لخطأ التنبؤ MAD, وکذلک متوسط النسبة MAEP أفضل من نظيرتها في أسلوب بوکس-جينکنز, وذلک کمتوسط عام بالنسبة لکل العينات وکل النماذج وللمشاهدات المستقبلية الثلاث الأولى. وبناءا على ذلک يمکن القول بصلاحية الطريقة المقترحة للتنبؤ بالقيم المستقبلية للسلاسل الزمنية المولدة من نماذج ARMAX. This paper has two objects. First, we present artificial neural networks method for forecasting linear and nonlinear ARAMAX time series. Second, we compare the proposed method with the well known Box-Jenkins method through a simulation study . To achieve these objects 32000 samples, generated from different ARMAX models, different sizes (25,40,60,100,150), were used for the network training. Then the system was tested for generated data . The accuracy of the neural network forecasts(NNF) is compared with the corresponding Box-Jenkins forecasts(BJF) by using three tools: the mean square error (MSE) , the mean absolute deviation of error (MAD) and the ratio of closeness from the true values (MPE) . A suitable computer program was designed (MATLAB TOOLBOX) for NN training , testing and comparing with Box-Jenkins method . The forecasts of theproposed NN approach, as shown from three measures, seem to provide better results than the classical forecasting Box-Jenkins approach . The results suggest thatthe ANN approach may provide a superior alternative to the Box-Jenkins forecasting approach for developing forecasting models in situations that do not require modeling of the internal structure of the series . The numerical results show that the proposed approach has a good performance for the forecasting of ARMAX(p,h,q) models.

Practical Classification Methods for Indoor Positioning

Location awareness is of primary importance in a wealth of applications such as transportation, mobile health systems, augmented reality and navigation. For example, in busy transportation areas (such as airports) providing clear, personalised notifications and directions, can reduce delays and improve the passenger journeys. Currently some applications provide easy access to information. These travel related applications can become context aware via the availability of accurate indoor/outdoor positioning. However, there are barriers that still have to overcome. One such barrier is the time required to set up and calibrate indoor positioning systems, another is the challenge of scalability with regard to the processing requirements of indoor positioning algorithms. This paper investigates the relationship between the calibration data and positioning system accuracy and analyses the performance of a k-Nearest Neighbour (k-NN) based positioning algorithm using real GSM data. Furthermore, the paper proposes a positioning scheme based on Gaussian Mixture Models (GMM). Experimental results show that the proposed GMM algorithm (without post-filtering) provides high levels of localization accuracy and successfully copes with the scalability problems that the conventional k- NN approach faces.

Effective feature extraction approach for fused images of Cartosat-I and Landsat ETM+ satellite sensors

This work presents an effective hybrid classification approach for feature extraction from fused images of two different satellite sensors. Wavelet transform function was used to fuse the panchromatic Cartosat-I and multispectral Landsat ETM+ sensor’s images which could preserve both the spatial and spectral components of the original images. Multi-resolution segmentations based on homogeneity criterion formed the basis for the hybrid approach which uses supervised fuzzy NN approach of classifier in conjunction with knowledge classification system. Gaussian fuzzy membership function was defined on an optimal set of object features such as the Normalized Difference Vegetation Index, band mean, area, shape index and brightness derived from the segmented image objects for class description. Based on our kappa index analysis evaluation, the hybrid approach provides significantly better performance than its other counterparts such as artificial neural network, maximum likelihood classifier and support vector machine in terms of classification accuracy.

A modified artificial immune system based pattern recognition approach—An application to clinical diagnostics

This paper introduces a modified artificial immune system (AIS)-based pattern recognition method to enhance the recognition ability of the existing conventional AIS-based classification approach and demonstrates the superiority of the proposed new AIS-based method via two case studies of breast cancer diagnosis. Conventionally, the AIS approach is often coupled with the k nearest neighbor (k-NN) algorithm to form a classification method called AIS-kNN. In this paper we discuss the basic principle and possible problems of this conventional approach, and propose a new approach where AIS is integrated with the radial basis function--partial least square regression (AIS-RBFPLS). Additionally, both the two AIS-based approaches are compared with two classical and powerful machine learning methods, back-propagation neural network (BPNN) and orthogonal radial basis function network (Ortho-RBF network). The diagnosis results show that: (1) both the AIS-kNN and the AIS-RBFPLS proved to be a good machine leaning method for clinical diagnosis, but the proposed AIS-RBFPLS generated an even lower misclassification ratio, especially in the cases where the conventional AIS-kNN approach generated poor classification results because of possible improper AIS parameters. For example, based upon the AIS memory cells of "replacement threshold=0.3", the average misclassification ratios of two approaches for study 1 are 3.36% (AIS-RBFPLS) and 9.07% (AIS-kNN), and the misclassification ratios for study 2 are 19.18% (AIS-RBFPLS) and 28.36% (AIS-kNN); (2) the proposed AIS-RBFPLS presented its robustness in terms of the AIS-created memory cells, showing a smaller standard deviation of the results from the multiple trials than AIS-kNN. For example, using the result from the first set of AIS memory cells as an example, the standard deviations of the misclassification ratios for study 1 are 0.45% (AIS-RBFPLS) and 8.71% (AIS-kNN) and those for study 2 are 0.49% (AIS-RBFPLS) and 6.61% (AIS-kNN); and (3) the proposed AIS-RBFPLS classification approaches also yielded better diagnosis results than two classical neural network approaches of BPNN and Ortho-RBF network. In summary, this paper proposed a new machine learning method for complex systems by integrating the AIS system with RBFPLS. This new method demonstrates its satisfactory effect on classification accuracy for clinical diagnosis, and also indicates its wide potential applications to other diagnosis and detection problems.

Off-line Signature Verification (SV) using the Chi-square statistics

Off-line Signature Verification (SV) is performed using Particle Swarm Optimisation Neural Network (PSO NN) algorithm. The technique is based on NN approach trained with PSO algorithm. The presented verification system includes image-processing techniques and other mathematical tools in its structure. To test the performance of the proposed algorithm, three types of forgeries, namely random, unskilled and skilled, are examined. A database with 1350 skilled and genuine signatures taken from 25 volunteers is used for testing the algorithm. The experimental results are presented with comparisons on verification accuracy and statistical figures.

Intelligent Diagnosis of Heart Diseases using Neural Network Approach

Experiments with the Switzerland Heart Disease database have concentrated on attempting to distinguish presence and absence. The classifiers based on various neural networks, namely, MLP, PCA, Jordan, GFF, Modular, RBF, SOFM, SVM NNs and conventional statistical techniques such as DA and CART are optimally designed, thoroughly examined and performance measures are compared in this study. With chosen optimal parameters of MLP NN, when it is trained and tested over cross validation (unseen data sets), the average (and best respectively) classification of 98±2.83 % (and 100%), 96.67±4.56% overall accuracy, sensitivity 96±5.48, specificity 100% are achieved which shows consistent performance than other NN and statistical models. The results obtained in this work show the potentiality of the MLP NN approach for heart diseases classification.

The Contribution of Neural Networks to Solve Corrosion Related Problems

This paper summarizes the results of various attempts to implement a neural network for solving corrosion problems. The first activity was aimed to develop a model able to predict crevice corrosion of stainless steel and related alloys in chloride containing media from long-term exposure tests. Second, the preliminary evaluation of a neural network approach for rapid prediction of naphthenic acid corrosion performance (NAC) of carbon and stainless steels in a crude oil distillation unit will be described. In this work, the neural network was trained on the basis of experimental data from laboratory experience. Finally, non-deterministic models based on artificial neural network (ANN) were developed to predict the corrosion rate of carbon steel in CO2 environment by elaborating laboratory and field data. NN models were developed and tested using, as an input, pattern physico-chemical variables typically met in empiric and mechanistic models as well as parameters apparently not involved in the corrosion phenomenon. Results confirmed the validity of the NN approach

Granular RBF NN Approach and Statistical Methods Applied to Modelling and Forecasting High Frequency Data

We examine the ARCH-GARCH models for the forecasting of the bond price time series provided by VUB bank and make comparisons the forecast accuracy with the class of RBF neural network models. A limited statistical or computer science theory exists on how to design the architecture of RBF networks for some specific nonlinear time series, which allows for exhaustive study of the underlying dynamics, and determination of their parameters. To illustrate the forecasting performance of these approaches the learning aspects of RBF networks are presented and an application is included. We show a new approach of function estimation for nonlinear time series model by means of a granular neural network based on Gaussian activation function modelled by cloud concept. In a comparative study is shown, that the presented approach is able to model and predict high frequency data with reasonable accuracy and more efficient than statistical methods.

Granular RBF NN Approach and Statistical Methods Applied to Modelling and Forecasting High Frequency Data

We examine the ARCH-GARCH models for the forecasting of the bond price time series provided by VUB bank and make comparisons the forecast accuracy with the class of RBF neural network models. A limited statistical or computer science theory exists on how to design the architecture of RBF networks for some specific nonlinear time series, which allows for exhaustive study of the underlying dynamics, and determination of their parameters. To illustrate the forecasting performance of these approaches the learning aspects of RBF networks are presented and an application is included. We show a new approach of function estimation for nonlinear time series model by means of a granular neural network based on Gaussian activation function modelled by cloud concept. In a comparative study is shown, that the presented approach is able to model and predict high frequency data with reasonable accuracy and more efficient than statistical methods.

Differential Evolution and Levenberg Marquardt Trained Neural Network Scheme for Nonlinear System Identification

This paper proposes a new nonlinear system identification scheme using differential evolution (DE), neural network and Levenberg Marquardt algorithm (LM). Here, DE and LM in a combined framework are used to train a neural network for achieving better convergence of neural network weight optimization. A number of examples including a practical case-study have been considered for implementation of different system identification methods namely, only NN, DE+NN and DE+LM+NN. After, a series of simulation studies of these methods on the different nonlinear systems it has been confirmed that the proposed DE and LM trained NN approach to nonlinear system identification has yielded better identification results in terms of time of convergence and less identification error.

Comparison of MLP NN Approach with PCA and ICA for Extraction of Hidden Regulatory Signals in Biological Networks

The biologists now face with the masses of high dimensional datasets generated from various high-throughput technologies, which are outputs of complex inter-connected biological networks at different levels driven by a number of hidden regulatory signals. So far, many computational and statistical methods such as PCA and ICA have been employed for computing low-dimensional or hidden representations of these datasets, but in most cases the results are inconsistent with underlying real network. In this paper we have employed and compared three linear (PCA and ICA) and non-linear (MLP neural network) dimensionality reduction techniques to uncover these regulatory signals, from outputs of such networks. The three approaches were verified experimentally using the absorbance spectra of a network of seven hemoglobin solutions, and the results revealed the superiority of the MLP NN to PCA and ICA. This study shows the capability of the MLP NN approach to efficiently determine the regulatory components in biological networked systems.

A GA-based NN approach for makespan estimation

The completion time or makespan estimation of a set of jobs in batch process industries is studied; Because jobs interact with each other at the level of shop floor, five interaction variables are defined and an indication of their influence on the makespan is experimentally investigated. A back-propagation network (BPN) model combined with genetic algorithms (GAs) to makespan estimation is proposed. GAs are adopted in the BPN to determine the parameters of BPN and to improve the accuracy of makespan estimation. Previously, there have been no appropriate rules to determine these parameters, 1000 training instances were used for training and evaluating the performance of the model. The study shows that combine GA with NN approach is more effective and accurate in estimating makespan than the BPN model by using trial and error.

EXPLAINING INTERNATIONAL INVESTMENT PATTERNS: A NEURAL NETWORK APPROACH

This paper aims to explain foreign portfolio investment impact on the dynamics of regional (Central and East Europe) capitalization structure in terms of sectorial investment distribution. Complexity of the problem domain has predetermined the use of interdisciplinary research framework. The ultimate result of this data analysis is a universally applicable model aimed at the nontraditional estimation of sectorial indices weights according to the relative capitalization of the economic sectors. The novelty of the proposed model lays in the application of neural network methods, which were thoroughly designed for the sectorial indices’ weights estimation. There is no experiment in the field which would focus on such an experimental setting. The overall NN research framework has developed into the software capable considerably to automate the whole research process. The results of the new approach decidedly outperformed the multivariate linear regression forecasting performance. We argue that the proposed NN approach will extend the assessment and forecasting power of the nonlinearities present in a nowadays volatile investment environment.

Hopfield neural network approaches to economic dispatch problems

This paper aims to identify and investigate a new Hopfield neural network structure suitable for application to economic dispatch problems such as unconstrained economic dispatch, economic dispatch with transmission capacity constraints and multi-area economic dispatch. A new mapping technique is described for quadratic programming problems with equality and inequality constraints. Side constraints are handled by a modified activation function. The proposed method is tested on different sizes of systems having between 3 and 240 generating units. The results of the proposed method are compared with those of a classical method and the improved Hopfield NN approach (IEE Proc Gen Transm Distrib 144(2) (1997) 181).