Cascade Correlation Network Research Articles

Identification of rhubarbs by using NIR spectrometry and temperature-constrained cascade correlation networks

Temperature-constrained cascade correlation networks (TCCCNs) were used to identify powdered rhubarbs based on their near-infrared spectra. Different network configurations that used multiple network models with single output (Uni-TCCCN) and single networks with multiple outputs (Multi-TCCCN) were compared. Comparative studies were made by using Latin-partitions and leave-one-out cross-validation methods. Results showed that multiple networks with single output predicted generally better than single network with multiple outputs. Better results with TCCCN models were obtained compared with conventional back propagation neural networks (BPNNs). The effects of parameters on correct identification and parameter optimizations were discussed in detail. With optimized neural network training parameters, NIR spectra from powdered rhubarb samples were classified by a TCCCN model with 100% accuracy.

Multiple Harmonic Source Detection and Equipment Identification With Cascade Correlation Network

This paper presents the design of a harmonic source detection system with a cascade correlation network (CCN). Current-injection-based harmonic power flow was used to calculate bus voltages and total harmonic distortion (THD). THD of voltages is used as indices for meter placement in the power network. At metering buses, it shows that the harmonic components of voltages under various loads would form particular patterns in the frequency domain, and can be used to create training examples for CCN. Good meter planning is helpful to reduce the number of meters and training examples. With an IEEE 14-bus power system, computer simulations were conducted to show the effectiveness of the proposed system.

Quality control of the powder pharmaceutical samples of sulfaguanidine by using NIR reflectance spectrometry and temperature-constrained cascade correlation networks

Temperature-constrained cascade correlation networks (TCCCNs) were applied to the identification of the powder pharmaceutical samples of sulfaguanidine based on near infrared (NIR) diffuse reflectance spectra and their first derivative spectra. This work focused on the comparison of performances of the uni-output TCCCN (Uni-TCCCN) and multi-output (Multi-TCCCN) by near infrared diffuse reflectance spectra and their first derivative spectra of sulfaguanidine. The TCCCN models were verified with independent prediction samples by using the “cross-validation” method. The networks were used to discriminate qualified, un-qualified and counterfeit sulfaguanidines pharmaceutical powders. The results showed that single outputs network generally worked better than the multiple outputs networks, and the first derivative spectra were more suitable for the identification comparing with original diffuse reflectance spectra. With proper network parameters the pharmaceutical powders can be classified at rate of 100% in this work. Also, the effects of parameters and related problems were discussed.

An Emerging Consensus: Younger and Cohen Revisited.

From Aesop to Sun Tzu, the importance of working together has long been acknowledged. Yet as long as cooperation has existed, so have the difficulties associated with it. Pooling two fields might mean twice the power, but this union also brings twice the jargon, twice the competing theories, and twice the head butting. Nonetheless, in this collection, researchers have made a heroic effort to set aside their theoretical differences to produce three computational models for the influential set of empirical data showing that young infants have difficulty detecting correlations among features (Younger & Cohen, 1983, 1986). Specifically, Gureckis and Love (2004/this issue) fit a well-developed adult-learning model to the infant work (down to the exact order and number of test trials) in support of a common mechanism underlying categorization. Shultz and Cohen (2004/this issue) vary depth of processing in a cascade correlation network to show how older infants learn more than younger infants from the same amount of exposure. And Westermann and Mareschal (2004/this issue) introduce a model for their representational acuity hypothesis, which explains qualitative shifts as a decrease in the size of neural receptive fields. This commentary takes stock of this attempt at unifying computational modeling and developmental data by identifying common themes, clarifying points of disagreement, and providing a synthesis for this work.

Event-orthogonal error-insensitive multiple fault detection with cascade correlation network

This paper presents the design of a fault detection system with cascade correlation network (CCN) for a power system. Associate fault components with the states of protective devices would form symptomatic patterns to create training data. The proposed method makes use of information from both the primary and back-up devices involving single fault, multiple faults, data communication with errors, or fault with the failed operation of relays and circuit breakers. With a sample power system, computer simulations were conducted to show the effectiveness of the proposed system.

ECONOMIC VALUATION OF CAPITAL PROJECTS USING NEURAL NETWORK METAMODELS

ABSTRACT It is often useful to incorporate a metamodel into the sensitivity analysis of the economics of capital projects. While statistical models are straightforward and understandable, they are not always appropriate, especially where there are discontinuities or complex relationships in the economic data. This paper puts forth a neural network approach that uses design of experiments to generate metamodel training and validation data from the project(s) under consideration. Both backpropagation multi-layered perceptron networks and cascade correlation networks are explored and compared with the statistical approach. It is shown that the backpropagation networks work well for a variety of data sets and can be superior to regression in prediction accuracy when the economic data is especially complicated.

Event-Orthogonal Error-Insensitive Multiple Fault Detection with Cascade Correlation Network

This paper presents the design of a fault detection system with cascade correlation network (CCN) for a power system. Associate fault components with the states of protective devices would form symptomatic patterns to create training data. The proposed method makes use of information from both the primary and back-up devices involving single fault, multiple faults, data communication with errors, or fault with the failed operation of relays and circuit breakers. With a sample power system, computer simulations were conducted to show the effectiveness of the proposed system.

Validation using sensitivity and target transform factor analyses of neuralnetwork models for classifying bacteriafrom mass spectra

Temperature constrained cascade correlation networks (TCCCNs) are computational neural networks that configure their own architecture, train rapidly, and give reproducible prediction results. TCCCN classification models were built using the Latin-partition method for five classes of pathogenic bacteria. Neural networks are problematic in that the relationships among the inputs (i.e., mass spectra) and the outputs (i.e., the bacterial identities) are not apparent. In this study, neural network models were constructed that successfully classified the targeted bacteria and the classification model was validated using sensitivity and target transformation factor analysis (TTFA). Without validation of the classification model, it is impossible to ascertain whether the bacteria are classified by peaks in the mass spectrum that have no causal relationships with the bacteria, but instead randomly correlate with the bacterial classes. Multiple single output network models did not offer any benefits when compared to single network models that had multiple outputs. A multiple output TCCCN model achieved classification accuracies of 96 ± 2% and exhibited improved performance over multiple single output TCCCN models. Chemical ionization mass spectra were obtained from in situ thermal hydrolysis methylation of freeze-dried bacteria. Mass spectral peaks that pertain to the neural network classification model of the pathogenic bacterial classes were obtained by sensitivity analysis. A significant number of mass spectral peaks that had high sensitivity corresponded to known biomarkers, which is the first time that the significant peaks used by a neural network model to classify mass spectra have been divulged. Furthermore, TTFA furnishes a useful visual target as to which peaks in the mass spectrum correlate with the bacterial identities.

Estimating tree mortality of Norway spruce stands with neural networks

Within forest growth modeling LOGIT models are used to predict individual tree mortality. In this paper we present, Multi-Layer Perceptron, Learning Vector Quantization and Cascade Correlation networks as different formalisms for mortality predictions. The data set for parameterizing the LOGIT model and training the different neural network types comes from the Austrian National Forest Inventory. After training the different network types, we evaluate the resulting mortality predictions using an independent data set from the Litschau forest. The results indicate that Multi-Layer Perceptron with the learning algorithm resilient back-propagation and scaled conjugate gradient and Cascade Correlation with learning algorithm resilient back-propagation perform the best predictions. This suggests that neural networks are a viable alternative to the conventional LOGIT approach.

Growing Cascade Correlation Networks in Two Dimensions: A Heuristic Approach

Growing Cascade Correlation Networks in Two Dimensions: A Heuristic Approach

Using the Cascade-Correlation Algorithm to Evaluate Investment Projects

This paper considers some aspects of using a cascade-correlation network in the invest- ment task in which it is required to determine the most suitable project to invest money. This task is one of the most often met economical tasks. In various bibliographical sources on economics there are described different methods of choosing investment projects. However, they all use either one or a few criteria, i.e., out of the set of criteria there are chosen most valuable ones. With this, a lot of information contained in other choice criteria is omitted. A neural network enables one to avoid information losses. It accumulates information and helps to gain better results when choosing an investment project in comparison with classical methods. The cascade-correlation network archi- tecture that is used in this paper has been developed by Scott E. Fahlman and Cristian Lebiere at Carnegie Mellon University.

Evaluation of neural network models with generalized sensitivity analysis

A sensitivity analysis method for discovering characteristic features of the input data using neural network classification models has been devised. The sensitivity is the gradient of the neural network model response function, and because neural network models are nonlinear, the gradient depends on the point where it is evaluated. Two criteria are used for measuring the sensitivity. The first criterion calculates the sensitivity or gradient of the neural network output with respect to the average of the objects that comprise each class. The second criterion measures the average sensitivity of the class objects. The sensitivity analysis was applied to temperature-constrained cascade correlation network models and evaluated with sets of synthetic data and experimental mobility spectra. The neural network models were built using temperature-constrained cascade correlation networks (TCCCNs). A weight constraint was devised for the output units of the network models. This method implements weight decay with conjugate gradient training and yields more sensitive neural network models. Temperature-constrained hidden units furnish more sensitive network models than networks without constraints. By comparing the sensitivities of the class mean input and the mean sensitivity for all the inputs of a class, the individual input variables may be assessed for linearity. If these two sensitivities for an input variable differ by a constant factor, then that variable is modeled by a simple linear relationship. If the two sensitivities vary by a nonconstant scale factor, then the variable is modeled by higher order functions in the network. The sensitivity method was used to diagnose errors in the training data, and the test for linearity indicated a TCCCN architecture that had better predictability.

Screening GC-MS data for carbamate pesticides with temperature-constrained–cascade correlation neural networks

Aromatic carbamate pesticides are important agrochemicals. Mass spectral classification models were built for carbamates and their substructures using temperature-constrained–cascade correlation networks (TC–CCNs). The carbamate classifier was applied to the mass spectral scans of a GC-MS run. The classification models were built from reference and experimental mass spectra. Different network configurations were compared that used multiple network models with single outputs and single networks with multiple outputs. A major source of variation caused by randomly partitioning the training and prediction sets was reduced by an order of magnitude by using a method of Latin-partitions. This method also furnished a precision measure for comparing classification methods. Multiple networks with single outputs generally predicted better than single networks with multiple outputs. Hierarchical single output networks achieved better than 98% classification accuracy in one study. The TC–CCN models compared favorably to the K-nearest neighbors (KNN) and discriminant partial least squares (DPLS) reference methods.

Development of Children's Seriation: A Connectionist Approach

This paper presents a modular connectionist network model of the development of seriation (sorting) in children. The model uses the cascade-correlation generative connectionist algorithm. These cascade-correlation networks do better than existing rule-based models at developing through soft stage transitions, sorting more correctly with larger stimulus size increments and showing variation in seriation performance within stages. However, the full generative power of cascade-correlation was not found to be a necessary component for successfully modelling the development of seriation abilities. Analysis of network weights indicates that improvements in seriation are due to continuous small changes instead of the radical restructuring suggested by Piaget. The model suggests that seriation skills are present early in development and increase in precision during later development. The required learning environment has a bias towards smaller and nearly ordered arrays. The variability characteristic of children's performance arises from sorting subsets of the total array. The model predicts better sorting moves with more array disorder, and a dissociation between which element should be moved and where it should be moved.

Robust Neural Net-Based Inverse-Model Identification of an Induction Motor

Four artificial neural-net training models, Back-propagation, Logicon projection TM, Modular, and Cascade Correlation networks, were used for inversemodel identification of a 3-hp, three-phase induction motor. The fully trained networks were then tested on three similar machines but with distinct characteristics. Results show that the Modular neural network and the Cascade Correlation network were the least sensitive to the differences in the three motors. The two would therefore be considered the best candidates for model identification in adjustable drives when continuous on-line training is undesirable. It is expected that after drive commissioning, the model would maintain an acceptable dynamic and steady state performance through the working life of the motor. This would make the use of expensive on-line self-tuning apparatuses unnecessary for a wide range of adjustable drive applications.

Analysis of Knowledge Representations in Cascade Correlation Networks

Feed-forward neural network models approximate nonlinear functions connecting inputs to outputs. The cascade correlation (CC) learning algorithm allows networks to grow dynamically starting from the simplest network topology to solve increasingly more difficult problems. It has been demonstrated that the CC network can solve a wide range of problems including those for which other kinds of networks (e.g., back-propagation networks) have been found to fail. In this paper we show the mechanism and characteristics of nonlinear function learning and representations in CC networks, their generalization capabilities, the effects of environmental bias, etc., using a variety of knowledge representation analysis tools.

Temperature-Constrained Cascade Correlation Networks

A novel neural network has been devised that combines the advantages of cascade correlation and computational temperature constraints. The combination of advantages yields a nonlinear calibration method that is easier to use, stable, and faster than back-propagation networks. Cascade correlation networks adjust only a single unit at a time, so they train very rapidly when compared to back-propagation networks. Cascade correlation networks determine their topology during training. In addition, the hidden units are not readjusted once they have been trained, so these networks are capable of incremental learning and caching. With the cascade architecture, temperature may be optimized for each hidden unit. Computational temperature is a parameter that controls the fuzziness of a hidden unit's output. The magnitude of the change in covariance with respect to temperature is maximized. This criterion avoids local minima, forces the hidden units to model larger variances in the data, and generates hidden units that furnish fuzzy logic. As a result, models built using temperature-constrained cascade correlation networks are better at interpolation or generalization of the design points. These properties are demonstrated for exemplary linear interpolations, a nonlinear interpolation, and chemical data sets for which the numbers of chlorine atoms in polychlorinated biphenyl molecules are predicted from mass spectra.

Supervised neural networks for the classification of structures

Standard neural networks and statistical methods are usually believed to be inadequate when dealing with complex structures because of their feature-based approach. In fact, feature-based approaches usually fail to give satisfactory solutions because of the sensitivity of the approach to the a priori selection of the features, and the incapacity to represent any specific information on the relationships among the components of the structures. However, we show that neural networks can, in fact, represent and classify structured patterns. The key idea underpinning our approach is the use of the so called "generalized recursive neuron", which is essentially a generalization to structures of a recurrent neuron. By using generalized recursive neurons, all the supervised networks developed for the classification of sequences, such as backpropagation through time networks, real-time recurrent networks, simple recurrent networks, recurrent cascade correlation networks, and neural trees can, on the whole, be generalized to structures. The results obtained by some of the above networks (with generalized recursive neurons) on the classification of logic terms are presented.

On the Computational Power of Recurrent Neural Networks for Structures

Recurrent neural networks can simulate any finite state automata as well as any multi-stack Turing machine. When constraining the network architecture, however, this computational power may no longer hold. For example, recurrent cascade-correlation cannot simulate any finite state automata. Thus, it is important to assess the computational power of a given network architecture, since this characterizes the class of functions which, in principle, can be computed by it. We discuss the computational power of neural networks for structures. Elman-style networks, cascade-correlation networks and neural trees for structures are introduced. We show that Elman-style networks can simulate any frontier-to-root tree automation, while neither cascade-correlation networks nor neural trees can. As a special case of the latter result, we obtain that neural trees for sequences cannot simulate any finite state machine. © 1997 Elsevier Science Ltd. All Rights Reserved.

Application of the Cascade-Correlation network to the classification of circularly scanned images

This paper describes an application of the Cascade-Correlation (CC) network to pattern recognition. The pattern recognition task was to simulate an automatic vision inspection system that had to properly classify five different objects. The feature vectors were extracted from 2D images of circularly scanned images and used as inputs for a neural network that was then trained to classify an unknown presented object. The results show that the CC network is viable tool in pattern recognition tasks. It is able to classify partially occluded objects with high accuracy, and to considerably improve classification of noisy images based on simple histogram trimming preprocessing.