Black-box Neural Network Research Articles

Nonlinear Black-box System Identification through Neural Networks of a Hysteretic Piezoelectric Robotic Micromanipulator

Piezoelectric micromanipulators are used in applications with precise and high dynamics positioning. This recognition is thanks to their high resolution, bandwidth and stiffness. Its nonlinear behavior, however, complicates the design of robust control laws with respect to no or imprecise sensing. In this context, this work presents the identification of a piezoelectric micromanipulator through nonlinear black-box neural networks with data acquired in a laboratory setup. A comparison is made regarding the model complexity. The results show the accuracy of the models, their statistical validity and that they were able to capture the dynamics of the micromanipulator adequately.

Approximate Reliability Function Based on Wavelet Latin Hypercube Sampling and Bee Recurrent Neural Network

This work combines a Bee Recurrent Neural Network (BRNN) optimized by the Artificial Bee Colony (ABC) algorithm with Monte Carlo Simulation (MCS) to generate a novel approximate model for predicting network reliability. We utilize the Wavelet Transform (WT)-based Latin Hypercube Sampling (LHS) (WLHS) to select input training data, and open the black box of neural networks by constructing a limited space reliability function from neural network parameters. Furthermore, the proposed method compares favorably with existing methods in literature based on experimental results for a benchmark example. The result reveals that the novel WLHS-MCS based on BRNN (WLHS-BRNN-MCS for short) is an excellent estimator of the reliability function.

Application of Neural Network and Particle Swarm Optimization Algorithm in Slope Runoff and Sediment Yield Calculation

Neural network black box model for predicting the slope runoff and sediment yield and two empirical equations for calculating the slope runoff and sediment yield were established with the basis of practical field data of slope runoff and sediment amount by artificial simulated rainfall experiments. In additional, particle swarm optimization algorithm is used to inquire the empirical equation’s unknown parameters based on least square method. And results show that, neural network model might represent the nonlinear relationship between runoff, sediment amount and each impact factor excellently. Furthermore, predicted results are satisfactory and its relative error mean is around 10%. Empirical equations are reasonably and reliable, its relative error mean is less than 20%. These two methods provide an operable means for such intricate research of slope runoff and sediment yield predication and calculation.

Combining models from neural networks and inductive learning algorithms

The knowledge-based artificial neural network (KBANN) is composed of phases involving the expression of domain knowledge, the abstraction of domain knowledge at neural networks, the training of neural networks, and finally, the extraction of rules from trained neural networks. The KBANN attempts to open up the neural network black box and generates symbolic rules with (approximately) the same predictive power as the neural network itself. An advantage of using KBANN is that the neural network considers the contribution of the inputs towards classification as a group, while rule-based algorithms like C5.0 measure the individual contribution of the inputs one at a time as the tree is grown. The knowledge consolidation model (KCM) combines the rules extracted using KBANN (NeuroRule), frequency matrix (which is similar to the Naïve Bayesian technique), and C5.0 algorithm. The KCM can effectively integrate multiple rule sets into one centralized knowledge base. The cumulative rules from other single models can improve overall performance as it can reduce error-term and increase R-square. The key idea in the KCM is to combine a number of classifiers such that the resulting combined system achieves higher classification accuracy and efficiency than the original single classifiers. The aim of KCM is to design a composite system that outperforms any individual classifier by pooling together the decisions of all classifiers. Another advantage of KCM is that it does not need the memory space to store the dataset as only extracted knowledge is necessary in build this integrated model. It can also reduce the costs from storage allocation, memory, and time schedule. In order to verify the feasibility and effectiveness of KCM, personal credit rating dataset provided by a local bank in Seoul, Republic of Korea is used in this study. The results from the tests show that the performance of KCM is superior to that of the other single models such as multiple discriminant analysis, logistic regression, frequency matrix, neural networks, decision trees, and NeuroRule. Moreover, our model is superior to a previous algorithm for the extraction of rules from general neural networks.

Generalized Constraint Neural Network Model System Parameter Identification

By analyzing and deducing generalized constraint neural network (GCNN) with model some present theories, the identification method of the m-input n-output (MINO) and multiple-input multiple–output (MIMO) systems is acquired. It is possible to improve the transparency of the black box through the practical test. This identification method is useful to enhance identification of GCNN model’s parameters, moreover, the identification ability of the neural network black box system model is further made better.

Hybrid modeling of wire cable vibration isolation system through neural network

A hybrid modeling method based on neural network (NN) is developed and used to model the hysteretic restoring force of a wire cable vibration isolation system for electronic equipment. Firstly, a knowledge-based model for the nonlinear hysteretic restoring force is identified using the measured data obtained from period loading tests. Secondly, the remaining characteristic of hysteretic restoring force, which cannot be modeled in an easy way, is identified using the NN method through network training. By building up a parallel hybrid NN model for the nonlinear hysteretic restoring force, the dynamic responses of the vibration isolation system under harmonic and broad band random excitations are predicted. The predicted results are compared with the measured ones to validate the effectiveness and prediction accuracy of the model. The comparative studies show the developed hybrid NN model possesses good prediction accuracy and generalization capability in contrast with the pure black box NN model.

Comparison of black-, white-, and grey-box models to predict ultimate tensile strength of high-strength hot rolled coils at the Port Talbot hot strip mill

In the manufacture of rolled steel from a hot strip mill (HSM), the final mechanical properties, such as ultimate tensile strength, are important requirements specified by the customer. Traditional, physically based models are now being challenged by the development of black-box techniques such as artificial neural networks. In this present paper, an existing white-box physical equation for predicting ultimate tensile strength in high-strength steels is compared to an optimized black-box neural network model using the same model inputs. The predictive accuracy of both models is compared through the use of previously unseen data from a validation coil where the ultimate tensile strength throughout its length is known. In addition, a combination of the white-box equation and a black-box neural network connected in parallel, whose role is to predict the white-box approximation error, provides a grey-box solution for ultimate tensile strength prediction for high-strength coils at the Port Talbot HSM. It was found that this grey-box solution outperformed the stand-alone white-and black-box models in terms of predictive accuracy.

Biological reaction modeling using radial basis function networks

The difficulty associated with experimental studies of biochemical systems often makes the development of pure black-box neural network models particularly delicate. Hence, it is appealing to resort to a hybrid physical-neural network approach, which uses all the available a priori knowledge about the process, and combines a first-principles model with a partial neural network (NN) model describing the phenomena, which are (at least partly) unknown. In this work, this strategy is applied to a real-case experimental study, i.e. batch CHO animal cell cultures. Several alternative model formulations are considered, including serial model structures, in which neural networks are used to describe either the reaction kinetics or the complete reaction rates (globalizing pseudo-stoichiometry and kinetics), or parallel model structures, in which a NN compensates for the prediction errors of a first-principles model. Attention is focused on the procedure used to estimate the unknown NN parameters and initial conditions from experimental data, including a maximum likelihood approach to take account of all the measurement errors, and a weight decay technique to alleviate identifiability problems. The good model agreement is demonstrated with cross-validation tests.

Opening the black box of neural networks for remote sensing image classification

Neural networks, which make no assumption about data distribution, have achieved improved image classification results compared to traditional methods. Unfortunately, a neural network is generally perceived as being a ‘black box’. It is extremely difficult to document how specific classification decisions are reached. Fuzzy systems, on the other hand, have the capability to represent classification decisions explicitly in the form of fuzzy ‘if-then’ rules. However, the construction of a knowledge base, especially the fine-tuning of the fuzzy set parameters of the fuzzy rules in a fuzzy expert system, is a tedious and subjective process. This research has developed a new, improved neuro-fuzzy image classification system based on the synergism between neural networks and fuzzy expert systems. It incorporates the best of both technologies and compensates for the shortcomings of each. The learning algorithms of neural networks developed here are used to automate the derivation of fuzzy set parameters for the fuzzy ‘if-then’ rules in a fuzzy expert system. The rules obtained, in symbolic form, facilitate the understanding of the neural network based image classification system. In addition, the image classification accuracy obtained from the improved neuro-fuzzy system was significantly superior to those of the back-propagation based neural network and the maximum likelihood approaches.

Entering the black box of neural networks.

Artificial neural networks have proved to be accurate predictive instruments in several medical domains, but have been criticized for failing to specify the information upon which their predictions are based. We used methods of relevance analysis and sensitivity analysis to determine the most important predictor variables for a validated neural network for community-acquired pneumonia. We studied a feed-forward, back-propagation neural network trained to predict pneumonia among patients presenting to an emergency department with fever or respiratory complaints. We used the methods of full retraining, weight elimination, constant substitution, linear substitution, and data permutation to identify a consensus set of important demographic, symptom, sign, and comorbidity predictors that influenced network output for pneumonia. We compared predictors identified by these methods to those identified by a weight propagation analysis based on the matrices of the network, and by logistic regression. Predictors identified by these methods were clinically plausible, and were concordant with those identified by weight analysis, and by logistic regression using the same data. The methods were highly correlated in network error, and led to variable sets with errors below bootstrap 95% confidence intervals for networks with similar numbers of inputs. Scores for variable relevance tended to be higher with methods that precluded network retraining (weight elimination) or that permuted variable values (data permutation), compared with methods that permitted retraining (full retraining) or that approximated its effects (constant and linear substitution). Methods of relevance analysis and sensitivity analysis are useful for identifying important predictor variables used by artificial neural networks.

Entering the Black Box of Neural Networks

Summary Objectives: Artificial neural networks have proved to be accurate predictive instruments in several medical domains, but have been criticized for failing to specify the information upon which their predictions are based. We used methods of relevance analysis and sensitivity analysis to determine the most important predictor variables for a validated neural network for community-acquired pneumonia. Methods: We studied a feed-forward, back-propagation neural network trained to predict pneumonia among patients presenting to an emergency department with fever or respiratory complaints. We used the methods of full retraining, weight elimination, constant substitution, linear substitution, and data permutation to identify a consensus set of important demographic, symptom, sign, and comorbidity predictors that influenced network output for pneumonia. We compared predictors identified by these methods to those identified by a weight propagation analysis based on the matrices of the network, and by logistic regression. Results: Predictors identified by these methods were clinically plausible, and were concordant with those identified by weight analysis, and by logistic regression using the same data. The methods were highly correlated in network error, and led to variable sets with errors below bootstrap 95% confidence intervals for networks with similar numbers of inputs. Scores for variable relevance tended to be higher with methods that precluded network retraining (weight elimination) or that permuted variable values (data permutation), compared with methods that permitted retraining (full retraining) or that approximated its effects (constant and linear substitution). Conclusion: Methods of relevance analysis and sensitivity analysis are useful for identifying important predictor variables used by artificial neural networks.

Hybrid Physical - Neural Network Modeling of Animal Cell Cultures

ABSTRACT The sparsity and low quality of measurement data in biochemical applications often make the development of black-box neural network models particularly delicate. Hence, it is appealing to resort to a hybrid physical-neural network approach, which combines a first-principles model and a partial neural network model. In this study, the hybrid approach is applied to a real case study, e.g. batch CHO animal cell cultures. Two alternative model structures are developed, in which NNs are used to describe either the reaction kinetics or the complete reaction rates (including the reaction pseudostoichiometry). Parameters and initial conditions are estimated from experimental data using a maximum likelihood approach, which takes all the measurement errors into account. A special procedure for initializing the minimization of the objective function is devised. The good model agreement is demonstrated with cross-validation tests.

Real-time daily flow forecasting using black-box models, diffusion processes, and neural networks

The purpose of this study is to compare three modeling approaches used for the prediction of daily natural flows 1-7 days ahead. Linear black-box models, which have been commonly used for modeling flows, constitute the first approach. The second approach, a linear type in the context of our application, is less known in the water resources field and is identified by the term diffusion process. The third approach uses models called neural networks, which have gained interest in many fields. All these approaches were tested on 15 watersheds from the Saguenay - Lac-Saint-Jean hydrographic system, located in the province of Quebec, Canada. Because the watersheds possess different physical characteristics, the models were tested under several runoff conditions. In this article, the focus is on results; all approaches along with their conditions of use have been detailed elsewhere in the literature. The results obtained showed that neural networks constitute, for almost all the watersheds studied, the best approach to forecast daily natural flows. The more flexible structure of neural networks allows a best reproduction of complex runoff conditions. However, neural networks are more sensitive to outliers present in observed natural flow series, which are used as inputs in the three models tested. In practice, to model flows at specific periods of the year, it seems preferable to establish seasonal models. If a neural network has an inadequate structure for the period under consideration, then it may produce less convincing results than the other two modeling approaches tested in this study.Key words: forecasts, flows, black-box model, diffusion process, neural network.

Metabolism and Cell Cycle Modelling by Means of Neural Network Based Structures

Due to their importance to the pharmaceutical industry and implicitly to the health, the animal cell cultures and the research carried out over their behaviour has gained recently an increasing attention. The paper presents a case study concerning the application of the neural networks in the modelling of the main parameters of an animal cell culture: the metabolic components and the cell cycle phases. For the metabolism a hybrid (neural-classical) structure is described for the building of a continuous simulator capable to reconstruct the trajectory of the main components from the initial conditions. A special attention is paid to the choice of the cost function and to the "amount" of the classical contribution to the hybrid model. For the cell cycle the outputs of the above continuous simulator are used and a neural network black box approach is employed in order to obtain the evolution of the three phases. The structures were tested on two batch animal cell cultures for which few and asynchronous measurements are available, namely one culture growing on microcarriers, the other in suspension.

Real-time daily flow forecasting using black-box models, diffusion processes, and neural networks

Real-time daily flow forecasting using black-box models, diffusion processes, and neural networks

Neural networks for the prediction of the state of Zymomonas mobilis CP4 batch fermentations

The capability of different neural networks for predicting the main state variables (biomass, substrate and ethanol concentrations, the output variables) in Zymomonas mobilis CP4 batch fermentations has been tested. Experimental data recorded from batch fermentations carried out under different conditions (medium composition and temperature) were used to train the net and test its predictions. First a black-box neural network was designed for predicting the three output values given as input values of the present condition of the system. The configuration of the neural network that gives the best results is the one with 10 neurons in the hidden layer (total training error equal to 0.575×10 −3, total testing error equal to 0.719×10 −3). Better results can be obtained using different neural networks for each of the output variables (total testing error equal to 0.527×10 −3). Attempts were made to enhance the prediction capability of neural network models, using the mathematical model that describes the process and a neural network for computing the kinetic parameters in the model. Here, the error is higher than in previous cases. The best prediction is obtained using a neural network with 30 neurons in the hidden layer (total training error equal to 5.535×10 −3, total testing error equal to 2.225×10 −3).

Hybrid Modeling for Mechanical Systems: Methodologies and Applications

A serial hybrid modeling approach is applied to mechanical systems. Here, hybrid means that models are based on combined structural and empirical approaches. The main system behavior is described by a physical model, while complex internal forces are modeled by black box neural networks. For a special class of systems this methodology is extended and a novel approach is presented modeling the whole system behavior by hierarchical neural networks, that fit the relation between system outputs and internal system variables. Useful information about the nonlinear system can be extracted from the resulting models. The power of hybrid modeling is illustrated with experimental results and some important issues considering the practical implementation are dealt with.

Hybrid Neural Network-First Principles Models Applied to a Food Process

In the recent years hybrid modelling approaches combining neural networks and first principle models have attracted interest. The main reasons are that a black-box neural network model is not physically interpretable and not as extensive as a first principle model, while the development of an accurate mechanistic model for a specific plant is costly and difficult. In this paper a number of hybrid approaches are described and applied to develop models of a continuous food process plant. Results presented indicate that the hybrid approaches generate accurate predictions of food product quality.

Modeling of Activated Sludge Wastewater Treatment Processes Using Integrated Neural Networks and a First Principle Model

In recent years, hybrid modeling approaches which combine neural networks and first principle models have attracted interest. The main reasons are that a black-box neural network model possesses no physical basis and therefore it can be inaccurate compared to a first principle model. However, development of an accurate first principle model for complex process is costly and difficult. The hybrid modeling approach can take advantages of both neural network and first principle models and offer a better solution. This paper presents such an application in the modeling of activated sludge wastewater treatment plants. A parallel hybrid model is used to model an oxidation ditch where various biochemical reactions occur to remove organic matter and nitrogen. A neural network is integrated with a first principle model and is trained to compensate for the output errors of the first principle model. Simulated and plant data show that the trained neural network can reasonably capture information not included in the first principle model. As a result, the hybrid model generates accurate predictions in a number of different cases.