Knowledge-based Neural Network Research Articles

Knowledge-based approach to septic shock patient data using a neural network with trapezoidal activation functions

In this contribution we present an application of a knowledge-based neural network technique in the domain of medical research. We consider the crucial problem of intensive care patients developing a septic shock during their stay at the intensive care unit. Septic shock is of prime importance in intensive care medicine due to its high mortality rate. Our analysis of the patient data is embedded in a medical data analysis cycle, including preprocessing, classification, rule generation and interpretation. For classification and rule generation we chose an improved architecture based on a growing trapezoidal basis function network for our metric variables. Our results extend those of a black box classification and give a deeper insight in our patient data. We evaluate our results with classification and rule performance measures. For feature selection we introduce a new importance measure.

Some applications of neural networks in microwave modeling

This paper presents some applications of neural networks in the microwave modeling. The applications are related to modeling of either passive or active structures and devices. Modeling is performed using not only simple multilayer perception network (MLP) but also advanced knowledge based neural network (KBNN) structures.

On-line learning, reasoning, rule extraction and aggregation in locally optimized evolving fuzzy neural networks

Fuzzy neural networks are connectionist systems that facilitate learning from data, reasoning over fuzzy rules, rule insertion, rule extraction, and rule adaptation. The concept of a particular class of fuzzy neural networks, called FuNNs, is further developed in this paper to a new concept of evolving neuro-fuzzy systems (EFuNNs), with respective algorithms for learning, aggregation, rule insertion, rule extraction. EFuNNs operate in an on-line mode and learn incrementally through locally tuned elements. They grow as data arrive, and regularly shrink through pruning of nodes, or through node aggregation. The aggregation procedure is functionally equivalent to knowledge abstraction. EFuNNs are several orders of magnitude faster than FuNNs and other traditional connectionist models. Their features are illustrated on a bench-mark data set. EFuNNs are suitable for fast learning of on-line incoming data (e.g., financial time series, biological process control), adaptive learning of speech and video data, incremental learning and knowledge discovery from large databases (e.g., in Bioinformatics), on-line tracing processes over time, life-long learning. The paper includes also a short review of the most common types of rules used in the knowledge-based neural networks.

A knowledge-based neural network for fusing edge maps of multi-sensor images

With the goal of fusion prescribed as building an edge map that contains as many edges as possible from the given multi-spectral/sensor images, a new fusion scheme, called the knowledge-based neural network fusion (KBNNF), is proposed to fuse edge maps of these images in order to generate a combined edge map that has more complete and reliable edge information than what one can obtain from any single image. The KBNNF is used to fuse edge maps of images having mutually complementary edge information in the following sense: (i) the edges in the images are compatible, i.e., can be interpreted together; and (ii) the edges in the different images reveal different parts of the scene. More complete edge contours of the same object are obtained by linking the edge sections obtained from different images together. The resulting edge map can be used for subsequent study (like object recognition). The proposed scheme bases its confidence and reliability on the analysis of variance (ANOVA)-based edge detector that can address two important issues of edge based image fusion well: (i) the difference in edge position among the images because of the different characteristics of the images and the error in the image registration process; and (ii) the variance existing among the edge test values calculated from different images. The KBNNF has been applied to fuse: (i) radar (SAR)–optical (SPOT), (ii) optical–optical, (iii) infrared–infrared, and (iv) optical–infrared (satellite) image combinations. Comparisons are made with the relevant existing techniques in the literature. The paper concludes with some examples to illustrate the efficacy of the proposed scheme.

Knowledge-Based Artificial Neural Network Models for Finline

Finline plays an important role in millimeter-wave integrated-circuit design. In this paper, a knowledge-based artifcial neural network is used to model the finline. Using prior knowledge input method and Bayesian regularization technique make the neural network models for finline reduce the amount of training data needed and prevent overfitting in neural network training. The neural network is electromagnetically developed with a set of training data that are produced by the fnite element method, which is robust both from the angle of time of computation and accuracy.

Knowledge-Based Artificial Neural Network Models for Microstrip Radial Stubs

Radial stubs are a superior choice over low characteristic impedance rectangular stubs in terms of providing an accurate localized zero-impedance reference point and maintaining a low input impedance value over a wide frequency range. In this paper, knowledge-based artificial neural networks are used to model the microstrip radial stubs. Using space-mapping technology and Huber optimization make the neural network models for radial stubs decrease the number of training data, improve generalization ability, and reduce the complexity of the neural network topology with respect to the classical neuromodeling approach. The neural networks are developed for design and optimization of radial stubs, which are robust both from the angle of time of computation and accuracy.

Neuro-fuzzy prediction of uricase production

Recent biotechnology requires implementation of new modelling methods based on knowledge principles and learning structures, comprised in fuzzy knowledge-based systems (FKBS), neural networks (NN) and different hybrid methods. The intelligent modelling approaches solve sufficiently a very important problem – processing of scarce, uncertainty and incomplete numerical and linguistic information about multivariate non-linear and non-stationary systems as well as biotechnological processes. The paper deals with prediction of an enzyme oxidizing uric acid to alantoin – the uricase, produced by Candida utilis 90-12 employing neuro-fuzzy knowledge-based approach. The implemented predictive technique exploits the fact that the fuzzy model can be seen as a network structure, similar to artificial NN, which on computational level assure a high model accuracy. The predictors implemented are four different by nature variables. The developed predictive model shows that best predictors of uricase production are biomass and limiting substrate concentrations.

Feature Selection vs Theory Reformulation: A Study of Genetic Refinement of Knowledge-based Neural Networks

Expert classification systems have proven themselves effective decision makers for many types of problems. However, the accuracy of such systems is often highly dependent upon the accuracy of a human expert's domain theory. When human experts learn or create a set of rules, they are subject to a number of hindrances. Most significantly experts are, to a greater or lesser extent, restricted by the tradition of scholarship which has preceded them and by an inability to examine large amounts of data in a rigorous fashion without the effects of boredom or frustration. As a result, human theories are often erroneous or incomplete. To escape this dependency, machine learning systems have been developed to automatically refine and correct an expert's domain theory. When theory revision systems are applied to expert theories, they often concentrate on the reformulation of the knowledge provided rather than on the reformulation or selection of input features. The general assumption seems to be that the expert has already selected the set of features that will be most useful for the given task. That set may, however, be suboptimal. This paper studies theory refinement and the relative benefits of applying feature selection versus more extensive theory reformulation.

Knowledge-based neural network for voltage contingency selection and ranking

Voltage contingency screening and ranking is performed to choose the contingencies that cause the worst voltage problems. A knowledge-based conceptual neural network (KBCNN) is developed for fast voltage contingency selection and ranking. A recognised shortcoming of the ANN based approach is that the problem solving knowledge of ANN is represented in the connection weights, and hence is difficult for a human user to comprehend. One way to provide an understanding of the behaviour of neural networks is to extract rules that can be provided to the users. The rules extracted are used to build a knowledge-based connectionist network for learning and revision of rules. The knowledge-based neural network is applied for voltage contingency selection and ranking in an IEEE 30-bus system and a practical 75-bus Indian system. Once trained, the KBCNN gives accurate selection and ranking for unknown patterns. At the same time, the system user is able to validate the output of the ANN under all possible input conditions. The system user is provided with the capability to determine the set of conditions under which a line-outage is critical, and if critical, then how severe it is, thereby providing some degree of transparency of the ANN solution.

Extract rules by using rough set and knowledge-based NN

In this paper, rough set theory is used to extract roughly-correct inference rules from information systems. Based on this idea, the learning algorithm ERCR is presented. In order to refine the learned roughly-correct inference rules, the knowledge-based neural network is used. The method presented here sufficiently combines the advantages of rough set theory and neural network.

An Integrated Approach for the Computerized Production Process of Curved Hull Plates

The production procedure for curved hull plates follows a sequence of shell development, plate cutting, roller press bending, and line heating processes. The final accuracy of shell plates to be formed depends on eliminating errors accumulated during each process. To satisfy shipyard demand for improved accuracy, each process requires careful examination and the entire system should be concurrently integrated. However, previous research and development has been limited to each independent process. An integrated approach for a computerized production process is being developed. This paper presents the basic concept of the approach. The approach is developed based on engineering analysis in order to guarantee the desired accuracy. Thus, it includes mechanical simulation of cutting, roller press bending, and line heating, with kinematics of shell development. Practical experiences of shipyard experts are implemented into the proposed system by means of a knowledge-based neural network system. Numerical examples are provided to illustrate the present approach.

A Recency Inference Engine for Connectionist Knowledge Bases

Knowledge-based neural networks (KBNNs) can be used as expert system knowledge bases. This approach shifts the interests in using connectionist knowledge bases for inferencing in an interactive fashion and giving reasonable justifications for their conclusions. The primary goal of this article is to present a good inference and control mechanism for such knowledge bases. For this purpose, the article develops a stand alone inference engine that uses a connectionist knowledge base, seeks to reduce the amount of data requested in order to reach a conclusion, and explains how a particular conclusion was reached. The inference engine was evaluated on illustrative example applications. Results obtained demonstrate that in spite of its simplicity the presented technique is superior to other techniques over sparse input knowledge bases.

Connectionist Theory Refinement: Genetically Searching the Space of Network Topologies

An algorithm that learns from a set of examples should ideally be able to exploit the available resources of (a) abundant computing power and (b) domain-specific knowledge to improve its ability to generalize. Connectionist theory-refinement systems, which use background knowledge to select a neural network's topology and initial weights, have proven to be effective at exploiting domain-specific knowledge; however, most do not exploit available computing power. This weakness occurs because they lack the ability to refine the topology of the neural networks they produce, thereby limiting generalization, especially when given impoverished domain theories. We present the REGENT algorithm which uses (a) domain-specific knowledge to help create an initial population of knowledge-based neural networks and (b) genetic operators of crossover and mutation (specifically designed for knowledge-based networks) to continually search for better network topologies. Experiments on three real-world domains indicate that our new algorithm is able to significantly increase generalization compared to a standard connectionist theory-refinement system, as well as our previous algorithm for growing knowledge-based networks.

Knowledge-based neural models for microwave design

Neural networks have recently been introduced to the microwave area as a fast and flexible vehicle to microwave modeling, simulation and optimization. In this paper, a novel neural network structure, namely, knowledge-based neural network (KBNN), is proposed where microwave empirical or semi-analytical information is incorporated into the internal structure of neural networks. The microwave knowledge complements the capability of learning and generalization of neural networks by providing additional information which may not be adequately represented in a limited set of training data. Such knowledge becomes even more valuable when the neural model is used to extrapolate beyond training data region. A new training scheme employing gradient based l/sub 2/ optimization technique is developed to train the KBNN model. The proposed technique can be used to model passive and active microwave components with improved accuracy, reduced cost of model development and less need of training data over conventional neural models for microwave design.

Learning capacity and sample complexity on expert networks

A major development in knowledge-based neural networks is the integration of symbolic expert rule-based knowledge into neural networks, resulting in so-called rule-based neural (or connectionist) networks. An expert network here refers to a particular construct in which the uncertainty management model of symbolic expert systems is mapped into the activation function of the neural network. This paper addresses a yet-to-be-answered question: Why can expert networks generalize more effectively from a finite number of training instances than multilayered perceptrons? It formally shows that expert networks reduce generalization dimensionality and require relatively small sample sizes for correct generalization.

Creating advice-taking reinforcement learners

Learning from reinforcements is a promising approach for creating intelligent agents. However, reinforcement learning usually requires a large number of training episodes. We present and evaluate a design that addresses this shortcoming by allowing a connectionist Q-learner to accept advice given, at any time and in a natural manner, by an external observer. In our approach, the advice-giver watches the learner and occasionally makes suggestions, expressed as instructions in a simple imperative programming language. Based on techniques from knowledge-based neural networks, we insert these programs directly into the agent‘s utility function. Subsequent reinforcement learning further integrates and refines the advice. We present empirical evidence that investigates several aspects of our approach and shows that, given good advice, a learner can achieve statistically significant gains in expected reward. A second experiment shows that advice improves the expected reward regardless of the stage of training at which it is given, while another study demonstrates that subsequent advice can result in further gains in reward. Finally, we present experimental results that indicate our method is more powerful than a naive technique for making use of advice.

Introduction to knowledge-based neural networks

Attempts have been made to incorporate symbolic knowledge into neural networks, resulting in a class of networks known as knowledge-based neural networks. These constructions inherit the merits of both the knowledge-based and neural network approaches to the modeling of human intelligence, and yet the hybrid holds out a major promise of being more successful than its parents. The paper introduces three kinds of knowledge-based neural network. These are based on production rules, decision trees, and semantic constraints, respectively.

Special issue: Knowledge-based neural networks

Special issue: Knowledge-based neural networks

Dynamically adding symbolically meaningful nodes to knowledge-based neural networks

Traditional connectionist theory-refinement systems map the dependencies of a domain-specific rule base into a neural network, and then refine this network using neural learning techniques. Most of these systems, however, lack the ability to refine their network's topology and are thus unable to add new rules to the (reformulated) rule base. Therefore, with domain theories that lack rules, generalization is poor, and training can corrupt the original rules — even those that were initially correct. The paper presents TopGen, an extension to the KBANN algorithm, which heuristically searches for possible expansions to the KBANN network. TopGen does this by dynamically adding hidden nodes to the neural representation of the domain theory, in a manner that is analogous to the adding of rules and conjuncts to the symbolic rule base. Experiments indicate that the method is able to heuristically find effective places to add nodes to the knowledge bases of four real-world problems, as well as an artificial chess domain. The experiments also verify that new nodes must be added in an intelligent manner. The algorithm showed statistically significant improvements over the KBANN algorithm in all five domains.

Integrating knowledge-based systems and artificial neural networks for engineering

AbstractThe feasibility and relative merits of integrating knowledge-based systems (KBSs) and artificial neural networks (ANNs) for application to engineering problems are presented and evaluated. The strength of KBSs lies in their ability to represent human judgment and solve problems by providing explanations from and reasoning with heuristic knowledge. ANNs demonstrate problem solving characteristics not inherent in KBSs, including an ability to learn from example, develop a generalized solution applicable to a range of examples of the problem, and process information extremely rapidly. In this respect, KBSs and ANNs are complementary, rather than alternatives, and may be integrated into a system that exploits the advantages of both technologies. The scope of application and quality of solutions produced by such a hybrid extend beyond the boundaries of the individual technologies. This paper identifies and describes how KBSs and ANNs can be integrated, and provides an evaluation of the advantages that will accrue in engineering applications.