Set Of Input Patterns Research Articles

Neural classifiers using one-time updating

The linear threshold element (LTE), or perceptron, is a linear classifier with limited capabilities due to the problems arising when the input pattern set is linearly nonseparable. Assuming that the patterns are presented in a sequential fashion, we derive a theory for the detection of linear nonseparability as soon as it appears in the pattern set. This theory is based on the precise determination of the solution region in the weight space with the help of a special set of vectors. For this region, called the solution cone, we present a recursive computation procedure which allows immediate detection of nonseparability. The separability-violating patterns may be skipped so that, at the end, we derive a totally separable subset of the original pattern set along with its solution cone. The intriguing aspect of this algorithm is that it can be directly cast into a simple neural-network implementation. In this model the synaptic weights are committed (they are updated only once, and the only change that may happen after that is their destruction). This bears resemblance to the behavior of biological neural networks, and it is a feature unlike those of most other artificial neural techniques. Finally, by combining many such neural models we develop a learning procedure capable of separating convex classes.

Neural-network feature selector

Feature selection is an integral part of most learning algorithms. Due to the existence of irrelevant and redundant attributes, by selecting only the relevant attributes of the data, higher predictive accuracy can be expected from a machine learning method. In this paper, we propose the use of a three-layer feedforward neural network to select those input attributes that are most useful for discriminating classes in a given set of input patterns. A network pruning algorithm is the foundation of the proposed algorithm. By adding a penalty term to the error function of the network, redundant network connections can be distinguished from those relevant ones by their small weights when the network training process has been completed. A simple criterion to remove an attribute based on the accuracy rate of the network is developed. The network is retrained after removal of an attribute, and the selection process is repeated until no attribute meets the criterion for removal. Our experimental results suggest that the proposed method works very well on a wide variety of classification problems.

Development of an Artificial Skin Ridge for Pattern Recognition

This paper describes a sensing technique which utilizes artificial neural networks and a distributed sensor. The concept resembles the manner in which humans use their fingertips to touch objects, and subsequently use their brains to classify and recognize images detected. There are three major components of the designed sensor: a compliant skin is made of rubber with wedge-shaped ridge, an energy transducer which is a piezoelectric polyvinyledene fluoride film, and neural networks which are used to characterize signals through a learning process. The backpropagation is used as a learning rule, thus, learning required a set of input patterns each paired with a desired output pattern. The objective for the networks is to learn and memorize, after proper training, in order to reproduce desired outputs. Experimental results show that the sensor with artificial neural nets is very effective at recognizing Braille characters including SIX, SEVEN, EIGHT, UP, DOWN, LEFT, and RIGHT.

Effects of LTP on Response Selectivity of Simulated Cortical Neurons

We report here on specific ways in which synaptic long-term potentiation (LTP) affects the response selectivity of primary sensory cortical cells. LTP increases synaptic efficacy by incremental "steps," up to a "ceiling" at which additional bursts of afferent stimulation cause no further potentiation. Endogenous and exogenous agents have been shown to modulate these two paramenters of LTP, raising the question of the functional implications associated with the sizes of steps and ceiling. We provide an analytical treatment of the effects of these two physiological LTP parameters on the behavior of simulated olfactory (piriform) cortex target cells in response to a range of inputs. A target cell's receptive field, i.e., the set of input patterns to which the cell responds, is broadened with potentiation of the cell's synapses, and is broadened more when the LTP step size is smaller, and when the LTP ceiling is higher. Moreover, the effects of step size and ceiling interact, and their joint relationship to receptive field breadth is nonlinear. Values of step size and ceiling are identified that balance the tradeoff between learning rate and receptive field breadth for particular sensory recognition tasks, and these model values are compared to corresponding known and inferred physiological values.

Reorganizing knowledge in neural networks: an explanatory mechanism for neural networks in data classification problems

We propose an explanatory mechanism for multilayered neural networks (NN). In spite of the effective learning capability as a uniform function approximator, the multilayered NN suffers from unreadability, i.e., it is difficult for the user to interpret or understand the "knowledge" that the NN has by looking at the connection weights and thresholds obtained by backpropagation (BP). This unreadability comes from the distributed nature of the knowledge representation in the NN. In this paper, we propose a method that reorganizes the distributed knowledge in the NN to extract approximate classification rules. Our rule extraction method is based on the analysis of the function that the NN has learned, rather than on the direct interpretation of connection weights as correlation information. More specifically, our method divides the input space into "monotonic regions" where a monotonic region is a set of input patterns that belongs to the same class with the same sensitivity pattern. Approximate classification rules are generated by projecting these monotonic regions.

Adaptive Pattern Matching

Pattern matching is an important operation used in many applications such as functional programming, rewriting, and rule-based expert systems. By preprocessing the patterns into a DFA-like automaton, we can rapidly select the matching pattern(s) in a single scan of the relevant portions of the input term. This automaton is typically based on left-to-right traversal of the patterns. By adapting the traversal order to suit the set of input patterns, it is possible to considerably reduce the space and matching time requirements of the automaton. The design of such adaptive automata is the focus of this paper. We first formalize the notion of an adaptive traversal. We then present several strategies for synthesizing adaptive traversal orders aimed at reducing space and matching time complexity. In the worst case, however, the space requirements can be exponential in the size of the patterns. We show this by establishing an exponential lower bounds on space that is independent of the traversal order used. We then discuss an orthogonal approach to space minimization based on direct construction of optimal dag automata. Finally, our work brings forth the impact of typing in pattern matching. In particular, we show that several important problems (e.g., lazy pattern matching in ML) are computationally hard in the presence of type disciplines, whereas they can be solved efficiently in the untyped setting.

Regression methods for automated colour image classification and thresholding

SummaryRegression methods are used to perform automated image thresholding and colour pixel classification. This is done by considering threshold levels and pixel classification labels as pattern attributes. A regression equation that performs a mapping from the J dimensional feature‐pattern space to the K dimensional attribute space is derived. The approach is non‐parametric and deterministic, hence no assumptions about the statistical properties of the input patterns or images need be made. Initially a known set of input patterns with associated attributes are used to constitute a training set. A mapping function is then determined from the training patterns and used for estimating attribute values from unknown input patterns, such as images.

Statistical mechanics of unsupervised Hebbian learning

A model describing the dynamics of the synaptic weights of a single neuron performing Hebbian learning is described. The neutron is repeatedly excited by a set of input patterns. Its response is modelled as a continuous, nonlinear function of its excitation. The authors study how the model forms a self-organized representation of the set of input patterns. The dynamical equations ae solved directly in a few simple cases. The model is studied for random patterns by a signal-to-noise analysis and by introducing a partition function and applying the replica approach. As the number of patterns is increased a first-order phase transition occurs where the neuron becomes unable to remember one pattern but learns instead a mixture of very many patterns. The critical number of patterns for this transition scales as Nb, where N is the number of synapses and b is the degree of nonlinearity. The leading order finite-size corrections are calculated and compared with numerical simulations. It is shown how the representation of the input patterns learned by the neutron depends upon the nonlinearity in the neuron's response. Two types of behaviour can be identified depending on the degree of nonlinearity; either the neuron learns to discriminate one pattern from all the others, or it will learn to discriminate a complex mixture of many of the patterns.

Optical competitive learning with VLSI/liquid-crystal winner-take-all modulators

A new approach to self-aligning unsupervised optical learning based on the competitive learning algorithm and adaptive holographic interconnectións is introduced. A volume hologram is used to diffract the light from an input spatial light modulator so that it focuses upon a custom winner-take-all very-large-scale-integrated circuit liquid-crystal spatial light modulator. The units that receive the most light switch to a reflective state, and the light reflected from the winning pixels interferes in the volume of the dynamic hologram with the phase conjugate of the input pattern in order to add an outer-product perturbation to the current holographic interconnection. As a set of input patterns is cycled through many times, the system learns to diffract the light from a particular input class upon a self-organized set of detectors that recognize similar input patterns without the aid of a teacher or any required alignment. Beam-propagation simulations are used to show that the holographic optical learning network faithfully reproduces the behavior of the ideal competitive learning algorithm. A winner-takeall detector/modulator device containing a total of 576 optical neurons grouped into 31 separate competitive patches in the required sparse-grid topology has been successfully fabricated, and experimental results are presented.

Self‐Organization of Recognition Cell Groups in Nerve Fields

AbstractThis paper proposes a basic model for the self‐organization of the recognition cell group in the nerve field. The model is composed of nerve cell layers with mutual lateral inhibition. Also described is the condition for the self‐organization of the recognition cell groups for the n‐dimensional input and, based on that condition, the properties of the model are discussed. Then the behavior of the model is verified through computer simulation. As a result, the following observations are made. When a certain condition is satisfied, recognition cells are formed in a group. These cells respond only to the particular set of input patterns. By varying the training parameters, the recognition cell group is formed depending on the similarity between input patterns. For highly similar patterns, the recognition cell groups are formed in nearby locations in the nerve field. The recognition cell group is formed according to the frequency of the input pattern and its norm.

Distributed memory and the representation of general and specific information.

We describe a distributed model of information processing and memory and apply it to the representation of general and specific information. The model consists of a large number of simple processing elements which send excitatory and inhibitory signals to each other via modifiable connections. Information processing is thought of as the process whereby patterns of activation are formed over the units in the model through their excitatory and inhibitory interactions. The memory trace of a processing event is the change or increment to the strengths of the interconnections that results from the processing event. The traces of separate events are superimposed on each other in the values of the connection strengths that result from the entire set of traces stored in the memory. The model is applied to a number of findings related to the question of whether we store abstract representations or an enumeration of specific experiences in memory. The model simulates the results of a number of important experiments which have been taken as evidence for the enumeration of specific experiences. At the same time, it shows how the functional equivalent of abstract representations--prototypes, logogens, and even rules--can emerge from the superposition of traces of specific experiences, when the conditions are right for this to happen. In essence, the model captures the structure present in a set of input patterns; thus, it behaves as though it had learned prototypes or rules, to the extent that the structure of the environment it has learned about can be captured by describing it in terms of these abstractions.

The Self-Testing of a Solid State Safety System

An automatic, on-line self-test system is employed in the General Electric BWR/6 solid state Nuclear System Protection System (NSPS) control room cabinets to rapidly detect hardware failures and facilitate repair. Each of the four NSPS cabinets in the control room contains a microprocessor based selftest controller in whose memory are stored sets of input stimulii test patterns and expected output results. For each set of input patterns the expected and actual outputs are compared and discrepancies annunciated to the plant's operators. These sets of test patterns comprehensively test all essential circuits and wiring within and between the NSPS cabinets such that the self-test system is able to determine the exact location of all failures in safety related hardware. The cabinets were designed to ensure that operation of and failures in the selftest system do not influence the correctness of the safety system's outputs.

Detecting Races with a Production Environment Logic Simulator

A logic simulator serves three functions: computing the output of a circuit board for a given set of input patterns, indicating faults which have been detected by a set of input patterns (tests), and predicting race conditions which may occur. Race detection is important because races prevent faults from being detected and produce unknown states during testing, causing failed boards to pass or good boards to fail. If a simulator fails to detect a race condition, the test engineer does not become aware of its effect until he observes anomalies during production testing. This paper describes a race detection technique which is quite conservative, in that it predicts potential race conditions which are ignored by commonly used simulators. It requires neither modeling of tolerances of components, nor the computation of delays on all parts through the board. Additionally, because the approach is based on a mathematical technique, each logic element state computed by the simulator is available as a dynamic trace printout which helps isolate the source of the race. An example is given in which a race ignored by the unit delay method is detected in a circuit where performance is adversely affected.