Neural Net Structure Research Articles

A novel, fast and clean 3D permeability measurement method for multi-layer textiles and fabrics

Permeability quantifies flow conductance of textile reinforcements, is required for process simulations, and can be used for a range of materials and process monitoring by industry. Current permeability measurement techniques are destructive, either because a liquid penetrates the sample or the sample must be cut from products to a specific size. A non-destructive measurement concept is introduced based on air flow established between flat circular patterns containing flow rate and pressure sensors. The presented experimental studies show that from flow rate and pressure distribution data, the technique clearly distinguishes changes in volume fraction and layup while being applicable to a range of woven and non-crimp textiles. The concept interprets permeability from a neural network trained using a large set of air flow simulations. A preliminary set of 250,000 simulated cases was applied to train seven different neural network types. Limited predictive accuracy has been achieved, utilizing a CGB neural net structure.

ANN-Based DEM Fusion and DEM Improvement Frameworks in Regions of Assam and Meghalaya using Remote Sensing Datasets

The Digital Elevation Models (DEMs) are a key and primary input to a large number of modelling processes such as disaster risk monitoring, flood modelling, hydrology, geology, geomorphology, climatology, and environmental study applications. The DEM serves as an important source of topographic information representing the continuous surface of the earth in 3 dimensions with x, y and z coordinates of any point in a grided raster form or a vector TIN form. This study is based on developing a new method using the universal approximation capability of neural networks for the fusion and improvement of L-band and X-band SAR (Synthetic Aperture Radar) DEMs in the complex terrain of Assam and Meghalaya states of the Indian geographic region using the DEM fusion technique. The high-spatial-resolution ALOS PALSAR RTC HR 12.5 m DEM products are used in a fusion framework designed with neural network models. The network adaptively learns the terrain information to produce fused DEM products. The neural network models generate the relationship between the input elevation information from ALOS PALSAR DEMs and precise reference elevation from ICESat-2 spaceborne altimetry as target data. The training and testing data samples are prepared and filtered by checking the correct range of elevation from the toposheets of this region. Different models are used to separately train for the relatively plain valley portion of the Assam region and mountainous portions of the highland Shillong plateau. The obtained fused DEMs from the developed neural net structure are assessed for their quality and accuracy by estimating the RMSE parameter. The fused DEM attains an RMSE value of 7 meters for the complete region which is a significant improvement over the input DEM RMSE of approximately 11 meters. The plain area points and mountainous region points are assessed separately to analyze the predictions from neural nets in the two types of terrains observed in this study site. Moreover, TanDEM-X 90 m DEM is improved using the neural network modeling in the geometric distortions affected areas, which shows an improvement of around 33% overall at the study site. The assessment of plain and mountainous region points for near-ground points shows an improvement of 47% and 55% respectively. The fusion framework designed using the neural network models is an effective and efficient method for obtaining fused DEMs as well as for the improvement of the existing DEMs for complex terrain.

LEAP nets for system identification and application to power systems

Using neural network modeling, we address the problem of system identification for continuous multivariate systems, whose structures vary around an operating point. Structural changes in the system are of combinatorial nature, and some of them may be very rare; they may be actionable for the purpose of controlling the system. Although our ultimate goal is both system identification and control, we only address the problem of identification in this paper. We propose and study a novel neural network architecture called LEAP net, for Latent Encoding of Atypical Perturbation. Our method maps system structure changes to neural net structure changes, using structural actionable variables. We demonstrate empirically that LEAP nets can be trained with a natural observational distribution, very concentrated around a “reference” operating point of the system, and yet generalize to rare (or unseen) structural changes. We validate the generalization properties of LEAP nets theoretically in particular cases. We apply our technique to power transmission grids, in which high voltage lines are disconnected and re-connected with one-another from time to time, either accidentally or willfully. We discuss extensions of our approach to actionable variables, which are continuous (instead of discrete, in the case of our application) and make connections between our problem setting, transfer learning, causal inference, and reinforcement learning.

Combining a DSGE Model with Neural Networks for Forecasting Economic Variables

This paper examines the possibility of combining a DSGE model and neural networks to supplement each other, in order to yield a good performance in out-of-sample forecasts for economic variables. The novel neural-net structure of TDVAE (Temporal Difference Variational Auto-Encoder) proposed by Gregor et.al [2019] helps to realize this idea. TDVAE virtually replicates a stochastic state-space model through combinations of neural networks within the framework of variational Bayesian inference. Because a DSGE model provides theoretical restrictions on the state transition matrices of a state-space model, I choose to transplant those DSGE-oriented matrices into the probabilistic model of state transition in TDVAE. This TDVAE-DSGE approach certainly achieved the superior performance in the task of out-of-sample forecasts on Japan's major macroeconomic variables during 1Q/2011 and 4Q/2019.

Wavelet-based convolutional neural networks for gender classification

We develop a gender classification method using convolutional neural networks. We train Alexnet Architecture using the luminance (Y) component of the facial image (YCbCr) for the SoF, groups, and face recognition technology datasets. The Y component is reduced to a size of 32 × 32 via discrete wavelet transform (DWT). The use of the Y plane and a low-resolution subband image of the DWT significantly reduce the amount of processed data. We are able to achieve better results than other machine learning, rule-based approaches and the traditional convolutional neural net structure that are trained with three-dimensional RGB images. We are able to maintain comparably high recognition accuracy, even with the reduction of the number of network layers. We have also compared our structure with the state-of-the-art methods and provided the recognition rates.

Parameter tuning of boiler thermal process based on SVM neural net optimisation

Because of complex characteristics, such as multivariable coupling in boiler thermal process of circulating fluid bed, parameter turning, there is relatively large difficulty in automatic accurate control so that a kind of self-adaptive controller algorithm is put forward. Fuse fuzzy control and equivalent method of BP neural net usage structure to fuzzy BP neural net and bring in weight of genetic algorithm optimisation BP neural net by aiming at defects, such long convergence time of neutral net and realise self-adaptive accuracy control to boiler thermal process of circulating fluid bed by feed-forward compensation decoupling device. It is showed from experiment results that the algorithm can adapt to working condition of variable parameter boiler thermal process of circulating fluid bed and it has realised uncoupling of bed temperature and main steam pressure.

Fuzzy neural net digital filtering: A MIMO structure

The paper is an analysis of the neural fuzzy filtering with multivariate description and realtime properties in order to give the elements of this kind of filters called RTMNFDF (Real Time Multivariate Neuro-Fuzzy Digital Filtering). This kind of MIMO (multiple inputs and multiple outputs) filter uses an adaptive inference mechanism into its own architecture with a fuzzy neural net structure to get the best answers in accordance to the corresponding parameter matrix values from the knowledge base (KB); actualizing the filter weights to give the answers in natural linguistic sense. The advantage with respect to classical filtering methods, is that dynamically deduce the reference system conditions and take a decision from its knowledge base in accordance with the desired signal at the filter input giving the best answer through the time. The filter structure requires that all of the state sequences bound into RTMNFDF time limits as a real-time system considering the Nyquist and Shannon criterion. The paper describes the characterization of the membership functions into the knowledge base with probabilistic description with respect to the rules set decisions, performing the RTFNDF. In addition, the paper shows schematically the structure of the neural net into the filter description. The results formally integrate the concepts exposed in the paper references. Finally, into the simulation show illustratively the RTMNFDF operations graphics using as a tool the Matlab® software. The paper has eight sections conformed as follows: 1. Introduction, 2. Neural fuzzy description, 3. Parameter properties, 4. Filter mechanism, 5. Neural net architecture, 6. Real time descriptions, 7. Results, Conclusions and References.

A Gaussian adaptive resonance theory neural network classification algorithm applied to supervised land cover mapping using multitemporal vegetation index data

Neural network classifiers have been shown to provide supervised classification results that significantly improve on traditional classification algorithms such as the Bayesian (maximum likelihood [ML]) classifier. While the predominant neural network architecture has been the feedforward multilayer perceptron known as backpropagation. Adaptive resonance theory (ART) neural networks offer advantages to the classification of optical remote sensing data for vegetation and land cover mapping. A significant advantage is that it does not require prior specification of the neural net structure, creating as many internal nodes as are needed to represent the calibration (training) data. The Gaussian ARTMAP classification algorithm bases the probability that input training samples belong to specific classes on the parameters of its Gaussian distributions: the means, standard deviations, and a priori probabilities. The performance of the Gaussian ARTMAP classification algorithm in terms of classification accuracy using independent validation data indicated was over 70% accurate when applied to an annual series of monthly 1-km advanced very high resolution radiometer (AVHRR) satellite normalized difference vegetation index (NDVI) data. The accuracies were comparable to those of fuzzy ARTMAP and a univariate decision tree, and significantly higher than a Bayesian classification algorithm. Algorithm testing is based on calibration and validation data developed and applied to Central America to map the International Geosphere-Biosphere Programme (IGBP) land cover classification system.

Damping function estimation based on modal receptance models and neural nets

A method for the estimation of an isotropic material damping based on the isotropic, augmented Hooke's law (AHL) and the concept of isotropic, modal and material damping functions is proposed. The method is a straightforward application of standard neural net (NN) back propagation techniques combined with back propagation of simulation errors using “response-to-damping parameter” mapping based on an analytical, modal receptance model incorporating AHL damping. Damping parameters are estimated by fitting the receptance model iteratively, using NN technique, to measured receptance data. Owing to the response model used, the proposed estimation technique provides new possibilities to completely separate pure damping properties from geometry, elastic (static) data and undamped modal data. The method is applicable to homogeneous materials and cases where cross coupling, due to damping, between the modes can be neglected. Even though in this sense restricted, the technique may be applied to a wide class of cases, including cases with very high damping and highly overlapping damped modes. Necessary background theory, including a suitable NN structure, for application of the method is presented. The estimation procedure is demonstrated for a plexiglas (PMMA) plate (modal loss factors ≈ 0.1 at room temperature) used as experimental test case. Very good agreement between measured receptances and responses predicted using the estimated damping function parameters is obtained. Validation was done using both modal and direct finite element (FE) computations.

Statistical analysis of a two-layer backpropagation algorithm used for modeling nonlinear memoryless channels: the single neuron case

Neural networks have been used for modeling the nonlinear characteristics of memoryless nonlinear channels using backpropagation (BP) learning with experimental training data. In order to better understand this neural network application, this paper studies the transient and convergence properties of a simplified two-layer neural network that uses the BP algorithm and is trained with zero mean Gaussian data. The paper studies the effects of the neural net structure, weights, initial conditions, and algorithm step size on the mean square error (MSE) of the neural net approximation. The performance analysis is based on the derivation of recursions for the mean weight update that can be used to predict the weights and the MSE over time. Monte Carlo simulations display good to excellent agreement between the actual behavior and the predictions of the theoretical model.

A neural-net based short term load forecasting using moving window procedure

An improved neural-net approach based on a combined unsupervised/supervised learning concept is proposed. A ‘moving window’ procedure is applied to the most recent load and weather information for creating training set data base. A forecasting lead time that varies from 16 hours to 88 hours is introduced to produce the short term electric load forecasting that meets requirements of real electric utility operating practice. The unsupervised learning (UL) is used to identify days with similar daily load patterns. A feed forward three-layer neural net is designed to predict 24-hour loads within the supervised learning (SL) phase. The effectiveness of proposed methods is demonstrated by comparison of forecasted hourly loads in every single day during 1991 with data realized in the same period in the Electric Power Utility of Serbia (EPS). A better choice of input features and more appropriate training set selection procedure allow significant improvement in forecasting results comparing with our previous UL/SL concept characterized by a fixed neural-net structure and absence of re-training procedure. The improvement is illustrated by reduction of average error in daily energy forecasting for 0.83% and reduction of 90th percentile of 2.04%.

Efficient identification of RBF neural net models for nonlinear discrete-time multivariable dynamical systems

In this paper, we propose a recursive identification technique for nonlinear discrete-time multivariable dynamical systems. Extending an early result to multivariable systems [15], the technique approaches a nonlinear system identification problem in two stages: One is to build up recursively a RBF (Radial-Basis-Function) neural net model structure including the size of the neural net and the parameters in the RBF neurons; the other is to design a stable recursive weight updating algorithm to obtain the weights of the net in an efficient way. Heuristics are employed to analyze the effect of RBF net parameters to the error of identification, leading to a simple but effective means to establish these parameters. The weight updating algorithm is developed based on ideas in the theory of adaptive control. Key stability results are proved in the paper along with illustrative examples to show the effectiveness of applying such a technique and other practical considerations.

A technique for on-line parameter estimation based on an analog artificial neural net structure

In this paper, we present a technique for on-line parameter estimation based on an analog artificial neural net structure. The architecture adopted is the so-called Hopfield net and the continuous state, or analog, version is used. The fundamental algorithm constructed is shown to be globally stable in the sense of Liapunov. This is further extended and developed to incorporate on-line recursion, and for systems with parameters that change over time, the most general version of our neural net algorithm incorporates recursion using data in exponentially weighted windows. Simulation results are provided to show the performance of the on-line estimator and its ability to track changing parameters with the use of exponential weighting. Additional simulation results are also included which shows the robustness of the estimator in the presence of unmodelled dynamics.

The use of neural nets to combine equalization with decoding for severe intersymbol interference channels

This paper deals with the problem of combining equalization with decoding in channels which have severe intersymbol interference. A multilayer neural net structure is proposed to achieve the process of equalization and decoding simultaneously. Experimental examples show that this method results in a substantial improvement over the more conventional methods of performing equalization and decoding.

Optimization of the neural net connectivity pattern using a backpropagation algorithm

This paper presents an approach to using a backpropagation learning algorithm in defining an optimal connectivity pattern given a fixed number of neurons. A neural net structure suitable for this approach is defined and the associated backpropagation algorithm is presented. The training of the neural network is divided into two phases: an exploration phase and a fixation phase. In the exploration phase, the backpropagation algorithm is utilized in finding optimal connections between neurons. In the fixation phase, the non-optimal connections are de-linked and learning is continued. In both the phases, weight updates are carried out similar to the regular backpropagation algorithm. The end result is a neural network with both optimal weights and optimal connections. Two test cases are used to show the effectiveness of this technique in terms of both generalization and accuracy of the network copying. Comparisons are made with the traditional multilayered feedforward neural networks.

Neural Nets for Job-shop Scheduling, will they do the Job?

A neural net structure has been developed which is capable of solving deterministic job-shop scheduling problems, part of the large class of np-complete problems. The problem was translated in an integer linear programming format which facilitated translation in an adequate neural structure. Use of the presented structure eliminated the need for integer adjustments. The search space was reduced by the use of precalculation, allowing the rapid calculation of feasible solutions. The neural net structure was reliable in simulated operation and its performance was superior to structures which have been presented previously.

Cluster network for recognition of handwritten, cursive script characters

A neural net structure was developed to recognize isolated cursive script characters. Characters were composed of features suggested by an established model of handwriting. The model assumes that handwritten characters are formed from a limited number of primitive hand motions characterized by the X-Y oscillations in the vertical ( Y) and horizontal ( X) directions. Consistent with this model, representation for handwritten characters contain features that are invariant under rotation, translation, changes in size and/or style. Besides their applicability to categorization of handwritten characters, these features are also assumed to control neural “motor” activities during writing. The initial application of a single backward error propagation network to handwritten script recognition resulted in slow learning and was limited to approximately 50% correct classification. Changing the architecture to a cluster of smaller networks and adding some simple heuristics improved performance dramatically. In its current form, the approach yields recognition up to 80% for our test data.

Parameter Estimation Using Artificial Neural Nets

Artificial neural nets have become increasingly popular in solving optimization problems such as the Travelling Salesman Problem, A/D converter realization, linear programming and pattern recognition. In many of these optimization problems, the neural net structure is based on the Hopfield model and each neuron is assumed to have two distinct outputs, 0 or 1. In this paper, a continuous Hopfield net is used to formulate and solve a least squares minimization problem associated with the parameter estimation of a physical system. The algorithm shows good convergence and a certain amount of “recursion” can be incorporated which makes it useful for real time applications.

The outcome advisor ®

A new classification method called The Outcome Advisor ® (OA) is presented which is an outgrowth of statistical pattern recognition and the Patrick-Fischer Generalized K-nearest Neighbor Decision Rule. Involved are new definitions of relative frequency and correlation. Training examples are store and processing begins once findings (a focus) are presented. An almost unlimited number of inferences can be made as an inference system and any feature can be used to define categories as a classification system. Implementable as a new neural net structure which is distribution free, multi-dimensional dependencies in the feature space for each category are learned utilizing a new definition of relative frequency. The new method may help explain how certain neural net structures may be estimating multidimensional dependencies. The OA has been trained and tested on established data bases and has improved performance as measured by experimental probability of error.