Net2Tab: Tabularizing neural networks with applications to data prefetching

The Sixth International Symposium on Neural Networks (ISNN 2009)

The atmospheric radiative transfer calculations are among the most time‐consuming components of the numerical weather prediction (NWP) models. Deep learning (DL) models have recently been increasingly applied to accelerate radiative transfer modeling. Besides, a physical relationship exists between the output variables, including fluxes and heating rate profiles. Integration of such physical laws in DL models is crucial for the consistency and credibility of the DL‐based parameterizations. Therefore, we propose a physics‐incorporated framework for the radiative transfer DL model, in which the physical relationship between fluxes and heating rates is encoded as a layer of the network so that the energy conservation can be satisfied. It is also found that the prediction accuracy was improved with the physic‐incorporated layer. In addition, we trained and compared various types of DL model architectures, including fully connected (FC) neural networks (NNs), convolutional‐based NNs (CNNs), bidirectional recurrent‐based NNs (RNNs), transformer‐based NNs, and neural operator networks, respectively. The offline evaluation demonstrates that bidirectional RNNs, transformer‐based NNs, and neural operator networks significantly outperform the FC NNs and CNNs due to their capability of global perception. A global perspective of an entire atmospheric column is essential and suitable for radiative transfer modeling as the changes in atmospheric components of one layer/level have both local and global impacts on radiation along the entire vertical column. Furthermore, the bidirectional RNNs achieve the best performance as they can extract information from both upward and downward directions, similar to the radiative transfer processes in the atmosphere.

The subject of research in the article are the processes of formalization of the pixel-by-pixel classification problem using the modified fuzzy neural production network of Wang-Mendel for segmentation of urban structures in the automated analysis of space and aerial photographs of the city. The purpose of the work is to develop the architecture of the modified fuzzy neural production network of Wang-Mendel as a classifier for image segmentation to increase the values of efficiency and reliability of urban monitoring. The following tasks are solved in the article: analysis of possibilities of Wang-Mendel network modification based on representation of membership functions in terms of interval fuzzy sets of the second type (IFST2) and realization of phasing, aggregation and activation operations using IFST 2 operations, development of the architecture of the modified fuzzy neural production network of Wang-Mendel as a classifier for image segmentation. The following methods and models are used: methods and models of fuzzy set theory (fuzzy Wang-Mendel neural network, interval fuzzy sets of the second type), methods and models of deep learning methodology (convolutional neural network for image segmentation (auto coder) U-net). The following results were obtained: the use of a fuzzy Wang-Mendel neural network as a classifier of a modified U-Net decoder based on the representation of membership functions in IFST2 and the implementation of phasing, aggregation and activation operations using operations on IFST2; introduction of an additional operation of type reduction in the phase of dephasification of the original variable based on the classical method of the center of gravity (centroid); introduction of several outputs of the network to recognize the appropriate number of classes (subclasses) of the subject area. To do this, the third layer is represented as a set of several pairs of adder neurons, and the fourth implements several normalizing neurons, the number of which corresponds to the number of pairs of the third layer. Conclusions: the use in the architecture of a convolutional neural network for segmentation of U-net images as a classifier of the modified fuzzy neural production network of Wang-Mendel will provide an additional increase in the accuracy of pixel-by-pixel classification of certain objects. Instead of fuzzy sets of the first type (FST1) in this network IFST2 are used. The proposed IFST2, on the one hand, provide a formalization of more additional degrees of uncertainty compared to FST1, on the other hand, are "implemented" in the development of fuzzy systems (models) and have less computational complexity, compared to fuzzy sets of the second type (FST2).

Neural networks have been availably applied to the simulations of the mechanical behaviors of many materials. In this work, a neural network material model is built for the simulation of the inelastic behavior of biocomposite insect cuticle. Radial basis function neural network is adopted in the simulation for that the neural network has the characteristic of fast and exactly completing the simulation. In the construction of the neural network, the network is trained based on the experimental data of the load-displacement relationship of a chafer cuticle. A strain-controlled mode and the iterative method of data are adopted in the training process of the neural network. The obtained neural network model is used for the simulation of the inelastic behavior of another kind of insect cuticle. It is shown that the obtained material model of the radial basis function neural network can satisfactorily simulate the inelastic behavior of insect cuticle.KeywordsNeural NetworkArtificial Neural NetworkHide LayerNeural Network ModelMaterial BehaviorThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The learning strategy employed in neural networks offers a good performance even in the situations where a model is presented with incomplete and noisy data. However, neural networks are known as ‘black boxes’ as how the outputs are produced is not clear. In this study, a hybrid learning strategy, namely RDC-ANNE (Rules Driven by Consistency in Artificial Neural Networks Ensemble) is proposed. This paper looks at the use of RDC-ANNE in the graft outcome prediction domain as a prototypical medical application. At first, for a better generalization, a committee of binary neural networks is trained. Then, a partial C4.5 decision tree is built from a specifically selected dataset, generated based on the graft data used to test the trained neural networks ensemble. Finally the most appropriate leaf in every path is converted into an understandable rule. In this approach, for the rule generation process, we enforced the model to mainly consider the patterns that their class labels were consistently causing agreement across the neural network classifiers. Experimental results show that the RDC-ANNE method is able to extract partial rules from an ensemble model and reveal the important embedded information of a trained neural network ensemble.KeywordsArtificial Neural Network ModelClass LabelEnsemble ModelNeural Network EnsembleArtificial Neural Network ClassifierThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Artificial intelligent provides diverse solutions for the complex problems in agriculture research. The study aimed to use three models of artificial neural networks (Feed Forward Neural Network (FFNN), Generalized Regression Neural Network (GRNN) and Radial-Basis Neural Network (RBNN)) in the field of wheat yield prediction. 27-year data for the period (1986-2012) were utilized to improve the models and four-year data (2013 and 2016) were used to estimate the models, to compare their outputs with the measured data. Prediction data was not entered in the process of building neural network models. The results showed that the optimal configuration of the FFNN model consists of 40 neurons in the hidden layer (8-40-1). The Tan Sigmoid activation function was used in both the hidden layer and the output layer using all of these models (anterior neural feeding network and the regression neural network and radial base neural network) in the 4-year wheat yield forecast field for production (2013-2016) by applying 8 input parameters that were result of NMMS (8.6%, 7.6% and 15.7% resp.), To find that FFNN and GRNN provide the best result from BRNN because while the information set was large or in a wide range, then the range data ranges from -1 to +1 (normalization data) , GRNN gives better outcomes after the information or sample data were in large range ConclusionsThe research assessment in concerning MLP, GRNN and RBNN in the field of wheat yield forecast. The forecast was worked utilizing the climate variables namely; Rain (R), maximum temperature (Tmax), mean temperature (Taver) ,minimum temperature (Tmin), potential evapotranspiration (PET), dew point data (DP), wind speed (WS) and irrigation requirement (IR) for wheat. Data of historical 31 years (1986 to 2012) were collected from standard agricultural meteorological stations of the Agricultural Research Center in the tests station of Sakha Province, Kafer el Shikh Governorate, Egypt, Data of 27 years for the period (1986-2012) were retained to develop the models and the data of four years (2013 and 2016) were expended to evaluate the models, to compare their outputs with the data measured these data did not affect in the process of building neural networks models. Results discovered that the ideal conformation for the FFNN model involved of one layer (8-40-1). The hidden layers had 40 nodes in the hidden layer for the ANN model. Hyperbolic tangent transfer function was engaged in hidden and output layers of the ANN display. The learning rate and the momentum parameter were 0.005 and 0.9 resp. for the ANN model. Iterations were 1000 epochs during training process for the ANN model. The outcome represented that GRNN extant well forecast outcomes as competed to FFNN and RBNN.

Engineering Applications of Bio-Inspired Artificial Neural Networks

Connectionist Models of Neurons, Learning Processes, and Artificial Intelligence

OBJECTIVES/GOALS: A particularly debilitating consequence of stroke is alexia, an acquired impairment in reading. Cognitive models aim to characterize how information is processed based on behavioral data. If we can concurrently characterize how neural networks process that information, we can enhance the models to reflect the neuronal interactions that drive them. METHODS/STUDY POPULATION: There will be 10 unimpaired adult readers. Two functional localizer tasks, deigned to consistently activate robust language areas, identify the regions of interest that process the cognitive reading functions (orthography, phonology, semantics). Another task, designed for this experiment, analyses the reading-related functional-connectivity between these areas by presenting words classified along the attributes of frequency, concreteness, and regularity, which utilize specific cognitive routes, and a visual control. Connectivity is analyzed during word reading overall vs. a control condition to determine overall reading-related connectivity, and while reading words that have high vs. low attribute values, to determine if cognitive processing routes bias the neural reading network connectivity. RESULTS/ANTICIPATED RESULTS: The localizer analysis is expected to result in the activation of canonical reading areas. The degree of functional connectivity observed between these regions is expected to depend on the degree to which each cognitive route is utilized to read a given word. After orthographic, phonologic, and semantic areas have been identified, the connectivity analysis should show that there is high correlation between all three types of areas during reading compared to the control condition. Then the frequency, regularity, and concreteness of the words being read should alter the reliance on the pathways between these area types. This would support the hypothesized pattern of connectivity as predicted by the cognitive reading routes. Otherwise, it will show how the neural reading network differs from the cognitive model. DISCUSSION/SIGNIFICANCE OF IMPACT: The results will determine the relationship between the cognitive reading model and the neural reading network. Cognitive models show what processes occur in the brain, but neural networks show how these processes occur. By relating these components, we obtain a more complete view of reading in the brain, which can inform future alexia treatments.

A pattern classification method called neural tree networks (NTNs) is presented. The NTN consists of neural networks connected in a tree architecture. The neural networks are used to recursively partition the feature space into subregions. Each terminal subregion is assigned a class label which depends on the training data routed to it by the neural networks. The NTN is grown by a learning algorithm, as opposed to multilayer perceptrons (MLPs), where the architecture must be specified before learning can begin. A heuristic learning algorithm based on minimizing the L1 norm of the error is used to grow the NTN. It is shown that this method has better performance in terms of minimizing the number of classification errors than the squared error minimization method used in backpropagation. An optimal pruning algorithm is given to enhance the generalization of the NTN. Simulation results are presented on Boolean function learning tasks and a speaker independent vowel recognition task. The NTN compares favorably to both neural networks and decision trees.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Artificial Neural Nets Problem Solving Methods

Contents: D. Sobajic, Foreword. Part I:Perspectives. Y-H. Pao, G-H. Park, Learning and Generalization Characteristics of the Random Vector Functional-Link Net. C-C. Liu, M. Damborg, Artificial Neural Networks and Expert Systems in the Power System Operation Environment. E. Bradley, A Utility Perspective on Neural Networks, Fuzzy Logic, and Artificial Intelligence. Part II:Neural Network Methodologies. B. Widrow, M.A. Lehr, Backpropagation and Its Applications. F. Beaufays, E.A. Wan, Using Flow Graph Interreciprocity to Relate Recurrent-Backpropagation and Backpropagation-Through-Time. A. Guha, Neural Network Based Inferential Sensing and Instrumentation. S.A. Harp, T. Samad, Optimizing Neural Networks Using Genetic Algorithms. Part III:Nuclear Power Plants. R. Uhrig, Potential Use of Neural Networks in Nuclear Power Plants. M. Khadem, A. Ipakchi, F.J. Alexandro, R.W. Colley, Sensor Validation in Power Plants Using Neural Networks. A. Ikonomopoulos, L. Tsoukalas, R. Uhrig, Measuring Fuzzy Variables in a Nuclear Reactor Using Artificial Neural Networks. Y.D. Lukic, C.R. Stevens, J. Si, Application of a Real Time Artificial Neural Network for Classifying Nuclear Power Plant Transient Events. J.A. Boshers, C.H.M. Saylor, S. Kamadolli, R. Wood, C. Isik, Control Rod Wear Recognition Using Neural Nets. R. Doremus, Severe Accident Management System On-Line Network (SAMSON). Part IV:Power System Operation. H. Ren-mu, A.J. Germond, Comparison of Dynamic Load Models Extrapolation Using Neural Networks and Traditional Methods. B. Avramovic, On Neural Network Voltage Assessment. D. Sobajic, Y-H. Pao, M. Djukanovic, Neural Network Synthesis of Tangent Hypersurfaces for Transient Security Assessment of Electric Power Systems. D. Niebur, A.J. Germond, Power System Static Security Assessment Using the Kohonen Neural Network Classifier. H. Mori, Voltage Stability Monitoring with Artificial Neural Networks. D. Novosel, A.B. Boveri, R.L. King, Intelligent Load Shedding. E. Chan, N. Markushevich, R. Adapa, Considerations in Intelligent Alarm Processing. Part V:Modeling and Prediction. D.J. Sobajic, Y-H. Pao, D.T. Lee, Predictive Security Monitoring with Neural Networks. A.G. Parlos, A.D. Patton, Empirical Modeling in Power Engineering Using the Recurrent Multilayer Perceptron Network. T. Samad, Modeling and Identification with Neural Networks. E. Wan, Autoregressive Neural Network Prediction: Learning Chaotic Time Series and Attractors. Part VI:Control. B. Widrow, F. Beaufays, Neural Control Systems. R.L. King, M.L. Oatts, Potential Uses of Intelligent and Adaptive Controls for Electric Power System Operations in the Year 2000 and Beyond. F. Beaufays, B. Widrow, Load-Frequency Control Using Neural Networks. L.L. Adams, Reinforcement Learning for Adaptive Control. Part VII:Load Forecasting. A.J. Germond, N. Macabrey, T. Baumann, Application of Artificial Neural Networks to Load Forecasting. M. Khadem, A. Lago, E. Dobrowolski, Short-Term Electric Load Forecasting Using Neural Networks. J.Y. Cheung, J. Fagan, D.C. Chance, Load Forecasting by Hierarchical Neural Networks that Incorporate Known Load Characteristics. Part VIII:Scheduling and Optimization. H. Sasaki, Y. Takiuchi, J. Kubokawa, A Solution Method for Maintenance Scheduling of Thermal Units by Artificial Neural Networks. H. Saitoh, Y. Shimotori, J. Toyoda, Generation Dispatch Algorithm Coordinating Economy and Stability by Using Artificial Neural Networks. Part IX:Fault Diagnosis. T. Baumann, A.J. Germond, D. Tschudi, Impulse Test Fault Diagnosis on Power Transformers Using Kohonen's Self-Organizing Neural Network. Y. Du, F. Wang, T.C. Cheng, A Case Study of Neural Network Application: Power Equipment Application Failure. A. Agogino, M-L. Tseng, P. Jain, Integrating Neural Networks with Influence Diagrams for Power Plant Monitoring and Diagnostics. W.L. Biach, Use of Neural Network in Optimizing RPV Bolting Procedures.

Neural networks, inspired by the functioning of the human brain, are a cornerstone of modern artificial intelligence (AI) research.This document traces the history, foundational concepts, types, and key applications of neural networks.Beginning with the pioneering work of McCulloch and Pitts, it outlines the significant developments that have shaped neural network models, from the Perceptron to advanced Deep Learning architectures like Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs).It also discusses essential neural network training techniques, including backpropagation, optimization strategies, and the role of hyperparameter tuning.Additionally, it contrasts Machine Learning and Deep Learning, highlighting their respective roles and computational requirements.Finally, the document introduces popular neural network frameworks such as TensorFlow and Keras, enabling the practical implementation of these models.

Artificial intelligent provides diverse solutions for the complex problems in agriculture research. The study aimed to use three models of artificial neural networks (Feed Forward Neural Network (FFNN), Generalized Regression Neural Network (GRNN) and Radial-Basis Neural Network (RBNN)) in the field of wheat yield prediction. 27-year data for the period (1986-2012) were utilized to improve the models and four-year data (2013 and 2016) were used to estimate the models, to compare their outputs with the measured data. Prediction data was not entered in the process of building neural network models. The results showed that the optimal configuration of the FFNN model consists of 40 neurons in the hidden layer (8-40-1). The Tan Sigmoid activation function was used in both the hidden layer and the output layer using all of these models (anterior neural feeding network and the regression neural network and radial base neural network) in the 4-year wheat yield forecast field for production (2013-2016) by applying 8 input parameters that were result of NMMS (8.6%, 7.6% and 15.7% resp.), To find that FFNN and GRNN provide the best result from BRNN because while the information set was large or in a wide range, then the range data ranges from -1 to +1 (normalization data) , GRNN gives better outcomes after the information or sample data were in large range.

The development of effective gastrointestinal diseases computer‐aided diagnosis tools and automatic image quality assessment algorithms is very important to improve the effectiveness of diagnosis and treatment. In order to further study the application of neural network algorithm in the endoscopic image of upper digestive tract, improve the efficiency of neural network algorithm in the field of endoscopic image. In this study, neural network algorithms were used to identify endoscopic images of the upper digestive tract. 1335 cases with upper gastrointestinal endoscopic images were collected. After the data was enlarged, it was randomly divided into training set and test set according to the proportion, and the obtained training set was input. After convolutional neural network training, an algorithm model was established in the institute. 1653 test set data samples were input into the neural network to verify the accuracy. Finally, the accuracy of the neural root network model constructed in this study reached 0.0942. Through horizontal comparison, it can be concluded that the neural network model proposed in this study not only has a higher accuracy rate, but also is better than the current existing related neural network algorithms. Based on the above experimental verification, it can be concluded that the upper gastrointestinal endoscopic image recognition algorithm based on neural network proposed in this study can more accurately and effectively identify the lesions in the upper gastrointestinal endoscopic images.