Neural Network Theory Research Articles

Learning Heuristics With Different Representations for Stochastic Routing.

Uncertainty is ubiquitous in real-world routing applications. The automated design of the routing policy by hyperheuristic methods is an effective technique to handle the uncertainty and to achieve online routing for dynamic or stochastic routing problems. Currently, the tree representation routing policy evolved by genetic programming is commonly adopted because of the remarkable flexibility. However, numeric representations have never been used. Considering the practicability of the numeric representations and the capability of the numeric optimization methods, in this article, we investigate two numeric representations on a representative stochastic routing problem and uncertain capacitated arc routing problem. Specifically, a linear representation and an artificial neural-network (ANN) representation are implemented and compared with the tree representation to reveal the potential of the numeric representations and the characteristics of their optimization. Experimental results show that the tree representation is the best choice, but on a majority of the test instances, the numeric representations, especially the ANN representation, can provide competitive performance. Further analyses also show that training a good ANN representation policy requires more training data than the tree representation. Finally, a guideline of representation selection is given.

Agricultural products price prediction based on improved RBF neural network model

ABSTRACT The agricultural products price has been affecting people’s livelihood issues and economic and social security and stability. The cyclical fluctuation of the price of agricultural products often indirectly affects the inner psychological demand of agricultural consumers. If the prices of production and processing industries, which rely on cheap raw materials as basic raw materials, are subject to frequent and abnormal fluctuations, it is likely to cause further widespread concerns about people’s lives, and ultimately lead to a vicious cycle of falling commodity prices. In recent years, as a result of lack of timely authoritative information all the time, the market price of agricultural products appeared a lot of varieties before short-term rise and fall repeatedly phenomenon. This paper attempts to take the quantity of garlic produced and sold by pork in China as the key object of analysis and research, and analyzes the level of market price index and the main factors influencing the price of pork sales and edible garlic demand in China in recent ten years. In information economics, financial market theory, statistical methods and other relevant mathematical model as the main guidance, combined with China’s agricultural prices during the period of policy, select the RBF neural network theory and the analysis methods for technical improvements, in a variety of market factors affecting economic operation under the mixed operation China period in our country agricultural prices technology wave prediction rule, broken Based on the traditional artificial statistics and forecasting model method of agricultural product price, the short-term forecasting model of Chinese agricultural product market price theory based on information technology innovation method was established.

Evaluation of Federated Learning in Phishing Email Detection.

The use of artificial intelligence (AI) to detect phishing emails is primarily dependent on large-scale centralized datasets, which has opened it up to a myriad of privacy, trust, and legal issues. Moreover, organizations have been loath to share emails, given the risk of leaking commercially sensitive information. Consequently, it has been difficult to obtain sufficient emails to train a global AI model efficiently. Accordingly, privacy-preserving distributed and collaborative machine learning, particularly federated learning (FL), is a desideratum. As it is already prevalent in the healthcare sector, questions remain regarding the effectiveness and efficacy of FL-based phishing detection within the context of multi-organization collaborations. To the best of our knowledge, the work herein was the first to investigate the use of FL in phishing email detection. This study focused on building upon a deep neural network model, particularly recurrent convolutional neural network (RNN) and bidirectional encoder representations from transformers (BERT), for phishing email detection. We analyzed the FL-entangled learning performance in various settings, including (i) a balanced and asymmetrical data distribution among organizations and (ii) scalability. Our results corroborated the comparable performance statistics of FL in phishing email detection to centralized learning for balanced datasets and low organizational counts. Moreover, we observed a variation in performance when increasing the organizational counts. For a fixed total email dataset, the global RNN-based model had a 1.8% accuracy decrease when the organizational counts were increased from 2 to 10. In contrast, BERT accuracy increased by 0.6% when increasing organizational counts from 2 to 5. However, if we increased the overall email dataset by introducing new organizations in the FL framework, the organizational level performance improved by achieving a faster convergence speed. In addition, FL suffered in its overall global model performance due to highly unstable outputs if the email dataset distribution was highly asymmetric.

A manifold two-sample test study: integral probability metric with neural networks

Abstract Two-sample tests are important areas aiming to determine whether two collections of observations follow the same distribution or not. We propose two-sample tests based on integral probability metric (IPM) for high-dimensional samples supported on a low-dimensional manifold. We characterize the properties of proposed tests with respect to the number of samples $n$ and the structure of the manifold with intrinsic dimension $d$. When an atlas is given, we propose a two-step test to identify the difference between general distributions, which achieves the type-II risk in the order of $n^{-1/\max \{d,2\}}$. When an atlas is not given, we propose Hölder IPM test that applies for data distributions with $(s,\beta )$-Hölder densities, which achieves the type-II risk in the order of $n^{-(s+\beta )/d}$. To mitigate the heavy computation burden of evaluating the Hölder IPM, we approximate the Hölder function class using neural networks. Based on the approximation theory of neural networks, we show that the neural network IPM test has the type-II risk in the order of $n^{-(s+\beta )/d}$, which is in the same order of the type-II risk as the Hölder IPM test. Our proposed tests are adaptive to low-dimensional geometric structure because their performance crucially depends on the intrinsic dimension instead of the data dimension.

Joint-product representation learning for domain generalization in classification and regression

In this work, we study the problem of generalizing a prediction (classification or regression) model trained on a set of source domains to an unseen target domain, where the source and target domains are different but related, i.e, the domain generalization problem. The challenge in this problem lies in the domain difference, which could degrade the generalization ability of the prediction model. To tackle this challenge, we propose to learn a neural network representation function to align a joint distribution and a product distribution in the representation space, and show that such joint-product distribution alignment conveniently leads to the alignment of multiple domains. In particular, we align the joint distribution and the product distribution under the L^{2}-distance, and show that this distance can be analytically estimated by exploiting its variational characterization and a linear variational function. This allows us to comfortably align the two distributions by minimizing the estimated distance with respect to the network representation function. Our experiments on synthetic and real-world datasets for classification and regression demonstrate the effectiveness of the proposed solution. For example, it achieves the best average classification accuracy of 82.26% on the text dataset Amazon Reviews, and the best average regression error of 0.114 on the WiFi dataset UJIIndoorLoc.

The power of typed affine decision structures: a case study

AbstractTADS are a novel, concise white-box representation of neural networks. In this paper, we apply TADS to the problem of neural network verification, using them to generate either proofs or concise error characterizations for desirable neural network properties. In a case study, we consider the robustness of neural networks to adversarial attacks, i.e., small changes to an input that drastically change a neural networks perception, and show that TADS can be used to provide precise diagnostics on how and where robustness errors a occur. We achieve these results by introducing Precondition Projection, a technique that yields a TADS describing network behavior precisely on a given subset of its input space, and combining it with PCA, a traditional, well-understood dimensionality reduction technique. We show that PCA is easily compatible with TADS. All analyses can be implemented in a straightforward fashion using the rich algebraic properties of TADS, demonstrating the utility of the TADS framework for neural network explainability and verification. While TADS do not yet scale as efficiently as state-of-the-art neural network verifiers, we show that, using PCA-based simplifications, they can still scale to medium-sized problems and yield concise explanations for potential errors that can be used for other purposes such as debugging a network or generating new training samples.

Two-Module Neural Network Method of Information Processing in Gas Analysis Systems

Purpose of research: reduction of additional errors in measuring gas concentrations in gas analytical systems (GS) caused by the sensitivity of semiconductor sensors to non-target components of gas mixtures, ambient temperature and humidity. To develop and test a two-module neural network method for processing information in a GS, which allows automating the processes of generating training data and searching for the optimal structure of artificial neural networks (ANNs), reducing errors in reproducing the characteristics of sensors by replacing their mathematical models with neural networks.Methods. Theory of artificial neural networks, numerical methods, simulation methods. To evaluate the effectiveness of the proposed solution, the relative error (d), standard deviation (RMS) were calculated, and comparison with analogues was carried out.Results: a two-module neural network method for processing information in a GS has been studied. Numerical modeling was used to carry out experimental studies on the choice of optimal ANN structures, the volume and composition of training data. In the course of experimental studies, the errors of generating training data using ANN (less than 5%) and determining the concentrations of detected gases under conditions of fluctuations in the parameters of the air environment and the composition of the gas mixture (less than 4%) were calculated.Conclusion. A two-module neural network method for information processing is proposed, which is distinguished by the use of two successive modules of multilayer neural networks for generating training data and processing information coming from the GS sensor unit. The use of an auxiliary module makes it possible to compress the initial data, unify and automate the process of their generation, as well as improve the accuracy of reproduction of multiparameter sensor conversion functions, in comparison with alternative methods. Results of experimental studies of the effectiveness of using the information processing method to reduce additional errors in the quantitative determination of the composition of the air environment under conditions of parameter fluctuations are presented.

Modeling density-driven flow in porous media by physics-informed neural networks for CO2 sequestration

Accurate prediction of density-driven convection of CO2 overlaying brine in porous media is crucial to the applications of geological carbon sequestration. In this paper, we introduce the physics-informed neural network (PINN) method to investigate the mass transfer of CO2 injected into a homogeneous subsurface porous formation, where the concentration and the streamfunction are approximated by neural network representations. To mitigate the computational burden and training difficulties arising from learning the long-term diffusion and convection processes, we implement a time domain decomposition-based scheme so that the PINN solution can represent short-interval dynamics in a sequential manner. Furthermore, Fourier-type basis functions are encoded in the PINN architecture to map the spatial coordinates onto an embedding feature space characterized by various frequencies. The numerical results show that the use of Fourier feature embedding improves the training performance by handling the notorious spectral bias in deep neural networks and enhances the stability for long-term prediction of flow and transport. The proposed PINN method is applied to a benchmark problem to understand the effect of natural convection on the simulation performance. Compared to the high-fidelity stabilized FEM solution, the PINN approach presents agreeable statistical characteristics of CO2-rich brine fingers, demonstrating the effectiveness of PINNs in modeling density-driven flow.

Dynamic-Pix2Pix: Medical image segmentation by injecting noise to cGAN for modeling input and target domain joint distributions with limited training data

Learning to translate images from a source to a target domain with applications such as converting simple line drawing to oil painting has attracted significant attention. The quality of translated images is directly related to two crucial issues. First, the consistency of the output distribution with that of the target is essential. Second, the generated output should have a high correlation with the input. Conditional Generative Adversarial Networks, cGANs, are the most common models for translating images. The performance of a cGAN drops when we use a limited training dataset. In this work, we study the obstacles that prevent a U-shaped model from learning the target domain distribution from limited data by using noise as input. This study helps to increase the Pix2Pix (a form of cGAN) target distribution modeling ability from limited data with the help of dynamic neural network theory. Our model has two learning cycles. The model learns the correlation between input and ground truth in the first cycle. Then, the model's architecture is refined in the second cycle to learn the target distribution from noise input. These processes are executed in each iteration of the training procedure. Helping the cGAN learn the target distribution from noise input results in a better model generalization during the test time and allows the model to fit almost perfectly to the target domain distribution. As a result, our model surpasses the Pix2Pix model in segmenting HC18 and Montgomery's chest x-ray images. Both qualitative and Dice scores show the superiority of our model. Although our proposed method does not use thousand of additional data for pretraining, it produces comparable results for the in and out-domain generalization compared to the state-of-the-art methods.

The moving load identification method on asphalt roads based on the BP neural network and FBG sensor monitoring

This study to accurately identify the load information of vehicles on asphalt roads is based on the measured data of the fiber Bragg grating (FBG) sensor and the Back Propagation (BP) neural network. First, the input parameters were determined by the BP neural network and FBG sensor theory. Second, the internal mechanical response of asphalt roads under different working conditions was calculated by using finite element software and by using the results of numerical simulation trained in the neural network. Third, the accuracy of the BP neural network used to identify the moving load on asphalt roads was verified by the field-measured data of the FBG sensor. Fourth, the influential factors of moving load identification accuracy were analyzed. The main conclusions are as follows: (a) the sensitive factors of moving load identification on asphalt roads include vehicle speed, asphalt layer temperature, the longitudinal strain between asphalt layers, the number of wheels, and load position; (b) the average absolute error of moving load identification is about 4.11kN, and the average recognition accuracy is up to 90.57%; (c) the measurement accuracy is sensitive to the speed of the vehicle. When vehicle speed exceeds 60 km/h, recognition accuracy may decrease to less than 90%; and (d) the loads at different locations have a significant influence on moving load identification.

Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems

We explore using neural operators, or neural network representations of nonlinear maps between function spaces, to accelerate infinite-dimensional Bayesian inverse problems (BIPs) with models governed by nonlinear parametric partial differential equations (PDEs). Neural operators have gained significant attention in recent years for their ability to approximate the parameter-to-solution maps defined by PDEs using as training data solutions of PDEs at a limited number of parameter samples. The computational cost of BIPs can be drastically reduced if the large number of PDE solves required for posterior characterization are replaced with evaluations of trained neural operators. However, reducing error in the resulting BIP solutions via reducing the approximation error of the neural operators in training can be challenging and unreliable. We provide an a priori error bound result that implies certain BIPs can be ill-conditioned to the approximation error of neural operators, thus leading to inaccessible accuracy requirements in training. To reliably deploy neural operators in BIPs, we consider a strategy for enhancing the performance of neural operators: correcting the prediction of a trained neural operator by solving a linear variational problem based on the PDE residual. We show that a trained neural operator with error correction can achieve a quadratic reduction of its approximation error, all while retaining substantial computational speedups of posterior sampling when models are governed by highly nonlinear PDEs. The strategy is applied to two numerical examples of BIPs based on a nonlinear reaction–diffusion problem and deformation of hyperelastic materials. We demonstrate that posterior representations of the two BIPs produced using trained neural operators are greatly and consistently enhanced by error correction.

Automatic design and optimization of educational space for autistic children based on deep neural network and affordance theory.

In recent years, the incidence of autistic children has shown rapid growth worldwide. The rapid development of education and rehabilitation institutions for autistic children is of great significance to the rehabilitation of this group. However, the research on indoor space environments and functional facilities for autistic children in China is still in its infancy. Reasonably and effectively, zoning the education and rehabilitation space for autistic children can promote better communication and learning between autistic children and rehabilitation therapists and effectively promote the rehabilitation progress of autistic children. However, the existing education and rehabilitation space for autistic children has some problems, such as unscientific indoor partition, indoor space layouts mainly relying on manual work, heavy workload and low efficiency. Therefore, it is of great research value and practical significance to explore the intuitive design and optimization of the education and rehabilitation space layout for autistic children. This study first evaluates and optimizes the educational space for autistic children based on the affordability theory. Then, this study proposes a layout recommendation algorithm based on deep learning, which is used to improve the layout efficiency of the education and rehabilitation space for autistic children and realize real-time online layout. The scene information is digitized in binary code. The segmentation and layout network models are constructed through bidirectional long short-term memory (LSTM) to discover the long segment pre-segmentation of house type and obtain the layout results. The word embedding algorithm is used to abstract the cross features between each vector segment, and the dimension of the feature matrix is reduced to improve the speed and accuracy of the layout scheme recommendation. The experimental results show that our method can learn the design rules from the data set and has achieved better results than the existing methods. This study provides an adequate theoretical basis and design reference for the research of residential education space for autistic children.

CompLaB v1.0: a scalable pore-scale model for flow, biogeochemistry, microbial metabolism, and biofilm dynamics

Abstract. Microbial activity and chemical reactions in porous media depend on the local conditions at the pore scale and can involve complex feedback with fluid flow and mass transport. We present a modeling framework that quantitatively accounts for the interactions between the bio(geo)chemical and physical processes and that can integrate genome-scale microbial metabolic information into a dynamically changing, spatially explicit representation of environmental conditions. The model couples a lattice Boltzmann implementation of Navier–Stokes (flow) and advection–diffusion-reaction (mass conservation) equations. Reaction formulations can include both kinetic rate expressions and flux balance analysis, thereby integrating reactive transport modeling and systems biology. We also show that the use of surrogate models such as neural network representations of in silico cell models can speed up computations significantly, facilitating applications to complex environmental systems. Parallelization enables simulations that resolve heterogeneity at multiple scales, and a cellular automaton module provides additional capabilities to simulate biofilm dynamics. The code thus constitutes a platform suitable for a range of environmental, engineering and – potentially – medical applications, in particular ones that involve the simulation of microbial dynamics.

Driver Behavior Recognition Based on Neural Networks Theory

The article deals with the problem of driver behavior while driving the vehicle. Driver distraction can lead to serious accidents that threaten human life and public property around the world. Solving the problem of preventing dangerous driving behavior will reduce the risk of getting into an accident in the future. Thus, there is a need for a smart vehicle that will support driver behavior recognition functionality. A possible solution to the problem using an artificial neural network for automatic recognition of driver behavior on a real set of driver behavior data is considered. The high accuracy and efficiency of the developed model recognition is obtained.

APPLICATION OF ARTIFICIAL NEURAL NETWORKS AND ACHIEVEMENTS OF TECHNOLOGICAL RE-ENGINEERING OF MODERN INDUSTRIAL PRODUCTION

The article deals with the issue of the justification mechanism for the choice of directions of technological reengineering. It was determined that the effectiveness of the transformations primarily depends on the methods, tools for choosing directions for the radical transformation of the technological basis of the production enterprise and, in general, on the technological policy. The justification of the directions of technological training engineering is connected with a large volume of interdependent variables of innovative transformations and requires a fundamental analysis of their behaviour and impact on production process. Today, neural networks are one of the most famous and effective tools for intelligent data analysis, which is developed thanks to achievements in the field of artificial intelligence theory and computer science. Intelligent systems based on artificial neural networks and fuzzy logic allow solving the tasks of forecasting, optimization, pattern recognition and management. To train the network, it is necessary to have a set of values of input values (factors) and corresponding to each separate set of values of the desired output value. Such an approach completely coincides with the tasks of choosing the directions of technological reengineering at the enterprises of the innovation cluster. For this purpose, significant external and internal factors influencing the effectiveness of the cluster policy have been determined. In contrast to internal, external to the greatest extent allow solving the tasks of forecasting, optimization, pattern recognition and management. In the conditions of an imperfect regional innovation system and taking into account this factor, for the efficiency of solving the tasks of technological reengineering of industrial production, a methodology is proposed that combines foresight forecasting of the prospects of technological reengineering of enterprises with the theory of artificial neural networks. The mechanism of interaction of artificial neural networks and the foresight method in determining directions of technological reengineering based on regional cluster policy are proposed.

RGBT tracking using randomly projected CNN features

Integrating multiple complementary features has been proved to be an effective way for improving tracking results. In this paper, we exploit how to perform robust visual tracking in challenging situations by adaptively integrating information from RGB and thermal videos. Specifically, convolutional neural network (CNN) representation with random projection is proposed to depict RGB and thermal images, respectively. Furthermore, an adaptive fusion strategy based on a period of time is developed. We jointly optimize the reliable weights of different modalities. In addition, we explore the random projection to CNN features. Extensive experiments against other state-of-the-art methods demonstrate the effectiveness of the proposed method. Through analyzing quantitative tracking results, we provide basic insights in RGB and thermal data tracking.

Intermediate acoustic-to-semantic representations link behavioral and neural responses to natural sounds

Recognizing sounds implicates the cerebral transformation of input waveforms into semantic representations. Although past research identified the superior temporal gyrus (STG) as a crucial cortical region, the computational fingerprint of these cerebral transformations remains poorly characterized. Here, we exploit a model comparison framework and contrasted the ability of acoustic, semantic (continuous and categorical) and sound-to-event deep neural network representation models to predict perceived sound dissimilarity and 7 T human auditory cortex functional magnetic resonance imaging responses. We confirm that spectrotemporal modulations predict early auditory cortex (Heschl’s gyrus) responses, and that auditory dimensions (for example, loudness, periodicity) predict STG responses and perceived dissimilarity. Sound-to-event deep neural networks predict Heschl’s gyrus responses similar to acoustic models but, notably, they outperform all competing models at predicting both STG responses and perceived dissimilarity. Our findings indicate that STG entails intermediate acoustic-to-semantic sound representations that neither acoustic nor semantic models can account for. These representations are compositional in nature and relevant to behavior.

Retinex-MPCNN: A Retinex and Modified Pulse coupled Neural Network based method for low-illumination visible and infrared image fusion

To overcome detail loss problem of infrared and low-illumination visible light image fusion, this paper proposes a novel fusion framework based on Modified Pulse Coupled Neural Network (MPCNN) and Retinex theory. First, MPCNN is designed to segment original images into regions with different weights. Second, a novel Retinex-MPCNN algorithm is proposed to enhance low-illumination visible light image, details of which can be clearer. Then, a specific weighted fusion strategy based on region segmentations of MPCNN is designed to fuse the infrared and enhanced visible light images. Different from average fusion strategy, we introduce an illumination item to increase the attention for low-illumination areas, thereby preserves more details to the fusion image. Experimental results on TNO dataset demonstrate that our proposed method can generate fusion images with clear contour and structure information. Compared with existing fusion methods, our method achieves better performance both in subjective and objective assessment.

Improving the Performance of Open-Set Recognition with Generated Fake Data

Open-set recognition models, in addition to generalizing to unseen instances of known categories, have to identify samples of unknown classes during the training phase. The main reason the latter is much more complicated is that there is very little or no information about the properties of these unknown classes. There are methodologies available to handle the unknowns. One possible method is to construct models for them by using generated inputs labeled as unknown. Generative adversarial networks are frequently deployed to generate synthetic samples representing unknown classes to create better models for known classes. In this paper, we introduce a novel approach to improve the accuracy of recognition methods while reducing the time complexity. Instead of generating synthetic input data to train neural networks, feature vectors are generated using the output of a hidden layer. This approach results in a less complex structure for the neural network representation of the classes. A distance-based classifier implemented by a convolutional neural network is used in our implementation. Our solution’s open-set detection performance reaches an AUC value of 0.839 on the CIFAR-10 dataset, while the closed-set accuracy is 91.4%, the highest among the open-set recognition methods. The generator and discriminator networks are much smaller when generating synthetic inner features. There is no need to run these samples through the first part of the classifier with the convolutional layers. Hence, this solution not only gives better performance than generating samples in the input space but also makes it less expensive in terms of computational complexity.

Improving Accuracy of Integrated Neuro-Fuzzy Classifier with FCM based Clustering for Diagnosis of Psychiatric Disorder

Parkinson’s disease (PD) is a progressive neurodegenerative disorder. Autism spectrum disorder (ASD) is a neurodevelopment disorder. Clinical decision-making process is complex. Due to complex nature of disease sign and its symptoms clinical decision making may lead to misclassification. To deal with such complex medical problems methods or approaches of soft computing play an important role. This paper will focus on presenting an integrated Neuro-fuzzy model. This integrated model has the learning strength of neural network and knowledge representation ability of fuzzy logic. Modified Adaptive Neuro –Fuzzy inference system (M-ANFIS) is used here for classification and predication. Here Fuzzy C-mean (FCM) Clustering is used first to make classes of data before presenting in to ANFIS. This FCM based class will reduce the classifier computational overhead. Precision error and recall, F-measure and accuracy matrices are used to compare the experimental results with other classic methods.