Neural Network Solution Research Articles

Radial basis function neural networks solution for stationary probability density function of nonlinear stochastic systems

A radial basis function neural networks (RBF-NN) solution of the reduced Fokker–Planck-Kolmogorov (FPK) equation is proposed in this paper. The activation functions consist of normalized Gaussian probability density functions (PDFs). The use of normalized Gaussian PDFs leads to a simple constraint on the coefficients for normalization of the RBF-NN solution, which as a constraint is imposed with the help of the method of Lagrange multiplier. The relationship between the proposed RBF-NN PDF solution and the generalized cell mapping with short-time Gaussian approximation is discussed, which provides a justification for Gaussian PDFs with varying means and variances in the state space. The optimal number of neurons or activation functions, which leads to the smallest error, is investigated. Four examples are presented to show the effectiveness of the proposed solution method. The results indicate that the proposed solution method is a very efficient and accurate way to compute the stationary PDF of nonlinear stochastic systems. It is also found that the distribution of the optimal coefficients as a function of the mean of Gaussian activation functions is similar to the steady-state PDF solution. Finally, we should point out that an important advantage of the RBF-NN method over methods such as finite element and finite difference is its ability to obtain solutions of the FPK equation for multi-degree-of-freedom stochastic systems.

New insight into bifurcation of fractional-order 4D neural networks incorporating two different time delays

Delay has a vital influence on the dynamics of neural networks. Exploring the effect of time delay on the dynamics of neural networks has become a hot issue in mathematics and engineering fields. In this current manuscript, on the basis of the earlier publications, we put forward a new fractional-order 4D neural networks incorporating two different time delays. First of all, the existence and uniqueness, boundedness of the solution of the fractional-order 4D neural networks incorporating two different time delays. are analyzed by applying contraction mapping principle, construct of an adaptive function, respectively. Next, the stability and the emergence of Hopf bifurcation are explored by making use of the stability and bifurcation theory of fractional-order dynamical system. A series of novel stability criteria and bifurcation conditions guaranteeing the stability and the emergence of Hopf bifurcation of the considered fractional-order 4D neural networks under the different delay cases are built. What,s more, the impact of delay on stabilizing neural networks and controlling the emergence of Hopf bifurcation of neural networks is adequately uncovered. At last, Matlab simulation figures are presented to confirm scientificness of the derived prime conclusions. The derived prime conclusions of this manuscript are perfectly innovative and own momentous theoretical reference value in the control issue and design aspect of neural networks.

Segmented Spline Curve Neural Network for Low Latency Digital Predistortion of RF Power Amplifiers

At present, we are the dawn of a new era for wireless communication systems. The new dynamic approach to the operation of cellular networks requires an extension of the essential input–output hardware mechanisms, to include intelligence. Traditional neural network structures can be used to embed artificial intelligence into cellular network base stations; however, standard network structures are not an optimal solution for these use cases. In this article, we present a neural network structure, which is specifically designed to provide a more accurate and computationally efficient solution compared with the previous neural network solutions for predistortion of RF power amplifiers (PAs). The proposed network structure is directly compared with alternative neural network solutions, which have been successfully employed for digital predistortion (DPD). The operation of this network is validated for DPD with experimental measurements with wideband signals using the latest generation of commercially available RF hardware. The novel network structure proposed in this work is demonstrated, in practice, to have better performance for a normalized mean square error (NMSE), an adjacent channel leakage ratio (ACLR), and an error vector magnitude (EVM) compared with the most popular previously published neural network.

Physics-Informed Neural Network Solution of Thermo–Hydro–Mechanical Processes in Porous Media

Physics-informed neural networks (PINNs) have received increased interest for forward, inverse, and surrogate modeling of problems described by partial differential equations (PDEs). However, their application to multiphysics problem, governed by several coupled PDEs, presents unique challenges that have hindered the robustness and widespread applicability of this approach. Here we investigate the application of PINNs to the forward solution of problems involving thermo–hydro–mechanical (THM) processes in porous media that exhibit disparate spatial and temporal scales in thermal conductivity, hydraulic permeability, and elasticity. In addition, PINNs are faced with the challenges of the multiobjective and nonconvex nature of the optimization problem. To address these fundamental issues, we (1) rewrote the THM governing equations in dimensionless form that is best suited for deep learning algorithms, (2) propose a sequential training strategy that circumvents the need for a simultaneous solution of the multiphysics problem and facilitates the task of optimizers in the solution search, and (3) leveraged adaptive weight strategies to overcome the stiffness in the gradient flow of the multiobjective optimization problem. Finally, we applied this framework to the solution of several synthetic problems in one and two dimensions.

A Framework for Neural Network Architecture and Compile Co-optimization

The efficiency of deep neural network (DNN) solutions on real hardware devices are mainly decided by the DNN architecture and the compiler-level scheduling strategy on the hardware. When we try to fully exploit the underlying hardware and obtain the optimal tradeoff between DNN accuracy and runtime performance, we discovered that the two optimization goals of DNN architecture and scheduling policy are intimately related to each other. However, current hardware-aware Neural Architecture Search (NAS) methods primarily focus on the DNN architecture search process, ignoring the effects of various compiler-level scheduling strategies (e.g., graph-level optimization, loop transformations, parallelization, etc.) on network candidates being evaluated in the search process. As a result, they may overlook the true-optimal DNN implementations on hardware, which can only be discovered by trying-out different combinations of scheduling strategies and DNN architectures. This work proposes a NAS framework (CHaNAS) that searches for not only the network architecture but also the dedicated compiler-level scheduling policy, as the optimal co-design solution on the target hardware. We propose to use a block-based pre-scheduling methodology to reduce the co-design search space and enable the automatic generation of the optimal co-design, including the network architecture and the tensor programs that practice the scheduling policy. Further, we introduce a new search objective function based on the generalization gap to prevent the selection of architectures that are prone to overfitting. We evaluate CHaNAS on Imagenet on different hardware back-ends against the state-of-the-art hardware-aware search method based on the MobileNet-v3 search space. Experimental results show that the co-design solutions obtained by ChaNAS show up to 1.6×, 1.9×, and 1.7×, 24 performance boost on NVIDIA P100 GPU, Intel Xeon 8163 CPU, and Samsung Note 10 Mobile, respectively, over the baselines of the same-level accuracy.

An application of Artificial Neural Network solution in the apparel industry for Job distribution to subcontractors

In this study, the results of the work distribution made with the TOPSIS method, which is frequently used in the distribution of work to the subcontractor workshops, were estimated using the Artificial Neural Networks Method (ANN). Here, the C* values used in the work distribution with the TOPSIS method were estimated by ANN. The correlation coefficient of the data obtained from the created model was found to be 99.99895% for learning. According to the results, it has been concluded that work distribution can be made by using the ANN method without making complex mathematical calculations in the distribution of work to the contract workshops.

Deep Neural Network Solution for Finite State Mean Field Game with Error Estimation

We discuss the numerical solution to a class of continuous time finite state mean field games. We apply the deep neural network (DNN) approach to solving the fully coupled forward and backward ordinary differential equation system that characterizes the equilibrium value function and probability measure of the finite state mean field game. We prove that the error between the true solution and the approximate solution is linear to the square root of DNN loss function. We give an example of applying the DNN method to solve the optimal market making problem with terminal rank-based trading volume reward.

Meshfree-based physics-informed neural networks for the unsteady Oseen equations

We propose the meshfree-based physics-informed neural networks for solving the unsteady Oseen equations. Firstly, based on the ideas of meshfree and small sample learning, we only randomly select a small number of spatiotemporal points to train the neural network instead of forming a mesh. Specifically, we optimize the neural network by minimizing the loss function to satisfy the differential operators, initial condition and boundary condition. Then, we prove the convergence of the loss function and the convergence of the neural network. In addition, the feasibility and effectiveness of the method are verified by the results of numerical experiments, and the theoretical derivation is verified by the relative error between the neural network solution and the analytical solution.

Physics-Informed Neural Network Solution of Point Kinetics Equations for a Nuclear Reactor Digital Twin

A digital twin (DT) for nuclear reactor monitoring can be implemented using either a differential equations-based physics model or a data-driven machine learning model. The challenge of a physics-model-based DT consists of achieving sufficient model fidelity to represent a complex experimental system, whereas the challenge of a data-driven DT consists of extensive training requirements and a potential lack of predictive ability. We investigate the performance of a hybrid approach, which is based on physics-informed neural networks (PINNs) that encode fundamental physical laws into the loss function of the neural network. We develop a PINN model to solve the point kinetic equations (PKEs), which are time-dependent, stiff, nonlinear, ordinary differential equations that constitute a nuclear reactor reduced-order model under the approximation of ignoring spatial dependence of the neutron flux. The PINN model solution of PKEs is developed to monitor the start-up transient of Purdue University Reactor Number One (PUR-1) using experimental parameters for the reactivity feedback schedule and the neutron source. The results demonstrate strong agreement between the PINN solution and finite difference numerical solution of PKEs. We investigate PINNs performance in both data interpolation and extrapolation. For the test cases considered, the extrapolation errors are comparable to those of interpolation predictions. Extrapolation accuracy decreases with increasing time interval.

A Practical Approach to the Analysis and Optimization of Neural Networks on Embedded Systems.

The exponential increase in internet data poses several challenges to cloud systems and data centers, such as scalability, power overheads, network load, and data security. To overcome these limitations, research is focusing on the development of edge computing systems, i.e., based on a distributed computing model in which data processing occurs as close as possible to where the data are collected. Edge computing, indeed, mitigates the limitations of cloud computing, implementing artificial intelligence algorithms directly on the embedded devices enabling low latency responses without network overhead or high costs, and improving solution scalability. Today, the hardware improvements of the edge devices make them capable of performing, even if with some constraints, complex computations, such as those required by Deep Neural Networks. Nevertheless, to efficiently implement deep learning algorithms on devices with limited computing power, it is necessary to minimize the production time and to quickly identify, deploy, and, if necessary, optimize the best Neural Network solution. This study focuses on developing a universal method to identify and port the best Neural Network on an edge system, valid regardless of the device, Neural Network, and task typology. The method is based on three steps: a trade-off step to obtain the best Neural Network within different solutions under investigation; an optimization step to find the best configurations of parameters under different acceleration techniques; eventually, an explainability step using local interpretable model-agnostic explanations (LIME), which provides a global approach to quantify the goodness of the classifier decision criteria. We evaluated several MobileNets on the Fudan Shangai-Tech dataset to test the proposed approach.

Deployment of convolutional neural network solutions for image computing in semiconductor manufacturing environment

BackgroundSimilar to many other industries, semiconductor manufacturing is undergoing a digital transformation. The wafer fabs have been highly automated for a few years, and data are everywhere, in high volume, heterogeneous, and not always structured. Data analytics for manufacturing is becoming a key competence to be embedded in the daily lives of wafer fabs engineers. Among the wide variety of data, this paper focuses on images and their classification by convolutional neural networks (CNN), which are illustrated by various use cases in the manufacturing environment.AimPractically, in over a year in a wafer fab, millions of pictures are generated. Most of the time these images are treated directly by metrology and inspection tools, and humans eventually look at these only if a measurement alarm (measurement quality or out-of-specification value) is reported. Is there any interest in reviewing all of them? Technically, images are underused data sources of information because they contain a lot of relevant information that is not captured. The return on investment of inspecting all images is highly questionable if the review needs to be done by humans. When done by the operator, this task is time-consuming and prone to human interpretation, which can lead to variability in the result. The aim of this work is to show that CNNs are perfectly adapted to wafer fabs and can take on this workload in a very efficient way.ApproachSince the middle of the last decade, new tools (software and hardware) have emerged to solve these massive classification needs, including (1) deep learning algorithms and particularly CNNs or region-based CNNs (RCNN), (2) graphical processor unit (GPU) to speed up training time, (3) distributed database to store high volumes of data, and (4) open-source community driven by python eco-systems to support proof of concept build-ups and customized solutions.ResultsThis paper will emphasize the integration of these deep learning solutions into the daily life of semiconductor manufacturing with a focus on transfer learning, which provides high versatility of CNNs/R-CNNs solutions with respect to different use cases.ConclusionsA unique CNN infrastructure has been set up to offer wafer fab engineers a versatile solution adapted to many different kinds of images and use cases to demonstrate that CNN image classification can be smoothly embedded in daily tasks.

Anti-periodic solutions of Clifford-valued fuzzy cellular neural networks with delays

Anti-periodic solutions of Clifford-valued fuzzy cellular neural networks with delays

Development of an EEG artefact detection algorithm and its application in grading neonatal hypoxic-ischemic encephalopathy

ObjectiveThe primary aim of this study is to develop and evaluate algorithms for neonatal EEG artefact detection. The secondary aim is to subsequently assess its application as a post-processing routine for automated EEG grading of background abnormalities in neonatal hypoxic-ischemic encephalopathy (HIE). Methods: A database of neonatal EEG with expertly annotated artefacts was used to train and validate machine learning models to automatically identify EEG epochs containing artefacts. Three approaches were developed and compared, specifically, a simple threshold-based digital signal processing (DSP) method, a machine learning method, and a deep learning method. The artefact detection classifier was subsequently assessed as a post-processing tool to assist in the application of automated EEG grading of HIE. A new deep learning model for grading the EEG was developed by training an existing network on a large, multi-centre dataset. The artefact detection algorithm was integrated into the grading algorithm through a post-processing routine. Results: Using a database containing 19 h of EEG from 51 patients with per-channel and per-second annotations of artefacts, a deep learning convolutional neural network solution achieved best performance for artefact detection with an area under the operating characteristic curve (AUC) of 0.84, compared to an AUC of 0.68 and 0.82 for a DSP method and a random-kernel ridge-classifier model, respectively. The automated EEG grading algorithm was trained and tested on 653 h of EEG from 181 patients, which achieved an accuracy of 82.8 % (95 % CI: 80.5 % to 85.2 %). The percentage of detected artefacts in the misclassified epochs was not statistically different (p = 0.568) compared to that of correctly classified epochs. Using artefact detection, a small number of epochs were removed from grading, resulting in a minor increase in accuracy for the EEG grading algorithm from 82.6 % to 83.6 %. Conclusion: Deep learning methods achieved highest classification performance for neonatal EEG artefact detection, although a ridge classifier using random kernels achieved comparable performance without significant parameter tuning or training time. The inclusion of artefact detection in automated EEG grading does not significantly improve accuracy in our curated dataset, but does allow for a quality measure to be presented alongside the automated EEG grades which may increase user confidence in its real-world application.

A New Approach to Stability Analysis for Stochastic Hopfield Neural Networks With Time Delays

This article is devoted to the existence and the global stability of stationary solutions for stochastic Hopfield neural networks with time delays and additive white noises. Using the method of random dynamical systems, we present a new approach to guarantee that the infinite-dimensional stochastic flow generated by stochastic delay differential equations admits a globally attracting random equilibrium in the state-space of continuous functions. An example is given to illustrate the effectiveness of our results, and the forward trajectory synchronization will occur.

Data-Driven Neural Predictors-Based Robust MPC for Power Converters

This article proposes a novel data-driven neural predictors-based robust finite control-set model predictive control (FCS-MPC) methodology for power converters, which aims to enhance the robustness of the control system and the flexibility of multiple control objectives. To be specific, the data-driven predictors-based neural network solution, which has a good potential to estimate the unknown nonlinear system dynamics by deploying real-time and historical data, is incorporated into the proposed design with a cost function derived intuitively from Lyapunov’s control theory. The key feature of this work is that the undesired effects of the uncertainties and the complex multiobjective optimization procedure can be explicitly dealt with, without involving accurate modeling information and weighting factor combinations, while guaranteeing adaptability to unknown nonlinear system dynamics, model variations, and environment changes. Finally, the stability analysis is given, and the effectiveness of the proposed FCS-MPC methodology is verified analytically and confirmed by comprehensive results that prove the theoretical investigations for active front-end modular multilevel converter.

A priori and a posteriori error estimates for the Deep Ritz method applied to the Laplace and Stokes problem

We analyze neural network solutions to partial differential equations obtained with Physics Informed Neural Networks. In particular, we apply tools of classical finite element error analysis to obtain conclusions about the error of the Deep Ritz method applied to the Laplace and the Stokes equations. Further, we develop an a posteriori error estimator for neural network approximations of partial differential equations. The proposed approach is based on the dual weighted residual estimator. It is destined to serve as a stopping criterion that guarantees the accuracy of the solution independently of the design of the neural network training. The result is equipped with computational examples for Laplace and Stokes problems.

Quantitative evaluation of the influence of multiple MRI sequences and of pathological tissues on the registration of longitudinal data acquired during brain tumor treatment.

Registration methods facilitate the comparison of multiparametric magnetic resonance images acquired at different stages of brain tumor treatments. Image-based registration solutions are influenced by the sequences chosen to compute the distance measure, and the lack of image correspondences due to the resection cavities and pathological tissues. Nonetheless, an evaluation of the impact of these input parameters on the registration of longitudinal data is still missing. This work evaluates the influence of multiple sequences, namely T1-weighted (T1), T2-weighted (T2), contrast enhanced T1-weighted (T1-CE), and T2 Fluid Attenuated Inversion Recovery (FLAIR), and the exclusion of the pathological tissues on the non-rigid registration of pre- and post-operative images. We here investigate two types of registration methods, an iterative approach and a convolutional neural network solution based on a 3D U-Net. We employ two test sets to compute the mean target registration error (mTRE) based on corresponding landmarks. In the first set, markers are positioned exclusively in the surroundings of the pathology. The methods employing T1-CE achieves the lowest mTREs, with a improvement up to 0.8 mm for the iterative solution. The results are higher than the baseline when using the FLAIR sequence. When excluding the pathology, lower mTREs are observable for most of the methods. In the second test set, corresponding landmarks are located in the entire brain volumes. Both solutions employing T1-CE obtain the lowest mTREs, with a decrease up to 1.16 mm for the iterative method, whereas the results worsen using the FLAIR. When excluding the pathology, an improvement is observable for the CNN method using T1-CE. Both approaches utilizing the T1-CE sequence obtain the best mTREs, whereas the FLAIR is the least informative to guide the registration process. Besides, the exclusion of pathology from the distance measure computation improves the registration of the brain tissues surrounding the tumor. Thus, this work provides the first numerical evaluation of the influence of these parameters on the registration of longitudinal magnetic resonance images, and it can be helpful for developing future algorithms.

Dynamics on time scales of wave solutions for nonlinear neural networks

A time scale is an arbitrary non-empty closed subset of denoted by . In this work, the traveling wave solutions on time scales for a class of nonlinear delayed dynamical neural networks are investigated. The outcomes of this study are new and have been obtained by inequality technique, Ikehara's theorem, the weighted energy method, Taylor formula, comparison principle and the first integral mean value theorem. This is the first work focused for the wave solution in time space scales for a dynamical delayed neural networks model.

CENN: Conservative energy method based on neural networks with subdomains for solving variational problems involving heterogeneous and complex geometries

We propose a conservative energy method based on neural networks with subdomains for solving variational problems (CENN), where the admissible function satisfying the essential boundary condition without boundary penalty is constructed by the radial basis function (RBF), particular solution neural network, and general neural network. Loss term is the potential energy, optimized based on the principle of minimum potential energy. The loss term at the interfaces has the lower order derivative compared to the strong form PINN with subdomains. The advantage of the proposed method is higher efficiency, more accurate, and less hyperparameters than the strong form PINN with subdomains. Another advantage of the proposed method is that it can apply to complex geometries based on the special construction of the admissible function. To analyze its performance, the proposed method CENN is used to model representative PDEs, the examples include strong discontinuity, singularity, complex boundary, non-linear, and heterogeneous problems. Furthermore, it outperforms other methods when dealing with heterogeneous problems.

Optimization and Evaluation of Multidetector Deep Neural Network for High-Accuracy Wi-Fi Fingerprint Positioning

To fulfill the need for high-accuracy indoor positioning in many location-based services (LBSs) and the emerging Internet of Things (IoT) applications, in this article, we propose a novel scene-analysis positioning solution of the multidetector deep neural network (DNN) architecture, with preprocessing steps, model optimization techniques, and variance estimation methods. During the offline site-surveying phase in our approach, fingerprint databases are created by purposely built robotic surveying devices traversing the target site to gather perceivable Wi-Fi and other signals including to create spatial positioning models for further use in the online positioning phase. The intricate nonlinear relationship between fingerprints and spatial positions are thus resolved by the multidetector DNN in our approach. Hyperparameter analyses were conducted to further optimize our proposed multidetector model in terms of complexity, achieving at least 6.7 times of parameter complexity reduction while retaining < 1% degradation of 0.9-m (3 ft) positioning accuracy level.