Neural Network Solution Research Articles

OpPINN: Physics-informed neural network with operator learning to approximate solutions to the Fokker-Planck-Landau equation

We propose a hybrid framework opPINN: physics-informed neural network (PINN) with operator learning for approximating the solution to the Fokker-Planck-Landau (FPL) equation. The opPINN framework is divided into two steps: Step 1 and Step 2. After the operator surrogate models are trained during Step 1, PINN can effectively approximate the solution to the FPL equation during Step 2 by using the pre-trained surrogate models. The operator surrogate models greatly reduce the computational cost and boost PINN by approximating the complex Landau collision integral in the FPL equation. The operator surrogate models can also be combined with the traditional numerical schemes. It provides a high efficiency in computational time when the number of velocity modes becomes larger. Using the opPINN framework, we provide the neural network solutions for the FPL equation under the various types of initial conditions, and interaction models in two and three dimensions. Furthermore, based on the theoretical properties of the FPL equation, we show that the approximated neural network solution converges to the a priori classical solution of the FPL equation as the pre-defined loss function is reduced.

Finite-time stability of almost periodic solutions of Clifford-valued RNNs with time-varying delays and [formula omitted] operator on time scales

The aim of this study is to present the finite-time stability of almost periodic solutions for Clifford-valued recurrent neural networks (RNNs) with time-varying delays and the D operators on time scales using a direct method. In real-world networks, the interactions between network elements are inherently time-delayed, which causes the neural network to oscillate and become unstable. First, some lemmas are obtained by the definitions of almost periodic. Second, by using the theory of calculus for time scales and the Banach fixed point theorem, some sufficient conditions to ensure the finite-time stability of almost periodic solutions for this class of neural networks are obtained. Furthermore, a numerical example is provided to demonstrate the feasibility of the results.

Besicovitch-like almost periodic solutions of Clifford-valued fuzzy cellular neural networks with time-varying delays

In this paper, we are concerned with a class of Clifford-valued fuzzy cellular neural networks with time-varying delays. Based on the Banach fixed point theorem and inequality technique, we obtain the existence and uniqueness of Besicovitch-like almost periodic solutions and global exponential stability of the fuzzy cellular neural network under consideration. Finally, we give a numerical example to illustrate the effectiveness of our results. Even when the systems we consider degenerate to quaternion-valued, complex-valued and real-valued ones, our results in this paper are completely new.

Deep-Reinforcement-Learning-Based IoT Sensor Data Cleaning Framework for Enhanced Data Analytics.

The Internet of things (IoT) combines different sources of collected data which are processed and analyzed to support smart city applications. Machine learning and deep learning algorithms play a vital role in edge intelligence by minimizing the amount of irrelevant data collected from multiple sources to facilitate these smart city applications. However, the data collected by IoT sensors can often be noisy, redundant, and even empty, which can negatively impact the performance of these algorithms. To address this issue, it is essential to develop effective methods for detecting and eliminating irrelevant data to improve the performance of intelligent IoT applications. One approach to achieving this goal is using data cleaning techniques, which can help identify and remove noisy, redundant, or empty data from the collected sensor data. This paper proposes a deep reinforcement learning (deep RL) framework for IoT sensor data cleaning. The proposed system utilizes a deep Q-network (DQN) agent to classify sensor data into three categories: empty, garbage, and normal. The DQN agent receives input from three received signal strength (RSS) values, indicating the current and two previous sensor data points, and receives reward feedback based on its predicted actions. Our experiments demonstrate that the proposed system outperforms a common time-series-based fully connected neural network (FCDQN) solution, with an accuracy of around 96% after the exploration mode. The use of deep RL for IoT sensor data cleaning is significant because it has the potential to improve the performance of intelligent IoT applications by eliminating irrelevant and harmful data.

Pseudo almost periodic solutions for Clifford-valued neutral-type fuzzy neural networks with multi-proportional delay and D operator1

In this paper, a class of Clifford-valued neutral fuzzy neural-type networks with proportional delay and D operator and whose self feedback coefficients are also Clifford numbers are considered. By using the Banach fixed point theorem and some differential inequality techniques, we directly study the existence and global asymptotic stability of pseudo almost periodic solutions by not decomposing the considered Clifford-valued systems into real-valued systems. Finally, two examples are given to illustrate our main results. Our results of this paper are new.

Constructing Physics-Informed Neural Networks with Architecture Based on Analytical Modification of Numerical Methods by Solving the Problem of Modelling Processes in a Chemical Reactor.

A novel type of neural network with an architecture based on physics is proposed. The network structure builds on a body of analytical modifications of classical numerical methods. A feature of the constructed neural networks is defining parameters of the governing equations as trainable parameters. Constructing the network is carried out in three stages. In the first step, a neural network solution to an equation corresponding to a numerical scheme is constructed. It allows for forming an initial low-fidelity neural network solution to the original problem. At the second stage, the network with physics-based architecture (PBA) is further trained to solve the differential equation by minimising the loss function, as is typical in works devoted to physics-informed neural networks (PINNs). In the third stage, the physics-informed neural network with architecture based on physics (PBA-PINN) is trained on high-fidelity sensor data, parameters are identified, or another task of interest is solved. This approach makes it possible to solve insufficiently studied PINN problems: selecting neural network architecture and successfully initialising network weights corresponding to the problem being solved that ensure rapid convergence to the loss function minimum. It is advisable to use the devised PBA-PINNs in the problems of surrogate modelling and modelling real objects with multi-fidelity data. The effectiveness of the approach proposed is demonstrated using the problem of modelling processes in a chemical reactor. Experiments show that subsequent retraining of the initial low-fidelity PBA model based on a few high-accuracy data leads to the achievement of relatively high accuracy.

Piecewise pseudo almost periodic solutions of interval general BAM neural networks with mixed time-varying delays and impulsive perturbations

<abstract><p>This paper is concerned with piecewise pseudo almost periodic solutions of a class of interval general BAM neural networks with mixed time-varying delays and impulsive perturbations. By adopting the exponential dichotomy of linear differential equations and the fixed point theory of contraction mapping. The sufficient conditions for the existence of piecewise pseudo almost periodic solutions of the interval general BAM neural networks with mixed time-varying delays and impulsive perturbations are obtained. By adopting differential inequality techniques and mathematical methods of induction, the global exponential stability for the piecewise pseudo almost periodic solutions of the interval general BAM neural networks with mixed time-varying delays and impulsive perturbations is discussed. An example is given to illustrate the effectiveness of the results obtained in the paper.</p></abstract>

Mittag-Leffler stabilization of anti-periodic solutions for fractional-order neural networks with time-varying delays

<abstract> <p>Mittag-Leffler stabilization of anti-periodic solutions for fractional-order neural networks with time-varying delays are investigated in the article. We derive the relationship between the fractional-order integrals of the state function with and without delays through the division of time interval, using the properties of fractional calculus, and initial conditions. Moreover, by constructing the sequence solution of the system function which converges to a continuous function uniformly with the Arzela-Asoli theorem, a sufficient condition is obtained to ensure the existence of an anti-periodic solution and Mittag-Leffler stabilization of the system. In the final, we verify the correctness of the conclusion by numerical simulation.</p> </abstract>

Weighted pseudo almost periodic solutions of octonion-valued neural networks with mixed time-varying delays and leakage delays

<abstract><p>In this paper, we propose a class of octonion-valued neural networks with leakage delays and mixed delays. Considering that the multiplication of octonion algebras does not satisfy the associativity and commutativity, we can obtain the existence and global exponential stability of weighted pseudo almost periodic solutions for octonion-valued neural networks with leakage delays and mixed delays by using the Banach fixed point theorem, the proof by contradiction and the non-decomposition method. Finally, we will give one example to illustrate the feasibility and effectiveness of the main results.</p></abstract>

The stability of anti-periodic solutions for fractional-order inertial BAM neural networks with time-delays

<abstract><p>The dynamic signal transmission process can be regarded as an anti-periodic process, and fractional-order inertial neural networks are widely used in signal processing and other fields, so anti-periodicity is also regarded as an important dynamic feature of inertial neural networks. This paper mainly studies the existence and Mittag-Leffler stability of anti-periodic solutions for a class of fractional-order inertial BAM neural networks with time-delays. By introducing variable substitution, the model with two different fractional-order derivatives is transformed into a model with only one fractional-order derivative of the same order. Using the properties of fractional-order calculus, the relationship between the fractional-order integral of the state function with and without time-delays is given. Firstly, the sufficient conditions for the boundedness and the Mittag-Leffler stability of the solutions for the system are derived. Secondly, by constructing the sequence solution of the function for the system and applying Ascoli-Arzela theorem, the sufficient conditions for the existence and Mittag-Leffler stability of the anti-periodic solution are given. Finally, the correctness of the conclusion is verified by a numerical example.</p></abstract>

Stability analysis of periodic solutions of the neutral-type neural networks with impulses and time-varying delays

This paper is concerned with a class of neutral-type neural networks with impulses and delays. By using continuation theorem due to Mawhin and constructing the appropriate Lyapunov-Krasovskii functional, several new sufficient conditions ensuring the existence and global exponential stability of the periodic solution are obtained. Moreover, a numerical example is provided to illustrate the main results. Our results can extend and improve some earlier publications.

Global attractive periodic solutions of neutral-type neural networks with delays in the leakage terms

<abstract><p>In this paper, we introduce a class of neutral-type neural networks with delay in the leakage terms. Using coincidence degree theory, Lyapunov functional method and the properties of neutral operator, we establish some new sufficient criteria for the existence and global attractiveness of periodic solutions. Finally, an example demonstrates our findings.</p></abstract>

Besicovitch almost periodic stochastic processes and almost periodic solutions of Clifford-valued stochastic neural networks

<p style='text-indent:20px;'>In this paper, to begin with, we introduce the concept of Besicovitch almost periodic stochastic processes in distribution sense and study the relationship between it and the concept of Besicovitch almost periodic stochastic processes in <inline-formula><tex-math id="M1">\begin{document}$ p $\end{document}</tex-math></inline-formula>-th mean sense. In addition, we take a class of Clifford-valued stochastic neural networks with time-varying delays as an example to investigate the existence and uniqueness of Besicovitch almost periodic solutions in distribution sense of this class of neural networks by using Banach fixed point theorem and a variant of Gronwall lemma. Moreover, we study the global exponential stability of this unique Besicovitch almost periodic solution in distribution sense by using inequality techniques. Finally, we give an example to illustrate our results. The results of this paper are completely new.</p>

Homoclinic solutions and periodic solutions of complex-valued neural networks with time-varying delays

In this paper, a class of complex-valued neural networks with time-varying delays is studied. By employing an extension of Mawhin?s continuation theorem and an approximation technique, several sufficient conditions of the new results on the existence of homoclinic solutions and periodic solutions are established. Moreover, the asymptotic behavior of solutions via the Lyapunov function is also investigated. Finally, for the purpose of validity, an example is given to illustrate the effectiveness of main results.

Finite-time stability of equilibrium solutions for the inertial neural networks via the figure analysis method.

The existence and finite-time stability (FTS) of equilibrium point (EP) for a kind of inertial neural networks (INNS) with varying-time delays is studied. Firstly, by adopting the degree theory and the maximum-valued method, a sufficient condition in the existence of EP is attained. Then by adopting the maximum-valued approach and the figure analysis approach, without adopting the matrix measure theory, linear matrix inequality (LMI), and FTS theorems, a sufficient condition in the FTS of EP for the discussed INNS is proposed.

A neural network for solving the generalized inverse mixed variational inequality problem in Hilbert Spaces

<abstract><p>In this paper, we study and analyze the generalized inverse mixed variational inequality. The existence and uniqueness of the solution of such problem are proposed. The neural network associated with the generalized inverse mixed variational inequality is presented, and moreover, the Wiener-Hopf equation which the solution of the equation is equivalent to the solution of the generalized inverse mixed variational inequality, is considered. The stability and existence of solution of such neural network are proved. Finally, we introduce some algorithms which are constructed by the concept of the neural network and display a numerical example for understanding our results.</p></abstract>

An Interpretable CNN for the Segmentation of the Left Ventricle in Cardiac MRI by Real-Time Visualization

The interpretability of deep learning models has emerged as a compelling area in artificial intelligence research. The safety criteria for medical imaging are highly stringent, and models are required for an explanation. However, existing convolutional neural network solutions for left ventricular segmentation are viewed in terms of inputs and outputs. Thus, the interpretability of CNNs has come into the spotlight. Since medical imaging data are limited, many methods to fine-tune medical imaging models that are popular in transfer models have been built using massive public ImageNet datasets by the transfer learning method. Unfortunately, this generates many unreliable parameters and makes it difficult to generate plausible explanations from these models. In this study, we trained from scratch rather than relying on transfer learning, creating a novel interpretable approach for autonomously segmenting the left ventricle with a cardiac MRI. Our enhanced GPU training system implemented interpretable global average pooling for graphics using deep learning. The deep learning tasks were simplified. Simplification included data management, neural network architecture, and training. Our system monitored and analyzed the gradient changes of different layers with dynamic visualizations in real-time and selected the optimal deployment model. Our results demonstrated that the proposed method was feasible and efficient: the Dice coefficient reached 94.48%, and the accuracy reached 99.7%. It was found that no current transfer learning models could perform comparably to the ImageNet transfer learning architectures. This model is lightweight and more convenient to deploy on mobile devices than transfer learning models.

Specializing Neural Networks for Cryptographic Code Completion Applications

Similarities between natural languages and programming languages have prompted researchers to apply neural network models to software problems, such as code generation and repair. However, program-specific characteristics pose unique prediction challenges that require the design of new and specialized neural network solutions. In this work, we identify new prediction challenges in application programming interface (API) completion tasks and find that existing solutions are unable to capture complex program dependencies in program semantics and structures. We design a new neural network model Multi-HyLSTM to overcome the newly identified challenges and comprehend complex dependencies between API calls. Our neural network is empowered with a specialized dataflow analysis to extract multiple global API dependence paths for neural network predictions. We evaluate Multi-HyLSTM on 64,478 Android Apps and predict 774,460 Java cryptographic API calls that are usually challenging for developers to use correctly. Our Multi-HyLSTM achieves an excellent top-1 API completion accuracy at 98.99%. Moreover, we show the effectiveness of our design choices through an ablation study and have released our dataset.

SLeNN-ELM: A shifted Legendre neural network method for fractional delay differential equations based on extreme learning machine

<abstract><p>In this paper, we introduce a shifted Legendre neural network method based on an extreme learning machine algorithm (SLeNN-ELM) to solve fractional differential equations with constant and proportional delays. Based on the properties of Caputo fractional derivatives and shifted Legendre polynomials, the fractional derivatives of SLeNN can be represented analytically without other numerical techniques. SLeNN, in terms of neural network architecture, uses a function expansion block to replace the hidden layer, and thus improving the computational efficiency by reducing parameters. In terms of solving technology of neural networks, the extreme learning machine algorithm is used to replace the traditional gradient-based training algorithm. It dramatically improves our solution efficiency. In addition, the proposed method does not require parameter initialization randomly, making the neural network solution stable. Finally, three examples with constant delays and three examples with proportional delays are given, and the effectiveness and superiority of the proposed method are verified by comparison with other numerical methods.</p></abstract>

Machine learning enables precise holographic characterization of colloidal materials in real time.

Holographic particle characterization uses in-line holographic video microscopy to track and characterize individual colloidal particles dispersed in their native fluid media. Applications range from fundamental research in statistical physics to product development in biopharmaceuticals and medical diagnostic testing. The information encoded in a hologram can be extracted by fitting to a generative model based on the Lorenz-Mie theory of light scattering. Treating hologram analysis as a high-dimensional inverse problem has been exceptionally successful, with conventional optimization algorithms yielding nanometer precision for a typical particle's position and part-per-thousand precision for its size and index of refraction. Machine learning previously has been used to automate holographic particle characterization by detecting features of interest in multi-particle holograms and estimating the particles' positions and properties for subsequent refinement. This study presents an updated end-to-end neural-network solution called CATCH (Characterizing and Tracking Colloids Holographically) whose predictions are fast, precise, and accurate enough for many real-world high-throughput applications and can reliably bootstrap conventional optimization algorithms for the most demanding applications. The ability of CATCH to learn a representation of Lorenz-Mie theory that fits within a diminutive 200 kB hints at the possibility of developing a greatly simplified formulation of light scattering by small objects.