Neural Network Solution Research Articles

Physics-informed neural network solution for thermo-elastic cavity expansion problem

ABSTRACT This paper proposes a physics-informed neural network (PINN) to solve the thermo-elastic coupled cavity expansion problem under plane strain conditions. The solid is modelled as a linear thermo-elastic material, and cavity expansion is driven by a constant temperature and a radial displacement subjected to the inner cavity wall. Governing partial differential equations (PDEs) for the problem is firstly presented, and an analytical solution is available to help benchmark the PINN approach. Then, the PDEs are normalised into dimensionless forms and neural network details for solving the coupled PDEs are demonstrated. Finally, the performance of the PINN approach is shown by comparison with the analytical benchmark solution and it is found that the PINN can predict thermo-elastic cavity expansion behaviour with high accuracy. The PINN approach and the analytical solution can also serve as a benchmark for solving thermo-elastic coupled problems in geotechnical engineering by more advanced PINN.

A Fast Algorithm for All-Pairs-Shortest-Paths Suitable for Neural Networks.

Given a directed graph of nodes and edges connecting them, a common problem is to find the shortest path between any two nodes. Here we show that the shortest path distances can be found by a simple matrix inversion: if the edges are given by the adjacency matrix Aij, then with a suitably small value of γ, the shortest path distances are Dij=ceil(logγ[(I-γA)-1]ij).We derive several graph-theoretic bounds on the value of γ and explore its useful range with numerics on different graph types. Even when the distance function is not globally accurate across the entire graph, it still works locally to instruct pursuit of the shortest path. In this mode, it also extends to weighted graphs with positive edge weights. For a wide range of dense graphs, this distance function is computationally faster than the best available alternative. Finally, we show that this method leads naturally to a neural network solution of the all-pairs-shortest-path problem.

Learning to construct a solution for UAV path planning problem with positioning error correction

Unmanned aerial vehicles (UAVs) are advanced flight systems. However, their positioning systems cause distance-dependent errors during flight. This study seeks to solve the UAV path planning problem with positioning error correction (UPEC) with an end-to-end method. Traditional methods struggle to balance solution quality and computational overload, and often have limited utilisation of scenario information. To overcome these issues, we propose a path planning model (PPM) based on deep reinforcement learning to solve the UPEC. The model has a complete structure that includes a mathematical model, feature engineering, solution process, neural policy network, scenario generation, training process, and test solution mechanism. Specifically, we first establish a Markov decision process (MDP) for UPEC and apply feature engineering with effective features to support decision-making. We then introduce a path planning neural network (PPNN) to represent the MDP policy. Based on the dataset generated from the multi-rule combination validation, we train the PPNN using the proposed RL algorithm with storage pool. Furthermore, we propose a backtracking mechanism to guarantee solution feasibility during the construction process. Extensive experiments demonstrate that the proposed PPM outperforms existing state-of-the-art algorithms in terms of solution quality and timeliness, and the backtracking mechanism effectively improves the scenario completion rate. The model study indicates the efficacy of our training algorithm and the generalisation of the PPNN. Additionally, our construction process is problem-tailored and more suitable for addressing UPEC than iterative search algorithms, because it effectively mitigates the impact of invalid nodes.

Advanced neural network approaches for coupled equations with fractional derivatives

We investigate numerical solutions and compare them with Fractional Physics-Informed Neural Network (FPINN) solutions for a coupled wave equation involving fractional partial derivatives. The problem explores the evolution of functions u and v over time t and space x. We employ two numerical approximation schemes based on the finite element method to discretize the system of equations. The effectiveness of these schemes is validated by comparing numerical results with exact solutions. Additionally, we introduce the FPINN method to tackle the coupled equation with fractional derivative orders and compare its performance against traditional numerical methods.Key findings reveal that both numerical approaches provide accurate solutions, with the FPINN method demonstrating competitive performance in terms of accuracy and computational efficiency. Our study highlights the significance of employing FPINNs in solving fractional differential equations and underscores their potential as alternatives to conventional numerical methods. The novelty of this work lies in its comparative analysis of traditional numerical techniques and FPINNs for solving coupled wave equations with fractional derivatives, offering insights into advancing computational methods for complex physical systems.

Parameterized physics-informed neural networks (P-PINNs) solution of uniform flow over an arbitrarily spinning spherical particle

Neural network-based approaches have emerged as alternatives to conventional computational fluid dynamics (CFD) in solving multiphase flow problems. However, most of these approaches are based on data-driven machine learning methods that require training datasets prepared beforehand through CFD simulations or experiments and cannot be used independently. This study presents an unsupervised parameterized physics-informed neural networks (P-PINNs) approach for obtaining the full flow field information over a multi-dimensional parametric space. This approach is then applied to solve the three-dimensional flow around an arbitrarily rotating sphere subjected to a cross-flow. This solution is obtained for a continuous range of three parameters: the Reynolds number based on cross-flow (Re∈[1,400]), non-dimensional streamwise and spanwise angular velocity components (Ωx∗,Ωy∗∈[−3,3]). The predictions from the P-PINNs model are compared against particle-resolved direct numerical simulation (PR-DNS) results, demonstrating that the neural network model predicts all flow variables (velocity, pressure, and their spatial derivatives) accurately with R2≳0.9. The force and torque on the particle obtained from the P-PINNs model are also compared well against the corresponding PR-DNS results with R2>0.94. The key observation is that the computational cost of the unsupervised parameterized neural network model training and deployment is substantially lower than performing numerous conventional CFD simulations to systematically vary the parameters. Furthermore, once trained, the resultant neural network model is substantially more compact than the huge database of discretized numerical solutions. The P-PINNs model is then used to reveal several interesting flow features and phenomena about the rotating sphere, including flow symmetry, secondary drag and lift, critical Reynolds number, flow bifurcation, and flow separation. This study presents P-PINNs as an efficient alternative for solving parameterized flow problems, demonstrating its practical usage in flow physics discovery with the example of flow past an arbitrarily rotating sphere.

Generalized-Type Multistability of Almost Periodic Solutions for Memristive Cohen-Grossberg Neural Networks.

This article investigates a generalized type of multistability about almost periodic solutions for memristive Cohen-Grossberg neural networks (MCGNNs). As the inevitable disturbances in biological neurons, almost periodic solutions are more common in nature than equilibrium points (EPs). They are also generalizations of EPs in mathematics. According to the concepts of almost periodic solutions and Ψ -type stability, this article presents a generalized-type multistability definition of almost periodic solutions. The results show that (K+1)n generalized stable almost periodic solutions can coexist in a MCGNN with n neurons, where K is a parameter of the activation functions. The enlarged attraction basins are also estimated based on the original state space partition method. Some comparisons and convincing simulations are given to verify the theoretical results at the end of this article.

AI-Personalized Elegance: Optimizing Your Appearance with Smart Hairstyle Recommendations

Many machine learning algorithms have been introduced to solve different types of problems. Recently, many of these algorithms have been applied to deep architecture models and showed very impressive performances. In general, deep architecture models suffer from the over-fitting problem when there is a small number of training data. In this article the attempt is made to remedy this problem in deep architecture with regularization techniques including overlap pooling and flipped image augmentation and dropout; the authors also compared a deep structure model (convolutional neural network (CNN)) with shallow structure models (support vector machine and artificial neural network with one hidden layer) on a small dataset. It was statistically confirmed that the shallow models achieved better performance than the deep model that did not use a regularization technique. Faces represent complex multidimensional meaningful visual stimuli and developing a computational model for face recognition is difficult. The authors present a hybrid neural-network solution which compares favorably with other methods. Keyword Hair style recommendation; Face recognition; Machine learning; Software Development

НЕЙРОСЕТЕВОЕ ПРОГНОЗИРОВАНИЕ ПАРАМЕТРОВ ТРАНСПОРТНЫХ ПОТОКОВ В ИНТЕЛЛЕКТУАЛЬНЫХ СИСТЕМАХ СВЕТОФОРНОГО РЕГУЛИРОВАНИЯ

A method is proposed for neural network prediction of congestion on road sections outside the control zones of transport detectors in the systems of “flexible” transport flow control at intersections with the traffic light regulation. The paper describes the operating principle of the original control system based on fuzzy logic. To develop a universal neural network solution that can be used to predict traffic on most road sections with common characteristics without the need for training for each case separately, it is proposed to identify 9 main types of sections and, accordingly, 9 neural networks. Each typical neural network differs in the amount of the input data and parameters. All neural networks are trained on samples obtained from analyzing traffic flow on road sections of each type. Typical characteristics of road sections, architecture, and parameters of neural networks are determined. Neural networks are trained and tested on the model data. Using computer simulation methods, the author assesses the control quality of the original intelligent traffic light control system, which has complete information about the transport flow (however, it is difficult to implement in practice), as well as a control system that uses a set of data obtained using transport detectors and supplemented with a neural network forecast. According to the experiment results, a system with a neural network forecast of congestion on road sections is inferior in quality of control by no more than 1.92% compared to an idealized system, which gives grounds to consider this method applicable in intelligent transport flow control systems at intersections with traffic light regulation.

Automated hearing loss type classification based on pure tone audiometry data

Hearing problems are commonly diagnosed with the use of tonal audiometry, which measures a patient’s hearing threshold in both air and bone conduction at various frequencies. Results of audiometry tests, usually represented graphically in the form of an audiogram, need to be interpreted by a professional audiologist in order to determine the exact type of hearing loss and administer proper treatment. However, the small number of professionals in the field can severely delay proper diagnosis. The presented work proposes a neural network solution for classification of tonal audiometry data. The solution, based on the Bidirectional Long Short-Term Memory architecture, has been devised and evaluated for classifying audiometry results into four classes, representing normal hearing, conductive hearing loss, mixed hearing loss, and sensorineural hearing loss. The network was trained using 15,046 test results analysed and categorised by professional audiologists. The proposed model achieves 99.33% classification accuracy on datasets outside of training. In clinical application, the model allows general practitioners to independently classify tonal audiometry results for patient referral. In addition, the proposed solution provides audiologists and otolaryngologists with access to an AI decision support system that has the potential to reduce their burden, improve diagnostic accuracy, and minimise human error.

A brief overview of existing neural network solutions and services for photographers

This article focuses on neural networks and artificial intelligence systems used to enhance images. It attempted to analyse the maximum range of existing systems that can be used for photography. The aim of the article was to find as many systems and services used to enhance photos, restore details, remove noise and improve image quality, in order to understand how neural networks are becoming useful to a wide range of professionals - photographers, photo editors, bloggers, marketers, designers, photo restorers and some other specialists. This finding of 146 neural networks and AI-based systems is summed up in a table compiled by the author. There is also an analysis of a number of neural network-based solutions that allow photographers to improve their images.

Detection and Type Recognition of SAR Artificial Modulation Targets Based on Multi-Scale Amplitude-Phase Features

With respect to the detection and recognition of an Artificial Modulation Target (AMT) with different modulated types, the state-of-the-art methods generally suffer the deficiencies of overfitting and insufficient generalization of existing neural network solutions. To address these problems, this paper proposes a multi-scale amplitude-phase feature discrimination method for AMTs in SAR images. First, a multi-type modulated AMT Dataset is generated (AMT Detection and Modulation Type Recognition Dataset, ADMTR Dataset), wherein the factors of jamming position, jamming-to-signal ratio (JSR), and the modulated parameter are considered to enhance the generalization. Second, a Multi-Input Multi-Output Fusion Wavelet Neural Network (MIMOFWTNN) is established, which not only uses the amplitude information of the scene but also adequately makes use of the phase and high-frequency information. This empowers us to detect the AMT in a higher dimensional feature space such that the type recognition can be implemented with more certainty. Analysis and discussions conducted on comparison experiments and ablation experiments demonstrate that the proposed network can achieve an average accuracy of 96.96% on the cross-validation set and a correct rate of 99.0% on the completely independent test set, which outperforms the compared methods.

Real-time fusion multi-tier DNN-based collaborative IDPS with complementary features for secure UAV-enabled 6G networks

UAV-enabled Integrated Sensing and Communication (ISAC) in sixth-generation (6G) wireless networks has sparked significant research interest. UAVs are positioned as aerial wireless platforms, extending coverage and improving Sensing and Communication (S&C) services. However, integrating UAVs introduces vulnerabilities due to multi-connectivity, necessitating robust security measures. To address this, efforts are focused on developing effective Intrusion Detection Systems (IDS). Here ML-based IDSs depend on the most suited Machine Learning (ML) algorithms for improved detection accuracy. However, inadequate detection features frequently contribute to the limitations of detection accuracy in various emerging cyber-attacks. Our study explores attack traits in UAV-enabled 6G networks, using complementary detection features to enhance ML-based attack detection. Additionally, existing Deep Neural Network (DNN) solutions for UAVs-enabled 6G networks mainly use single models (e.g., CNN, SVM, CGAN) instead of multi-tier models, (e.g., Hybrid, Fusion, Ensemble). Here, although the use of a single model can be faster for computer systems, it is difficult to understand the growing complexity of intrusion patterns in data. Also, these single models might not be good at recognizing the unique patterns of less common issues in datasets. Thus, we propose a fusion multi-tier DNN-based Collaborative Intrusion Detection and Prevention System (CIDPS) for critical UAV-enabled 6G networks. This system boosts accuracy without sacrificing latency reductions. Our approach learns decision boundaries from imbalanced data points using preceding DNNs sequentially. Moreover, most existing research works do not focus on intrusion prevention methods and deployment frameworks for UAV networks. This research proposes effective IPS mechanisms and deployment architecture for CIDPS in UAV-enabled 6G networks. It incorporates a CIDPS with an emergency response protocol, neutralizing attacks upon anomaly detection. We validate our solution through diverse dataset experiments (UAVIDS-2020, NF-UQ-NIDS-v2, 5G-NIDD). Unlike traditional practices where IDS are simulated, we implement CIDPS on actual UAV devices (PX4 Vision Dev Kit V1.5, DJI mini se, DJI mini 3 pro) and real-world UAV networks. Our approach outperforms prior algorithms with a 99.25% attack classification accuracy, higher detection efficiency, and lower resource usage, exceeding 99.05% detection rates.

Hyperbolic Tangent-Type Variant-Parameter and Robust ZNN Solutions for Resolving Time-Variant Sylvester Equation in Preassigned-Time

To solve a general time-variant Sylvester equation, two novel zeroing neural networks (ZNNs) solutions are designed and analyzed. In the foregoing ZNN solutions, the design convergent parameters (CPs) before the nonlinear stimulated functions are very pivotal because CPs basically decide the convergent speeds. Nonetheless, the CPs are generally set to be constants, which is not feasible because CPs are generally time-variant in practical hardware conditions particularly when the external noises invade. So, a lot of variant-parameter ZNNs (VP-ZNNs) with time-variant CPs have been come up with. Comparing with fixed-parameter ZNNs, the foregoing VP-ZNNs have been illustrated to own better convergence, the downside is that the CPs generally increases over time, and will be probably infinite at last. Obviously, infinite large CPs would lead to be non-robustness of the ZNN schemes, which are not permitted in reality when the exterior noises inject. Moreover, even though VP-ZNNs are convergent over time, the growth of CPs will waste tremendous computing resources. Based on these factors, 2 hyperbolic tangent-type variant-parameter robust ZNNs (HTVPR-ZNNs) have been proposed in this paper. Both the convergent preassigned-time of the HTVPR-ZNN and top-time boundary of CPs are theoretically investigated. Many numerical simulations substantiated the admirable validity of the HTVPR-ZNN solutions.

Automatic Particle Recognition Based on Digital lmage Processing

The purpose of the research is to develop and compare various methods and algorithms for effective particle analysis based on their visual characteristics. Тhe purpose of this article is to develop and compare various methods and algorithms for effective particle analysis based on their visual characteristics. Тhe paper considers two fundamentally different approaches: the analysis of grayscale gradients and the machine learning method.Methods.Тhe research methodology includes the analysis of particle images obtained by precipitation from colloidal solutions after laser ablation and images of powder particles for selective laser melting. Тhe materials were obtained using a Quanta 200 3D electron microscope (FЕ/). For the analysis, threshold brightness binarization, contour recognition methods by the Kenny operator and the Hough algorithm are used to combine boundary points into connected contours. For comparison, the U-Net neural network solution was used, and a dataset generator was created to train the neural network. Hand-cut images of aluminum alloy powder particles and micro and nanoparticles of various metals are used as data for generation.Results.Тhe results of the study show that the Hough method provides recognition of the number of particles at the level of 80%, and the machine learning method achieves 95% accuracy in recognizing the shape of particles. Both methods can be used to analyze microand nanoparticles, including irregularly shaped particles.Conclusion.Тhe findings of the work confirm that neural networks are the optimal solution for automatic particle recognition in digital images. However, in order to create a dataset of sufficient volume, it is necessary to develop a generator of labeled images, which requires a detailed study of the subject area.

Leaky-integrate-fire neuron based on vertically extended drain Si[formula omitted]Ge[formula omitted] source TFET: Ultra-energy-efficient and high speed design

This article investigates the performance of a vertically extended drain double gate Si1−x Gex source tunnel field effect transistor (VD-DG-Si1−x Gex S-TFET) as a leaky integrate-and-fire (LIF) neuron. The proposed VD-DG-Si1−x Gex S-TFET structure has been designed and optimized by deploying the commercially available Silvaco TCAD. The LIF neuron using VD-DG-Si1−x Gex S-TFET requires a low spiking threshold voltage (i.e. 0.20 V) which is 5x, 3x, 5x, 1.3x, 1.55x, 7.5x, and 1.44x times less in comparison to MOSFET, Bulk FinFET, phase change device, SOI MOSFET, DG-JLFET, silicon nanowire, and DL-TFET silicon neuron. The proposed device based neuron demonstrates the least spiking energy (177 ZJ) reported to date which is 18192 times less as compared to BTBT based PDSOI MOSFET neuron. The threshold value of current is achieved at 0.20 V drain voltage which is 15x, 14x, 10x, and 1.5x less as compared to FinFET, PD-SOI MOSFET, LBIMOS, and DL-TFET, respectively. Moreover, the effects of gate metal work function, germanium mole fraction, and temperature on the spike current variations have also been investigated. Here, impact ionization is a key method for generating a neuron spike due to holes accumulated in the potential well of VD-DG-Si1−xGex S-TFET. In addition, a LIF neuron based on VD-DG-Si0.5Ge0.5 S-TFET device exhibits a 9 THz spiking frequency which is approximately 12 times higher than real nerves present in the human body. This work shows a remarkable reduction in the power consumption and the proposed device neuron is a potential start-up solution for a large scale spiking neural networks (SNN) implementation because of its better performance and compact circuitry.

Integrated Spatial and Temporal Features Based Network Intrusion Detection System Using SMOTE Sampling

With attackers discovering more inventive ways to take advantage of network weaknesses, the pace of attacks has drastically increased in recent years. As a result, network security has never been more important, and many network intrusion detection systems (NIDS) rely on old, out-of-date attack signatures. This necessitates the deployment of reliable and modern Network Intrusion Detection Systems that are educated on the most recent data and employ deep learning techniques to detect malicious activities. However, it has been found that the most recent datasets readily available contain a large quantity of benign data, enabling conventional deep learning systems to train on the imbalance data. A high false detection rate result from this. To overcome the aforementioned issues, we suggest a Synthetic Minority Over-Sampling Technique (SMOTE) integrated convolution neural network and bi-directional long short-term memory SCNN-BIDLSTM solution for creating intrusion detection systems. By employing the SMOTE, which integrates a convolution neural network to extract spatial features and a bi-directional long short-term memory to extract temporal information; difficulties are reduced by increasing the minority samples in our dataset. In order to train and evaluate our model, we used open benchmark datasets as CIC-IDS2017, NSL-KDD, and UNSW-NB15 and compared the results with other state of the art models.

Periodic solution problems of neutral-type stochastic neural networks with time-varying delays

This paper is devoted to investigating a class of stochastic neutral-type neural networks with delays. By using the fixed point theorem and the properties of neutral-type operator, we obtain the existence conditions for periodic solutions of stochastic neutral-type neural networks. Furthermore, we obtain the conditions for the exponential stability of periodic solutions using Gronwall-Bellman inequality and stochastic analysis technique. Finally, a numerical example is given to show the effectiveness and merits of the present results. Our results can be used to obtain the existence and exponential stability of periodic solution to the corresponding deterministic systems.

Physics-Informed Neural Networks for Solving High-Index Differential-Algebraic Equation Systems Based on Radau Methods

As is well known, differential algebraic equations (DAEs), which are able to describe dynamic changes and underlying constraints, have been widely applied in engineering fields such as fluid dynamics, multi-body dynamics, mechanical systems, and control theory. In practical physical modeling within these domains, the systems often generate high-index DAEs. Classical implicit numerical methods typically result in varying order reduction of numerical accuracy when solving high-index systems. Recently, the physics-informed neural networks (PINNs) have gained attention for solving DAE systems. However, it faces challenges like the inability to directly solve high-index systems, lower predictive accuracy, and weaker generalization capabilities. In this paper, we propose a PINN computational framework, combined Radau IIA numerical method with an improved fully connected neural network structure, to directly solve high-index DAEs. Furthermore, we employ a domain decomposition strategy to enhance solution accuracy. We conduct numerical experiments with two classical high-index systems as illustrative examples, investigating how different orders and time-step sizes of the Radau IIA method affect the accuracy of neural network solutions. For different time-step sizes, the experimental results indicate that utilizing a 5th-order Radau IIA method in the PINN achieves a high level of system accuracy and stability. Specifically, the absolute errors for all differential variables remain as low as 10−6, and the absolute errors for algebraic variables are maintained at 10−5. Therefore, our method exhibits excellent computational accuracy and strong generalization capabilities, providing a feasible approach for the high-precision solution of larger-scale DAEs with higher indices or challenging high-dimensional partial differential algebraic equation systems.

Neural network solutions of bosonic quantum systems in one dimension

Neural network solutions of bosonic quantum systems in one dimension

Can neural networks benefit from objectives that encourage iterative convergent computations? A case study of ResNets and object classification.

Recent work has suggested that feedforward residual neural networks (ResNets) approximate iterative recurrent computations. Iterative computations are useful in many domains, so they might provide good solutions for neural networks to learn. However, principled methods for measuring and manipulating iterative convergence in neural networks remain lacking. Here we address this gap by 1) quantifying the degree to which ResNets learn iterative solutions and 2) introducing a regularization approach that encourages the learning of iterative solutions. Iterative methods are characterized by two properties: iteration and convergence. To quantify these properties, we define three indices of iterative convergence. Consistent with previous work, we show that, even though ResNets can express iterative solutions, they do not learn them when trained conventionally on computer-vision tasks. We then introduce regularizations to encourage iterative convergent computation and test whether this provides a useful inductive bias. To make the networks more iterative, we manipulate the degree of weight sharing across layers using soft gradient coupling. This new method provides a form of recurrence regularization and can interpolate smoothly between an ordinary ResNet and a "recurrent" ResNet (i.e., one that uses identical weights across layers and thus could be physically implemented with a recurrent network computing the successive stages iteratively across time). To make the networks more convergent we impose a Lipschitz constraint on the residual functions using spectral normalization. The three indices of iterative convergence reveal that the gradient coupling and the Lipschitz constraint succeed at making the networks iterative and convergent, respectively. To showcase the practicality of our approach, we study how iterative convergence impacts generalization on standard visual recognition tasks (MNIST, CIFAR-10, CIFAR-100) or challenging recognition tasks with partial occlusions (Digitclutter). We find that iterative convergent computation, in these tasks, does not provide a useful inductive bias for ResNets. Importantly, our approach may be useful for investigating other network architectures and tasks as well and we hope that our study provides a useful starting point for investigating the broader question of whether iterative convergence can help neural networks in their generalization.