Neural Network Solution Research Articles

Application of physics-informed neural networks (PINNs) solution to coupled thermal and hydraulic processes in silty sands

The accurate modeling of water and heat transport in soils is crucial for both geo-environmental and geothermal engineering. Traditional modeling methods are problematic because they require well-defined boundaries and initial conditions. Recently, physics-informed neural networks (PINNs), which incorporate partial differential equations (PDEs) to solve forward and inverse problems, have attracted increasing attention in machine learning research. In this study, we applied PINNs to tackle hydraulic and thermal transport coupling forward problems in silty sands. A fully connected deep neural network was utilized for training. This neural network model leverages automatic differentiation to apply the governing equations as constraints, based on the mathematical approximations established by the neural network itself. We conducted forward problems and compared the solutions derived from PINNs with those from Finite Element Method (FEM) simulations. The forward problem results demonstrate the PINNs model’s capability in predicting hydraulic transport, heat transport, and thermal–hydraulic coupling in silty sands under various boundary conditions. The PINNs exhibited great performance in simulating the thermal–hydraulic coupling problem. The accuracy of the PINNs solutions shows its potential for simulation in geotechnical engineering.

Novel Framework for Artificial Bubble Image Generation and Boundary Detection Using Superformula Regression and Computer Vision Techniques

Bubble multiphase systems are crucial in industries such as biotechnology, medicine, oil and gas, and water treatment. Optical data analysis provides critical insights into bubble characteristics, such as the shape and size, complementing physical sensor data. Existing detection techniques rely on classical computer vision algorithms and neural network models. While neural networks achieve a higher accuracy, they require extensive annotated datasets, and classical methods often struggle with complex systems due to their lower accuracy. This study proposes a novel framework to address these limitations. Using Superformula parameter regression, we introduce an advanced border detection method for accurately identifying gas inclusions and complex-shaped objects in multiphase environments. The framework also includes a new approach for generating realistic artificial bubble images based on physical flow conditions, leveraging the Superformula to create extensive, labeled datasets without manual annotation. Tested on real bubble flows in mass transfer equipment, the algorithms enable bubble classification by shape and size, enhance detection accuracy, and reduce development time for neural network solutions. This work provides a robust method for object detection and dataset generation in multiphase systems, paving the way for more precise modeling and analysis.

Application of a Multicriteria Genetic Algorithm for Structural Parametric Synthesis of Convolutional Neural Networks

The paper identifies and describes promising architectural solutions for convolutional neural networks, along with key parameters for further structural and parametric synthesis. The advantages and disadvantages of different blocks are outlined, and their relevance in structural synthesis is substantiated. The use of a genetic evolutionary algorithm for structural-parametric synthesis is proposed, along with a review of contemporary approaches. The configuration process of the evolutionary algorithm is shown and described. Based on optimization criteria, the fitness function, selection, mutation, and crossover methods are defined. The results of the experimental evolutionary process are presented and analyzed. An example model created by evolutionary algorithms is considered, which is based on functional blocks aggregated from various architectural approaches in convolutional neural networks. For each model, performance criteria, including average reduction in training time, benefits, and architectural integration details, were calculated during the synthesis process. Experimental results demonstrate that using complex structural blocks instead of traditional layers with a flexible fitness function configuration according to quality and performance criteria yields significant improvements for the final model

Advancing Neural Networks: Innovations and Impacts on Energy Consumption

AbstractThe energy efficiency of Artificial Intelligence (AI) systems is a crucial and actual issue that may have an important impact on an ecological, economic and technological level. Spiking Neural Networks (SNNs) are strongly suggested as valid candidates able to overcome Artificial Neural Networks (ANNs) in this specific contest. In this study, the proposal involves the review and comparison of energy consumption of the popular Artificial Neural Network architectures implemented on the CPU and GPU hardware compared with Spiking Neural Networks implemented in specialized memristive hardware and biological neural network human brain. As a result, the energy efficiency of Spiking Neural Networks can be indicated from 5 to 8 orders of magnitude. Some Spiking Neural Networks solutions are proposed including continuous feedback‐driven self‐learning approaches inspired by biological Spiking Neural Networks as well as pure memristive solutions for Spiking Neural Networks.

Automatic Detection and Tracking of Objects of Interest in Video Data with Global Motion

Introduction. At present, automatic capture and tracking of moving objects in video data obtained by a video camera mounted on a mobile carrier represents a relevant research task. Its successful solution is challenged by such factors, as a non-uniform background, object overlapping between one another and the background, significant and rapid changes in the size of the object of interest, abrupt changes in the movement trajectory of the mobile carrier.Aim. To develop an automatic method for detecting moving objects followed by their tracking in video data obtained under difficult observation conditions. An additional requirement imposed on the tracking stage consists in the restriction of computing resources.Materials and methods. The method is based on a convolutional neural network with a YOLO architecture. Due to the restriction of computing resources, object tracking is implemented without neural network solutions. In order to ensure stable tracking, several detectors are used simultaneously with the subsequent analysis of the data obtained. The tracking stage involves a detector based on histograms of oriented gradients (HOG), supplemented by a detector based on correlation filtering and motion trajectory prediction based on the Kalman filter.Results. At the automatic detection stage, the TPR, averaged over all video files participating in the experiments, was equal to 0.81, with the FPR corresponding to 0.10. At the tracking stage, the failure rate (tracking failures) was 6·10 –5.Conclusion. The proposed method can be successfully used to detect and track objects at a distance of 1500 m with an object projection size on the frame of 5 × 5 pixels under the conditions of global motion, a non-uniform background, and significant changes in the properties of the object of interest.

WeldNet: An ultra fast measurement algorithm for precision laser stripe extraction in robotic welding

The integration of laser vision sensors in robotic welding improves seam tracking accuracy, but welding noise poses significant challenges. Our research introduces WeldNet, enhances laser stripe extraction, significantly outperforming traditional and deep neural network (DNN) solutions in efficiency and measurement precision. WeldNet comprises lightweight modules for optimal feature extraction, including Multi-Part Channel Convolution (MPC) blocks, Parallel Shift Multilayer Perceptrons (PS-MLP), and Serial Shift MLP (SS-MLP). A specially designed data augmentation strategy is also integrated to address the complex noise encountered in robotic welding. Experimental results demonstrate WeldNet’s effectiveness in reducing welding noise interference, achieving a real-time processing speed of 145 FPS on RTX 2080 Ti GPU, approximately 5x faster than existing state-of-the-art methods. With a Dice coefficient of 87.52% and an IoU value of 77.82%, WeldNet not only enhances operational efficiency but also markedly improves precision in industrial robotic welding.

Yielding onset in FGM thick-walled spherical shells under thermo-mechanical loading

Based on von Mises yield criterion, a thermoelasto-plastic analysis of a thick-walled spherical shell composed of functionally graded material (FGM) subjected to internal pressure and thermal gradient is conducted in order to accurately predict the onset radius of yielding within the vessel wall. The modulus of elasticity, thermal conductivity and thermal expansion coefficients, and yield stress of FGM are assumed to follow power-law functions in radius according to Erdogan’s model. The equilibrium differential equation of the shell under thermo-mechanical loading in steady-state conduction heat transfer are derived analytically and then solved to determine through-thickness variations of the radial and circumferential stresses and radial displacement in elastic and perfectly plastic zones in the vessel wall. The presented approach leads to the definition of new formulation to predict the onset radius of yielding. Moreover, a neural network (NN) solution to reliable quantify the influence of all uncertain input parameters with respect to uncertain output parameter is utilized. The numerical results showed that for various FGM parameters and specific thermal gradient, the plastic zone can commence from inside radius, outside radius, simultaneously in both radii, or may be launched in the intermediate radius of the spherical shell wall.

A neural network solution of first-passage problems

A neural network solution of first-passage problems

Physics-informed neural network solution for thermo-elastic cavity expansion problem

ABSTRACT This paper proposes a physics-informed neural network (PINN) to solve the thermo-elastic coupled cavity expansion problem under plane strain conditions. The solid is modelled as a linear thermo-elastic material, and cavity expansion is driven by a constant temperature and a radial displacement subjected to the inner cavity wall. Governing partial differential equations (PDEs) for the problem is firstly presented, and an analytical solution is available to help benchmark the PINN approach. Then, the PDEs are normalised into dimensionless forms and neural network details for solving the coupled PDEs are demonstrated. Finally, the performance of the PINN approach is shown by comparison with the analytical benchmark solution and it is found that the PINN can predict thermo-elastic cavity expansion behaviour with high accuracy. The PINN approach and the analytical solution can also serve as a benchmark for solving thermo-elastic coupled problems in geotechnical engineering by more advanced PINN.

A Fast Algorithm for All-Pairs-Shortest-Paths Suitable for Neural Networks.

Given a directed graph of nodes and edges connecting them, a common problem is to find the shortest path between any two nodes. Here we show that the shortest path distances can be found by a simple matrix inversion: if the edges are given by the adjacency matrix Aij, then with a suitably small value of γ, the shortest path distances are Dij=ceil(logγ[(I-γA)-1]ij).We derive several graph-theoretic bounds on the value of γ and explore its useful range with numerics on different graph types. Even when the distance function is not globally accurate across the entire graph, it still works locally to instruct pursuit of the shortest path. In this mode, it also extends to weighted graphs with positive edge weights. For a wide range of dense graphs, this distance function is computationally faster than the best available alternative. Finally, we show that this method leads naturally to a neural network solution of the all-pairs-shortest-path problem.

Learning to construct a solution for UAV path planning problem with positioning error correction

Unmanned aerial vehicles (UAVs) are advanced flight systems. However, their positioning systems cause distance-dependent errors during flight. This study seeks to solve the UAV path planning problem with positioning error correction (UPEC) with an end-to-end method. Traditional methods struggle to balance solution quality and computational overload, and often have limited utilisation of scenario information. To overcome these issues, we propose a path planning model (PPM) based on deep reinforcement learning to solve the UPEC. The model has a complete structure that includes a mathematical model, feature engineering, solution process, neural policy network, scenario generation, training process, and test solution mechanism. Specifically, we first establish a Markov decision process (MDP) for UPEC and apply feature engineering with effective features to support decision-making. We then introduce a path planning neural network (PPNN) to represent the MDP policy. Based on the dataset generated from the multi-rule combination validation, we train the PPNN using the proposed RL algorithm with storage pool. Furthermore, we propose a backtracking mechanism to guarantee solution feasibility during the construction process. Extensive experiments demonstrate that the proposed PPM outperforms existing state-of-the-art algorithms in terms of solution quality and timeliness, and the backtracking mechanism effectively improves the scenario completion rate. The model study indicates the efficacy of our training algorithm and the generalisation of the PPNN. Additionally, our construction process is problem-tailored and more suitable for addressing UPEC than iterative search algorithms, because it effectively mitigates the impact of invalid nodes.

Advanced neural network approaches for coupled equations with fractional derivatives

We investigate numerical solutions and compare them with Fractional Physics-Informed Neural Network (FPINN) solutions for a coupled wave equation involving fractional partial derivatives. The problem explores the evolution of functions u and v over time t and space x. We employ two numerical approximation schemes based on the finite element method to discretize the system of equations. The effectiveness of these schemes is validated by comparing numerical results with exact solutions. Additionally, we introduce the FPINN method to tackle the coupled equation with fractional derivative orders and compare its performance against traditional numerical methods.Key findings reveal that both numerical approaches provide accurate solutions, with the FPINN method demonstrating competitive performance in terms of accuracy and computational efficiency. Our study highlights the significance of employing FPINNs in solving fractional differential equations and underscores their potential as alternatives to conventional numerical methods. The novelty of this work lies in its comparative analysis of traditional numerical techniques and FPINNs for solving coupled wave equations with fractional derivatives, offering insights into advancing computational methods for complex physical systems.

Parameterized physics-informed neural networks (P-PINNs) solution of uniform flow over an arbitrarily spinning spherical particle

Neural network-based approaches have emerged as alternatives to conventional computational fluid dynamics (CFD) in solving multiphase flow problems. However, most of these approaches are based on data-driven machine learning methods that require training datasets prepared beforehand through CFD simulations or experiments and cannot be used independently. This study presents an unsupervised parameterized physics-informed neural networks (P-PINNs) approach for obtaining the full flow field information over a multi-dimensional parametric space. This approach is then applied to solve the three-dimensional flow around an arbitrarily rotating sphere subjected to a cross-flow. This solution is obtained for a continuous range of three parameters: the Reynolds number based on cross-flow (Re∈[1,400]), non-dimensional streamwise and spanwise angular velocity components (Ωx∗,Ωy∗∈[−3,3]). The predictions from the P-PINNs model are compared against particle-resolved direct numerical simulation (PR-DNS) results, demonstrating that the neural network model predicts all flow variables (velocity, pressure, and their spatial derivatives) accurately with R2≳0.9. The force and torque on the particle obtained from the P-PINNs model are also compared well against the corresponding PR-DNS results with R2>0.94. The key observation is that the computational cost of the unsupervised parameterized neural network model training and deployment is substantially lower than performing numerous conventional CFD simulations to systematically vary the parameters. Furthermore, once trained, the resultant neural network model is substantially more compact than the huge database of discretized numerical solutions. The P-PINNs model is then used to reveal several interesting flow features and phenomena about the rotating sphere, including flow symmetry, secondary drag and lift, critical Reynolds number, flow bifurcation, and flow separation. This study presents P-PINNs as an efficient alternative for solving parameterized flow problems, demonstrating its practical usage in flow physics discovery with the example of flow past an arbitrarily rotating sphere.

Generalized-Type Multistability of Almost Periodic Solutions for Memristive Cohen-Grossberg Neural Networks.

This article investigates a generalized type of multistability about almost periodic solutions for memristive Cohen-Grossberg neural networks (MCGNNs). As the inevitable disturbances in biological neurons, almost periodic solutions are more common in nature than equilibrium points (EPs). They are also generalizations of EPs in mathematics. According to the concepts of almost periodic solutions and Ψ -type stability, this article presents a generalized-type multistability definition of almost periodic solutions. The results show that (K+1)n generalized stable almost periodic solutions can coexist in a MCGNN with n neurons, where K is a parameter of the activation functions. The enlarged attraction basins are also estimated based on the original state space partition method. Some comparisons and convincing simulations are given to verify the theoretical results at the end of this article.

AI-Personalized Elegance: Optimizing Your Appearance with Smart Hairstyle Recommendations

Many machine learning algorithms have been introduced to solve different types of problems. Recently, many of these algorithms have been applied to deep architecture models and showed very impressive performances. In general, deep architecture models suffer from the over-fitting problem when there is a small number of training data. In this article the attempt is made to remedy this problem in deep architecture with regularization techniques including overlap pooling and flipped image augmentation and dropout; the authors also compared a deep structure model (convolutional neural network (CNN)) with shallow structure models (support vector machine and artificial neural network with one hidden layer) on a small dataset. It was statistically confirmed that the shallow models achieved better performance than the deep model that did not use a regularization technique. Faces represent complex multidimensional meaningful visual stimuli and developing a computational model for face recognition is difficult. The authors present a hybrid neural-network solution which compares favorably with other methods. Keyword Hair style recommendation; Face recognition; Machine learning; Software Development

НЕЙРОСЕТЕВОЕ ПРОГНОЗИРОВАНИЕ ПАРАМЕТРОВ ТРАНСПОРТНЫХ ПОТОКОВ В ИНТЕЛЛЕКТУАЛЬНЫХ СИСТЕМАХ СВЕТОФОРНОГО РЕГУЛИРОВАНИЯ

A method is proposed for neural network prediction of congestion on road sections outside the control zones of transport detectors in the systems of “flexible” transport flow control at intersections with the traffic light regulation. The paper describes the operating principle of the original control system based on fuzzy logic. To develop a universal neural network solution that can be used to predict traffic on most road sections with common characteristics without the need for training for each case separately, it is proposed to identify 9 main types of sections and, accordingly, 9 neural networks. Each typical neural network differs in the amount of the input data and parameters. All neural networks are trained on samples obtained from analyzing traffic flow on road sections of each type. Typical characteristics of road sections, architecture, and parameters of neural networks are determined. Neural networks are trained and tested on the model data. Using computer simulation methods, the author assesses the control quality of the original intelligent traffic light control system, which has complete information about the transport flow (however, it is difficult to implement in practice), as well as a control system that uses a set of data obtained using transport detectors and supplemented with a neural network forecast. According to the experiment results, a system with a neural network forecast of congestion on road sections is inferior in quality of control by no more than 1.92% compared to an idealized system, which gives grounds to consider this method applicable in intelligent transport flow control systems at intersections with traffic light regulation.

Automated hearing loss type classification based on pure tone audiometry data

Hearing problems are commonly diagnosed with the use of tonal audiometry, which measures a patient’s hearing threshold in both air and bone conduction at various frequencies. Results of audiometry tests, usually represented graphically in the form of an audiogram, need to be interpreted by a professional audiologist in order to determine the exact type of hearing loss and administer proper treatment. However, the small number of professionals in the field can severely delay proper diagnosis. The presented work proposes a neural network solution for classification of tonal audiometry data. The solution, based on the Bidirectional Long Short-Term Memory architecture, has been devised and evaluated for classifying audiometry results into four classes, representing normal hearing, conductive hearing loss, mixed hearing loss, and sensorineural hearing loss. The network was trained using 15,046 test results analysed and categorised by professional audiologists. The proposed model achieves 99.33% classification accuracy on datasets outside of training. In clinical application, the model allows general practitioners to independently classify tonal audiometry results for patient referral. In addition, the proposed solution provides audiologists and otolaryngologists with access to an AI decision support system that has the potential to reduce their burden, improve diagnostic accuracy, and minimise human error.

A brief overview of existing neural network solutions and services for photographers

This article focuses on neural networks and artificial intelligence systems used to enhance images. It attempted to analyse the maximum range of existing systems that can be used for photography. The aim of the article was to find as many systems and services used to enhance photos, restore details, remove noise and improve image quality, in order to understand how neural networks are becoming useful to a wide range of professionals - photographers, photo editors, bloggers, marketers, designers, photo restorers and some other specialists. This finding of 146 neural networks and AI-based systems is summed up in a table compiled by the author. There is also an analysis of a number of neural network-based solutions that allow photographers to improve their images.

Detection and Type Recognition of SAR Artificial Modulation Targets Based on Multi-Scale Amplitude-Phase Features

With respect to the detection and recognition of an Artificial Modulation Target (AMT) with different modulated types, the state-of-the-art methods generally suffer the deficiencies of overfitting and insufficient generalization of existing neural network solutions. To address these problems, this paper proposes a multi-scale amplitude-phase feature discrimination method for AMTs in SAR images. First, a multi-type modulated AMT Dataset is generated (AMT Detection and Modulation Type Recognition Dataset, ADMTR Dataset), wherein the factors of jamming position, jamming-to-signal ratio (JSR), and the modulated parameter are considered to enhance the generalization. Second, a Multi-Input Multi-Output Fusion Wavelet Neural Network (MIMOFWTNN) is established, which not only uses the amplitude information of the scene but also adequately makes use of the phase and high-frequency information. This empowers us to detect the AMT in a higher dimensional feature space such that the type recognition can be implemented with more certainty. Analysis and discussions conducted on comparison experiments and ablation experiments demonstrate that the proposed network can achieve an average accuracy of 96.96% on the cross-validation set and a correct rate of 99.0% on the completely independent test set, which outperforms the compared methods.

Real-time fusion multi-tier DNN-based collaborative IDPS with complementary features for secure UAV-enabled 6G networks

UAV-enabled Integrated Sensing and Communication (ISAC) in sixth-generation (6G) wireless networks has sparked significant research interest. UAVs are positioned as aerial wireless platforms, extending coverage and improving Sensing and Communication (S&C) services. However, integrating UAVs introduces vulnerabilities due to multi-connectivity, necessitating robust security measures. To address this, efforts are focused on developing effective Intrusion Detection Systems (IDS). Here ML-based IDSs depend on the most suited Machine Learning (ML) algorithms for improved detection accuracy. However, inadequate detection features frequently contribute to the limitations of detection accuracy in various emerging cyber-attacks. Our study explores attack traits in UAV-enabled 6G networks, using complementary detection features to enhance ML-based attack detection. Additionally, existing Deep Neural Network (DNN) solutions for UAVs-enabled 6G networks mainly use single models (e.g., CNN, SVM, CGAN) instead of multi-tier models, (e.g., Hybrid, Fusion, Ensemble). Here, although the use of a single model can be faster for computer systems, it is difficult to understand the growing complexity of intrusion patterns in data. Also, these single models might not be good at recognizing the unique patterns of less common issues in datasets. Thus, we propose a fusion multi-tier DNN-based Collaborative Intrusion Detection and Prevention System (CIDPS) for critical UAV-enabled 6G networks. This system boosts accuracy without sacrificing latency reductions. Our approach learns decision boundaries from imbalanced data points using preceding DNNs sequentially. Moreover, most existing research works do not focus on intrusion prevention methods and deployment frameworks for UAV networks. This research proposes effective IPS mechanisms and deployment architecture for CIDPS in UAV-enabled 6G networks. It incorporates a CIDPS with an emergency response protocol, neutralizing attacks upon anomaly detection. We validate our solution through diverse dataset experiments (UAVIDS-2020, NF-UQ-NIDS-v2, 5G-NIDD). Unlike traditional practices where IDS are simulated, we implement CIDPS on actual UAV devices (PX4 Vision Dev Kit V1.5, DJI mini se, DJI mini 3 pro) and real-world UAV networks. Our approach outperforms prior algorithms with a 99.25% attack classification accuracy, higher detection efficiency, and lower resource usage, exceeding 99.05% detection rates.