Neural Network Solution Research Articles

Advancing Neural Networks: Innovations and Impacts on Energy Consumption

AbstractThe energy efficiency of Artificial Intelligence (AI) systems is a crucial and actual issue that may have an important impact on an ecological, economic and technological level. Spiking Neural Networks (SNNs) are strongly suggested as valid candidates able to overcome Artificial Neural Networks (ANNs) in this specific contest. In this study, the proposal involves the review and comparison of energy consumption of the popular Artificial Neural Network architectures implemented on the CPU and GPU hardware compared with Spiking Neural Networks implemented in specialized memristive hardware and biological neural network human brain. As a result, the energy efficiency of Spiking Neural Networks can be indicated from 5 to 8 orders of magnitude. Some Spiking Neural Networks solutions are proposed including continuous feedback‐driven self‐learning approaches inspired by biological Spiking Neural Networks as well as pure memristive solutions for Spiking Neural Networks.

Automatic Detection and Tracking of Objects of Interest in Video Data with Global Motion

Introduction. At present, automatic capture and tracking of moving objects in video data obtained by a video camera mounted on a mobile carrier represents a relevant research task. Its successful solution is challenged by such factors, as a non-uniform background, object overlapping between one another and the background, significant and rapid changes in the size of the object of interest, abrupt changes in the movement trajectory of the mobile carrier.Aim. To develop an automatic method for detecting moving objects followed by their tracking in video data obtained under difficult observation conditions. An additional requirement imposed on the tracking stage consists in the restriction of computing resources.Materials and methods. The method is based on a convolutional neural network with a YOLO architecture. Due to the restriction of computing resources, object tracking is implemented without neural network solutions. In order to ensure stable tracking, several detectors are used simultaneously with the subsequent analysis of the data obtained. The tracking stage involves a detector based on histograms of oriented gradients (HOG), supplemented by a detector based on correlation filtering and motion trajectory prediction based on the Kalman filter.Results. At the automatic detection stage, the TPR, averaged over all video files participating in the experiments, was equal to 0.81, with the FPR corresponding to 0.10. At the tracking stage, the failure rate (tracking failures) was 6·10 –5.Conclusion. The proposed method can be successfully used to detect and track objects at a distance of 1500 m with an object projection size on the frame of 5 × 5 pixels under the conditions of global motion, a non-uniform background, and significant changes in the properties of the object of interest.

Yielding onset in FGM thick-walled spherical shells under thermo-mechanical loading

Based on von Mises yield criterion, a thermoelasto-plastic analysis of a thick-walled spherical shell composed of functionally graded material (FGM) subjected to internal pressure and thermal gradient is conducted in order to accurately predict the onset radius of yielding within the vessel wall. The modulus of elasticity, thermal conductivity and thermal expansion coefficients, and yield stress of FGM are assumed to follow power-law functions in radius according to Erdogan’s model. The equilibrium differential equation of the shell under thermo-mechanical loading in steady-state conduction heat transfer are derived analytically and then solved to determine through-thickness variations of the radial and circumferential stresses and radial displacement in elastic and perfectly plastic zones in the vessel wall. The presented approach leads to the definition of new formulation to predict the onset radius of yielding. Moreover, a neural network (NN) solution to reliable quantify the influence of all uncertain input parameters with respect to uncertain output parameter is utilized. The numerical results showed that for various FGM parameters and specific thermal gradient, the plastic zone can commence from inside radius, outside radius, simultaneously in both radii, or may be launched in the intermediate radius of the spherical shell wall.

WeldNet: An ultra fast measurement algorithm for precision laser stripe extraction in robotic welding

The integration of laser vision sensors in robotic welding improves seam tracking accuracy, but welding noise poses significant challenges. Our research introduces WeldNet, enhances laser stripe extraction, significantly outperforming traditional and deep neural network (DNN) solutions in efficiency and measurement precision. WeldNet comprises lightweight modules for optimal feature extraction, including Multi-Part Channel Convolution (MPC) blocks, Parallel Shift Multilayer Perceptrons (PS-MLP), and Serial Shift MLP (SS-MLP). A specially designed data augmentation strategy is also integrated to address the complex noise encountered in robotic welding. Experimental results demonstrate WeldNet’s effectiveness in reducing welding noise interference, achieving a real-time processing speed of 145 FPS on RTX 2080 Ti GPU, approximately 5x faster than existing state-of-the-art methods. With a Dice coefficient of 87.52% and an IoU value of 77.82%, WeldNet not only enhances operational efficiency but also markedly improves precision in industrial robotic welding.

Physics embedded neural network: Novel data-free approach towards scientific computing and applications in transfer learning

This paper introduces a novel approach for solving partial differential equations using neural networks for scientific computing applications. The proposed approach is referred to as physics embedded neural network (or PENN) due to its unique modified architecture designed to contain the partial differential equation information within the neural network, which is built upon a feed-forward neural network (NN) with fully connected layers. The key aspect of PENN is the numerical embedding of a differential equation associated with physical problem in the penultimate layer of the neural network. Here we propose that through such integration of PDE to the neural network’s penultimate layer, we can produce a fast numerical solution of the PDE, and with time performance comparable with Finite Element Method (FEM). Due to direct solution of the PDE, the NN solution occurs without training and testing phases which are normally applied when using NN in a data-based training. To demonstrate the effectiveness of the proposed architecture, we investigate variety of cases focused around second order in-homogeneous PDE in 2 dimensional (2D) problems. Additionally, we demonstrate the effectiveness of our approach by applying it to solve electromagnetic problem to illustrate its engineering applications. In this study, we also conducted a comparison of solution time and error achieved using our proposed network with a classical finite element method (FEM) solver. The convergence was found exponential with number of epochs proving an error significantly lower error compared to FEM and reported in other NN based algorithms such as Physics Informed Neural Networks. Furthermore, we explored the application of PENN in transfer learning. Our findings showed that by using transfer learning, up to 5 times additional reduction in simulation time is possible as compared to a direct solution through PENN. Overall, the study contributes to the development of efficient and accurate methods for solving PDEs through neural network optimization and provides important insights into the potential benefits of incorporating transfer learning techniques in numerical modeling of physical phenomenology.

A neural network solution of first-passage problems

A neural network solution of first-passage problems

Physics-informed neural network solution for thermo-elastic cavity expansion problem

ABSTRACT This paper proposes a physics-informed neural network (PINN) to solve the thermo-elastic coupled cavity expansion problem under plane strain conditions. The solid is modelled as a linear thermo-elastic material, and cavity expansion is driven by a constant temperature and a radial displacement subjected to the inner cavity wall. Governing partial differential equations (PDEs) for the problem is firstly presented, and an analytical solution is available to help benchmark the PINN approach. Then, the PDEs are normalised into dimensionless forms and neural network details for solving the coupled PDEs are demonstrated. Finally, the performance of the PINN approach is shown by comparison with the analytical benchmark solution and it is found that the PINN can predict thermo-elastic cavity expansion behaviour with high accuracy. The PINN approach and the analytical solution can also serve as a benchmark for solving thermo-elastic coupled problems in geotechnical engineering by more advanced PINN.

A Fast Algorithm for All-Pairs-Shortest-Paths Suitable for Neural Networks.

Given a directed graph of nodes and edges connecting them, a common problem is to find the shortest path between any two nodes. Here we show that the shortest path distances can be found by a simple matrix inversion: if the edges are given by the adjacency matrix Aij, then with a suitably small value of γ, the shortest path distances are Dij=ceil(logγ[(I-γA)-1]ij).We derive several graph-theoretic bounds on the value of γ and explore its useful range with numerics on different graph types. Even when the distance function is not globally accurate across the entire graph, it still works locally to instruct pursuit of the shortest path. In this mode, it also extends to weighted graphs with positive edge weights. For a wide range of dense graphs, this distance function is computationally faster than the best available alternative. Finally, we show that this method leads naturally to a neural network solution of the all-pairs-shortest-path problem.

Learning to construct a solution for UAV path planning problem with positioning error correction

Unmanned aerial vehicles (UAVs) are advanced flight systems. However, their positioning systems cause distance-dependent errors during flight. This study seeks to solve the UAV path planning problem with positioning error correction (UPEC) with an end-to-end method. Traditional methods struggle to balance solution quality and computational overload, and often have limited utilisation of scenario information. To overcome these issues, we propose a path planning model (PPM) based on deep reinforcement learning to solve the UPEC. The model has a complete structure that includes a mathematical model, feature engineering, solution process, neural policy network, scenario generation, training process, and test solution mechanism. Specifically, we first establish a Markov decision process (MDP) for UPEC and apply feature engineering with effective features to support decision-making. We then introduce a path planning neural network (PPNN) to represent the MDP policy. Based on the dataset generated from the multi-rule combination validation, we train the PPNN using the proposed RL algorithm with storage pool. Furthermore, we propose a backtracking mechanism to guarantee solution feasibility during the construction process. Extensive experiments demonstrate that the proposed PPM outperforms existing state-of-the-art algorithms in terms of solution quality and timeliness, and the backtracking mechanism effectively improves the scenario completion rate. The model study indicates the efficacy of our training algorithm and the generalisation of the PPNN. Additionally, our construction process is problem-tailored and more suitable for addressing UPEC than iterative search algorithms, because it effectively mitigates the impact of invalid nodes.

Advanced neural network approaches for coupled equations with fractional derivatives

We investigate numerical solutions and compare them with Fractional Physics-Informed Neural Network (FPINN) solutions for a coupled wave equation involving fractional partial derivatives. The problem explores the evolution of functions u and v over time t and space x. We employ two numerical approximation schemes based on the finite element method to discretize the system of equations. The effectiveness of these schemes is validated by comparing numerical results with exact solutions. Additionally, we introduce the FPINN method to tackle the coupled equation with fractional derivative orders and compare its performance against traditional numerical methods.Key findings reveal that both numerical approaches provide accurate solutions, with the FPINN method demonstrating competitive performance in terms of accuracy and computational efficiency. Our study highlights the significance of employing FPINNs in solving fractional differential equations and underscores their potential as alternatives to conventional numerical methods. The novelty of this work lies in its comparative analysis of traditional numerical techniques and FPINNs for solving coupled wave equations with fractional derivatives, offering insights into advancing computational methods for complex physical systems.

Parameterized physics-informed neural networks (P-PINNs) solution of uniform flow over an arbitrarily spinning spherical particle

Neural network-based approaches have emerged as alternatives to conventional computational fluid dynamics (CFD) in solving multiphase flow problems. However, most of these approaches are based on data-driven machine learning methods that require training datasets prepared beforehand through CFD simulations or experiments and cannot be used independently. This study presents an unsupervised parameterized physics-informed neural networks (P-PINNs) approach for obtaining the full flow field information over a multi-dimensional parametric space. This approach is then applied to solve the three-dimensional flow around an arbitrarily rotating sphere subjected to a cross-flow. This solution is obtained for a continuous range of three parameters: the Reynolds number based on cross-flow (Re∈[1,400]), non-dimensional streamwise and spanwise angular velocity components (Ωx∗,Ωy∗∈[−3,3]). The predictions from the P-PINNs model are compared against particle-resolved direct numerical simulation (PR-DNS) results, demonstrating that the neural network model predicts all flow variables (velocity, pressure, and their spatial derivatives) accurately with R2≳0.9. The force and torque on the particle obtained from the P-PINNs model are also compared well against the corresponding PR-DNS results with R2>0.94. The key observation is that the computational cost of the unsupervised parameterized neural network model training and deployment is substantially lower than performing numerous conventional CFD simulations to systematically vary the parameters. Furthermore, once trained, the resultant neural network model is substantially more compact than the huge database of discretized numerical solutions. The P-PINNs model is then used to reveal several interesting flow features and phenomena about the rotating sphere, including flow symmetry, secondary drag and lift, critical Reynolds number, flow bifurcation, and flow separation. This study presents P-PINNs as an efficient alternative for solving parameterized flow problems, demonstrating its practical usage in flow physics discovery with the example of flow past an arbitrarily rotating sphere.

Generalized-Type Multistability of Almost Periodic Solutions for Memristive Cohen-Grossberg Neural Networks.

This article investigates a generalized type of multistability about almost periodic solutions for memristive Cohen-Grossberg neural networks (MCGNNs). As the inevitable disturbances in biological neurons, almost periodic solutions are more common in nature than equilibrium points (EPs). They are also generalizations of EPs in mathematics. According to the concepts of almost periodic solutions and Ψ -type stability, this article presents a generalized-type multistability definition of almost periodic solutions. The results show that (K+1)n generalized stable almost periodic solutions can coexist in a MCGNN with n neurons, where K is a parameter of the activation functions. The enlarged attraction basins are also estimated based on the original state space partition method. Some comparisons and convincing simulations are given to verify the theoretical results at the end of this article.

AI-Personalized Elegance: Optimizing Your Appearance with Smart Hairstyle Recommendations

Many machine learning algorithms have been introduced to solve different types of problems. Recently, many of these algorithms have been applied to deep architecture models and showed very impressive performances. In general, deep architecture models suffer from the over-fitting problem when there is a small number of training data. In this article the attempt is made to remedy this problem in deep architecture with regularization techniques including overlap pooling and flipped image augmentation and dropout; the authors also compared a deep structure model (convolutional neural network (CNN)) with shallow structure models (support vector machine and artificial neural network with one hidden layer) on a small dataset. It was statistically confirmed that the shallow models achieved better performance than the deep model that did not use a regularization technique. Faces represent complex multidimensional meaningful visual stimuli and developing a computational model for face recognition is difficult. The authors present a hybrid neural-network solution which compares favorably with other methods. Keyword Hair style recommendation; Face recognition; Machine learning; Software Development

НЕЙРОСЕТЕВОЕ ПРОГНОЗИРОВАНИЕ ПАРАМЕТРОВ ТРАНСПОРТНЫХ ПОТОКОВ В ИНТЕЛЛЕКТУАЛЬНЫХ СИСТЕМАХ СВЕТОФОРНОГО РЕГУЛИРОВАНИЯ

A method is proposed for neural network prediction of congestion on road sections outside the control zones of transport detectors in the systems of “flexible” transport flow control at intersections with the traffic light regulation. The paper describes the operating principle of the original control system based on fuzzy logic. To develop a universal neural network solution that can be used to predict traffic on most road sections with common characteristics without the need for training for each case separately, it is proposed to identify 9 main types of sections and, accordingly, 9 neural networks. Each typical neural network differs in the amount of the input data and parameters. All neural networks are trained on samples obtained from analyzing traffic flow on road sections of each type. Typical characteristics of road sections, architecture, and parameters of neural networks are determined. Neural networks are trained and tested on the model data. Using computer simulation methods, the author assesses the control quality of the original intelligent traffic light control system, which has complete information about the transport flow (however, it is difficult to implement in practice), as well as a control system that uses a set of data obtained using transport detectors and supplemented with a neural network forecast. According to the experiment results, a system with a neural network forecast of congestion on road sections is inferior in quality of control by no more than 1.92% compared to an idealized system, which gives grounds to consider this method applicable in intelligent transport flow control systems at intersections with traffic light regulation.

Automated hearing loss type classification based on pure tone audiometry data

Hearing problems are commonly diagnosed with the use of tonal audiometry, which measures a patient’s hearing threshold in both air and bone conduction at various frequencies. Results of audiometry tests, usually represented graphically in the form of an audiogram, need to be interpreted by a professional audiologist in order to determine the exact type of hearing loss and administer proper treatment. However, the small number of professionals in the field can severely delay proper diagnosis. The presented work proposes a neural network solution for classification of tonal audiometry data. The solution, based on the Bidirectional Long Short-Term Memory architecture, has been devised and evaluated for classifying audiometry results into four classes, representing normal hearing, conductive hearing loss, mixed hearing loss, and sensorineural hearing loss. The network was trained using 15,046 test results analysed and categorised by professional audiologists. The proposed model achieves 99.33% classification accuracy on datasets outside of training. In clinical application, the model allows general practitioners to independently classify tonal audiometry results for patient referral. In addition, the proposed solution provides audiologists and otolaryngologists with access to an AI decision support system that has the potential to reduce their burden, improve diagnostic accuracy, and minimise human error.

A brief overview of existing neural network solutions and services for photographers

This article focuses on neural networks and artificial intelligence systems used to enhance images. It attempted to analyse the maximum range of existing systems that can be used for photography. The aim of the article was to find as many systems and services used to enhance photos, restore details, remove noise and improve image quality, in order to understand how neural networks are becoming useful to a wide range of professionals - photographers, photo editors, bloggers, marketers, designers, photo restorers and some other specialists. This finding of 146 neural networks and AI-based systems is summed up in a table compiled by the author. There is also an analysis of a number of neural network-based solutions that allow photographers to improve their images.

Detection and Type Recognition of SAR Artificial Modulation Targets Based on Multi-Scale Amplitude-Phase Features

With respect to the detection and recognition of an Artificial Modulation Target (AMT) with different modulated types, the state-of-the-art methods generally suffer the deficiencies of overfitting and insufficient generalization of existing neural network solutions. To address these problems, this paper proposes a multi-scale amplitude-phase feature discrimination method for AMTs in SAR images. First, a multi-type modulated AMT Dataset is generated (AMT Detection and Modulation Type Recognition Dataset, ADMTR Dataset), wherein the factors of jamming position, jamming-to-signal ratio (JSR), and the modulated parameter are considered to enhance the generalization. Second, a Multi-Input Multi-Output Fusion Wavelet Neural Network (MIMOFWTNN) is established, which not only uses the amplitude information of the scene but also adequately makes use of the phase and high-frequency information. This empowers us to detect the AMT in a higher dimensional feature space such that the type recognition can be implemented with more certainty. Analysis and discussions conducted on comparison experiments and ablation experiments demonstrate that the proposed network can achieve an average accuracy of 96.96% on the cross-validation set and a correct rate of 99.0% on the completely independent test set, which outperforms the compared methods.

Real-time fusion multi-tier DNN-based collaborative IDPS with complementary features for secure UAV-enabled 6G networks

UAV-enabled Integrated Sensing and Communication (ISAC) in sixth-generation (6G) wireless networks has sparked significant research interest. UAVs are positioned as aerial wireless platforms, extending coverage and improving Sensing and Communication (S&C) services. However, integrating UAVs introduces vulnerabilities due to multi-connectivity, necessitating robust security measures. To address this, efforts are focused on developing effective Intrusion Detection Systems (IDS). Here ML-based IDSs depend on the most suited Machine Learning (ML) algorithms for improved detection accuracy. However, inadequate detection features frequently contribute to the limitations of detection accuracy in various emerging cyber-attacks. Our study explores attack traits in UAV-enabled 6G networks, using complementary detection features to enhance ML-based attack detection. Additionally, existing Deep Neural Network (DNN) solutions for UAVs-enabled 6G networks mainly use single models (e.g., CNN, SVM, CGAN) instead of multi-tier models, (e.g., Hybrid, Fusion, Ensemble). Here, although the use of a single model can be faster for computer systems, it is difficult to understand the growing complexity of intrusion patterns in data. Also, these single models might not be good at recognizing the unique patterns of less common issues in datasets. Thus, we propose a fusion multi-tier DNN-based Collaborative Intrusion Detection and Prevention System (CIDPS) for critical UAV-enabled 6G networks. This system boosts accuracy without sacrificing latency reductions. Our approach learns decision boundaries from imbalanced data points using preceding DNNs sequentially. Moreover, most existing research works do not focus on intrusion prevention methods and deployment frameworks for UAV networks. This research proposes effective IPS mechanisms and deployment architecture for CIDPS in UAV-enabled 6G networks. It incorporates a CIDPS with an emergency response protocol, neutralizing attacks upon anomaly detection. We validate our solution through diverse dataset experiments (UAVIDS-2020, NF-UQ-NIDS-v2, 5G-NIDD). Unlike traditional practices where IDS are simulated, we implement CIDPS on actual UAV devices (PX4 Vision Dev Kit V1.5, DJI mini se, DJI mini 3 pro) and real-world UAV networks. Our approach outperforms prior algorithms with a 99.25% attack classification accuracy, higher detection efficiency, and lower resource usage, exceeding 99.05% detection rates.

Hyperbolic Tangent-Type Variant-Parameter and Robust ZNN Solutions for Resolving Time-Variant Sylvester Equation in Preassigned-Time

To solve a general time-variant Sylvester equation, two novel zeroing neural networks (ZNNs) solutions are designed and analyzed. In the foregoing ZNN solutions, the design convergent parameters (CPs) before the nonlinear stimulated functions are very pivotal because CPs basically decide the convergent speeds. Nonetheless, the CPs are generally set to be constants, which is not feasible because CPs are generally time-variant in practical hardware conditions particularly when the external noises invade. So, a lot of variant-parameter ZNNs (VP-ZNNs) with time-variant CPs have been come up with. Comparing with fixed-parameter ZNNs, the foregoing VP-ZNNs have been illustrated to own better convergence, the downside is that the CPs generally increases over time, and will be probably infinite at last. Obviously, infinite large CPs would lead to be non-robustness of the ZNN schemes, which are not permitted in reality when the exterior noises inject. Moreover, even though VP-ZNNs are convergent over time, the growth of CPs will waste tremendous computing resources. Based on these factors, 2 hyperbolic tangent-type variant-parameter robust ZNNs (HTVPR-ZNNs) have been proposed in this paper. Both the convergent preassigned-time of the HTVPR-ZNN and top-time boundary of CPs are theoretically investigated. Many numerical simulations substantiated the admirable validity of the HTVPR-ZNN solutions.

Automatic Particle Recognition Based on Digital lmage Processing

The purpose of the research is to develop and compare various methods and algorithms for effective particle analysis based on their visual characteristics. Тhe purpose of this article is to develop and compare various methods and algorithms for effective particle analysis based on their visual characteristics. Тhe paper considers two fundamentally different approaches: the analysis of grayscale gradients and the machine learning method.Methods.Тhe research methodology includes the analysis of particle images obtained by precipitation from colloidal solutions after laser ablation and images of powder particles for selective laser melting. Тhe materials were obtained using a Quanta 200 3D electron microscope (FЕ/). For the analysis, threshold brightness binarization, contour recognition methods by the Kenny operator and the Hough algorithm are used to combine boundary points into connected contours. For comparison, the U-Net neural network solution was used, and a dataset generator was created to train the neural network. Hand-cut images of aluminum alloy powder particles and micro and nanoparticles of various metals are used as data for generation.Results.Тhe results of the study show that the Hough method provides recognition of the number of particles at the level of 80%, and the machine learning method achieves 95% accuracy in recognizing the shape of particles. Both methods can be used to analyze microand nanoparticles, including irregularly shaped particles.Conclusion.Тhe findings of the work confirm that neural networks are the optimal solution for automatic particle recognition in digital images. However, in order to create a dataset of sufficient volume, it is necessary to develop a generator of labeled images, which requires a detailed study of the subject area.