Neural Network-based Technique Research Articles

Application of deep learning reduced-order modeling for single-phase flow in faulted porous media

AbstractOur research is positioned within the framework of subsurface resource utilization for sustainable economies. We concentrate on modeling the underground single-phase fluid flow affected by geological faults using numerical simulations. The study of such flows is characterized by strong uncertainites in the data defing the problem due to the difficulty of taking precise measurements in the subsoil. We aim to demonstrate the feasibility of a reduced order model that is both reliable and computationally efficient, thereby facilitating the incorporation of uncertainties. We account for the uncertainities of the properties of the rock and the geometry of the fault. The latter is achieved by using a radial basis function mesh deformation method. This approach benefits from a mixed-dimensional framework to model the rock matrix and faults as n and $${n-1}$$ n - 1 dimensional domains, allowing for non-conforming meshes. Our primary focus is on a reduced-order model capable of reproducing flow variables across the entire domain. We utilize the Deep Learning Reduced Order Model (DL-ROM), a nonintrusive neural network-based technique, and we compare it against the traditional Proper Orthogonal Decomposition (POD) method across various scenarios. The most relevant contributions of this work are: the proof of concept of the use of neural network for reduced order models for subsoil flow, dealing with non-affine problems and mixed dimensional domain. Additionally, we generalize an existing mesh deformation method for discontinuous deformation maps. Our analysis highlights the capability of reduced order model, highlighting DL-ROM’s capacity to expedite complex analyses with promising accuracy and efficiency, making multi-query analyses with various quantities of interest affordable.

Multi-timescale dispatch technology for islanded energy system in the Gobi Desert

The Gobi Desert has vast wilderness to utilize, and its renewable energy capacity experiencing rapid growth. To better allocate regulation resources for maintaining power balance and frequency regulation capacity, an islanded grid optimization model considering multi-timescale dispatch optimization is constructed for integrating chemical parks with thermal power units, energy storage, and green hydrogen production. In the day-ahead optimization stage, to improve the level of renewable energy consumption, the ladder-type regulation performance of thermal power units involved in deep peak load with frequency regulation capacity is derived and established. Moving to the real-time frequency regulation stage, a novel graph neural network-based technique with infeasible region modification is proposed to rapidly acquire the operational power scheme. The input features are the time series of total power command, regulation capacity, past operating power, and past mileage command. And the output features are the mileage commands received by the resources. Training data are generated through offline optimization with the genetic algorithm, which utilizes renewable energy generation and load data with a scale of three months in the Gobi Desert. In the one-month simulation test, the proposed method demonstrated a reduction in power deviation by approximately 32.7 % and an improvement in accuracy by roughly 16 %.

On-Device Neural Network for Object Train and Recognition using Mobile

A neural network is a machine learning (ML) program or model that processes information and recognizes patterns, similar to the human brain. The neural network algorithm operates by training on data to learn and enhance its accuracy. If there is any mistake in the learning process, the machine will react unexpectedly and produce incorrect information. So, whenever a neural network model is developed, it is mandatory to evaluate its performance and ensure its output is accurate. Cameras, touchscreens, internet connectivity, and powerful CPUs have contributed to the popularity of smartphones. Mobile apps are software applications that run on smartphones. ML enhances mobile app functionality by offering features such as voice recognition, image analysis, natural language processing, personalization, and recommendations. Training ML models using mobile apps is challenging due to limited resources, data, and privacy concerns. Object recognition is a neural network-based technique that enables the identification and localization of objects in an image or video. This technique is used in driverless cars, disease identification, industrial inspection, robotics, and more. In this paper, the author introduces a new neural network-based algorithm that utilizes mobile devices for training and recognizing images. Although mobile devices have some technological limitations for training, well-established guidelines for systematically mapping verification and validation techniques have been used in the proposed neural network to ensure performance and correctness in on-device object training and recognition.

Utilizing External Knowledge to Enhance Location Prediction for Twitter/X Users in Low Resource Settings

Accurate estimates of user location are important for many online services, including event detection, disaster management and determining public opinion. Neural network-based techniques have proven to be highly effective in predicting user location. However, these models typically require a large amount of labeled training data, which can be difficult to obtain in real-world scenarios. In this paper, we present two approaches to tackle the issue of limited training data when predicting city level location. First, we consider a self-supervised approach that trains a state level model without labeled data and then integrate this knowledge into the training data set used for city level predictions. Second, we explore the option of increasing the number of training examples by utilizing external resources to generate synthetic users . Finally, we combine these two strategies, exploiting the benefits of both. We empirically evaluate our proposed techniques on multiple Twitter/X data sets and show that our models perform significantly better than the state-of-the-art with improvements of up to 6% for Acc@161 and 8% for F1 score.

Detection and classification of human respiration under building debris model using VHF/UHF waves

Detecting live humans in buildings that have collapsed due to disasters and identifying their condition of health is of great importance for search and rescue operations. Although various methods have been used for this purpose, there are still critical challenges to ensure accurate and rapid life-saving operations. Immediate detection of the presence of living humans under debris combined with the assessment of their vital signs is a crucial factor. This research endeavors to introduce a previously unexplored method: the use of artificial neural network-based techniques to detect human respiration under building debris by generating novel simulation-derived electromagnetic data. To achieve this, a realistic three-dimensional debris model was integrated into an electromagnetic simulation program and complemented by a phantom simulating anterior–posterior body movements indicative of respiration. Measurements of magnitude and phase between 150 and 650 MHz were performed under different conditions. Using surrogate models based on artificial neural networks, noise with different signal-to-noise ratios within the selected frequencies was introduced. These models were used to perform two different steps. Firstly, the presence of respiration of living humans trapped under debris was successfully detected with a success rate of 99.97%. Secondly, the difficult task of classifying the respiration patterns of the human was accomplished with an impressive accuracy of 99.69%, providing a solid basis for proof of concept.

A Survey of Deep Learning for Detecting miRNA- Disease Associations: Databases, Computational Methods, Challenges, and Future Directions.

MicroRNAs (miRNAs) are an important class of non-coding RNAs that play an essential role in the occurrence and development of various diseases. Identifying the potential miRNA-disease associations (MDAs) can be beneficial in understanding disease pathogenesis. Traditional laboratory experiments are expensive and time-consuming. Computational models have enabled systematic large-scale prediction of potential MDAs, greatly improving the research efficiency. With recent advances in deep learning, it has become an attractive and powerful technique for uncovering novel MDAs. Consequently, numerous MDA prediction methods based on deep learning have emerged. In this review, we first summarize publicly available databases related to miRNAs and diseases for MDA prediction. Next, we outline commonly used miRNA and disease similarity calculation and integration methods. Then, we comprehensively review the 48 existing deep learning-based MDA computation methods, categorizing them into classical deep learning and graph neural network-based techniques. Subsequently, we investigate the evaluation methods and metrics that are frequently used to assess MDA prediction performance. Finally, we discuss the performance trends of different computational methods, point out some problems in current research, and propose 9 potential future research directions. Data resources and recent advances in MDA prediction methods are summarized in the GitHub repository https://github.com/sheng-n/DL-miRNA-disease-association-methods.

A Chebyshev neural network-based numerical scheme to solve distributed-order fractional differential equations

This study aims to develop a first-order Chebyshev neural network-based technique for solving ordinary and partial distributed-order fractional differential equations. The neural network is used as a trial solution to construct the loss function. The loss function is utilized to train the neural network via an extreme learning machine and obtain the solution. The novelty of this work is developing and implementing a neural network-based framework for distributed-order fractional differential equations via an extreme learning machine. The proposed method is validated on several test problems. The error metrics utilized in the study include the absolute error and the L2 error. A comparison with other previously available approaches is presented. Also, we provide the computation time of the method.

Improvement of Distributed Denial of Service Attack Detection through Machine Learning and Data Processing

The early and accurate detection of Distributed Denial of Service (DDoS) attacks is a fundamental area of research to safeguard the integrity and functionality of organizations’ digital ecosystems. Despite the growing importance of neural networks in recent years, the use of classical techniques remains relevant due to their interpretability, speed, resource efficiency, and satisfactory performance. This article presents the results of a comparative analysis of six machine learning techniques, namely, Random Forest (RF), Decision Tree (DT), AdaBoost (ADA), Extreme Gradient Boosting (XGB), Multilayer Perceptron (MLP), and Dense Neural Network (DNN), for classifying DDoS attacks. The CICDDoS2019 dataset was used, which underwent data preprocessing to remove outliers, and 22 features were selected using the Pearson correlation coefficient. The RF classifier achieved the best accuracy rate (99.97%), outperforming other classifiers and even previously published neural network-based techniques. These findings underscore the feasibility and effectiveness of machine learning algorithms in the field of DDoS attack detection, reaffirming their relevance as a valuable tool in advanced cyber defense.

A convolution neural network-based technique for health monitoring of connections of a multi-story 3D steel frame structure

A convolution neural network-based technique for health monitoring of connections of a multi-story 3D steel frame structure

Model Predictive Control of Magnetic Helical Swimmers in Two-Dimensional Plane

Magnetic micro/mini-swimmers have a great potential application in biomedical research and have gained broad attention. Recent research studies focus on the automatic control methods of magnetic micro/mini-swimmers, such as artificial intelligence methods. However, their autonomous manipulation remains a challenge since they are subject to various disturbances from the external environment and model uncertainties. The current methods employ a status observer to estimate these disturbances and uncertainties. In this paper, we apply a data-driven technique that utilizes the nonlinear approximation ability of neural networks (NNs). To be specific, a flexible structure of NNs, Broad Learning System (BLS), is employed to model the input-output mapping relationship between the direction of the rotating magnetic field and the swimming direction of the helical miniature swimmer. Then, according to the dual-variable decoupling control, a levitation controller is formulated and a path following controller is proposed based on a planar three-degree-of-freedom model of magnetic helical swimmers, which is inspired by the model of wheeled mobile robots. Simulations and experiments are conducted to quantitatively validate the proposed control method using different planar paths. The experiment results show that the mean absolute error of the path following control is about 3 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\%$</tex-math> </inline-formula> of the body length which is less than 0.5 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$mm$</tex-math> </inline-formula> . Our proposed control method provides a preliminary study to alleviate the impact of disturbances and uncertainties on the control performance of magnetic micro/mini-swimmers. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is mainly motivated by the potential application of artificial intelligence methods in magnetic micro/mini-robot community, especially for the applications that reject disturbances and uncertainties. In practice, a path planner is employed to compute a pre-defined reference path that connects the start and the targeted locations. Then our control method is applied to the magnetic helical swimmer and it is guided to follow the reference path. Tackling external disturbances and model uncertainties is still challenging during the control progress. Neural network-based technique is an intuitive approach since their nonlinear modeling ability and generalization are suitable for estimating these disturbances and uncertainties. As for training the neural-network model, the training data about the angle parameters should be recorded by manual control of the helical swimmer. These angle parameters define the rotating axis of the uniform rotating magnetic field and the self-rotating axis of the magnetic helical swimmer. According to the planar motion model, the optimal controller is formulated using the feedback distance error and angle error, and the sum of the control signal and the prediction of the compensating model is used as the final control input. Our electromagnetic coil system features easy operation and configuration of cameras or other sensors. Simulations and experiments validate the performance of the neural network-based compensating method and the proposed optimal control method using magnetic micro/mini-swimmers.

Convolutional neural network-based techniques and error level analysis for image tamper detection

&lt;span&gt;Photographs are the foremost powerful and trustworthy media of expression. At present, digital pictures not only serve forged information but also disseminate deceptive information. Users and experts with various objectives edit digital photographs. Images are frequently used as proof of reality or fact, therefore fake news or any publication that makes use of photos that have been altered in any way has a larger chance of deceiving readers. There is a need for a high-resolution image analysis model that processes individual pixels in images and a substantial amount of diverse image data, to detect image falsification. Convolutional neural network (CNN) with error level analysis (ELA) adopted in this research is found to be an ideal deep learning concept for detecting image manipulation. The model exhibited a validation accuracy of 99.6%, 99.7%, and 99.4% for CASIA V1.0, CASIA V2.0 and MICC datasets respectively. The accuracy for handmade tampered images was found to be 99.2%.&lt;/span&gt;

Linear Combinations of Patches are Unreasonably Effective for Single-Image Denoising.

In the past decade, deep neural networks have revolutionized image denoising in achieving significant accuracy improvements by learning on datasets composed of noisy/clean image pairs. However, this strategy is extremely dependent on training data quality, which is a well-established weakness. To alleviate the requirement to learn image priors externally, single-image (a.k.a., self-supervised or zero-shot) methods perform denoising solely based on the analysis of the input noisy image without external dictionary or training dataset. This work investigates the effectiveness of linear combinations of patches for denoising under this constraint. Although conceptually very simple, we show that linear combinations of patches are enough to achieve state-of-the-art performance. The proposed parametric approach relies on quadratic risk approximation via multiple pilot images to guide the estimation of the combination weights. Experiments on images corrupted artificially with Gaussian noise as well as on real-world noisy images demonstrate that our method is on par with the very best single-image denoisers, outperforming the recent neural network-based techniques, while being much faster and fully interpretable.

QPDE: Quantum Neural Network Based Stabilization Parameter Prediction for Numerical Solvers for Partial Differential Equations

We propose a Quantum Neural Network (QNN) for predicting stabilization parameter for solving Singularly Perturbed Partial Differential Equations (SPDE) using the Streamline Upwind Petrov Galerkin (SUPG) stabilization technique. SPDE-Q-Net, a QNN, is proposed for approximating an optimal value of the stabilization parameter for SUPG for 2-dimensional convection-diffusion problems. Our motivation for this work stems from the recent progress made in quantum computing and the striking similarities observed between neural networks and quantum circuits. Just like how weight parameters are adjusted in traditional neural networks, the parameters of the quantum circuit, specifically the qubits’ degrees of freedom, can be fine-tuned to learn a nonlinear function. The performance of SPDE-Q-Net is found to be at par with SPDE-Net, a traditional neural network-based technique for stabilization parameter prediction in terms of the numerical error in the solution. Also, SPDE-Q-Net is found to be faster than SPDE-Net, which projects the future benefits which can be earned from the speed-up capabilities of quantum computing.

Neural Network-Based Recent Research Developments in SLAM for Autonomous Ground Vehicles: A Review

The development of autonomous vehicles has prompted an interest in exploring various techniques in navigation. One such technique is simultaneous localization and mapping (SLAM), which enables a vehicle to comprehend its surroundings, build a map of the environment in real-time, and locate itself within that map. Although traditional techniques have been used to perform SLAM for a long time, recent advancements have seen the incorporation of neural network techniques into various stages of the SLAM pipeline. This review paper provides a focused analysis of the recent developments in neural network techniques for SLAM-based localization of autonomous ground vehicles. In contrast to the previous review studies that covered general navigation and SLAM techniques, this work specifically addresses the unique challenges and opportunities presented by the integration of neural networks in this context. Existing review studies have highlighted the limitations of conventional visual SLAM, and this paper aims to explore the potential of deep learning methods. The paper discusses the functions required for localization, as well as several neural network-based techniques proposed by researchers to carry out such functions, are discussed. Firstly, it presents a general background of the issue, the relevant review studies that have already been done, and the adopted methodology in this review. Then, it provides a thorough review of the findings regarding localization and odometry. Finally, it presents our analysis of the findings, open research questions in the field, and a conclusion. A semi-systematic approach is used to carry out the review.

Recurrent neural network-based technique for synchronization of fractional-order systems subject to control input limitations and faults

Despite the existence of promising methods for controlling complex systems, there is still a need for further advancement in controlling and synchronizing fractional systems. This is even more monumental when it comes to practical applications with faults and physical constraints present in their control actuators. To address this issue, the current study proposes a new control technique that utilizes a recurrent neural network-based finite-time super-twisting algorithm for fractional-order systems. The proposed controller is enhanced with an intelligent observer to account for faults and limitations that may be present in the control actuator of the fractional-order systems. The proposed method allows the system to be regulated even in the presence of control input constraints and faults. Moreover, the proposed technique guarantees that the closed-loop system will converge in a finite amount of time. The control design is explained in detail, and its finite-time stability is proven. To evaluate the performance of the controller, we applied it to two different fractional-order systems that were subject to control input limitations and faults. The outcomes of the proposed approach were then compared with those obtained using a state-of-the-art technique for fractional-order systems to further validate the effectiveness of our proposed approach.

Ultra-Attention: Automatic Recognition of Liver Ultrasound Standard Sections Based on Visual Attention Perception Structures.

Acquisition of a standard section is a prerequisite for ultrasound diagnosis. For a long time, there has been a lack of clear definitions of standard liver views because of physician experience. The accurate automated scanning of standard liver sections, however, remains one of ultrasonography medicine's most important issues. In this article, we enrich and expand the classification criteria of liver ultrasound standard sections from clinical practice and propose an Ultra-Attention structured perception strategy to automate the recognition of these sections. Inspired by the attention mechanism in natural language processing, the standard liver ultrasound views will participate in the global attention algorithm as modular local images in computer vision of ultrasound images, which will significantly amplify small features that would otherwise go unnoticed. In addition to using the dropout mechanism, we also use a Part-Transfer Learning training approach to fine-tune the model's rate of convergence to increase its robustness. The proposed Ultra-Attention model outperforms various traditional convolutional neural network-based techniques, achieving the best known performance in the field with a classification accuracy of 93.2%. As part of the feature extraction procedure, we also illustrate and compare the convolutional structure and the Ultra-Attention approach. This analysis provides a reasonable view for future research on local modular feature capture in ultrasound images. By developing a standard scan guideline for liver ultrasound-based illness diagnosis, this work will advance the research on automated disease diagnosis that is directed by standard sections of liver ultrasound.

Fault prediction using fuzzy convolution neural network on IoT environment with heterogeneous sensing data fusion

Using multi-source sensing data based on the Internet of things and fusion in conjunction with fuzzy convolutional neural networks to classify and predict mechanical failures has emerged as a very pertinent area of research. This study presents a fuzzy convolutional neural network-based technique to extract and diagnose the defect indicators utilizing the bearing dataset from Case Western Reserve University (CWRU). In addition, the proposed model's effectiveness is evaluated with the help of a model based on a direct convolution neural network. The strategy based on Fuzzification yielded superior results when applied to a fully linked layer for classification. When applied to the bearing dataset, the proposed approach produces an average classification accuracy of 99.87% using the fuzzy layer. The proposed method is evaluated compared to other methods based on machine learning and deep learning. The accuracy achieved using the proposed method is superior to that achieved using the existing approaches in every imaginable working environment. The proposed approach can be used to diagnose mechanical motor faults and provide earlier recommendations.

An analysis of the omega meson conversion decay using neural network-based technique with the CMD-3 experiment

The study of the conversion decay of the omega meson into π 0 e + e − state was performed with the CMD-3 detector at the VEPP-2000 electron-positron collider in Novosibirsk. The main physical background to the process under study is radiative decay ω → π 0 γ, where monochromatic photon converts to e + e − on the material in front of the detector. The deep neural network was used to suppress these background events. The neural network was trained based on Monte Carlo simulation. To control the systematic uncertainty of this approach the events of the process of inelastic scattering e + e − → e + e − γ (eeγ) and events of annihilation into two photons (γγ) were used since they are similar to signal and background events under study. The cross-sections of these processes are well calculated based on quantum electrodynamics. The difference in the calculated and measured ratio of the number of events Neeγ /Nγγ was used to determine a systematic uncertainty. Using integrated luminosity about 10 pb -1 collected at the c.m. energy range from 775 MeV to 800 MeV the visible cross-section of the process ω → π 0 e + e − was measured. The preliminary result for the branching ratio Br(ω → π 0 e + e −) was obtained.

MetNet: A Novel Low-Complexity Neural Network-Aided Detection for Faster-Than-Nyquist (FTN) Signaling in ISI Channels

This paper studies the application of neural networks to Viterbi detection of FTN signals in an intersymbol interference (ISI) channel. The main contribution of this paper is to propose a receiver structure for detecting FTN signals in unknown static ISI channel. In particular, we propose a novel low-complexity neural network structure for calculating the branch metrics, and we explore its suitability for FTN signalling with channel uncertainty. We compare the proposed network, which we call the Metric Net (MetNet), to a benchmark neural network-based technique for metric calculation, the ViterbiNet, which was originally designed for ISI channels. The simulation results confirm that the MetNet outperforms the ViterbiNet, with two orders of magnitude lower complexity, and is much more resilient to channel uncertainty than the traditional Viterbi detector, which uses Euclidean distance for metric calculations. We further show that the MetNet exhibits robustness to being trained at mismatched SNR values and FTN pulse acceleration factors, meaning that the number of trained models required can be significantly reduced. Additionally, the results show that the proposed MetNet remains a favorable alternative at much higher levels of channel uncertainties. The results also reflect that we can generalize the MetNet to work with different channel models defined by different decaying factors. Finally, we show that we succeed in achieving a bandwidth efficiency gain of 33% due to FTN by using the MetNet in the presence of channel uncertainty.

Accurate diagnosis of liver diseases through the application of deep convolutional neural network on biopsy images

&lt;abstract&gt; &lt;p&gt;Accurate detection of non-alcoholic fatty liver disease (NAFLD) through biopsies is challenging. Manual detection of the disease is not only prone to human error but is also time-consuming. Using artificial intelligence and deep learning, we have successfully demonstrated the issues of the manual detection of liver diseases with a high degree of precision. This article uses various neural network-based techniques to assess non-alcoholic fatty liver disease. In this investigation, more than five thousand biopsy images were employed alongside the latest versions of the algorithms. To detect prominent characteristics in the liver from a collection of Biopsy pictures, we employed the YOLOv3, Faster R-CNN, YOLOv4, YOLOv5, YOLOv6, YOLOv7, YOLOv8, and SSD models. A highlighting point of this paper is comparing the state-of-the-art Instance Segmentation models, including Mask R-CNN, U-Net, YOLOv5 Instance Segmentation, YOLOv7 Instance Segmentation, and YOLOv8 Instance Segmentation. The extent of severity of NAFLD and non-alcoholic steatohepatitis was examined for liver cell ballooning, steatosis, lobular, and periportal inflammation, and fibrosis. Metrics used to evaluate the algorithms' effectiveness include accuracy, precision, specificity, and recall. Improved metrics are achieved by optimizing the hyperparameters of the associated models. Additionally, the liver is scored in order to analyse the information gleaned from biopsy images. Statistical analyses are performed to establish the statistical relevance in evaluating the score for different zones.&lt;/p&gt; &lt;/abstract&gt;