Neural Network Research Articles

Sensitivity and feature importance of climate factors and evaluation of different machine learning models for predicting fire hotspots in Kalimantan, Indonesia

<p>Hotspots as indicators of forest fires capable of quickly monitoring large areas are often predicted using various machine learning methods. However, there is still little research that analyzes the sensitivity and feature importance of each predictor that forms a machine learning prediction model. This study evaluates and compares several machine learning methods to predict hotspots in Kalimantan. Using the most accurate machine learning model, each climate factor used as a predictor is analyzed for its sensitivity and feature importance. Some of the machine learning methods used include random forest, gradient boosting, Bayesian regression, and artificial neural networks. Meanwhile, several measures of sensitivity and feature importance used are variance-based, density-based, and distribution-based sensitivity indices, as well as permutation and Shapley feature importance. Evaluation of the ML model concluded that the Bayesian linear regression model outperformed other ML models, based on RMSE and explained variance score. Meanwhile, tree-based models, such as random forest and gradient boosting, are indicative of overfit. Based on the results of sensitivity analysis and feature importance, the number of dry days is the most important feature for the Bayesian linear regression model in predicting the number of hotspots in Kalimantan.</p>

B-Cos Alignment for Inherently Interpretable CNNs and Vision Transformers.

We present a new direction for increasing the interpretability of deep neural networks (DNNs) by promoting weight-input alignment during training. For this, we propose to replace the linear transformations in DNNs by our novel B-cos transformation. As we show, a sequence (network) of such transformations induces a single linear transformation that faithfully summarises the full model computations. Moreover, the B-cos transformation is designed such that the weights align with relevant signals during optimisation. As a result, those induced linear transformations become highly interpretable and highlight task-relevant features. Importantly, the B-cos transformation is designed to be compatible with existing architectures and we show that it can easily be integrated into virtually all of the latest state of the art models for computer vision-e.g. ResNets, DenseNets, ConvNext models, as well as Vision Transformers-by combining the B-cos-based explanations with normalisation and attention layers, all whilst maintaining similar accuracy on ImageNet. Finally, we show that the resulting explanations are of high visual quality and perform well under quantitative interpretability metrics.

Application of improved glomerular filtration rate estimation by a neural network model in patients with neurogenic lower urinary tract dysfunction.

Previous studies have indicated that creatinine (Cr)-based glomerular filtration rate (GFR) estimating equations - including the new Chronic Kidney Disease Epidemiology creatinine (CKD-EPIcr) equation without race and the estimated glomerular filtration rate (eGFR) equation developed for the Chinese population - displayed suboptimal performance in patients with neurogenic lower urinary tract dysfunction (NLUTD), which limited their clinical application for detecting changes in GFR levels in all cohorts. To develop a neural network model based on multilayer perceptron (MLP) for evaluating GFR in Chinese NLUTD patients, and compare the diagnostic performance with Cr-based multiple linear regression equations for Chinese and the CKD-EPIcr equation without race. Single-center, cross-sectional study of GFR estimation from serum Cr, demographic data, and clinical characteristics in Chinese patients with NLUTD. A total of 204 NLUTD patients, from 27 different geographic regions of China, were selected. A random sample of 141 of these subjects was included in the training sample set, and the remaining 63 patients were included in the testing sample set. The reference GFR (rGFR) was assessed by the technetium-99m-labeled diethylenetriaminepentaacetic acid (99mTc-DTPA) double plasma sample method. A neural network model based on MLP was developed to evaluate GFR in the training sample set, which was then validated in the testing sample set and compared with Cr-based GFR equations. The MLP-based model showed significant performance improvement in evaluating the difference, absolute difference, precision, and accuracy of GFR estimation compared with the Cr-based GFR equations. Additionally, compared with the rGFR, we found that the MLP-based model provided an acceptable level of accuracy (greater than 85%, which was within a 30% deviation from the rGFR). The MLP-based model offered significant advantages in estimating GFR in Chinese NLUTD patients, and its application could be suggested in clinical practice.

Inverse Reinforcement Learning With Graph Neural Networks for Full-Dimensional Task Offloading in Edge Computing

Inverse Reinforcement Learning With Graph Neural Networks for Full-Dimensional Task Offloading in Edge Computing

Rapid reactivation of long-term frozen preservation anammox granular sludge by biochar: Performance, mechanism and artificial neural network optimization

Anaerobic ammonia oxidation (Anammox) is an efficient nitrogen removal technology for wastewater, and has been successfully applied in practical wastewater treatment. The preservation and rapid reactivation of anammox bacteria (AnAOB) play an important role in the anammox application. Biochar has been widely used in the water treatment industry due to its stable structure, economic and environmental protection characteristics. Hence, this study used biochar to accelerate anammox granular sludge (AnGS) performance recovery after 60 d preservation under low-temperature frozen (−20 °C). The time of AnGS recovery in the group with biochar addition was shortened by 15 d than the natural recovery group. Meanwhile, biochar addition improved the shock loading resistance of the anammox system. EPS secretion, and the types and quantities of functional groups in EPS increased under biochar stimulation, especially tighter PN. The functional groups in biochar can act as electron shuttles during anammox process, accelerating the electron transfer rate. Biochar addition enhanced the relative abundance of AnAOB, especially Candidatus Brocadia. PICRUSt analysis indicated that biochar promoted key gene related to anammox process and accelerated the recovery of frozen sludge. In addition, the changes in effluent nitrogen removal efficiency were successfully predicted by artificial neural network modeling with R = 0.9895 and MSE = 0.0048. This result may provide a new technological idea for the rapid recovery of AnGS under long time frozen conditions.

TQU-HG dataset and comparative study for hand gesture recognition of RGB-based images using deep learning

Hand gesture recognition has great applications in human-computer interaction (HCI), human-robot interaction (HRI), and supporting the deaf and mute. To build a hand gesture recognition model using deep learning (DL) with high results then needs to be trained on many data and in many different conditions and contexts. In this paper, we publish the TQU-HG dataset of large RGB images with low resolution (640×480) pixels, low light conditions, and fast speed (16 fps). TQU-HG dataset includes 60,000 images collected from 20 people (10 male, 10 female) with 15 gestures of both left and right hands. A comparative study with two branches: i) based on Mediapipe TML and ii) Based on convolutional neural networks (CNNs) (you only look once (YOLO); YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLO-Nas, single shot multiBox detector (SSD) VGG16, residual network (ResNet)18, ResNext50, ResNet152, ResNext50, MobileNet V3 small, and MobileNet V3 large), the architecture and operation of CNNs models are also introduced in detail. We especially fine-tune the model and evaluate it on TQU-HG and HaGRID datasets. The quantitative results of the training and testing are presented (F1-score of YOLOv8, YOLO-Nas, MobileNet V3 small, ResNet50 is 98.99%, 98.98%, 99.27%, 99.36%, respectively on the TQU-HG dataset and is 99.21%, 99.37%, 99.36%, 86.4%, 98.3%, respectively on the HaGRID dataset). The computation time of YOLOv8 is 6.19 fps on the CPU and 18.28 fps on the GPU.

Anti-derivatives approximator for enhancing physics-informed neural networks

This study presents a novel strategy for constructing an approximator for arbitrary univariate functions. The proposed approximation utilizes the anti-derivatives of a Fourier series expansion for the presumed piecewise function, resulting in a remarkable feature that enables the simultaneous approximation of an arbitrary function and its anti-derivatives. These anti-derivatives can be employed to discover solution curves for systems of ordinary differential equations based on an optimization scheme, even in the presence of chaotic dynamics. Additionally, the anti-derivatives approximator is extended as an adaptive activation function for physics-informed neural networks, leveraging the high-order differentiability of the anti-derivatives. Systematic experiments have demonstrated the outstanding merits of the proposed anti-derivatives-based approximator, including its ability to construct regression models for volatile data and their anti-derivatives, solve differential equations, and enhance the capabilities of physics-informed neural networks.

Improved Quantile Convolutional and Recurrent Neural Networks for Electric Vehicle Battery Temperature Prediction

Improved Quantile Convolutional and Recurrent Neural Networks for Electric Vehicle Battery Temperature Prediction

Sentiment analysis of student feedback using attention-based RNN and transformer embedding

Sentiment analysis systems aim to assess people’s opinions across various domains by collecting and categorizing feedback and reviews. In our study, researchers put forward a sentiment analysis system that leverages three distinct embedding techniques: automatic, global vectors (GloVe) for word representation, and bidirectional encoder representations from transformers (BERT). This system features an attention layer, with the best model chosen through rigorous comparisons. In developing the sentiment analysis model, we employed a hybrid dataset comprising students’ feedback and comments. This dataset comprises 3,820 comments, including 2,773 from formal evaluations and 1,047 generated by ChatGPT and prompting engineering. Our main motivation for integrating generative AI was to balance both positive and negative comments. We also explored recurrent neural network (RNN), gated recurrent unit (GRU), long short-term memory (LSTM), and bidirectional long short-term memory (Bi-LSTM), with and without pre-trained GloVe embedding. These techniques produced F-scores ranging from 67% to 69%. On the other hand, the sentiment model based on BERT, particularly its KERAS implementation, achieved higher F-scores ranging from 83% to 87%. The Bi-LSTM architecture outperformed other models and the inclusion of an attention layer further enhanced the performance, resulting in F-scores of 89% and 88% from the Bi-LSTM-BERT sentiment models, respectively.

Research on Establishment of Digital-Twin System for Intelligent Control of Cutting Tools Sintering Process Driven by Data-Model Combination

In the manufacturing process of tungsten cemented carbide cutting tools, the changes in temperature, pressure and vacuum during the sintering process of the cutting tools can have a significant impact on the quality of the cutting tools. This effect is particularly significant in the vacuum sintering stage, so this paper focuses on cutting tools sintering in the vacuum stage. The cutting tools sintering process is carried out in a closed space in a sintering furnace with a complex internal structure. Therefore, it is extremely difficult to accurately predict the heating condition of the cutting tools in the furnace and monitor the sintering quality of the cutting tools. In this study, a digital twin system for the sintering process of cemented carbide cutting tools is proposed, which combines monitoring rolling data analysis and sintering quality prediction model. Firstly, the radiative thermodynamic model of cutting tools sintering temperature was established in this system. The model predicts the time-varying state of the heat distribution at any location in the furnace during the vacuum phase. Subsequently, based on the convolutional neural network, the mapping relationship between the sintering parameters and the grain size in the vacuum stage is constructed, and a dynamic evaluation model of cutting tools sintering quality is proposed. The model enables real-time prediction of sintering quality in the vacuum stage. The system realizes the prediction of the time-varying state of the sintering quality of the cutting tools. The digital twin system realizes the adaptive temperature control of the vacuum stage of cutting tools sintering based on the fuzzy control algorithm. The application of this system has increased the yield rate of cemented carbide cutting tools manufacturing to 98.9%. This breaks through the problem of timing modulation of cutting tools sintering quality in the vacuum stage of cutting tools sintering.

A benchmark of health insurance fraud detection using machine learning techniques

Health insurance fraud is a complex problem that also has a significant financial impact. Recently, with the availability of large volumes of data and the evolution of computing power, machine learning techniques have become the preferred method for fraud detection. However, the main difficulty facing researchers in this field is the lack of real data sets and the absence of reliable fraud labels. Most published studies use aggregated provider-level or simulated data to test fraud detection algorithms, which may not deliver accurate results. The present study aims to provide a more accurate assessment of fraud detection methods by using real detailed health insurance claims data to compare six of the most common supervised classification algorithms including neural networks and the use of two categorical feature preparation methods. The study was conducted under the guidance of insurance experts, who provided the fraud label inference rules and reviewed the results. A comprehensive description of the benchmarking process and an interpretation of the results are provided in this paper. The results show that supervised classification can be used effectively to detect health insurance fraud, improving detection accuracy by a factor of 4.2 (84% recall for a positive rate of 20%).<p> </p>

Face recognition for smart door security access with convolutional neural network method

Face recognition for smart door security access with convolutional neural network method

Nanobots: Revolutionising the Next Generation of Biomedical Technology and Drug Therapy

Abstract: Nanoscale machines called “nanorobots” that were hitherto only futuristic ideas are set to storm healthcare and pharmaceuticals with newer technologies for minimally invasive diagnosis, quick and precise surgeries, and targeted drug delivery, which is challenging to achieve by conventional drug delivery systems. Nanobots can be defined as controllable nano-sized mechanical or electromechanical devices which are easily incorporated into cells and used for a variety of cellular functions like combating bacteria and viruses, disposing away dead cells/tissue at the place of a wound, cell/tissue repair and destruction of cancer cells, and also for transporting drugs to cells. Nanorobots can help avoid the untoward effects of traditional drug delivery systems and ameliorate the efficiency of drug delivery by quickly entering the desired cells without affecting other organs. With the advent of mobile communication, artificial neural networks, and Information Technology, futuristic and more advanced nanobots with artificial intelligence are in the offing. However, the challenges to this revolutionary technology are umpteen, the major concern being their interaction inside the human body. This review explicitly expounds on nanobots and their applications to medicine, biomedical research, and drug delivery.

Enhancing speech emotion recognition with deep learning using multi-feature stacking and data augmentation

This study evaluates the effectiveness of data augmentation on 1D convolutional neural network (CNN) and transformer models for speech emotion recognition (SER) on the Ryerson audio-visual database of emotional speech and song (RAVDESS) dataset. The results show that data augmentation has a positive impact on improving emotion classification accuracy. Techniques such as noising, pitching, stretching, shifting, and speeding are applied to increase data variation and overcome class imbalance. The 1D CNN model with data augmentation achieved 94.5% accuracy, while the transformer model with data augmentation performed even better at 97.5%. This research is expected to contribute better insights for the development of accurate emotion recognition methods by using data augmentation with these models to improve classification accuracy on the RAVDESS dataset. Further research can explore larger and more diverse datasets and alternative model approaches.

Autism spectrum disorder identification with multi-site functional magnetic resonance imaging

Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by enduring difficulties in social interaction and communication. People analyzed with ASD may display repetitive behaviors and limited interests. Autism is classified as a spectrum disorder, implying that the symptom intensity might range from mild to severe depending on the individual. To detect ASD in this paper an attribute feature graph approach is designed by using the stastical dependencies features that necessarily accomplish the diagnosis of ASD. In the first phase the features extracted are designed based on the functional magnetic resonance imaging (fMRI) data, in the next-step the attribute feature graph layer learns the features of the node information of various nodes by ASD classification. Further, in the third step, it is employed to independently extract distinguishing features from the functional connectivity matrices of the brain that are derived from fMRI. The custom convolutional neural network (CNN) used in this study is trained on a comprehensive dataset comprising individuals diagnosed with ASD and typically developing individuals. In the fourth stage, a prototype learning is developed to augment the classification performance of the custom-CNN. The experimental analysis further carried out states that the proposed model works efficiently in comparison with the existing system.

Performance evaluation of bridgeless isolated SEPIC-Luo converter for EV battery charging using PI and ANN controller

Electric vehicle (EV) rechargeable battery packs that employ traditional power factor correction (PFC) circuit design have performance limitations due to their substantial conductivity loss that ensures at the input of a diode bridge rectifier (DBR). This study suggests a bridgeless (BL) isolated single ended primary inductance converter (SEPIC) - Luo converter to address the problem. As a result, the input current exhibits a power factor operation of unity throughout the charging process. DBR elimination and current conduction through a remarkably small number of circuits both significantly reduce conduction losses. The use of an artificial neural network (ANN) and proportional integral (PI) controller enhances the converter's performance with a stable DC link voltage. The suggested converter overall operation is thoroughly described in terms of variety of operating modes and simulation-based effectiveness. Here, with the assistance of the hysteresis current controller (HCC), the input current disruptions are reduced. Constant current and voltage management is used to successfully charge the EV battery, resulting in improved efficacy and inherent PFC. By utilizing simulation outcomes achieved from MATLAB, the performance of proposed BL isolated SEPIC-Luo in boosting the power quality of EV charger system is examined.

Asymptotic-Preserving Neural Networks for Multiscale Kinetic Equations

Asymptotic-Preserving Neural Networks for Multiscale Kinetic Equations

Aspect based sentiment analysis using a novel ensemble deep network

Aspect-based sentiment analysis (ABSA) is a fine-grained task in natural language processing, which aims to predict the sentiment polarity of several parts of a sentence or document. The essential aspect of sentiment polarity and global context have deep relationships that have not received enough attention. This research work design and develops a novel ensemble deep network (EDN) which comprises the various network and integrated to enhance the model performance. In the proposed work the words of the input sentence are converted into word vectors using the optimised bidirectional encoder representations from transformers (BERT) model and an optimised BERT-graph neural networks (GNN) model with convolutions is built that analyses the ABSA of the input sentence. The optimised GNN model with convolutions for context-based word representations is developed for the word-vector embedding. We propose a novel EDN for an ABSA model for optimised BERT over GNN with convolutions. The proposed ensemble deep network proposed system (EDN-PS) is evaluated with various existing techniques and results are plotted in terms of metrics for accuracy and F1- score, concluding that the proposed EDN-PS ensures better performance in comparison with the existing model.

A comparative analysis of convolutional neural networks for breast cancer prediction

Breast cancer continues to be a substantial worldwide health concern, affecting millions of individuals each year; this emphasizes the critical nature of early detection in order to enhance patient prognoses. The present study aims to assess the classification performance of three convolutional neural network (CNN) architectures-visual geometry group 19 (VGG19), AlexNet, and residual network 50 (ResNet50)-with respect to breast cancer detection in medical images. Thorough assessments, encompassing metrics such as accuracy, precision, recall, and F-score, were undertaken to evaluate the diagnostic performance of the models. ResNet50 consistently outperforms other models, as evidenced by its highest accuracy and F-score. The research highlights the significant importance of carefully choosing suitable architectures for medical image analysis, with a specific focus on the detection of breast cancer. In addition, it demonstrates the capacity of deep learning models, such as ResNet50, to improve the diagnosis of breast cancer with exceptional precision and sensitivity, which is critical for reducing the occurrence of false positives and negatives in clinical environments. In addition, computational efficiency is taken into account; AlexNet is recognized as the most efficient model, which is advantageous in environments with limited resources. This study advances medical image processing by demonstrating the potential of CNNs in the detection of breast cancer. The results of this study establish a fundamental basis for sub- sequent inquiries and suggest approaches to improve timely detection and treatment, which will ultimately be advantageous for both patients and healthcare professionals.

A new deep learning model with interface for fine needle aspiration cytology image-based breast cancer detection

Cytological evaluation through microscopic image analysis of fine needle aspiration cytology (FNAC) is pivotal in the initial screening of breast cancer. The sensitivity of FNAC as a screening tool relies on both image quality and the pathologist’s expertise. To enhance diagnostic accuracy and alleviate the pathologist’s workload, a computer-aided diagnosis (CAD) system was developed. A comparative study was conducted, assessing twelve candidate pre-trained models. Utilizing a locally gathered FNAC image dataset, three superior models-MobileNet-V2, DenseNet-121, and Inception-V3-were selected based on their training, validation, and testing accuracies. Further, these models underwent evaluation in four transfer learning scenarios to enhance testing accuracy. While the outcomes were promising, they left room for improvement, motivating us to create a novel deep convolutional neural network (CNN). The newly proposed model exhibited robust performance with testing accuracy at 85%. Our research concludes that the most lightweight, high-accuracy model is the one we propose. We’ve integrated it into our user-friendly Android App, “Breast Cancer Detection System,” in TensorFlow Lite format, with cloud database support, showcasing its effectiveness. Implementing an artificial intelligent (AI)-based diagnosis system with a user-friendly interface holds the potential to enhance early breast cancer detection using FNAC.