Neural Network Classification Model Research Articles

Automatic diagnosis of diabetic retinopathy using vision transformer based on wide-field optical coherence tomography angiography

Diabetic retinopathy (DR) is one of the major causes of visual impairment in adults with diabetes. Optical coherence tomography angiography (OCTA) is nowadays widely used as the golden criterion for diagnosing DR. Recently, wide-field OCTA (WF-OCTA) provided more abundant information including that of the peripheral retinal degenerative changes and it can contribute in accurately diagnosing DR. The need for an automatic DR diagnostic system based on WF-OCTA pictures attracts more and more attention due to the large diabetic population and the prevalence of retinopathy cases. In this study, automatic diagnosis of DR using vision transformer was performed using WF-OCTA images (12[Formula: see text]mm × 12[Formula: see text]mm single-scan) centered on the fovea as the dataset. WF-OCTA images were automatically classified into four classes: No DR, mild nonproliferative diabetic retinopathy (NPDR), moderate to severe NPDR, and proliferative diabetic retinopathy (PDR). The proposed method for detecting DR on the test set achieves accuracy of 99.55%, sensitivity of 99.49%, and specificity of 99.57%. The accuracy of the method for DR staging reaches up to 99.20%, which has been proven to be higher than that attained by classical convolutional neural network models. Results show that the automatic diagnosis of DR based on vision transformer and WF-OCTA pictures is more effective for detecting and staging DR.

A two-stage feature aggregation network for multi-category soybean leaf disease identification

Accurate identification of soybean leaf disease is of utmost importance for its cultivation and fine management, as it is a critical factor contributing to the decreased quality and yield of soybean. Nevertheless, several existing studies on the identification of soybean leaf disease encounter the challenge of striking a balance between model expressiveness and practical applicability. To address such issue, we propose a well-designed two-stage feature aggregation network framework (TFANet). The main component of it is the two-stage feature aggregation (TFA) module, which is designed to achieve great feature representation capability by aggregating feature information from different convolutional layers in two stages. Meanwhile, TFA module incorporates the efficient channel attention to extract crucial disease information. Afterwards, to limit the loss of feature information and expand the receptive field, a dilated convolution-based feature fusion (DCFF) module is developed. Additionally, the InceptionC module is introduced to further achieve better performance. The experimental results demonstrated that TFANet obtained an accuracy of 98.18% and an F1 scores of 98.39%, with merely 1.18 M parameters. In comparison with some classic convolutional neural network models, TFANet showed remarkable superiority in multiple evaluation metrics. Hence, TFANet has great potential to be used in practical soybean leaf disease identification.

Accuracy Improvement Based on Classic Neural Network: Voting, Restarting and Quantization

Since AlphaGo beat the world Go champion in 2016, which attracted wide attention, the neural network has become more and more popular in recent years, and people’s research on it has gradually improved and been applied in different fields. Today, artificial intelligence and machine learning have become an essential part of modern society and intelligent systems. We do image recognition, speech recognition, and visual learning, closely related to machine learning. However, in machine learning, unsatisfactory training results or even training failure is always encountered. Therefore, in machine learning, it is imperative to improve the accuracy of neural network training results. In this paper, speech recognition, image processing, MNIST, and other classical neural network models will be used to set the training parameters of the neural network better and improve the accuracy of training through voting, quantization, restart, and other methods. The part of research is aiming to find the relationship between restart numbers on the training process and the total extent of learning improvement. At the same time, several algorithms on utilizing these restart numbers are to be compared and selected. Finally, the conclusion is drawn that the more restart made in the training process with a convolutional neural network, the less profit on accuracy improvement we gain from restarting the process.

Age and gender classification with bone images using deep learning algorithms

In paediatrics, bone age is a crucial indicator of how a child's skeleton is developing. They have had great success ever since the creation of deep learning (DL)-based bone age prediction tools. Deep features learning, however, has a significant computing overhead problem. Deep convolution layers are used in this technique to learn representative features in the small yet useful regions that are extracted for feature learning. This work suggests using an extreme learning machine algorithm as the fundamental architecture in the final bone age assessment study to realise the rapid computation speed and feature interaction. The viability and efficacy of the suggested strategy have been verified by experiments using data that is openly accessible. The suggested model is explicitly trained using a cutting-edge end-to-end learning architecture employing bone scans to extract the most discriminative patches from the original high-resolution image. The bone picture is the foundation of the procedure. Our main objective is to categorize individuals by age using convolution neural network (CNN) classification models, such as the Xception and Mobile Net models. As a result, we have achieved results that are 90% and 94% accurate in classifying people by age using CNN models.

Classification of Brain Tumor as Benign or Malignant Using Image Processing and Deep Learning Techniques: A Comparative Study

bstract: In India, the dominance of brain tumors is 5-10 per 100,000 people. According to the International Association of Cancer Registries (IARC), over 28,000 cases of brain tumors are recorded each year in India, with over 24,000 individuals dying from them each year. The proposed approach includes pre-processing of the Magnetic Resonance Imaging (MRI) scans followed by the extraction of the Region of Interest (ROI) using image processing techniques. The extracted parts are then used as input to train a deep neural network model for classification. The Convolutional Neural(CNN) is trained on a large dataset of MRI images of brain tumors, and evaluated using various performance metrics. Deep Learning models like VGG16, ResNet50 are used for the classification of images. Deep learning model with VGG16 gives 98.5 % accuracy which is better than CNN and ResNet50.

Integrated analyses delineate distinctive immunological pathways and diagnostic signatures for Behcet's disease by leveraging gene microarray data.

Behcet's disease (BD) is a chronic inflammatory vasculitis and clinically heterogeneous disorder caused by immunocyte aberrations. Comprehensive research on gene expression patterns in BD illuminating its aetiology is lacking. E-MTAB-2713 downloaded from ArrayExpress was analysed to screen differentially expressed genes (DEGs) using limma. Random forest (RF) and neural network (NN) classification models composed of gene signatures were established using the E-MTAB-2713 training set and subsequently verified using GSE17114. Single sample gene set enrichment analysis was used to assess immunocyte infiltration. After identifying DEGs in E-MTAB-2713, pathogen-triggered, lymphocyte-mediated and angiogenesis- and glycosylation-related inflammatory pathways were discovered to be predominant in BD episodes. Gene signatures from the RF and NN diagnostic models, together with genes enriched in angiogenesis and glycosylation pathways, well discriminated the clinical subtypes of BD manifesting as mucocutaneous, ocular and large vein thrombosis involvement in GSE17114. Moreover, a distinctive immunocyte profile revealed T, NK and dendritic cell activation in BD compared to the findings in healthy controls. Our findings suggested that EPHX1, PKP2, EIF4B and HORMAD1 expression in CD14+ monocytes and CSTF3 and TCEANC2 expression in CD16+ neutrophils could serve as combined gene signatures for BD phenotype differentiation. Pathway genes comprising ATP2B4, MYOF and NRP1 for angiogenesis and GXYLT1, ENG, CD69, GAA, SIGLEC7, SIGLEC9 and SIGLEC16 for glycosylation also might be applicable diagnostic markers for subtype identification.

Quantum Graph Neural Network Models for Materials Search.

Inspired by classical graph neural networks, we discuss a novel quantum graph neural network (QGNN) model to predict the chemical and physical properties of molecules and materials. QGNNs were investigated to predict the energy gap between the highest occupied and lowest unoccupied molecular orbitals of small organic molecules. The models utilize the equivariantly diagonalizable unitary quantum graph circuit (EDU-QGC) framework to allow discrete link features and minimize quantum circuit embedding. The results show QGNNs can achieve lower test loss compared to classical models if a similar number of trainable variables are used, and converge faster in training. This paper also provides a review of classical graph neural network models for materials research and various QGNNs.

Evaluation of practical edge computing CNN‐based solutions for intelligent recycling bins

AbstractRapid economic growth has given rise to the urgent demand for more efficient waste recycling systems. An innovative smart recycling bin is proposed that automatically separates urban waste to increase the recycling rate. Over 1800 recycling waste images were collected and combined with an existing public dataset to train neural network classification models for two embedded systems, one incorporating a Jetson Nano and the other a K210 unit. The model developed reached an accuracy of 93.99% on the Jetson Nano and 94.61% on the K210. A user interface application was also designed to collect feedback from users during their interaction with the smart bin. In terms of power consumption, the system employing the Jetson Nano consumed 4.7 W, representing a 30% reduction in power consumption compared to previous work, while the K210 required just 0.89 W of power to operate. In summary, our work demonstrated a small‐scale, fully functional prototype of an energy‐efficient, high‐accuracy smart recycling bin, with the potential of commercialisation for the purpose of improving urban waste recycling.

Quantum Language Model With Entanglement Embedding for Question Answering.

Quantum language models (QLMs) in which words are modeled as a quantum superposition of sememes have demonstrated a high level of model transparency and good post-hoc interpretability. Nevertheless, in the current literature, word sequences are basically modeled as a classical mixture of word states, which cannot fully exploit the potential of a quantum probabilistic description. A quantum-inspired neural network (NN) module is yet to be developed to explicitly capture the nonclassical correlations within the word sequences. We propose a NN model with a novel entanglement embedding (EE) module, whose function is to transform the word sequence into an entangled pure state representation. Strong quantum entanglement, which is the central concept of quantum information and an indication of parallelized correlations among the words, is observed within the word sequences. The proposed QLM with EE (QLM-EE) is proposed to implement on classical computing devices with a quantum-inspired NN structure, and numerical experiments show that QLM-EE achieves superior performance compared with the classical deep NN models and other QLMs on question answering (QA) datasets. In addition, the post-hoc interpretability of the model can be improved by quantifying the degree of entanglement among the word states.

A data-driven approach to estimating post-discovery parameters of unexplored oilfields

Consider a typical situation where an investor is considering acquiring an unexplored oilfield. The oilfield has undergone a preliminary geological and geophysical study in which pre-discovery data such as lithology, depth, depositional system, diagenetic overprint, structural compartmentalization, and trap type are available. In this situation, investors usually estimate production rates using a volumetric approach. A more accurate estimation of production rates can be obtained using analytical methods, which require additional data such as net pay, porosity, oil formation volume factor, permeability, viscosity, and pressure. We call these data post-discovery parameters because they are only available after discovery through exploration drilling. A data-driven approach to estimating post-discovery parameters of an unexplored oilfield is developed based on its pre-discovery data by learning from proven reservoir data. Using the Gaussian mixture model, and a data-driven reservoir typology based on the joint probability distribution of post-discovery parameters is established. We came up with 12 reservoir types. Subsequently, an artificial neural network classification model with the resilient backpropagation algorithm is used to find relationships between pre-discovery data and reservoir types. Based on k-fold cross-validation with k = 10, the accuracy of the classification model is stable with an average of 87.9%. With our approach, an investor considering acquiring an unexplored oilfield can classify the oilfield's reservoir into a particular type and estimate its post-discovery parameters' joint probability distribution. The investor can incorporate this information into a valuation model to calculate the production rates more accurately, estimate the oilfield's value and risk, and make an informed acquisition decision accordingly.

Driver Assistant Using Deep Learning

Abstract: In the development of intelligent vehicles, accurate traffic sign detection and recognition are critical. This project proposes an improved algorithm for traffic sign detection and recognition, aiming to address limitations of traditional methods such as environmental sensitivity and poor real-time performance of deep learning-based approaches. The algorithm employs HSV-based spatial threshold segmentation for effective traffic sign detection based on shape features. Furthermore, the algorithm enhances the classical convolutional neural network model by utilizing Gabor kernel for initial convolution, incorporating batch normalization after pooling, and employing the Adam optimizer algorithm. The proposed algorithm is evaluated using the German Traffic Sign Recognition Benchmark, achieving a favorable prediction and accurate recognition rate of 99.75%, with an average processing time of 5.4ms per frame. Compared to other algorithms, the proposed approach demonstrates superior accuracy, real-time performance, generalization ability, and training efficiency. The findings of this project are expected to contribute to reducing accident rates and enhancing road traffic safety through improved traffic sign recognition in intelligent vehicles.

Real-Time sEMG Pattern Recognition of Multiple-Mode Movements for Artificial Limbs Based on CNN-RNN Algorithm

Currently, sEMG-based pattern recognition is a crucial and promising control method for prosthetic limbs. A 1D convolutional recurrent neural network classification model for recognizing online finger and wrist movements in real time was proposed to address the issue that the classification recognition rate and time delay cannot be considered simultaneously. This model could effectively combine the advantages of the convolutional neural network and recurrent neural network. Offline experiments were used to verify the recognition performance of 20 movements, and a comparative analysis was conducted with CNN and LSTM classification models. Online experiments via the self-developed sEMG signal pattern recognition system were established to examine real-time recognition performance and time delay. Experiment results demonstrated that the average recognition accuracy of the 1D-CNN-RNN classification model achieved 98.96% in offline recognition, which is significantly higher than that of the CNN and LSTM (85.43% and 96.88%, respectively, p < 0.01). In the online experiments, the average accuracy of the real-time recognition of the 1D-CNN-RNN reaches 91% ± 5%, and the average delay reaches 153 ms. The proposed 1D-CNN-RNN classification model illustrates higher performances in real-time recognition accuracy and shorter time delay with no obvious sense of delay in the human body, which is expected to be an efficient control for dexterous prostheses.

AAIN: Attentional aggregative interaction network for deep learning based recommender systems

Feature engineering is a classical problem in recommender systems, and feature interactions is one of the most important parts of feature engineering. Factorization based models are widely used for explicit feature interactions. However, most current works utilize separate features to model cross features. Such a pattern limits the significance of cross features, since realistic recommendation scenarios are rich in associations between features.In this paper, we classify the basic feature interactions into sum-interaction and product-interaction, and improve the current general strategy of explicit feature interactions. Based on these theoretical studies, we propose a novel explicit feature interactions model Attentional Aggregative Interaction Network (AAIN), which models higher-order features using a cyclic explicit module. Specifically, we introduce attention mechanism for the reorganization of separate features, followed by product-interaction and higher-order features’ compression and output. The model is efficient since: 1) AAIN automatically learns high-order feature interactions and filters them with different weights. 2) AAIN optimizes the interaction between features into the interaction between feature groups, which allows for other relevant information to be considered when performing interactions. Furthermore, we integrate AAIN model with the classical deep neural network (DNN) model into a new model Deep Attentional Aggregative Interaction Network (DAAIN). Experiments on real-world datasets show that our models achieve state-of-the-art results.

Multi-scale asphalt pavement deformation detection and measurement based on machine learning of full field-of-view digital surface data

Asphalt pavement deformation is a common phenomenon due to the material property and traffic loads. Aiming at ensuring traffic comfort and safety, it is essential to constantly monitor the multi-scale pavement deformation with a non-destructive and automatic system. This paper presents a full field-of-view asphalt pavement deformation inspection framework based on multi-dimensional surface data and machine learning, which can detect and measure pavement rutting, roughness and large-span deformation simultaneously. In this integrated system, one-dimensional convolutional neural network (1D CNN) classification and localization models are developed for two-step rutting detection. The quarter-car model with multiple measuring lines is employed for full-lane international roughness index (IRI) measurement and spatial analysis. The unsupervised K-means convolutional neural network (K-CNN) model is proposed for large-span deformation detection. The results show that the overall F1 scores of rutting classification and localization are 99.56% and 97.24%, respectively. The full-lane measurement suggests that IRI presents a bimodal distribution in transverse space due to the concentrated traffic loads on the wheel path. In addition, the unsupervised K-CNN achieves an average consistency index (CI) of 91.61% on large-span sunken and heave deformation segmentation.

LightMixer: A novel lightweight convolutional neural network for tomato disease detection.

Tomatoes are among the very important crops grown worldwide. However, tomato diseases can harm the health of tomato plants during growth and reduce tomato yields over large areas. The development of computer vision technology offers the prospect of solving this problem. However, traditional deep learning algorithms require a high computational cost and several parameters. Therefore, a lightweight tomato leaf disease identification model called LightMixer was designed in this study. The LightMixer model comprises a depth convolution with a Phish module and a light residual module. Depth convolution with the Phish module represents a lightweight convolution module designed to splice nonlinear activation functions with depth convolution as the backbone; it also focuses on lightweight convolutional feature extraction to facilitate deep feature fusion. The light residual module was built based on lightweight residual blocks to accelerate the computational efficiency of the entire network architecture and reduce the information loss of disease features. Experimental results show that the proposed LightMixer model achieved 99.3% accuracy on public datasets while requiring only 1.5 M parameters, an improvement over other classical convolutional neural network and lightweight models, and can be used for automatic tomato leaf disease identification on mobile devices.

Research on the Application of Neural Network Classification Model in English Grammar Error Correction

The field of English grammar error correction has developed rapidly in recent years, and many excellent research results have been obtained. However, errors are generally viewed as a whole for revision, which cannot be subdivided to identify the specific error types of sentences, lacks interpretability, and requires a lot of manpower to label the data. To address this, the study uses the CoreNLP tool to output the input text as a basic form of each word, and encodes and decodes the linguistic text in the corpus using the CNN-based Seq2Seq model. And word embedding and left-right GRU operations are performed on left-right text and target words in turn, and the merged new vector is obtained using two attention mechanism operations. It is fed into the MLP classifier for classification and error correction. To further improve the model performance, two model optimizers, stochastic gradient descent (SGD) and Adam, are used to tune the parameters of the classification prediction model. The study thus constructs an English grammar classification and error correction model based on a neural network classification method. The experimental analysis shows that the average accuracy of the model constructed in the study is 96.83%, the average recall rate is 78.93%, and the average overall evaluation F0.5 is 92.67%, which can effectively and accurately classify and correct the grammatical errors that occur in the process of English language learning.

A neural network for evaluating the concentration of leaked gas clouds detected by TDLAS at oil and gas stations

Due to the significant advantages of methane sensitivity and area-type leakage detection, tunable diode laser absorption spectroscopy (TDLAS) gas detection has been promoted as an effective method for microleakage monitoring in oil and gas stations. However, the output of the TDLAS detector is the integral concentration (IC). Based on the relevant research, the alarm threshold and risk are assessed via the gas concentration rather than the IC. How to evaluate the concentration of leaked gas clouds based on IC from TDLAS detectors is still a challenge. To address this problem, the characteristics of IC and the influence of the laser path, wind speed and leakage rate were studied via computational fluid dynamics (CFD). A neural network classification model (NNCM) was proposed to obtain the probability distribution of the maximal concentration along the laser path (Cmax ). The results indicated that the IC is strongly correlated with the Cmax . Considering the accuracy and operability, the NNCM with input features of IC, wind speed and angle of laser path was selected. Field tests showed that the developed model achieved the concentration evaluation of leaked gas clouds. Additionally, the NNCM can also quantify the uncertainty of the results, which avoids misjudgments caused by deviations.

Origin Identification of Saposhnikovia divaricata by CNN Embedded with the Hierarchical Residual Connection Block

This paper proposes a method for recognizing the origin of Saposhnikovia divaricata using the IResNet model to achieve computer vision-based classification. Firstly, we created a small sample dataset and applied data augmentation techniques to enhance its diversity. After that, we introduced the hierarchical residual connection block in the early stage of the original model to expand the perceptual field of the neural network and enhance the extraction of scale features. Meanwhile, a depth-separable convolution operation was adopted in the later stage of the model to replace the conventional convolution operation and further reduce the time cost of the model. The experimental results demonstrate that the improved network model achieved a 5.03% improvement in accuracy compared to the original model while also significantly reducing the number of parameters required for the model. In our experiments, we compared the accuracy of the proposed model with several classical convolutional neural network models, including ResNet50, Resnest50, Res2net50, RepVggNet_B0, and ConvNext_T. The results showed that our proposed model achieved an accuracy of 93.72%, which outperformed ResNet50 (86.68%), Resnest50 (89.38%), Res2net50 (91.83%), RepVggNet_B0 (88.68%), and ConvNext_T (92.18%). Furthermore, our proposed model achieved the highest accuracy among the compared models, with a transmission frame rate of 158.9 fps and an inference time of only 6.29 ms. The research methodology employed in this study has demonstrated the ability to reduce potential errors caused by manual observation, effectively improving the recognition ability of Saposhnikovia divaricata based on existing data. Furthermore, the findings of this study provide valuable reference and support for future efforts to develop lightweight models in this area.

Weld Defect Detection of a CMT Arc-Welded Aluminum Alloy Sheet Based on Arc Sound Signal Processing

The cold metal transfer (CMT) process is widely used in thin plate welding because of its characteristics of low heat input and stable arc. In actual production, a larger weld gap, misalignment, or other problems due to assembly error lead to serious welding defects, such as burn-through and a lack of fusion. The arc sound contains a wealth of information related to the quality of the weld. This work analyzes the mechanism of CMT arc sound generation, as well as the correlation between the time–frequency spectrum of the arc sound signal and welding quality. This paper studies the extraction of the multi-channel time–frequency spectrum of an arc sound and inputs it to a custom convolutional neural network for the CMT welding defect identification of thin aluminum alloy plates. The experimental result shows that the average accuracy of the proposed model is 91.49% in the defect identification of a CMT arc-welded aluminum alloy sheet, which is higher than that of the single-channel time–frequency convolutional neural network and other traditional classification models.

The Application of Hyperspectral Images in the Classification of Fresh Leaves’ Maturity for Flue-Curing Tobacco

The maturity of tobacco leaves directly affects their curing quality. However, no effective method has been developed for determining their maturity during production. Assessment of tobacco maturity for flue curing has long depended on production experience, leading to considerable variation. In this study, hyperspectral imaging combined with a novel algorithm was used to develop a classification model that could accurately determine the maturity of tobacco leaves. First, tobacco leaves of different maturity levels (unripe, under-ripe, ripe, and over-ripe) were collected. ENVI software was used to remove the hyperspectral imaging (HSI) background, and 11 groups of filtered images were obtained using Python 3.7. Finally, a full-band-based partial least-squares discriminant analysis (PLS-DA) classification model was established to identify the maturity of the tobacco leaves. In the calibration set, the model accuracy of the original spectrum was 88.57%, and the accuracy of the de-trending, multiple scattering correction (MSC), and standard normalization variable (SNV) treatments was 91.89%, 95.27%, and 92.57%, respectively. In the prediction set, the model accuracy of the de-trending, MSC, and SNV treatments was 93.85%, 96.92%, and 93.85%, respectively. The experimental results indicate that a higher model accuracy was obtained with the filtered images than with the original spectrum. Because of the higher accuracy, de-trending, MSC, and SNV treatments were selected as the candidate characteristic spectral bands, and a successive projection algorithm (SPA), competitive adaptive reweighted sampling (CASR), and particle swarm optimization (PSO) were used as the screening methods. Finally, a genetic algorithm (GA), PLS-DA, line support vector machine (LSVM), and back-propagation neural network (BPNN) classification and discrimination models were established. The combination SNV-SPA-PLS-DA model provided the best accuracy in the calibration and prediction sets (99.32% and 98.46%, respectively). Our findings highlight the efficacy of using visible/near-infrared (ViS/NIR) hyperspectral imaging for detecting the maturity of tobacco leaves, providing a theoretical basis for improving tobacco production.