Neural Network Classification Model Research Articles

Studying sleep: towards the identification of hypnogram features that drive expert interpretation.

Hypnograms contain a wealth of information and play an important role in sleep medicine. However, interpretation of the hypnogram is a difficult task and requires domain knowledge and "clinical intuition." This study aimed to uncover which features of the hypnogram drive interpretation by physicians. In other words, make explicit which features physicians implicitly look for in hypnograms. Three sleep experts evaluated up to 612 hypnograms, indicating normal or abnormal sleep structure and suspicion of disorders. ElasticNet and convolutional neural network classification models were trained to predict the collected expert evaluations using hypnogram features and stages as input. The models were evaluated using several measures, including accuracy, Cohen's kappa, Matthew's correlation coefficient, and confusion matrices. Finally, model coefficients and visual analytics techniques were used to interpret the models to associate hypnogram features with expert evaluation. Agreement between models and experts (Kappa between 0.47 and 0.52) is similar to agreement between experts (Kappa between 0.38 and 0.50). Sleep fragmentation, measured by transitions between sleep stages per hour, and sleep stage distribution were identified as important predictors for expert interpretation. By comparing hypnograms not solely on an epoch-by-epoch basis, but also on these more specific features that are relevant for the evaluation of experts, performance assessment of (automatic) sleep-staging and surrogate sleep trackers may be improved. In particular, sleep fragmentation is a feature that deserves more attention as it is often not included in the PSG report, and existing (wearable) sleep trackers have shown relatively poor performance in this aspect.

Integrated bulk and single-cell transcriptomes reveal pyroptotic signature in prognosis and therapeutic options of hepatocellular carcinoma by combining deep learning.

Although some pyroptosis-related (PR) prognostic models for cancers have been reported, pyroptosis-based features have not been fully discovered at the single-cell level in hepatocellular carcinoma (HCC). In this study, by deeply integrating single-cell and bulk transcriptome data, we systematically investigated significance of the shared pyroptotic signature at both single-cell and bulk levels in HCC prognosis. Based on the pyroptotic signature, a robust PR risk system was constructed to quantify the prognostic risk of individual patient. To further verify capacity of the pyroptotic signature on predicting patients' prognosis, an attention mechanism-based deep neural network classification model was constructed. The mechanisms of prognostic difference in the patients with distinct PR risk were dissected on tumor stemness, cancer pathways, transcriptional regulation, immune infiltration and cell communications. A nomogram model combining PR risk with clinicopathologic data was constructed to evaluate the prognosis of individual patients in clinic. The PR risk could also evaluate therapeutic response to neoadjuvant therapies in HCC patients. In conclusion, the constructed PR risk system enables a comprehensive assessment of tumor microenvironment characteristics, accurate prognosis prediction and rational therapeutic options in HCC.

AdaRes: A deep learning-based model for ultrasound image denoising: Results of image quality metrics, radiomics, artificial intelligence, and clinical studies.

The quality of ultrasound images is degraded by speckle and Gaussian noises. This study aims to develop a deep-learning (DL)-based filter for ultrasound image denoising. A novel DL-based filter using adaptive residual (AdaRes) learning was proposed. Five image quality metrics (IQMs) and 27 radiomics features were used to evaluate denoising results. The effect of our proposed filter, AdaRes, on four pre-trained convolutional neural network (CNN) classification models and three radiologists was assessed. AdaRes filter was tested on both natural and ultrasound image databases. IQMs results indicate that AdaRes could remove noises in three different noise levels with the highest performances. In addition, a radiomics study proved that AdaRes did not distort tissue textures and it could preserve most radiomics features. AdaRes could also improve the performance classification using CNNs in different settings. Finally, AdaRes also improved the mean overall performance (AUC) of three radiologists from 0.494 to 0.702 in the classification of benign and malignant lesions. AdaRes filtered out noises on ultrasound images more effectively and can be used as an auxiliary preprocessing step in computer-aided diagnosis systems. Radiologists may use it to remove unwanted noises and improve the ultrasound image quality before the interpretation.

An efficient system for water leak detection and localization based on IoT and lightweight deep learning

Nowadays, Water Distribution Networks (WDNs) efficiency is limited due to leakage issues, which result in large water losses that exceed about one third of the input volume. Considering the problems of growing population and water scarcity especially in Africa and Asia continents, an urgent humanitarian need arisen to efficiently detect and localize pipeline water leaks. To address this need, this paper proposes an end-to-end system for leak detection and localization in water pipelines based on IoT and deep learning. The IoT solution is deployed allowing real time monitoring of pipelines by using non-invasive IoT devices equipped with acoustic sensors which are installed on the pipeline surface. A lightweight one-Dimensional Convolutional Neural Network (1D-CNN) classification model is embedded on these devices to detect leaks. This model uses raw audio signals generated by sensors as input information. Information about the detected leaks is then remotely transmitted using long range and low energy LoRaWAN protocol to the LoRa gateway. The gateway relays messages to a data processing Server. A 1D-CNN regression model has been then created and deployed in this server to directly estimate the 2D position of the leak. As an illustration, An IoT device prototype is designed and used as a case study to test the performance of the proposed system. We demonstrated that our proposal achieves an accuracy of 97 % for detecting leaks and can fix the 2D position of leak with an error of 0.0006 on the X coordinate and 0.0004 on the Y coordinate. Experiments confirm the effectiveness of the proposed solution.

3D superstructure based metabolite profiling for glaucoma diagnosis

Metabolome analysis has gained widespread application in disease diagnosis owing to its ability to provide comprehensive information, including disease phenotypes. In this study, we utilized 3D superstructures fabricated through evaporation-induced microprinting to analyze the metabolome for glaucoma diagnosis. 3D superstructures offer the following advantages: high hotspot density per unit volume of the structure extending from two to three dimensions, excellent signal repeatability due to the reproducibility and defect tolerance of 3D printing, and high thermal stability due to the PVP-enclosed capsule form. Leveraging the superior optical properties of the 3D superstructure, we aimed to classify patients with glaucoma. The signal obtained from the 3D superstructure was employed in a Deep Neural Network (DNN) classification model to accurately classify glaucoma patients. The sensitivity and specificity of the model were determined as 92.05% and 93.51%, respectively. Additionally, the fabrication of 3D superstructures can be performed at any stage, significantly reducing measurement time. Furthermore, their thermal stability allows for the analysis of smaller samples. One notable advantage of 3D superstructures is their versatility in accommodating different target materials. Consequently, they can be utilized for a wide range of metabolic analyses and disease diagnoses.

SE-VisionTransformer: Hybrid Network for Diagnosing Sugarcane Leaf Diseases Based on Attention Mechanism.

Sugarcane is an important raw material for sugar and chemical production. However, in recent years, various sugarcane diseases have emerged, severely impacting the national economy. To address the issue of identifying diseases in sugarcane leaf sections, this paper proposes the SE-VIT hybrid network. Unlike traditional methods that directly use models for classification, this paper compares threshold, K-means, and support vector machine (SVM) algorithms for extracting leaf lesions from images. Due to SVM's ability to accurately segment these lesions, it is ultimately selected for the task. The paper introduces the SE attention module into ResNet-18 (CNN), enhancing the learning of inter-channel weights. After the pooling layer, multi-head self-attention (MHSA) is incorporated. Finally, with the inclusion of 2D relative positional encoding, the accuracy is improved by 5.1%, precision by 3.23%, and recall by 5.17%. The SE-VIT hybrid network model achieves an accuracy of 97.26% on the PlantVillage dataset. Additionally, when compared to four existing classical neural network models, SE-VIT demonstrates significantly higher accuracy and precision, reaching 89.57% accuracy. Therefore, the method proposed in this paper can provide technical support for intelligent management of sugarcane plantations and offer insights for addressing plant diseases with limited datasets.

PTS-LSTM: Temperature prediction for fused filament fabrication using thermal image time series

In-situ thermography for Fuse Filament Fabrication (FFF) processes reveals the dynamic thermal behavior during printing. The data collected are thermal image time series. Their infrared (IR) intensity is visual evidence of heat-affected zone (HAZ) temperatures, which can be leveraged to train deep learning models, e.g., Long Short-Term Memory (LSTM), for real-time temperature prediction and process monitoring. Nonetheless, the data collection method and printing path may pose challenges for data modeling. Typically, the IR camera has a fixed position while the HAZ moves per the predetermined printing path. Consequently, the HAZ shifts in images and the features extracted from these thermal images show a “periodic” behavior over time. Such periodic patterns do not reflect any useful information about HAZ temperatures. Instead, they are noise hiding and interrupting the true temperature information, thus must be removed before using the data to train an LSTM model for temperature prediction. This study integrates a time series model, i.e., ARIMA, with Stacked LSTM to build a pTS-LSTM model that eliminates noisy patterns and predicts temperatures during FFF printing. The case study results show the outperformance of pTS-LSTM over conventional LSTM and classic Recurrent Neural Network models. pTS-LSTM is demonstrated to be promising for in-situ process monitoring with low-quality thermal images. In FFF practices, pTS-LSTM will be a preferred option over the commonly used deep learning models for thermal-image-based temperature prediction.

Recognizing Pneumonia Infection in Chest X-Ray Using Deep Learning

One of the diseases that attacks the lungs is pneumonia. Pneumonia is inflammation and fluid in the lungs making it difficult to breathe. This disease is diagnosed using X-Ray. Against the darker background of the lungs, infected tissue shows denser areas, which causes them to appear as white spots called infiltrates. In the image processing approach, pneumonia-infected X-rays can be detected using machine learning as well as deep learning. The convolutional neural network model is able to recognize images well and focus on points that are invisible to the human eye. Previous research using a convolutional neural network model with 10 convolution layers and 6 convolution layers has not achieved optimal accuracy. The aim of this research is to develop a convolutional neural network with a simpler architecture, namely two convolution layers and three convolution layers to solve the same problem, as well as examining the combination of various hyperparameter sizes and regularization techniques. We need to know which convolutional neural network architecture is better. As a result, the convolutional neural network classification model can recognize chest x-rays infected with pneumonia very well. The best classification model obtained an average accuracy of 89.743% with a three-layer convolution architecture, batch size 32, L2 regularization 0.0001, and dropout 0.2. The precision reached 94.091%, recall 86.456%, f1-score 89.601%, specificity 85.491, and error rate 10.257%. Based on the results obtained, convolutional neural network models have the potential to diagnose pneumonia and other diseases.

Wearable Sleep Monitoring System Based on Machine Learning Using Snoring Sound Signal

Abstract According to the obstructive sleep apnea Syndrome (OSAS), a wearable sleep monitoring system is designed based on machine learning using snoring sound signal. The system picks up snoring signal via bone conduction sensor, and calculates the apnea-hypopnea index (AHI). By analyzing the snoring signal in frequency domain, spectral entropy and other frequency-domain features are selected. Finally, the neural network classifier model is established. In the model, the input variables are eight frequency-domain features, and the output response is related to AHI value. Trained by machine learning, the result shows that the average accuracy in identifying the severity of the four kinds of OSAS categories is 59%. The system uses the measured data of snoring to analyze the symptoms of OSAS, so as to realize the preliminary forecast based on the snoring data. The system proposed in this paper has a good application development prospect in intelligent monitoring and medical instruments.

Revealing the diagnostic value and immune infiltration of senescence-related genes in endometriosis: a combined single-cell and machine learning analysis.

Introduction: Endometriosis is a prevalent and recurrent medical condition associated with symptoms such as pelvic discomfort, dysmenorrhea, and reproductive challenges. Furthermore, it has the potential to progress into a malignant state, significantly impacting the quality of life for affected individuals. Despite its significance, there is currently a lack of precise and non-invasive diagnostic techniques for this condition. Methods: In this study, we leveraged microarray datasets and employed a multifaceted approach. We conducted differential gene analysis, implemented weighted gene co-expression network analysis (WGCNA), and utilized machine learning algorithms, including random forest, support vector machine, and LASSO analysis, to comprehensively explore senescence-related genes (SRGs) associated with endometriosis. Discussion: Our comprehensive analysis, which also encompassed profiling of immune cell infiltration and single-cell analysis, highlights the therapeutic potential of this gene assemblage as promising targets for alleviating endometriosis. Furthermore, the integration of these biomarkers into diagnostic protocols promises to enhance diagnostic precision, offering a more effective diagnostic journey for future endometriosis patients in clinical settings. Results: Our meticulous investigation led to the identification of a cluster of genes, namely BAK1, LMNA, and FLT1, which emerged as potential discerning biomarkers for endometriosis. These biomarkers were subsequently utilized to construct an artificial neural network classifier model and were graphically represented in the form of a Nomogram.

Multi-center atrial fibrillation electrocardiogram (ECG) classification using Fourier space convolutional neural networks (FD-CNN) and transfer learning

There has been a proliferation of machine learning (ML) electrocardiogram (ECG) classification algorithms reaching >85% accuracy for various cardiac pathologies. Despite the high accuracy at individual institutions, challenges remain when it comes to multi-center deployment. Transfer learning (TL) is a technique in which a model trained for a specific task is repurposed for another related task, in this case ECG ML model trained at one institution is fine-tuned to be utilized to classify ECGs at another institution. Models trained at one institution, however, might not be generalizable for accurate classification when deployed broadly due to differences in type, time, and sampling rate of traditional ECG acquisition. In this study, we evaluate the performance of time domain (TD) and frequency domain (FD) convolutional neural network (CNN) classification models in an inter-institutional scenario leveraging three different publicly available datasets. The larger PTB-XL ECG dataset was used to initially train TD and FD CNN models for atrial fibrillation (AFIB) classification. The models were then tested on two different data sets, Lobachevsky University Electrocardiography Database (LUDB) and Korea University Medical Center database (KURIAS). The FD model was able to retain most of its performance (>0.81 F1-score), whereas TD was highly affected (<0.53 F1-score) by the dataset variations, even with TL applied. The FD CNN showed superior robustness to cross-institutional variability and has potential for widespread application with no compromise to ECG classification performance.

A Convolutional Neural Network (CNN) Classification Model for Web Page: A Tool for Improving Web Page Category Detection Accuracy

Game and Online Video Streaming are the most viewed web pages. Users who spend too much time on these types of web pages may suffer from internet addiction. Access to Game and Online Video Streaming web pages should be restricted to combat internet addiction. A tool is required to recognise the category of web pages based on the text content of the web pages. Due to the unavailability of a matrix representation that can handle long web page text content, this study employs a document representation known as word cloud image to visualise the words extracted from the text content web page after data pre-processing. The most popular words are shown in large size and appear in the centre of the word cloud image. The most common words are the words that appear frequently in the text content web page and are related to describing what the web page content is about. The Convolutional Neural Network (CNN) recognises the pattern of words presented in the core portions of the word cloud image to categorise the category to which the web page belongs. The proposed model for web page classification has been compared with the other web page classification models. It shows the good result that achieved an accuracy of 85.6%. It can be used as a tool that helps to make identifying the category of web pages more accurate

Determination of moisture content and contaminated blank dried figs (Ficus carica L.) using dielectric property and artificial neural network

AbstractDried figs are a garden produce that must be graded after harvesting. Moisture levels and contaminated blank are two of the most critical effective elements on the marketability of dried figs, and they are highly related to fig quality. In the present research, an intelligent system was employed to classify dried figs based on moisture content levels and infected blank fruits. Capacitance characteristics, average diameter, and fruit area were all taken into account in this study. The dried fig dielectric constant was measured at six different frequency levels: 12, 22, 32, 42, 52, and 62 MHz. The best frequency was then chosen using the improved distance evaluation feature selection approach. Image processing was also used to determine the average diameter and area of the figures. Following that, the dielectric constant of the most effective frequency, the average diameter, and the area of the fruit were used as input parameters in the artificial neural network classification model to classify and describe the moisture and porosity level of the dried fig. The most essential dielectric constant information relating to moisture and porosity level was at frequencies of 22 and 52 MHz, respectively. Finally, classification accuracy of 95.7% for moisture and 91.3% for porosity level was attained. The results demonstrated the excellent performance and capabilities of the proposed approach for rating the internal quality of dried figs.

Optimization of Gearbox Fault Detection Method Based on Deep Residual Neural Network Algorithm.

Because of its long running time, complex working environment, and for other reasons, a gear is prone to failure, and early failure is difficult to detect by direct observation; therefore, fault diagnosis of gears is very necessary. Neural network algorithms have been widely used to realize gear fault diagnosis, but the structure of the neural network model is complicated, the training time is long and the model is not easy to converge. To solve the above problems and combine the advantages of the ResNeXt50 model in the extraction of image features, this paper proposes a gearbox fault detection method that integrates the convolutional block attention module (CBAM). Firstly, the CBAM is embedded in the ResNeXt50 network to enhance the extraction of image channels and spatial features. Secondly, the different time-frequency analysis method was compared and analyzed, and the method with the better effect was selected to convert the one-dimensional vibration signal in the open data set of the gearbox into a two-dimensional image, eliminating the influence of the redundant background noise, and took it as the input of the model for training. Finally, the accuracy and the average training time of the model were obtained by entering the test set into the model, and the results were compared with four other classical convolutional neural network models. The results show that the proposed method performs well both in fault identification accuracy and average training time under two working conditions, and it also provides some references for existing gear failure diagnosis research.

A novel approach to the development of neural network architecture based on metaheuristic protis approach

Determining the best model for the neural network architecture and how to optimize the architecture with the metaheuristic Protis Approach is a subject of the study. A comprehensive investigation and utilization of metaheuristic methods are necessary. These methods aim to solve problems and adapt from the lifestyle of the amoeba protis. In this study, the proposed method modifies the life cycle of the amoeba, which consists of four phases: prophase, metaphase, anaphase, and telophase. These four phases are modified in the neural network architecture to optimize the appropriate number of hidden layers and produce an efficient architecture model. The results show that the protis approach optimized the neural network architecture, especially in generating hidden layers to improve the neural network model. Distinctive features of the results obtained are that the average range of degenerate neurons in the hidden layer is 0 to 35 neurons in each layer. The standard number of neurons makes it possible to solve the problem of determining the best model on the neural network architecture. The protis algorithm embedded in the protis recurrent neural network for categorical data measurements produces an average RMSE value, representing the difference between actual measurements and predictions, equal to 0.066. Consequently, the developed model surpasses the current classical neural network model in terms of performance. Regarding accuracy, the protis algorithm embedded in the neural network for categorical and time series data achieves an average precision of 0.952 and a recall of 0.950. The protis convolutional neural network achieves an accuracy of 95.9 %. Therefore, from the three tested datasets, the protis convolutional neural network exhibits the highest accuracy value

A Comprehensive Implementation of Road Surface Classification for Vehicle Driving Assistance: Dataset, Models, and Deployment

The prior monitoring of the road surface conditions provides valuable information to vehicle trajectory planning and active control systems. Road surface perception with vision sensors is an emerging technology that has recently gained much attention. However, there are few reports on an exhaustive technical scheme and application practice for driving assistance. This study first creates a large-scale road surface image dataset containing one million samples with detailed road friction level, material, and unevenness level annotations. A convolutional neural network (CNN) classification model constrained by a combined loss function and adapted optimization strategies is trained on the dataset. Then we propose a decision-level fusion method based on the improved Dempster-Shafer evidence theory to enhance the robustness of the classification algorithm. Finally, the developed models are deployed on the embedded hardware platform. The top-1 accuracy for classifying road surface images reaches 92.05% by the CNN model and 97.50% after fusion. Onboard experiments illustrate the advantage and superiority of the developed technical framework, which has excellent potential in real-vehicle applications for driving assistance.

Predicting ischemic stroke risk from atrial fibrillation based on multi-spectral fundus images using deep learning.

Ischemic stroke (IS) is one of the most common serious secondary diseases of atrial fibrillation (AF) within 1 year after its occurrence, both of which have manifestations of ischemia and hypoxia of the small vessels in the early phase of the condition. The fundus is a collection of capillaries, while the retina responds differently to light of different wavelengths. Predicting the risk of IS occurring secondary to AF, based on subtle differences in fundus images of different wavelengths, is yet to be explored. This study was conducted to predict the risk of IS occurring secondary to AF based on multi-spectrum fundus images using deep learning. A total of 150 AF participants without suffering from IS within 1 year after discharge and 100 IS participants with persistent arrhythmia symptoms or a history of AF diagnosis in the last year (defined as patients who would develop IS within 1 year after AF, based on fundus pathological manifestations generally prior to symptoms of the brain) were recruited. Fundus images at 548, 605, and 810 nm wavelengths were collected. Three classical deep neural network (DNN) models (Inception V3, ResNet50, SE50) were trained. Sociodemographic and selected routine clinical data were obtained. The accuracy of all DNNs with the single-spectral or multi-spectral combination images at the three wavelengths as input reached above 78%. The IS detection performance of DNNs with 605 nm spectral images as input was relatively more stable than with the other wavelengths. The multi-spectral combination models acquired a higher area under the curve (AUC) scores than the single-spectral models. The probability of IS secondary to AF could be predicted based on multi-spectrum fundus images using deep learning, and combinations of multi-spectrum images improved the performance of DNNs. Acquiring different spectral fundus images is advantageous for the early prevention of cardiovascular and cerebrovascular diseases. The method in this study is a beneficial preliminary and initiative exploration for diseases that are difficult to predict the onset time such as IS.

Automatic diagnosis of diabetic retinopathy using vision transformer based on wide-field optical coherence tomography angiography

Diabetic retinopathy (DR) is one of the major causes of visual impairment in adults with diabetes. Optical coherence tomography angiography (OCTA) is nowadays widely used as the golden criterion for diagnosing DR. Recently, wide-field OCTA (WF-OCTA) provided more abundant information including that of the peripheral retinal degenerative changes and it can contribute in accurately diagnosing DR. The need for an automatic DR diagnostic system based on WF-OCTA pictures attracts more and more attention due to the large diabetic population and the prevalence of retinopathy cases. In this study, automatic diagnosis of DR using vision transformer was performed using WF-OCTA images (12[Formula: see text]mm × 12[Formula: see text]mm single-scan) centered on the fovea as the dataset. WF-OCTA images were automatically classified into four classes: No DR, mild nonproliferative diabetic retinopathy (NPDR), moderate to severe NPDR, and proliferative diabetic retinopathy (PDR). The proposed method for detecting DR on the test set achieves accuracy of 99.55%, sensitivity of 99.49%, and specificity of 99.57%. The accuracy of the method for DR staging reaches up to 99.20%, which has been proven to be higher than that attained by classical convolutional neural network models. Results show that the automatic diagnosis of DR based on vision transformer and WF-OCTA pictures is more effective for detecting and staging DR.

A two-stage feature aggregation network for multi-category soybean leaf disease identification

Accurate identification of soybean leaf disease is of utmost importance for its cultivation and fine management, as it is a critical factor contributing to the decreased quality and yield of soybean. Nevertheless, several existing studies on the identification of soybean leaf disease encounter the challenge of striking a balance between model expressiveness and practical applicability. To address such issue, we propose a well-designed two-stage feature aggregation network framework (TFANet). The main component of it is the two-stage feature aggregation (TFA) module, which is designed to achieve great feature representation capability by aggregating feature information from different convolutional layers in two stages. Meanwhile, TFA module incorporates the efficient channel attention to extract crucial disease information. Afterwards, to limit the loss of feature information and expand the receptive field, a dilated convolution-based feature fusion (DCFF) module is developed. Additionally, the InceptionC module is introduced to further achieve better performance. The experimental results demonstrated that TFANet obtained an accuracy of 98.18% and an F1 scores of 98.39%, with merely 1.18 M parameters. In comparison with some classic convolutional neural network models, TFANet showed remarkable superiority in multiple evaluation metrics. Hence, TFANet has great potential to be used in practical soybean leaf disease identification.

Accuracy Improvement Based on Classic Neural Network: Voting, Restarting and Quantization

Since AlphaGo beat the world Go champion in 2016, which attracted wide attention, the neural network has become more and more popular in recent years, and people’s research on it has gradually improved and been applied in different fields. Today, artificial intelligence and machine learning have become an essential part of modern society and intelligent systems. We do image recognition, speech recognition, and visual learning, closely related to machine learning. However, in machine learning, unsatisfactory training results or even training failure is always encountered. Therefore, in machine learning, it is imperative to improve the accuracy of neural network training results. In this paper, speech recognition, image processing, MNIST, and other classical neural network models will be used to set the training parameters of the neural network better and improve the accuracy of training through voting, quantization, restart, and other methods. The part of research is aiming to find the relationship between restart numbers on the training process and the total extent of learning improvement. At the same time, several algorithms on utilizing these restart numbers are to be compared and selected. Finally, the conclusion is drawn that the more restart made in the training process with a convolutional neural network, the less profit on accuracy improvement we gain from restarting the process.