Deep Neural Network Classifier Research Articles

From Clustering to Cluster Explanations via Neural Networks.

A recent trend in machine learning has been to enrich learned models with the ability to explain their own predictions. The emerging field of explainable AI (XAI) has so far mainly focused on supervised learning, in particular, deep neural network classifiers. In many practical problems, however, the label information is not given and the goal is instead to discover the underlying structure of the data, for example, its clusters. While powerful methods exist for extracting the cluster structure in data, they typically do not answer the question why a certain data point has been assigned to a given cluster. We propose a new framework that can, for the first time, explain cluster assignments in terms of input features in an efficient and reliable manner. It is based on the novel insight that clustering models can be rewritten as neural networks-or "neuralized." Cluster predictions of the obtained networks can then be quickly and accurately attributed to the input features. Several showcases demonstrate the ability of our method to assess the quality of learned clusters and to extract novel insights from the analyzed data and representations.

Deep learning application to automated classification of recommendations made by hospital pharmacists during medication prescription review.

Recommendations to improve therapeutics are proposals made by pharmacists during the prescription review process to address suboptimal use of medicines. Recommendations are generated daily as text documents but are rarely reused beyond their primary use to alert prescribers and caregivers. If recommendation data were easier to summarize, they could be used retrospectively to improve safeguards for better prescribing. The objective of this work was to train a deep learning algorithm for automated recommendation classification to valorize the large amount of recommendation data. The study was conducted in a French university hospital, at which recommendation data were collected throughout 2017. Data from the first 6 months of 2017 were labeled by 2 pharmacists who assigned recommendations to 1 of the 29 possible classes of the French Society of Clinical Pharmacy classification. A deep neural network classifier was trained to predict the class of recommendations. In total, 27,699 labeled recommendations from the first half of 2017 were used to train and evaluate a classifier. The prediction accuracy calculated on a validation dataset was 78.0%. We also predicted classes for unlabeled recommendations collected during the second half of 2017. Of the 4,460 predictions reviewed, 67 required correction. When these additional labeled data were concatenated with the original dataset and the neural network was retrained, accuracy reached 81.0%. To facilitate analysis of recommendations, we have implemented an automated classification system using deep learning that achieves respectable performance. This tool can help to retrospectively highlight the clinical significance of daily medication reviews performed by hospital clinical pharmacists.

Design a novel hybrid optimization with tuned deep convolutional neural network classifier for brain tumor segmentation and classification

Design a novel hybrid optimization with tuned deep convolutional neural network classifier for brain tumor segmentation and classification

Likelihood-based feature representation learning combined with neighborhood information for predicting circRNA-miRNA associations.

Connections between circular RNAs (circRNAs) and microRNAs (miRNAs) assume a pivotal position in the onset, evolution, diagnosis and treatment of diseases and tumors. Selecting the most potential circRNA-related miRNAs and taking advantage of them as the biological markers or drug targets could be conducive to dealing with complex human diseases through preventive strategies, diagnostic procedures and therapeutic approaches. Compared to traditional biological experiments, leveraging computational models to integrate diverse biological data in order to infer potential associations proves to be a more efficient and cost-effective approach. This paper developed a model of Convolutional Autoencoder for CircRNA-MiRNA Associations (CA-CMA) prediction. Initially, this model merged the natural language characteristics of the circRNA and miRNA sequence with the features of circRNA-miRNA interactions. Subsequently, it utilized all circRNA-miRNA pairs to construct a molecular association network, which was then fine-tuned by labeled samples to optimize the network parameters. Finally, the prediction outcome is obtained by utilizing the deep neural networks classifier. This model innovatively combines the likelihood objective that preserves the neighborhood through optimization, to learn the continuous feature representation of words and preserve the spatial information of two-dimensional signals. During the process of 5-fold cross-validation, CA-CMA exhibited exceptional performance compared to numerous prior computational approaches, as evidenced by its mean area under the receiver operating characteristic curve of 0.9138 and a minimal SD of 0.0024. Furthermore, recent literature has confirmed the accuracy of 25 out of the top 30 circRNA-miRNA pairs identified with the highest CA-CMA scores during case studies. The results of these experiments highlight the robustness and versatility of our model.

A bearing fault diagnosis method with an improved residual Unet diffusion model under extreme data imbalance

As a vital constituent of rotating machinery, rolling bearings assume a pivotal function in ensuring the stable operation of equipment. Recently, deep learning (DL)-based methods have been able to diagnose bearing faults accurately. However, in practical applications, the severe data imbalance caused by the limited availability of fault data compared to the abundance of healthy data poses challenges to the effective training of DL models, leading to a decrease in diagnostic accuracy. In this paper, a bearing fault diagnosis method with the improved residual Unet diffusion model (IResUnet-DM) based on a data generation strategy is proposed to solve the extreme data imbalance. Initially, a deep feature extraction network named improved residual Unet is built, which effectively enhances the information learning ability from vibration signals of the Unet network by one-dimensional residual block and self-attention block. Furthermore, the IResUnet-DM is constructed, which generates vibration signals under extreme data imbalance based on a probability model. The variational bound on the negative log-likelihood of the distribution of generated data was optimized to make the generated data similar to the real data distribution. Finally, wide deep convolutional neural network and one-dimensional ResNet classification networks were used for fault identification to verify the validity and generalization of the IResUnet-DM. Experiment results at different data imbalance rates on two bearing datasets demonstrate that the proposed method can effectively improve fault diagnosis accuracy under extreme data imbalances and outperform the comparison method.

Method of Improving the Management of Cancer Risk Groups by Coupling a Features-Attention Mechanism to a Deep Neural Network

(1) Background: Lung cancers are the most common cancers worldwide, and prostate cancers are among the second in terms of the frequency of cancers diagnosed in men. Automatic ranking of the risk groups of such diseases is highly in demand, but the clinical practice has shown us that, for a sensitive screening of the clinical parameters using an artificial intelligence system, a customarily defined deep neural network classifier is not sufficient given the usually small size of medical datasets. (2) Methods: In this paper, we propose a new management method of cancer risk groups based on a supervised neural network model that is further enhanced by using a features attention mechanism in order to boost its level of accuracy. For the analysis of each clinical parameter, we used local interpretable model-agnostic explanations, which is a post hoc model-agnostic technique that outlines feature importance. After that, we applied the feature-attention mechanism in order to obtain a higher weight after training. We tested the method on two datasets, one for binary-class in cases of thoracic cancer and one for multi-class classification in cases of urological cancer, to demonstrate the wide availability and versatility of the method. (3) Results: The accuracy levels of the models trained in this way reached values of more than 80% for both clinical tasks. (4) Conclusions: Our experiments demonstrate that, by using explainability results as feedback signals in conjunction with the attention mechanism, we were able to increase the accuracy of the base model by more than 20% on small medical datasets, reaching a critical threshold for providing recommendations based on the collected clinical parameters.

LMHistNet: Levenberg–Marquardt Based Deep Neural Network for Classification of Breast Cancer Histopathological Images

LMHistNet: Levenberg–Marquardt Based Deep Neural Network for Classification of Breast Cancer Histopathological Images

Effective IoT-based crop disease prediction using localise search traversing coupled with deep convolutional neural network classifier

Effective IoT-based crop disease prediction using localise search traversing coupled with deep convolutional neural network classifier

Towards Hardware Supported Domain Generalization in DNN-based Edge Computing Devices for Health Monitoring.

Deep neural network (DNN) models have shown remarkable success in many real-world scenarios, such as object detection and classification. Unfortunately, these models are not yet widely adopted in health monitoring due to exceptionally high requirements for model robustness and deployment in highly resource-constrained devices. In particular, the acquisition of biosignals, such as electrocardiogram (ECG), is subject to large variations between training and deployment, necessitating domain generalization (DG) for robust classification quality across sensors and patients. The continuous monitoring of ECG also requires the execution of DNN models in convenient wearable devices, which is achieved by specialized ECG accelerators with small form factor and ultra-low power consumption. However, combining DG capabilities with ECG accelerators remains a challenge. This article provides a comprehensive overview of ECG accelerators and DG methods and discusses the implication of the combination of both domains, such that multi-domain ECG monitoring is enabled with emerging algorithm-hardware co-optimized systems. Within this context, an approach based on correction layers is proposed to deploy DG capabilities on the edge. Here, the DNN fine-tuning for unknown domains is limited to a single layer, while the remaining DNN model remains unmodified. Thus, computational complexity (CC) for DG is reduced with minimal memory overhead compared to conventional fine-tuning of the whole DNN model. The DNN model-dependent CC is reduced by more than 2.5 × compared to DNN fine-tuning at an average increase of F1 score by more than 20% on the generalized target domain. In summary, this article provides a novel perspective on robust DNN classification on the edge for health monitoring applications.

Research on Adversarial Sample Detection Method Based on Image Similarity

<p>With the widespread application of deep neural networks in image detection, adversarial sample attacks have gradually become a hot issue of concern for researchers. In this paper we propose a new adversarial sample detection approach called AdvDetector, which combines image generation through label fusion with image similarity detection. AdvDetector enhances sample quality and effectively identifies adversarial samples. Specifically, the method generates images by selecting seed pixels, the labels of deep neural network classification, and the pixel distribution learned from training data, and detects them using image similarity comparison methods. During the sample generation process, we introduce the AdvDetector method for adversarial sample detection to improve the quality of generated samples. We evaluated the effectiveness of the method on three publicly available image datasets, MNIST, Cifar-10, and GTSR, and the results show that the method is superior to existing baseline methods in terms of adversarial sample detection rate and sample generation quality.</p> <p> </p>

Towards interpretable learned representations for ecoacoustics using variational auto-encoding

Ecoacoustics is an emerging science that seeks to understand the role of sound in ecological processes. Passive acoustic monitoring is being used to collect vast quantities of soundscape audio recordings to study variations in acoustic community and monitor biodiversity. However, extracting relevant information from soundscape recordings is non-trivial. Recent approaches to machine-learned acoustic features appear promising but are limited by at least three issues: inductive biases, lack of interpretability and crude temporal integration. In this paper we introduce a novel self-supervised representation learning algorithm for ecoacoustics - a convolutional Variational Auto-encoder (VAE) - and directly address these shortcomings. Firstly, we train the network on soundscape recordings from temperate and tropical field sites along a gradient of ecological degradation to provide a more relevant inductive bias than prior approaches. Secondly, we present a new method that allows interpretation of the latent space for the first time, giving insight into the basis of classification. Thirdly, we advance existing methods for temporal aggregation of learned embeddings by encoding latent features as a distribution over time. Under our approach to increase interpretability, we provide insight into how learned features drive habitat classification for the first time: inspection of latent space confirms that varying combinations of biophony, geophony and anthrophony are used to infer sites along a degradation gradient. Our novel temporal encoding method increases sensitivity to periodic signals and improves on previous research that uses time-averaged representations for site classification. This approach also reveals the contribution of hardware-specific frequency response that create a potential bias; we demonstrate how a simple linear transformation can be used to mitigate the effect of hardware variance on the learned representation under our approach. Our novel approach paves the way for development of a new class of deep neural networks that afford more interpretable learned ecoacoustic representations to advance both fundamental and applied science and support global conservation efforts.

An Integrated Active Deep Learning Approach for Image Classification from Unlabeled Data with Minimal Supervision

The integration of active learning (AL) and deep learning (DL) presents a promising avenue for enhancing the efficiency and performance of deep learning classifiers. This article introduces an approach that seamlessly integrates AL principles into the training process of DL models to build robust image classifiers. The proposed approach employs a unique methodology to select high-confidence unlabeled data points for immediate labeling, reducing the need for human annotation and minimizing annotation costs. Specifically, by combining uncertainty sampling with the pseudo-labeling of confident data, the proposed approach expands the training set efficiently. The proposed approach uses a hybrid active deep learning model that selects the most informative data points that need labeling based on an uncertainty measure. Then, it iteratively retrains a deep neural network classifier on the newly labeled samples. The model achieves high accuracy with fewer manually labeled samples than traditional supervised deep learning by selecting the most informative samples for labeling and retraining in a loop. Experiments on various image classification datasets demonstrate that the proposed model outperforms conventional approaches in terms of classification accuracy and reduced human annotation requirements. The proposed model achieved accuracy of 98.9% and 99.3% for the Cross-Age Celebrity and Caltech Image datasets compared to the conventional approach, which achieved 92.3% and 74.3%, respectively. In summary, this work presents a promising unified active deep learning approach to minimize the human effort in manually labeling data while maximizing classification accuracy by strategically labeling only the most valuable samples for the model.

Efficient machine learning model to detect early stage Parkinson’s disease

<p class="root-block-node">Parkinson’s disease (PD) often manifests itself in memory loss and cognitive decline. The decline is inexorable, and damage to the brain’s cortex has already occurred. Numerous studies have shown that by detecting dementia early and beginning treatment, the disease’s course can be slowed, and any further atrophy can be prevented. Brain imaging data, such as from an MRI, is frequently used in the diagnosis of Parkinson’s disease (PD). In recent years, utilizing deep convolutional neural networks has greatly improved Parkinson’s disease diagnosis. However, getting to the level of quality needed for clinical use is still challenging. In this study, we introduce a machine learning-based approach for more accurately diagnosing Parkinson’s disease. This research makes use of information gleaned from single-photon emission computerized tomography (SPECT) scan and positron emission tomography (PET) scans performed on patients with Parkinson’s disease (PD) and healthy controls. The most crucial characteristics of these datasets are isolated with the aid of the Fisher discriminate ratio (FDR) and non-negative matrix factorization (NMF). The K-nearest neighbor, Decision Tree, Support vector machine (SVM), and Deep Convolution neural network (DCCN) classifiers with confidence bounds classify the NMF-transformed data sets with a decreased number of features. The proposed DCCN technique has a classification accuracy of up to 93.7 percent when compared to decision trees, K-Nearest Neighbor (KNN)s, and SVMs. The DCCN is now a reliable approach for classifying SPECT and PET, PD images.</p>

The impact of gut microbiome enterotypes on ulcerative colitis: identifying key bacterial species and revealing species co-occurrence networks using machine learning

ABSTRACT Ulcerative colitis (UC) is a chronic inflammatory intestinal disease affecting the colon and rectum, with its pathogenesis attributed to genetic background, environmental factors, and gut microbes. This study aimed to investigate the role of enterotypes in UC by conducting a hierarchical analysis, determining differential bacteria using machine learning, and performing Species Co-occurrence Network (SCN) analysis. Fecal bacterial data were collected from UC patients, and a 16S rRNA metagenomic analysis was performed using the QIIME2 bioinformatics pipeline. Enterotype clustering was conducted at the family level, and deep neural network (DNN) classification models were trained for UC and healthy controls (HC) in each enterotype. Results from eleven 16S rRNA gut microbiome datasets revealed three enterotypes: Bacteroidaceae (ET-B), Lachnospiraceae (ET-L), and Clostridiaceae (ET-C). Ruminococcus (R. gnavus) abundance was significantly higher in UC subjects with ET-B and ET-C than in those with ET-L. R. gnavus also showed a positive correlation with Clostridia in UC SCN for ET-B and ET-C subjects, with a higher correlation in ET-C subjects. Conversely, Odoribacter (O.) splanchnicus and Bacteroides (B.) uniformis exhibited a positive correlation with tryptophan metabolism and AMP-activated protein kinase (AMPK) signaling pathways, while R. gnavus showed a negative correlation. In vitro co-culture experiments with Clostridium (C.) difficile demonstrated that fecal microbiota from ET-B subjects had a higher abundance of C. difficile than ET-L subjects. In conclusion, the ET-B enterotype predisposes individuals to UC, with R. gnavus as a potential risk factor and O. splanchnicus and B. uniformis as protective bacteria, and those with UC may have ultimately become ET-C.

Description of tea quality using deep learning and multi-sensor feature fusion

To address the problem of the lack of rapid discrimination technology for comprehensive evaluation of product quality and grade in the field of tea quality control, this study uses near-infrared spectroscopy, electronic eye, electronic tongue, and electronic nose technology to sense the external and internal quality sensing characteristics of tea leaves, and realizes comprehensive evaluation of four quality factors such as "color, aroma, taste and shape" of tea leaves through cross-sensor multimodal quality characteristics data fusion. For the first time, a comprehensive method for assessing the grade quality of tea based on deep learning algorithms combining cross-sensor multimodal fusion features such as color, texture, shape, optical, electronic tongue, and electronic nose is established. The results showed that the constructed deep convolutional neural network (CNN) classification model had a misclassification rate of only 0.86% in the prediction set, which was better than the modeling results based on traditional classification methods. This study reveals that the CNN algorithm can effectively evaluate the quality grade of tea samples and realize the digital evaluation of multiple evaluation factors of tea sensory quality.

A Cryptographic based I2ADO-DNN Security Framework for Intrusion Detection in Cloud Systems

Cloud computing's popularity and success are directly related to improvements in the use of Information and Communication Technologies (ICT). The adoption of cloud implementation and services has become crucial due to security and privacy concerns raised by outsourcing data and business applications to the cloud or a third party. To protect the confidentiality and security of cloud networks, a variety of Intrusion Detection System (IDS) frameworks have been developed in the conventional works. However, the main issues with the current works are their lengthy nature, difficulty in intrusion detection, over-fitting, high error rate, and false alarm rates. As a result, the proposed study attempts to create a compact IDS architecture based on cryptography for cloud security. Here, the balanced and normalized dataset is produced using the z-score preprocessing procedure. The best attributes for enhancing intrusion detection accuracy are then selected using an Intelligent Adorn Dragonfly Optimization (IADO). In addition, the trained features are used to classify the normal and attacking data using an Intermittent Deep Neural Network (IDNN) classification model. Finally, the Searchable Encryption (SE) mechanism is applied to ensure the security of cloud data against intruders. In this study, a thorough analysis has been conducted utilizing various parameters to validate the intrusion detection performance of the proposed I2ADO-DNN model.

Differentiable matrix product states for simulating variational quantum computational chemistry

Quantum Computing is believed to be the ultimate solution for quantum chemistry problems. Before the advent of large-scale, fully fault-tolerant quantum computers, the variational quantum eigensolver (VQE) is a promising heuristic quantum algorithm to solve real world quantum chemistry problems on near-term noisy quantum computers. Here we propose a highly parallelizable classical simulator for VQE based on the matrix product state representation of quantum state, which significantly extend the simulation range of the existing simulators. Our simulator seamlessly integrates the quantum circuit evolution into the classical auto-differentiation framework, thus the gradients could be computed efficiently similar to the classical deep neural network, with a scaling that is independent of the number of variational parameters. As applications, we use our simulator to study commonly used small molecules such as HF, HCl, LiH and H2O, as well as larger molecules CO2, BeH2 and H4 with up to 40 qubits. The favorable scaling of our simulator against the number of qubits and the number of parameters could make it an ideal testing ground for near-term quantum algorithms and a perfect benchmarking baseline for oncoming large scale VQE experiments on noisy quantum computers.

Differentiation of frontotemporal dementia subtypes using neuroimaging‐based multi‐type parallel feature embedding

AbstractBackgroundFrontotemporal dementia (FTD) represents a collection of neurocognitive syndromes with frontotemporal lobar degeneration (FTLD) neuropathology, and is associated with significant clinical, pathological, and genetic heterogeneity. We trained a deep neural network (DNN) classifier to differentiate behavioral‐variant FTD (bvFTD), semantic variant primary progressive aphasia (svPPA), and non‐fluent variant (nfvPPA) patients using MRI scans drawn from two multi‐site neuroimaging consortiums.MethodBvFTD (N = 173), nfvPPA (N = 63), and svPPA (N = 41) patients with T1‐MRI were extracted from the FTLD Neuroimaging Initiative (FTLDNI) and the ARTFL‐LEFFTDS Longitudinal Frontotemporal Lobar Degeneration (ALLFTD) databases. MRI data were preprocessed with FreeSurfer, with cortical thickness, cortical volume, and subcortical volumes extracted. Cortical measures were parcellated into 360 patches (i.e., ROIs) according to the HCP‐MMP1 atlas. Both the patch‐based cortical thickness and volume features were harmonized to control confounding effects of sex, age, total intracranial volume (TIV), cohort, and scanner. Multi‐type features were parallelly fed into a multi‐layer‐perceptron (MLP)‐based classifier. Weighted cross‐entropy loss function was used to account for unbalanced sample sizes across FTD subtypes. 10‐fold nested cross‐validation was used to evaluate the robustness of the classification model, with data split into 80/10/10 of training/validation/test data sets.ResultVisual evaluation z‐scores of cortical features among FTLDNI and ALLFTD groups revealed site‐specific differences significantly reduced by feature harmonization, especially for volume (Figure 1). The balanced accuracy of the ensembled DNN classifier of each FTD subtype on test sets across all ten folds reached 0.76 ± 0.09 for bvFTD, 0.79 ± 0.07 for nfvPPA, and 0.88 ± 0.07 for svPPA (Figure 2). Figure 3 shows confusion matrices for classification performance in the test set across ten cross‐validation folds.ConclusionWe developed a deep‐learning‐based framework to classify three FTD subtypes: bvFTD, nfvPPA, and svPPA. The combination of feature harmonization and parallel multi‐type feature embedding framework showed promising differentiation power. This work can be used to recognize at‐risk populations for early and precise diagnosis, to aid intervention planning. Future studies will compare classification based on different input features and use visualization methods to identify the most discriminative regions and explore their clinical relevance.

A Deep Stacked Ensemble Model for Microarray Data Classification with Boosted Meta Classifier

Classification of microarray data is one of the major research interests in the biomedical field. It allows physicians for early detection of cancer through analysis of the Deoxyribonucleic acid (DNA). Classification of these sensitive data is still challenging due to the small sample and more feature size. In this paper, the authors have used an ensemble model for classifying two types of leukemia as Acute lymphocytic leukemia (ALL) and Acute myelocytic leukemia (AML). The work is carried out with other types of genetic data such as Leukemia, lung tumor, liver cancer, and liver Cirrhosis. The biomedical data are imbalanced. The ensemble classifier is based on a stacked approach where deep neural network (DNN) classifiers are used as the base classifier. The structure of each DNN is chosen as the homogenous type for the same training process for all classifiers. Because of the adaptive nature and random weight initialization, it provides different results for each classifier. The outputs of the base classifiers are again fed to a gradient boosting ensemble model termed a meta classifier. The meta classifier provides the final classification output. For comparison purposes, two types of meta-classifiers such as support vector machine (SVM) and ensemble gradient boosting are used in the proposed work. The performance of the model is verified well and the results are provided in the result section. From the experimental result, it is observed that the classification accuracy is 96% with SVM and 98% with boosted meta classifier for leukemia data, whereas 99.04% for lung tumor and 99.03% for liver Cirrhosis.

Hybrid Sneaky algorithm-based deep neural networks for Heart sound classification using phonocardiogram

ABSTRACT In the diagnosis of cardiac disorders Heart sound has a major role, and early detection is crucial to safeguard the patients. Computerized strategies of heart sound classification advocate intensive and more exact results in a quick and better manner. Using a hybrid optimization-controlled deep learning strategy this paper proposed an automatic heart sound classification module. The parameter tuning of the Deep Neural Network (DNN) classifier in a satisfactory manner is the importance of this research which depends on the Hybrid Sneaky optimization algorithm. The developed sneaky optimization algorithm inherits the traits of questing and societal search agents. Moreover, input data from the Phonocardiogram (PCG) database undergoes the process of feature extraction which extract the important features, like statistical, Heart Rate Variability (HRV), and to enhance the performance of this model, the features of Mel frequency Cepstral coefficients (MFCC) are assisted. The developed Sneaky optimization-based DNN classifier’s performance is determined in respect of the metrics, namely precision, accuracy, specificity, and sensitivity, which are around 97%, 96.98%, 97%, and 96.9%, respectively.