Deep Classification Research Articles

A novel random fast multi-label deep forest classification algorithm

With the development of big data, a large amount of multi-label data has emerged in various fields. The use of deep classification models to solve multi-label classification problems is the current research hotspot. However, the existing deep classification algorithms still face two great challenges in solving the multi-label classification problem: the poor multi-label classification accuracy and the high time cost. In order to solve the above challenges, this paper proposes a novel random fast multi-label deep forest classification algorithm (RFDF). Firstly, the label-by-label approach is adopted for the training and prediction of base classifiers, while the multi-label cascade model is constructed, so as to obtain better multi-label classification results. Then, an adaptive method is proposed to determine the number of base classifiers, and a random features partitioning strategy is provided to generate the initial base classifier, achieving parallel training of the base classifier. Finally, the proposed RFDF is compared with the existing advanced algorithms, and the experimental results show that the RFDF achieves better results on most of the metrics of the ten datasets, and obtains lower time cost compared to VDSDF and better stability.

Libby-Novick Beta-Liouville Distribution for Enhanced Anomaly Detection in Proportional Data

We consider the problem of anomaly detection in proportional data by investigating the Libby-Novick Beta-Liouville distribution, a novel distribution merging the salient characteristics of Liouville and Libby-Novick Beta distributions. Its main benefit, compared to the typical distributions dedicated to proportional data such as Dirichlet and Beta-Liouville, is its adaptability and explanatory power when dealing with this kind of data. Our goal is to exploit this appropriateness for modeling proportional data to achieve great performance in the anomaly detection task. First, we develop generative models, namely finite mixture models of Libby-Novick Beta-Liouville distributions. Then, we propose two discriminative techniques: Normality scores based on selecting the given distribution to approximate the softmax output vector of a deep classifier and an improved version of Support Vector Machine (SVM) by suggesting a feature mapping approach. We demonstrate the benefits of the presented approaches through a variety of experiments on both image and non-image datasets. The results demonstrate that the proposed anomaly detectors based on the Libby-Novick Beta-Liouville distribution outperform the classical distributions as well as the baseline techniques.

Automated Classification of Coronary Plaque on Intravascular Ultrasound by Deep Classifier Cascades.

Intravascular ultrasound (IVUS) is the gold standard modality for in vivo visualization of coronary arteries and atherosclerotic plaques. Classification of coronary plaques helps to characterize heterogeneous components and evaluate the risk of plaque rupture. Manual classification is time-consuming and labor-intensive. Several machine learning-based classification approaches have been proposed and evaluated in recent years. In the current study, we develop a novel pipeline composed of serial classifiers for distinguishing IVUS images into five categories: normal, calcified plaque, attenuated plaque, fibrous plaque, and echolucent plaque. The cascades comprise densely connected classification models and machine learning classifiers at different stages. Over 100,000 IVUS frames of five different lesion types were collected and labeled from 471 patients for model training and evaluation. The overall accuracy of the proposed classifier is 0.877, indicating that the proposed framework has the capacity to identify the nature and category of coronary plaques in IVUS images. Further, it may provide real-time assistance on plaque identification and facilitate clinical decision-making in routine practice.

Independent Vector Analysis for Feature Extraction in Motor Imagery Classification.

Independent vector analysis (IVA) can be viewed as an extension of independent component analysis (ICA) to multiple datasets. It exploits the statistical dependency between different datasets through mutual information. In the context of motor imagery classification based on electroencephalogram (EEG) signals for the brain-computer interface (BCI), several methods have been proposed to extract features efficiently, mainly based on common spatial patterns, filter banks, and deep learning. However, most methods use only one dataset at a time, which may not be sufficient for dealing with a multi-source retrieving problem in certain scenarios. From this perspective, this paper proposes an original approach for feature extraction through multiple datasets based on IVA to improve the classification of EEG-based motor imagery movements. The IVA components were used as features to classify imagined movements using consolidated classifiers (support vector machines and K-nearest neighbors) and deep classifiers (EEGNet and EEGInception). The results show an interesting performance concerning the clustering of MI-based BCI patients, and the proposed method reached an average accuracy of 86.7%.

Correction: IoT based smart agri system: deep classifiers for black gram disease classification with modified feature set

Correction: IoT based smart agri system: deep classifiers for black gram disease classification with modified feature set

Detection and comparison of reversible shape transformations in responsive polymers using deep learning and knowledge transfer by identifying stimulus-triggering characteristic points

Responsive polymers can alter their properties or shape according to stimuli such as stress, light, among others. The stimuli may be strategically programmed at assigned triggering points in a structure using additive manufacturing, which enables controlled deformation that is crucial for reversible shape transformation. Understanding and controlling shape transformation holds significance in topology optimization, or generative design, among other applications, where a material may not be reduced by machining; it can instead be reversibly transformed to perform evolving functions. However, handling responsive polymers can be challenging, as they react to various stimuli. The present research introduces a contactless technique using deep classification to detect and measure shape changes in responsive polymers. The application of deep network in responsive structural element (characteristic points) detection has not been attempted before. The classification categorizes polymers according to the degree of shape change, which aids in the verification of selectively programmed stimulus-responsive properties. By correlating the output with trained data, accurate classifications were made regarding the degree of shape change. The shape changes were then compared with different deformation angles, lengths, or heights through a novel formula-based correlation. The classification helps reconfigure the material if the deformation is not within the desired range. The classification and comparison were performed with AlexNet, which possesses a simple architecture. An accuracy of 100% was achieved by fine-tuning the hyperparameters of the same network. The present work thus shows the possibility of combining multiple tasks more efficiently with less data and computational resources using simple network.

A Rapid Detection Method for Coal Ash Content in Tailings Suspension Based on Absorption Spectra and Deep Feature Extraction

Traditional visual detection methods that employ image data are often unstable due to environmental influences like lighting conditions. However, microfiber spectrometers are capable of capturing the specific wavelength characteristics of tail coal suspensions, effectively circumventing the instability caused by lighting variations. Utilizing spectral analysis techniques for detecting ash content in tail coal appears promising as a more stable method of indirect ash detection. In this context, this paper proposes a rapid detection method for the coal ash content in tailings suspensions based on absorption spectra and deep feature extraction. Initially, a preprocessing method, the inverse time weight function (ITWF), is presented, focusing on the intrinsic connection between the sedimentation phenomena of samples. This enables the model to learn and retain spectral time memory features, thereby enhancing its analytical capabilities. To better capture the spectral characteristics of tail coal suspensions, we designed the DSFN (DeepSpectraFusionNet) model. This model has an MSCR (multi-scale convolutional residual) module, addressing the conventional models’ oversight of the strong correlation between adjacent wavelengths in the spectrum. This facilitates the extraction of relative positional information. Additionally, to uncover potential temporal relationships in sedimentation, we propose a CLSM-CS (convolutional long-short memory with candidate states) module, designed to strengthen the capturing of local information and sequential memory. Ultimately, the method employs a fused convolutional deep classifier to integrate and reconstruct both temporal memory and positional features. This results in a model that effectively correlates the ash content of suspensions with their absorption spectral characteristics. Experimental results confirmed that the proposed model achieved an accuracy of 80.65%, an F1-score of 80.45%, a precision of 83.43%, and a recall of 80.65%. These results outperformed recent coal recognition models and classical temporal models, meeting the high standards required for industrial on-site ash detection tasks.

IoT based smart agri system: deep classifiers for black gram disease classification with modified feature set

IoT based smart agri system: deep classifiers for black gram disease classification with modified feature set

A novel robust black-box fingerprinting scheme for deep classification neural networks

With the growing value and high training cost of deep neural network (DNN) models, there has been increasing research interest in deep neural network model watermarking. However, the existing literature lacks robust model watermarking methods with retrospective functionality, which hinders multi-user distribution and fine-grained copyright verification of deep neural network models. To achieve the model watermarking with retrospective functionality that is generally denoted the model fingerprinting, this study proposes a novel robust black-box fingerprinting scheme for deep classification neural networks, in which a fingerprint that is unique for each user is inserted in the given DNN. Specifically, a DWT-DCT-SVD-based embedding method is introduced to construct a high-quality poisoned image by spreading the fingerprint as a poisoned trigger to the entire image, which both leads to the traceability of the to-be-protected DNN and significantly enhances the stealthiness and anti-forgery of the embedded poisoned trigger. As the amplitude of the embedded fingerprint is rather small, it poses a challenge that makes the model hard to capture features of the embedded fingerprint during the training process. To tackle this challenge, this study then proposes a poisoned feature enhancement module, which groups classification categories of clean images without the fingerprint as one type while taking the poisoned images with the correct fingerprint and the poisoned adversarial images containing the other incorrect fingerprints as another two different types. This module, which can be removed during fingerprint verification, improves the reliability and fidelity of the model fingerprinting. Accordingly, an adversarial training strategy is designed, aiming at enhancing the effective training of classification neural network models that contain the fingerprint with small embedding strength. Combining these strategies then gives rise to the proposed scheme, which is denoted the UfNet for notational convenience. Extensive experimental simulation results demonstrate the excellent invisibility of the fingerprint in the poisoned image and the desirable verification of the fingerprint in case of even a one-bit difference. Additionally, it also exhibits robustness against the related attacks such as STRIP, Neural Cleanse, Fine-Pruning, and GradCAM, which outperforms the state-of-the-arts (SOTAs). Furthermore, the proposed UfNet is robust to collusion attacks and frame the innocent. These results show the feasibility and effectiveness of the introduced model fingerprinting framework and the proposed implementation methodology in this paper.

Electron ionization mass spectrometry feature peak relationships combined with deep classification model to assist similarity algorithm for fast and accurate identification of compounds.

Gas chromatography-mass spectrometry (GC-MS) combines chromatography and MS, providing full play to the advantages of high separation efficiency of GC, strong qualitative ability of MS, and high sensitivity of detector. In GC-MS data processing, determining the experimental compounds is one of the most important analytical steps, which is usually realized by one-to-one similarity calculations between the experimental mass spectrum and the standard mass spectrum library. Although the accuracy of the algorithm has been improved in recent years, it is still difficult to distinguish structurally similar mass spectra, especially isomers. At the same time, the library capacity is very large and increasing every year, and the algorithm needs to perform large numbers of calculations with irrelevant compounds in the library to recognize unknown compounds, which leads to a significant reduction in efficiency. This work proposed to exclude a large number of irrelevant mass spectra by presearching, perform preliminary similarity calculations using similarity algorithms, and finally improve the accuracy of similarity calculations using deep classification models. The replica library of NIST17 is used as the query data, and the master library is used as the reference database. Compared with the traditional recognition algorithm, the preprocessing algorithm has reduced the time by 4.2h, and by adding the deep learning models 1 and 2 as the final determination, the recognition accuracy has been improved by 1.9% and 6.5%, respectively, based on the original algorithm. This method improves the recognition efficiency compared to conventional algorithms and at the same time has better recognition accuracy for structurally similar mass spectra and isomers.

Customized m-RCNN and hybrid deep classifier for liver cancer segmentation and classification

Diagnosing liver disease presents a significant medical challenge in impoverished countries, with over 30 billion individuals succumbing to it each year. Existing models for detecting liver abnormalities suffer from lower accuracy and higher constraint metrics. As a result, there is a pressing need for improved, efficient, and effective liver disease detection methods. To address the limitations of current models, this method introduces a deep liver segmentation and classification system based on a Customized Mask-Region Convolutional Neural Network (cm-RCNN). The process begins with preprocessing the input liver image, which includes Adaptive Histogram Equalization (AHE). AHE helps dehaze the input image, remove color distortion, and apply linear transformations to obtain the preprocessed image. Next, a precise region of interest is segmented from the preprocessed image using a novel deep strategy called cm-RCNN. To enhance segmentation accuracy, the architecture incorporates the ReLU activation function and the modified sigmoid activation function. Subsequently, a variety of features are extracted from the segmented image, including ResNet features, shape features (area, perimeter, approximation, and convex hull), and enhanced median binary pattern. These extracted features are then used to train a hybrid classification model, which incorporates classifiers like SqueezeNet and DeepMaxout models. The final classification outcome is determined by averaging the scores obtained from both classifiers.

Ensemble activation enabled deep classifier for Alzheimer's disease detection in the blockchain-enabled distributed edge environment

Ensemble activation enabled deep classifier for Alzheimer's disease detection in the blockchain-enabled distributed edge environment

Electrocardiogram-based heart disease prediction using hybrid deep feature engineering with sequential deep classifier

Electrocardiogram-based heart disease prediction using hybrid deep feature engineering with sequential deep classifier

Q-SupCon: Quantum-Enhanced Supervised Contrastive Learning Architecture within the Representation Learning Framework

In the evolving landscape of data privacy regulations, the challenge of providing extensive data for robust deep classification models arises. The accuracy of these models relies on the amount of training data, due to the multitude of parameters that require tuning. Unfortunately, obtaining such ample data proves challenging, particularly in domains like medical applications, where there is a pressing need for robust models for early disease detection but a shortage of labeled data. Nevertheless, the classical supervised contrastive learning models, have shown the potential to address this challenge up to a certain limit, by utilizing deep encoder models. However, recent advancements in quantum machine learning enable the extraction of meaningful representations from extremely limited and simple data. Thus, replacing classical counterparts in classical or hybrid quantum-classical supervised contrastive models enhances feature learning capability with minimal data. Therefore, this work proposes the Q-SupCon model, a fully quantum-powered supervised contrastive learning model comprising a quantum data augmentation circuit, quantum encoder, quantum projection head, and quantum variational classifier, enabling efficient image classification with minimal labeled data. Furthermore, the novel model attains 80%, 60%, and 80% test accuracy on MNIST, KMNIST, and FMNIST datasets, marking a significant advancement in addressing the data scarcity challenge.

Synthetic data augmentation for high-resolution X-ray welding defect detection and classification based on a small number of real samples

Deep learning has become the dominant technology in most computer vision tasks. These methods often rely on a large number of labeled sample datasets for training, and in the field of non-destructive testing of welds in industrial manufacturing, weld images with defects are very scarce, and it is still a challenging challenge to construct high-resolution weld defect datasets that meet the requirements. To overcome this limitation, a new data augmentation method for high-resolution X-ray welding defect classification and synthesis based on a small number of real samples is proposed to realize the data augmentation of industrial nondestructive inspection X-ray film defect images. Firstly, to overcome the scarcity of the weld X-ray defect classification dataset, the weld X-ray defect classification dataset (Weld Defect Classification, WDC) is constructed. Secondly, the performance of 16 common deep classification models on WDC datasets is explored. Then, the images of the real local welding defects and the non-defective weld area are fused at random locations, and two data augmentation modes, (Single Image Single Defect, SISD) and (Single Image Multi Defects, SIMD), can generate defect files and annotation files (Visual Object Classes, VOC) at the same time, which can save a lot of time for manual marking. Finally, compared with the traditional data augmentation method, the proposed method can effectively improve the accuracy of defect detection and generalization, the mAP (Mean Average Precision, mAP) @0.5 of YOLOV8X (You Only Look Once, YOLO) and YOLOV5.6.1X is 66.6% and 72.8%, which provides an effective solution for data sample generation in the industrial field.

HayCAMJ: A new method to uncover the importance of main filter for small objects in explainable artificial intelligence

Visual XAI methods enable experts to reveal importance maps highlighting intended classes over input images. This research paper presents a novel approach to visual explainable artificial intelligence (XAI) for object detection in deep learning models. The study investigates the effectiveness of activation maps generated by five different methods, namely GradCAM, GradCAM++, EigenCAM, HayCAM, and a newly proposed method called "HayCAMJ", in detecting objects within images. The experiments were conducted on two datasets (Pascal VOC 2007 and Pascal VOC 2012) and three models (ResNet18, ResNet34, and MobileNet). Zero padding was applied to resize and center the objects due to the large objects in the images. The results show that HayCAMJ performs better than other XAI techniques in detecting small objects. This finding suggests that HayCAMJ has the potential to become a promising new approach for object detection in deep classification models.

A Unique Identification-Oriented Black-Box Watermarking Scheme for Deep Classification Neural Networks

Given the substantial value and considerable training costs associated with deep neural network models, the field of deep neural network model watermarking has come to the forefront. While black-box model watermarking has made commendable strides, the current methodology for constructing poisoned images in the existing literature is simplistic and susceptible to forgery. Notably, there is a scarcity of black-box model watermarking techniques capable of discerning a unique user in a multi-user model distribution setting. For this reason, this paper proposes a novel black-box model watermarking method for unique identity identification, which is denoted as the ID watermarking of neural networks (IDwNet). Specifically, to enhance the distinguishability of deep neural network models in multi-user scenarios and mitigate the likelihood of poisoned image counterfeiting, this study develops a discrete cosine transform (DCT) and singular value decomposition (SVD)-based symmetrical embedding method to form the poisoned image. As this ID embedding method leads to indistinguishable deep features, the study constructs a poisoned adversary training strategy by simultaneously inputting clean images, poisoned images with the correct ID, and poisoned adversary images with incorrect IDs to train a deep neural network. Extensive simulation experiments show that the proposed scheme achieves excellent invisibility for the concealed ID, surpassing remarkably the state-of-the-art. In addition, the proposed scheme obtains a validation success rate exceeding 99% for the poisoned images at the cost of a marginal classification accuracy reduction of less than 0.5%. Moreover, even though there is only a 1-bit discrepancy between IDs, the proposed scheme still results in an accurate validation of user copyright. These results indicate that the proposed scheme is promising.

Meta-Transfer Metric Learning for Time Series Classification in 6G-Supported Intelligent Transportation Systems

Deep learning-based time series classification in 6G-supported Intelligent Transportation Systems (ITS) helps transport decision-making. Deep learning classifier training necessitates a large amount of labeled data for feature extraction. Labeling time series data in 6G-supported ITS is tough. Meta-learning can be used to train deep classifiers with limited data. However, in meta-learning, the tasks are frequently modeled by a low-complexity base learner. It is unable to use more complicate and powerful structures. The meta-learning-pretrained classifier can only perform new classification problems with the same number of classes. Most pre-training strategies do not prioritize enhancing the pre-training phase’s convergence rate and lowering the computational cost. Most research work aims to improve classification performance by increasing the complexity of the classification model. However, this raises computing costs. In this paper, we propose a one-dimensional Multi-Scale Dilated Convolution Neural Network time series classifier (MSDCNN). MSDCNN combines multi-scale CNN and dilated CNN. It can extract multi-scale characteristics from time series and reduce the complexity of the classifier. Furthermore, we propose a pre-training strategy, called Meta-transfer metric Learning using Scale function (MLS). MLS allows the classifier to gain experience from different tasks with various numbers of classes. Experiments show that MLS reduces pre-training computation costs during the pre-training phase. The pre-trained classifier, without using any fine tuning techniques, achieves the highest accuracy by comparing with the state-of-the-art methods. Finally, we present a case study of applying MSDCNN and MLS to detect road accidents in 6G-supported transportation systems.

Leveraging BERT to Improve Spoken Language Identification of Code-Switching Speech

Language identification (LID) involves automatically determining the language being spoken in a given segment. In code-switching speech, there are rapid switches between two or more languages within a single conversation. Despite LID attaining high accuracy on medium or long utterances, the performance on short utterances of code-switching speech is currently unsatisfactory. We propose a bidirectional encoder representation from transformers (BERT)-based LID system (BERT-LID) to enhance LID performance, especially for short-duration code-switching speech segments. The original BERT model is expanded by incorporating phonetic posterior grams (PPGs) extracted from a front-end phone recognizer as input. Then it is followed by the deployment of an optimal deep classifier for LID. Our BERT-LID model demonstrates a significant improvement of approximately 6.5% in accuracy for long-segment identification and 19.9% for short-segment identification, thereby demonstrating its effectiveness in code-switching speech LID tasks.

TSCF: An Improved Deep Forest Model for Time Series Classification

The deep forest presents a novel approach that yields competitive performance when compared to deep neural networks. Nevertheless, there are limited studies on the application of deep forest to time series classification (TSC) tasks, and the direct use of deep forest cannot effectively capture the relevant characteristics of time series. For that, this paper proposes time series cascade forest (TSCF), a model specifically designed for TSC tasks. TSCF relies on four base classifiers, i.e., random forest, completely random forest, random shapelet forest, and diverse representation canonical interval forest, allowing for feature learning on the original data from three granularities: point, subsequence, and summary statistics calculated based on intervals. The major contribution of this work, is to define an ensemble and deep classifier that significantly outperforms the individual classifiers and the original deep forest. Experimental results show that TSCF outperforms other forest-based algorithms for solving TSC problems.