Discrimination Ability Of Features Research Articles

An attention-based deep learning for acute lymphoblastic leukemia classification

The bone marrow overproduces immature cells in the malignancy known as Acute Lymphoblastic Leukemia (ALL). In the United States, about 6500 occurrences of ALL are diagnosed each year in both children and adults, comprising nearly 25% of pediatric cancer cases. Recently, many computer-assisted diagnosis (CAD) systems have been proposed to aid hematologists in reducing workload, providing correct results, and managing enormous volumes of data. Traditional CAD systems rely on hematologists’ expertise, specialized features, and subject knowledge. Utilizing early detection of ALL can aid radiologists and doctors in making medical decisions. In this study, Deep Dilated Residual Convolutional Neural Network (DDRNet) is presented for the classification of blood cell images, focusing on eosinophils, lymphocytes, monocytes, and neutrophils. To tackle challenges like vanishing gradients and enhance feature extraction, the model incorporates Deep Residual Dilated Blocks (DRDB) for faster convergence. Conventional residual blocks are strategically placed between layers to preserve original information and extract general feature maps. Global and Local Feature Enhancement Blocks (GLFEB) balance weak contributions from shallow layers for improved feature normalization. The global feature from the initial convolution layer, when combined with GLFEB-processed features, reinforces classification representations. The Tanh function introduces non-linearity. A Channel and Spatial Attention Block (CSAB) is integrated into the neural network to emphasize or minimize specific feature channels, while fully connected layers transform the data. The use of a sigmoid activation function concentrates on relevant features for multiclass lymphoblastic leukemia classification The model was analyzed with Kaggle dataset (16,249 images) categorized into four classes, with a training and testing ratio of 80:20. Experimental results showed that DRDB, GLFEB and CSAB blocks’ feature discrimination ability boosted the DDRNet model F1 score to 0.96 with minimal computational complexity and optimum classification accuracy of 99.86% and 91.98% for training and testing data. The DDRNet model stands out from existing methods due to its high testing accuracy of 91.98%, F1 score of 0.96, minimal computational complexity, and enhanced feature discrimination ability. The strategic combination of these blocks (DRDB, GLFEB, and CSAB) are designed to address specific challenges in the classification process, leading to improved discrimination of features crucial for accurate multi-class blood cell image identification. Their effective integration within the model contributes to the superior performance of DDRNet.

An automatic sparse-based deep cascade framework with multilayer representation for detecting breast cancer

This work proposes an effective sparse-based hybrid alternating deep-layer cascade model (HADLCM) for breast cancer detection. It is achieved by cascading sparse and collaborative representation alternatively, with a softmax vector representation at the interface of each cascade segment. Here, alternating deep-layer cascade model (ADLCM) followed by Laplacian of Gaussian-based modified high boosting filter (LoGMHBF) integrated with projection to discriminative subclasses (PDS)-based pre-processing is hybridized with ADLCM subsequent to PDS to improve feature discrimination ability. A class discriminant softmax vector representation is employed to cascade sparse and collaborative representations, leveraging the benefits of both. It improves hierarchical learning abilities by expanding shallow-sparse representation to an effective multi-layer learning approach for extracting deep-rooted discriminative information. The experimental results show the superiority of HADLCM to identify abnormality and malignancy with the accuracy of 98.17% and 97.75% in mini-DDSM (Digital Database for Screening Mammography); 94.87% and 95.00% in BUSI (Breast Ultrasound Image)BUSI, respectively.

Anti-noise transfer adversarial convolutions with adaptive threshold for rotating machine fault diagnosis

Fault diagnosis based on deep learning (DL) has been a research hotspot in recent years. However, the current neural networks are getting larger and larger, with more and more parameters and insufficient noise resistance, making it difficult to effectively apply these methods to real working conditions. To address these issues, we propose a novel deep learning method with fewer parameters and better noise resistance based on transfer adversarial subnetwork (TAS) and channel-wise thresholds (CWT), namely, anti-noise transfer adversarial convolutions (ANTAC). In the proposed method, the original data and feature vectors are mapped to reproducing kernel Hilbert space (RKHS) and processed by maximum mean discrepancy (MMD) and Wasserstein distance (WD), which makes the method more capable to distinguish the similar features without producing any additional training parameters. Furthermore, white Gaussian noise (WGN) and the soft thresholding method with CWT are used to reduce data noise and improve the robustness and noise resistance of the network. Finally, the superiority of the proposed method is validated through experiments on different datasets, network structures and the data with different SNRs. The results show that the proposed method has better feature discrimination ability, noise resistance, and fewer parameters compared with other methods. The highest accuracy of the proposed method is 99.90% on the test set.

Oral mucosal disease recognition based on dynamic self-attention and feature discriminant loss.

To develop a dynamic self-attention and feature discrimination loss function (DSDF) model for identifying oral mucosal diseases presented to solve the problems of data imbalance, complex image background, and high similarity and difference of visual characteristics among different types of lesion areas. In DSDF, dynamic self-attention network can fully mine the context information between adjacent areas, improve the visual representation of the network, and promote the network model to learn and locate the image area of interest. Then, the feature discrimination loss function is used to constrain the diversity of channel characteristics, so as to enhance the feature discrimination ability of local similar areas. The experimental results show that the recognition accuracy of the proposed method for oral mucosal disease is the highest at 91.16%, and is about 6% ahead of other advanced methods. In addition, DSDF has recall of 90.87% and F1 of 90.60%. Convolutional neural networks can effectively capture the visual features of the oral mucosal disease lesions, and the distinguished visual features of different oral lesions can be extracted better using dynamic self-attention and feature discrimination loss function, which is conducive to the auxiliary diagnosis of oral mucosal diseases.

Multiscale Fusion Network Based on Global Weighting for Hyperspectral Feature Selection

Feature selection is an important way to achieve high precision and efficient classification of hyperspectral remote sensing images. However, most existing FS methods use a fixed scale to extract features and the relationship between spatial and spectral dimensions is ignored. In fact, this correlation is useful for classification. In this paper, a multi-scale feature fusion network (MSFGW) is proposed in which a global weighting mechanism is explored to catch spatial-spectral information at multiple scales. First, the multi-scale feature extraction module composed of group convolution and dilated convolution is utilized to extract the multi-scale features. With the increase of the dilation rate, the module takes the spatial differences at varying scales. Secondly, a 3D weighting mechanism is used to combine the spatial and spectral correlated information for reducing the interference of homologous and heterologous and boosting the feature discrimination ability. Then, multi-scale weighted features are fused to integrate the internal information of all bands at different scales. Finally, the band reconstruction network is used to select representative bands according to their entropy. The experimental results with the state-of-the-art feature selection algorithms on three widely hyperspectral datasets demonstrate that the features selected by MSFGW have obvious advantages in classification with only a few training samples.

SNCF-Net: Scale-aware neighborhood correlation feature network for hotspot defect detection of photovoltaic farms

The photovoltaic hotspot defect detection is challenging due to the features vanishing as the network deepens and the poor feature discrimination ability under complex background interference. To address this challenging problem, we first designed a novel neighborhood correlation feature module (NCFM) to adaptively Integrate information from different scales based on the correlation between features to alleviate the problem of small defects feature loss. Then, we constructed a Scale-aware Attention Mechanism (SAM), which adaptively reweights the channel features and adjusts the supervision signals at different scales to enhance the utilization of features. Finally, we designed a Scale-aware Neighborhood Correlation Feature Network (SNCF-Net), which performs well in photovoltaic inspection hotspot defect localization. The experimental results demonstrated that SNCF-Net achieves 95.2 % (F-measure), 89.7 % (mAP), and 54.3 % (IoU) in terms of hotspot defects classification and detection results in infrared images of photovoltaic farms, which outperforms the current state-of-the-art methods.

Ship Classification in SAR Imagery by Shallow CNN Pre-Trained on Task-Specific Dataset with Feature Refinement

Ship classification based on high-resolution synthetic aperture radar (SAR) imagery plays an increasingly important role in various maritime affairs, such as marine transportation management, maritime emergency rescue, marine pollution prevention and control, marine security situational awareness, and so on. The technology of deep learning, especially convolution neural network (CNN), has shown excellent performance on ship classification in SAR images. Nevertheless, it still has some limitations in real-world applications that need to be taken seriously by researchers. One is the insufficient number of SAR ship training samples, which limits the learning of satisfactory CNN, and the other is the limited information that SAR images can provide (compared with natural images), which limits the extraction of discriminative features. To alleviate the limitation caused by insufficient training datasets, one of the widely adopted strategies is to pre-train CNNs on a generic dataset with massive labeled samples (such as ImageNet) and fine-tune the pre-trained network on the target dataset (i.e., a SAR dataset) with a small number of training samples. However, recent studies have shown that due to the different imaging mechanisms between SAR and natural images, it is hard to guarantee that the pre-trained CNNs (even if they perform extremely well on ImageNet) can be finely tuned by a SAR dataset. On the other hand, to extract the most discriminative ship representation features from SAR images, the existing methods have carried out fruitful research on network architecture design, attention mechanism embedding, feature fusion, etc. Although these efforts improve the performance of SAR ship classification to some extent, they are usually based on more complex network architecture and higher dimensional features, accompanied by more time-consuming storage expenses. Through the analysis of SAR image characteristics and CNN feature extraction mechanism, this study puts forward three hypotheses: (1) Pre-training CNN on a task-specific dataset may be more effective than that on a generic dataset; (2) a shallow CNN may be more suitable for SAR image feature extraction than a deep one; and (3) the deep features extracted by CNNs can be further refined to improve the feature discrimination ability. To validate these hypotheses, we propose to learn a shallow CNN which is pre-trained on a task-specific dataset, i.e., the optical remote sensing ship dataset (ORS) instead of on the widely adopted ImageNet dataset. For comparison purposes, we designed 28 CNN architectures by changing the arrangement of the CNN components, the size of convolutional filters, and pooling formulations based on VGGNet models. To further reduce redundancy and improve the discrimination ability of the deep features, we propose to refine deep features by active convolutional filter selection based on the coefficient of variation (COV) sorting criteria. Extensive experiments not only prove that the above hypotheses are valid but also prove that the shallow network learned by the proposed pre-training strategy and the feature refining method can achieve considerable ship classification performance in SAR images like the state-of-the-art (SOTA) methods.

Emotion Recognition of Online Education Learners by Convolutional Neural Networks.

At present, the facial expression recognition model in video communication has problems such as weak network generalization ability and complex model structure, which leads to a large amount of computation. Firstly, the Inception architecture is adopted as a design philosophy. The Visual Geometry Group Network (VGGNet) model is improved. Multiscale kernel convolutional layers are constructed to obtain more expressive features. Secondly, the attention mechanism is integrated into a multiscale feature fusion network to form a multiattention mechanism Convolutional Neural Network (CNN) model. Novel spatial and multichannel attention models are designed. The effects of redundant information and noise are reduced. Finally, experiments are carried out on the Fer2013 dataset and the Extended Cohn-Kanade Dataset (CK+) to verify the detection accuracy of the model. The results show that the Delivered Duty Unpaid (DDU) loss can be used for facial expression recognition in complex environments. After the attention module is added, the overall recognition accuracy of the network on Fer2013 and CK+ has been improved to varying degrees. The addition of the channel attention module has a more obvious effect on the recognition accuracy compared with the spatial attention module. The addition of the attention module enables the network to increase the attention to error-prone samples. The improved network model can better extract the key features of facial expressions, enhance the feature discrimination ability, and improve the recognition accuracy of error-prone expressions. The accuracy rate of facial expression recognition with larger movements is over 98%. Facial expressions are an important way of communication between people, and online video has greatly limited this communication method. The proposed CNN model based on multiscale feature fusion will effectively solve these network limitations and have an important and positive impact on future network information exchange.

CDFormer: A Hyperspectral Image Change Detection Method Based on Transformer Encoders

Hyperspectral image (HSI) change detection (CD) has gained much attention in remote sensing. However, most deep learning methods are restricted by a limited receptive field, without leveraging temporal information, and the need for many training samples. In this letter, we proposed a Transformer Encoder-based HSI CD framework called CDFormer. First, space and time encodings are added to the pixel sequence to guide transformers to exploit change information of space and time by the pixel embedding (PE) module. Second, the self-attention component of Transformer Encoder module has a global space-time receptive field to mine the correlation and interaction between bi-temporal features, enhancing the utilization of temporal dependencies. Next, the multi-head attention mechanism learns several attentions and extracts the joint weighted spatial-spectral-temporal features, which improves the feature discrimination ability of the changes. Finally, the detection result is predicted using a fully connected network. It is notable to mention that the proposed method only uses a few labeled samples to train the network. Experiments on two HSI datasets demonstrate that our proposed method can get effective performance in HSI CD.

Fusion-Based Feature Attention Gate Component for Vehicle Detection Based on Event Camera

In the field of autonomous vehicles, various heterogeneous sensors, such as LiDAR, Radar, camera, etc, are combined to improve the vehicle ability of sensing accuracy and robustness. Multi-modal perception and learning has been proved to be an effective method to help vehicle understand the nature of complex environments. Event camera is a bio-inspired vision sensor that captures dynamic changes in the scene and filters out redundant information with high temporal resolution and high dynamic range. These characteristics of the event camera make it have a certain application potential in the field of autonomous vehicles. In this paper, we introduce a fully convolutional neural network with feature attention gate component (FAGC) for vehicle detection by combining frame-based and event-based vision. Both grayscale features and event features are fed into the feature attention gate component (FAGC) to generate the pixel-level attention feature coefficients to improve the feature discrimination ability of the network. Moreover, we explore the influence of different fusion strategies on the detection capability of the network. Experimental results demonstrate that our fusion method achieves the best detection accuracy and exceeds the accuracy of the method that only takes single-mode signal as input.

Fuzzy rough discrimination and label weighting for multi-label feature selection

Fuzzy rough set is a theoretical framework of fuzzy uncertainty management, and discernibility matrix offers a mathematical foundation for algorithm construction of feature learning. The approaches of fuzzy rough set and discernibility matrix have been successfully applied in single-label learning. However, few works have been done on investigating the foundation of fuzzy rough discernibility matrix on multi-label data. There will be two pivotal problems to be addressed when using fuzzy rough discernibility matrix for multi-label data analysis. One is how to extract sample-level and label-level correlations; and the other is how to utilize the discernibility matrix for algorithm construction. For this reason, in this paper the fuzzy rough discrimination matrix is introduced to deal with the problem of multi-label feature selection. First, the significance of labels in the label space is captured based on the label correlation. Labels with different significances contribute to different weights for measuring the similarity between samples. Hence, a sample similarity matrix in the label space is computed based on the label weighting strategy. Then, a framework of a fuzzy decision system is formalized, in which the discernibility matrix of fuzzy rough sets is introduced as a foundation to evaluate the sample discrimination ability of features. Under the discernibility matrix criterion, a multi-label learning algorithm is developed to select discriminative features from multi-label data. A series of experimental analysis verifies the effectiveness and efficiency of the proposed method.

CloudU-Netv2: A Cloud Segmentation Method for Ground-Based Cloud Images Based on Deep Learning

Accurately acquiring cloud information through cloud images segmentation is of great importance for weather forecasting, environmental monitoring, sites selection of observatory and analysis of climate evolution. In this paper, a cloud segmentation method based on deep learning, called CloudU-Netv2, is proposed to segment daytime and nighttime ground-based cloud images. The CloudU-Netv2 includes encoder, dual attention modules and decoder. The contributions in this paper are four folds as follows. Firstly, it replaces the ‘upsampling’ in CloudU-Net with ‘bilinear upsampling’. Secondly, position and channel attention modules are added to the structure to improve the discrimination ability of features’ representation. Thirdly, it chooses rectified Adam as the optimizer in the CloudU-Netv2 structure. Finally, we conduct ablation experiments on the key components of CloudU-Netv2 and compare with the existing four advanced methods using six evaluation metrics. Results show that the key components of the model play the pivotal role in improving the segmentation performance, and the proposed CloudU-Netv2 has the best segmentation performance for daytime and nighttime ground-based cloud images compared with four other methods.

Multi-feature Fusion for High Resolution Aerial Scene Image Classification

Remote sensing scene classification plays an important role in many applications. Obtaining a high discriminative feature representation is the key of scene classification. The information hidden in different layers of convolutional neural network (CNN) has great potential for enhancing the feature discrimination ability. In this paper, the features from convolutional layers and a set of local features are combined for scene classification. Specifically, deep hierarchical features from different convolutional layers are extracted by a pretrained CNN model, which is used as a feature extractor. A patch-based MS-CLBP method is adopted to acquire local representations. Then the holistic hierarchical and local visual representation is obtained after fisher vector (FV) encoding. Finally, an improved extreme learning machine (ELM) is adopted to classify the scene images based on the obtained FVs. Experimental results show that the proposed methods achieves excellent performance compared with the state-of-the-art classification methods.

Broad Attribute Prediction Model With Enhanced Attribute and Feature

For the zero-shot image classification without intersection between training and testing sets, the high-quality representation of image attributes and features plays a key role to improve the classification performance. In order to overcome the limitations related to insufficient attribute and feature expression in zero-shot image classification, we propose a broad attribute prediction model with enhanced attribute and feature (EAF-BAP) based on broad learning and elastic net constraint. Firstly, the EAF-BAP enhances pre-defined attributes by elastic net constraint to obtain hybrid attributes, which effectively improves the finiteness of semantic attributes. Secondly, the enhanced features are constructed by broad learning to increase the discrimination ability of features in different classes. Meanwhile, the broad learning is employed to train multiple attribute classifiers synchronously, which is more efficient compared to traditional support vector machines. Finally, the similarity between predicted attributes and hybrid attributes in testing classes is calculated by Manhattan distance, which is further used to implement image classification. Experiments on both AwA and Shoes datasets show that the proposed EAF-BAP model is capable of improving the accuracy of zero-shot image classification efficiently.

Local Intensity Ordering based Binary Patterns for Image Region Description

Local image region description is a fundamental task for image feature matching in the field of Computer Vision. A good image region descriptor should have the ability to discriminate image features even though the images differ due to photometric variations and geometric transformations. Over these years, many local region descriptors have been proposed to tackle the aforementioned challenges. Achieving rotation invariance in keypoint description is considered one of the main challenges in local region description and matching. Previous approaches proposed to tackle rotation variations depend on unreliable dominant orientation estimation. In this paper, two novel local image region descriptors called Local Intensity Order-based Center Symmetric Local Binary Patterns (LIOCSLBP) and Local Intensity Order-based Orthogonally Combined Local Binary Patterns (LIOOCLBP) are proposed to build rotation invariant local region descriptions. The rotation invariance characteristic of the proposed binary pattern-based local region description is achieved by applying a simple and efficient mechanism called Local Intensity Ordering (LIO). The proposed descriptors use double interest regions for each interest point to improve feature discrimination. In order to further improve the feature discrimination ability RGBLIOCSLBP, RGBLIOOCLBP, HSVLIOCSLBP and HSVLIOOCLBP are also proposed exploiting RGB and HSV color models. Extensive experiments are conducted to evaluate the performance of the proposed descriptors on standard benchmark datasets for image matching, object recognition and scene recognition against the state-of-the-art descriptors. The experimental results show that the proposed descriptors are highly competitive to several state-of-the-art local region descriptors where the proposed descriptors outperformed the comparative approaches in many cases.

Improving time–frequency domain sleep EEG classification via singular spectrum analysis

BackgroundManual sleep scoring is deemed to be tedious and time consuming. Even among automatic methods such as time–frequency (T–F) representations, there is still room for more improvement. New methodTo optimise the efficiency of T–F domain analysis of sleep electroencephalography (EEG) a novel approach for automatically identifying the brain waves, sleep spindles, and K-complexes from the sleep EEG signals is proposed. The proposed method is based on singular spectrum analysis (SSA). The single-channel EEG signal (C3-A2) is initially decomposed and then the desired components are automatically separated. In addition, the noise is removed to enhance the discrimination ability of features. The obtained T–F features after preprocessing stage are classified using a multi-class support vector machines (SVMs) and used for the identification of four sleep stages over three sleep types. Furthermore, to emphasise on the usefulness of the proposed method the automatically-determined spindles are parameterised to discriminate three sleep types. ResultThe four sleep stages are classified through SVM twice: with and without preprocessing stage. The mean accuracy, sensitivity, and specificity for before the preprocessing stage are: 71.5±0.11%, 56.1±0.09% and 86.8±0.04% respectively. However, these values increase significantly to 83.6±0.07%, 70.6±0.14% and 90.8±0.03% after applying SSA. Comparison with existing methodThe new T–F representation has been compared with the existing benchmarks. Our results prove that, the proposed method well outperforms the previous methods in terms of identification and representation of sleep stages. ConclusionExperimental results confirm the performance improvement in terms of classification rate and also representative T–F domain.

OPTIMIZED NEURAL INCREMENTAL ATTRIBUTE LEARNING FOR CLASSIFICATION BASED ON STATISTICAL DISCRIMINABILITY

Feature ordering is a significant data preprocessing method in incremental attribute learning (IAL), where features are gradually trained according to a given order. Previous research showed feature ordering is crucial to the IAL performance. It is relevant to each feature's discrimination ability, which can be calculated by single discriminability (SD). However, when feature dimensions increase, feature discrimination ability should also be calculated incrementally, because discrimination ability in lower dimensional spaces is different from that in higher spaces. Thus based on SD, accumulative discriminability (AD), a new statistical metric for incremental feature discrimination ability estimation, is designed. Moreover, a criterion that summarizes all the produced values of AD is employed to obtain the optimum feature ordering for classification problems based on neural networks by means of IAL. In addition, in order to reduce the time consumption, an effective feature ordering approach is developed. Compared with the feature ordering obtained by other approaches, the method outlined in this paper obtained good final classification results, which indicates that, firstly, feature discrimination ability should be incrementally estimated in IAL; and secondly, feature ordering derived by AD and its corresponding approaches are applicable with IAL.

Statistical Discriminability Estimation for Pattern Classification Based on Neural Incremental Attribute Learning

Feature ordering is a significant data preprocessing method in Incremental Attribute Learning (IAL), a novel machine learning approach which gradually trains features according to a given order. Previous research has shown that, similar to feature selection, feature ordering is also important based on each feature's discrimination ability, and should be sorted in a descending order of their discrimination ability. However, such an ordering is crucial for the performance of IAL. As the number of feature dimensions in IAL is increasing, feature discrimination ability also should be calculated in the corresponding incremental way. Based on Single Discriminability (SD), where only the feature discrimination ability is computed, a new filter statistical feature discrimination ability predictive metric, called the Accumulative Discriminability (AD), is designed for the dynamical feature discrimination ability estimation. Moreover, a criterion that summarizes all the produced values of AD is employed with a GA (Genetic Algorithm)-based approach to obtain the optimum feature ordering for classification problems based on neural networks by means of IAL. Compared with the feature ordering obtained by other approaches, the method proposed in this paper exhibits better performance in the final classification results. Such a phenomenon indicates that, (i) the feature discrimination ability should be incrementally estimated in IAL, and (ii) the feature ordering derived by AD and its corresponding approaches are applicable with IAL.

Guilt-by-association feature selection: Identifying biomarkers from proteomic profiles

In recent years, proteomic profiling by mass spectrometry has opened up a new realm of methods for identifying potential biomarkers. Mass spectrometry data, like other proteomic and genomic data, are challenging to analyze because of their high dimensionality and the availability of few samples. Hence, feature selection is extremely important because it directly provides a list of potential biomarkers by choosing a subset of effective features that separate diseased samples from healthy ones. The rule of thumb for feature selection is that features must be discriminant and independent for the best separation of the two groups. However, in general, existing feature selection algorithms only take into account the discrimination ability of features. In this paper, we present a novel method for feature selection, guilt-by-association feature selection (GBA-FS). The algorithm makes it possible to select features that are independent as well as discriminant. After measuring similarities between features, the algorithm groups together similar features using a clustering algorithm, and selects the best representative feature from each group. As a result, it produces a list of discriminant and independent features. The efficacy of GBA-FS was extensively tested on two real-world SELDI TOF data sets. The experimental results demonstrate that GBA-FS assists in selecting more independent features as compared to a common filter type feature selection method, the t test. The results also show that GBA-FS can be used to deconvolve multiply charged states of the same protein molecules. As GBA-FS successfully identifies feature groups with similar mass values, it can also be employed as an alternative to peak detection for preprocessing the mass spectrometry data.