Top-1 Accuracy Research Articles

Discrete Cosine Transformed Images Are Easy to Recognize in Vision Transformers

Deep learning models for image classification with adequate parameters show excellent classification performance because they can effectively extract the features of input images. On the other hand, there is a limit to the abilities of deep learning models to interpret images using only spatial information because an image is a signal with great spatial redundancy. Therefore, in this study, the discrete cosine transform was applied to an input image in units of an N×N block size to allow the deep learning model to employ both frequency and spatial information. The proposed method was implemented and verified by selecting a vision transformer using a 16×16 nonoverlapping patch as a baseline and training various datasets of Cifar-10, Cifar-100, and Tiny-ImageNet from the very beginning without pre-trained weights. The experimental results showed that the top-1 accuracy is improved by approximately 3-5% for every dataset with little increase in computational cost.

Deep Contrastive Learning-Based Model for ECG Biometrics

The electrocardiogram (ECG) signal is shown to be promising as a biometric. To this end, it has been demonstrated that the analysis of ECG signals can be considered as a good solution for increasing the biometric security levels. This can be mainly due to its inherent robustness against presentation attacks. In this work, we present a deep contrastive learning-based system for ECG biometric identification. The proposed system consists of three blocks: a feature extraction backbone based on short time Fourier transform (STFT), a contrastive learning network, and a classification network. We evaluated the proposed system on the Heartprint dataset, a new ECG biometrics multi-session dataset. The experimental analysis shows promising capabilities of the proposed method. In particular, it yields an average top1 accuracy of 98.02% on a new dataset built by gathering 1539 ECG records from 199 subjects collected in multiple sessions with an average interval between sessions of 47 days.

Shape-aware fine-grained classification of erythroid cells

Fine-grained classification and counting of bone marrow erythroid cells are vital for evaluating the health status and formulating therapeutic schedules for leukemia or hematopathy. Due to the subtle visual differences between different types of erythroid cells, it is challenging to apply existing image-based deep learning models for fine-grained erythroid cell classification. Moreover, there is no large open-source datasets on erythroid cells to support the model training. In this paper, we introduce BMEC (Bone Morrow Erythroid Cells), the first large fine-grained image dataset of erythroid cells, to facilitate more deep learning research on erythroid cells. BMEC contains 5,666 images of individual erythroid cells, each of which is extracted from the bone marrow erythroid cell smears and professionally annotated to one of the four types of erythroid cells. To distinguish the erythroid cells, one key indicator is the cell shape which is closely related to the cell growth and maturation. Therefore, we design a novel shape-aware image classification network for fine-grained erythroid cell classification. The shape feature is extracted from the shape mask image and aggregated to the raw image feature with a shape attention module. With the shape-attended image feature, our network achieved superior classification performance (81.12% top-1 accuracy) on the BMEC dataset comparing to the baseline methods. Ablation studies also demonstrate the effectiveness of incorporating the shape information for the fine-grained cell classification. To further verify the generalizability of our method, we tested our network on two additional public white blood cells (WBC) datasets and the results show our shape-aware method can generally outperform recent state-of-the-art works on classifying the WBC. The code and BMEC dataset can be found on https://github.com/wangye8899/BMEC .

Toward Interpretable Graph Tensor Convolution Neural Network for Code Semantics Embedding

Intelligent deep learning-based models have made significant progress for automated source code semantics embedding, and current research works mainly leverage natural language-based methods and graph-based methods. However, natural language-based methods do not capture the rich semantic structural information of source code, and graph-based methods do not utilize rich distant information of source code due to the high cost of message-passing steps. In this paper, we propose a novel interpretable model, called graph tensor convolution neural network (GTCN), to generate accurate code embedding, which is capable of comprehensively capturing the distant information of code sequences and rich code semantics structural information. Firstly, we propose to utilize a high-dimensional tensor to integrate various heterogeneous code graphs with node sequence features, such as control flow, data flow. Secondly, inspired by the current advantages of graph-based deep learning and efficient tensor computations, we propose a novel interpretable graph tensor convolution neural network for learning accurate code semantic embedding from the code graph tensor. Finally, we evaluate three popular applications on the GTCN model, variable misuse detection, source code prediction, and vulnerability detection. Compared with current state-of-the-art methods, our model achieves higher scores with respect to the top-1 accuracy while costing less training time.

Efficient Lung Cancer Image Classification and Segmentation Algorithm Based on an Improved Swin Transformer

With the advancement of computer technology, transformer models have been applied to the field of computer vision (CV) after their success in natural language processing (NLP). In today’s rapidly evolving medical field, radiologists continue to face multiple challenges, such as increased workload and increased diagnostic demands. The accuracy of traditional lung cancer detection methods still needs to be improved, especially in realistic diagnostic scenarios. In this study, we evaluated the performance of the Swin Transformer model in the classification and segmentation of lung cancer. The results showed that the pre-trained Swin-B model achieved a top-1 accuracy of 82.26% in the classification mission, outperforming ViT by 2.529%. In the segmentation mission, the Swin-S model demonstrated improvement over other methods in terms of mean Intersection over Union (mIoU). These results suggest that pre-training can be an effective approach for improving the accuracy of the Swin Transformer model in these tasks.

Model Construction and System Design of Natural Grassland-Type Recognition Based on Deep Learning

As an essential basic function of grassland resource surveys, grassland-type recognition is of great importance in both theoretical research and practical applications. For a long time, grassland-type recognition has mainly relied on two methods: manual recognition and remote sensing recognition. Among them, manual recognition is time-consuming and laborious, and easily affected by the level of expertise of the investigator, whereas remote sensing recognition is limited by the spatial resolution of satellite images, and is not suitable for use in field surveys. In recent years, deep learning techniques have been widely used in the image recognition field, but the application of deep learning in the field of grassland-type recognition needs to be further explored. Based on a large number of field and web-crawled grassland images, grassland-type recognition models are constructed using the PyTorch deep learning framework. During model construction, a large amount of knowledge learned by the VGG-19 model on the ImageNet dataset is transferred to the task of grassland-type recognition by the transfer learning method. By comparing the performances of models with different initial learning rates and whether or not data augmentation is used, an optimal grassland-type recognition model is established. Based on the optimal model, grassland resource-type map, and meteorological data, PyQt5 is used to design and develop a grassland-type recognition system that uses user-uploaded grassland images and the images’ location information to comprehensively recognize grassland types. The results of this study showed that: (1) When the initial learning rate was set to 0.01, the model recognition accuracy was better than that of the models using initial learning rates of 0.1, 0.05, 0.005, and 0.001. Setting a reasonable initial learning rate helps the model quickly reach optimal performance and can effectively avoid variations in the model. (2) Data augmentation increases the diversity of data, reducing the overfitting of the model; recognition accuracies of the models constructed using the augmented data can be improved by 3.07–4.88%. (3) When the initial learning rate was 0.01, modeling with augmented data and with a training epoch = 30, the model performance reached its peak—the TOP1 accuracy of the model was 78.32% and the TOP5 accuracy of the model was 91.27%. (4) Among the 18 grassland types, the recognition accuracy of each grassland type reached over 70.00%, and the probability of misclassification among most of the grassland types was less than 5.00%. (5) The grassland-type recognition system incorporates two reference grassland types to further improve the accuracy of grassland-type recognition; the accuracy of the two reference grassland types was 72.82% and 75.01%, respectively. The recognition system has the advantages of convenient information acquisition, good visualization, easy operation, and high stability, which provides a new approach for the intelligent recognition of grassland types using grassland images taken in a field survey.

An Intelligent Framework for Oversubscription Management in CPU-GPU Unified Memory

This paper proposes a novel intelligent framework for oversubscription management in CPU-GPU UVM. We analyze the current rule-based methods of GPU memory oversubscription with unified memory, and the current learning-based methods for other computer architectural components. We then identify the performance gap between the existing rule-based methods and the theoretical upper bound. We also identify the advantages of applying machine intelligence and the limitations of the existing learning-based methods. This paper proposes a novel intelligent framework for oversubscription management in CPU-GPU UVM. It consists of an access pattern classifier followed by a pattern-specific Transformer-based model using a novel loss function aiming for reducing page thrashing. A policy engine is designed to leverage the model's result to perform accurate page prefetching and pre-eviction. We evaluate our intelligent framework on a set of 11 memory-intensive benchmarks from popular benchmark suites. Our solution outperforms the state-of-the-art (SOTA) methods for oversubscription management, reducing the number of pages thrashed by 64.4\% under 125\% memory oversubscription compared to the baseline, while the SOTA method reduces the number of pages thrashed by 17.3\%. Our solution achieves an average IPC improvement of 1.52X under 125\% memory oversubscription, and our solution achieves an average IPC improvement of 3.66X under 150\% memory oversubscription. Our solution outperforms the existing learning-based methods for page address prediction, improving top-1 accuracy by 6.45\% (up to 41.2\%) on average for a single GPGPU workload, improving top-1 accuracy by 10.2\% (up to 30.2\%) on average for multiple concurrent GPGPU workloads.

Lambertian-based adversarial attacks on deep-learning-based underwater side-scan sonar image classification

Deep convolutional neural networks (CNNs) are extensively applied to the classification tasks for Side-scan sonar (SSS) images. However, state-of-the-art neural networks are prone to be confused by adversarial attacks that generate a tiny modification of the images, threatening the security of SSS classification. The robustness of CNN to adversarial attacks can be improved by introducing adversarial examples through adversarial training. Practical adversarial examples are often generated from elaborate adversarial attackers. For the underwater scenario of sonar, a specially designed adversarial attack method to weaken SSS image classification can make the research community better understand the weakness of CNN in this scenario and improve the security measures in a well-directed way. Thus, exploring adversarial attack methods for SSS image classification is essential. Nevertheless, the existing adversarial attack methods are designed for optical images, reflecting no physical characteristics of sonar images. To fill this gap and investigate the adversarial attack related to real-world conditions, in this paper, we propose an adversarial attack method named Lambertian Adversarial Sonar Attack (LASA). It initially leverages the Lambertian model to simulate the formation of the SSS image, factoring the image to three parameters, then updates the parameters on the direction of gradients by the chain rule. Finally, the parameters regenerate the adversarial example to fool the classifier. To validate the performance of LASA, we constructed a diversified SSS image dataset containing three categories. On our dataset, LASA reduces the Top-1 accuracy of a well-trained ResNet-101 to 7.31%±0.21 (one-shot version) and 0.00% (iterative version), the success rate of targeted attack reaches 97.03±2.24, far beyond the performance of the existing state-of-the-art adversarial attack methods. Meanwhile, we show that the adversarial training using examples generated from LASA makes the classifier more robust. We expect that our methods can be applied as a benchmark of adversarial attacks on SSS images, motivating future research to design novel neural networks or defensive methods to resist real-world adversarial attacks on SSS images.

Adaptive pooling-based convolution factorization for deploying CNNs on energy-constrained IoT edge devices

Convolutional neural networks (CNNs) involve a tremendous amount of multiply-and-accumulate (MAC) computations, leading to energy issues when deployed on energy-constrained IoT edge devices for computer vision purposes. However, a great number of these computations are ineffectual since they lead to meaningless zero activations, which could be avoided if predicted appropriately. This has been the focus of many studies, whereas it has only been done through zero-activation simplifications. This paper goes beyond those simplifications through a convolution factorization approach to convert the MAC-intensive convolution operations into less complex pooling operations. The main characteristic of the proposed idea is that, given a target accuracy, the applicability and type of factorization are adaptively decided for each individual activation. This is achieved by providing a lightweight multi-class CNN-based predictor, namely RedZAP. The selected types of pooling operations for our convolution factorizations are supported by existing CNN architectures. The experimental results show that, given a top-1 accuracy loss constraint of 1%, the proposed approach saves another 8% MAC operations in comparison to the existing zero-prediction approaches in the literature.

DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning

Neural Architecture Search (NAS) has demonstrated state-of-the-art performance on various computer vision tasks. Despite the superior performance achieved, the efficiency and generality of existing methods are highly valued due to their high computational complexity and low generality. In this paper, we propose an efficient and unified NAS framework termed DDPNAS via dynamic distribution pruning, facilitating a theoretical bound on accuracy and efficiency. In particular, we first sample architectures from a joint categorical distribution. Then the search space is dynamically pruned and its distribution is updated every few epochs. With the proposed efficient network generation method, we directly obtain the optimal neural architectures on given constraints, which is practical for on-device models across diverse search spaces and constraints. The architectures searched by our method achieve remarkable top-1 accuracies, 97.56 and 77.2 on CIFAR-10 and ImageNet (mobile settings), respectively, with the fastest search process, i.e., only 1.8 GPU hours on a Tesla V100. Codes for searching and network generation are available at: https://openi.pcl.ac.cn/PCL_AutoML/XNAS.

Autonomous landing scene recognition based on transfer learning for drones

In this paper, we study autonomous landing scene recognition with knowledge transfer for drones. Considering the difficulties in aerial remote sensing, especially that some scenes are extremely similar, or the same scene has different representations in different altitudes, we employ a deep convolutional neural network (CNN) based on knowledge transfer and fine-tuning to solve the problem. Then, LandingScenes-7 dataset is established and divided into seven classes. Moreover, there is still a novelty detection problem in the classifier, and we address this by excluding other landing scenes using the approach of thresholding in the prediction stage. We employ the transfer learning method based on ResNeXt-50 backbone with the adaptive momentum (ADAM) optimization algorithm. We also compare ResNet-50 backbone and the momentum stochastic gradient descent (SGD) optimizer. Experiment results show that ResNeXt-50 based on the ADAM optimization algorithm has better performance. With a pre-trained model and fine-tuning, it can achieve 97.845 0% top-1 accuracy on the LandingScenes-7 dataset, paving the way for drones to autonomously learn landing scenes.

Predicting origin for bone metastatic cancer using deep learning-based pathology.

Predicting origin for bone metastatic cancer using deep learning-based pathology.

APIRO: A Framework for Automated Security Tools API Recommendation

Security Orchestration, Automation, and Response (SOAR) platforms integrate and orchestrate a wide variety of security tools to accelerate the operational activities of Security Operation Center (SOC). Integration of security tools in a SOAR platform is mostly done manually using APIs, plugins, and scripts. SOC teams need to navigate through API calls of different security tools to find a suitable API to define or update an incident response action. Analyzing various types of API documentation with diverse API format and presentation structure involves significant challenges such as data availability, data heterogeneity, and semantic variation for automatic identification of security tool APIs specific to a particular task. Given these challenges can have negative impact on SOC team’s ability to handle security incident effectively and efficiently, we consider it important to devise suitable automated support solutions to address these challenges. We propose a novel learning-based framework for automated security tool API R ecommendation for security O rchestration, automation, and response, APIRO . To mitigate data availability constraint, APIRO enriches security tool API description by applying a wide variety of data augmentation techniques. To learn data heterogeneity of the security tools and semantic variation in API descriptions, APIRO consists of an API-specific word embedding model and a Convolutional Neural Network (CNN) model that are used for prediction of top three relevant APIs for a task. We experimentally demonstrate the effectiveness of APIRO in recommending APIs for different tasks using three security tools and 36 augmentation techniques. Our experimental results demonstrate the feasibility of APIRO for achieving 91.9% Top-1 Accuracy. Compared to the state-of-the-art baseline, APIRO is 26.93%, 23.03%, and 20.87% improved in terms of Top-1, Top-2, and Top-3 Accuracy and outperforms the baseline by 23.7% in terms of Mean Reciprocal Rank (MRR).

Classification of racehorse limb radiographs using deep convolutional neural networks.

To assess the capability of deep convolutional neural networks to classify anatomical location and projection from a series of 48 standard views of racehorse limbs. Radiographs (N = 9504) of horse limbs from image sets made for veterinary inspections by 10 independent veterinary clinics were used to train, validate and test (116, 40 and 42 radiographs, respectively) six deep learning architectures available as part of the open source machine learning framework PyTorch. The deep learning architectures with the best top-1 accuracy had the batch size further investigated. Top-1 accuracy of six deep learning architectures ranged from 0.737 to 0.841. Top-1 accuracy of the best deep learning architecture (ResNet-34) ranged from 0.809 to 0.878, depending on batch size. ResNet-34 (batch size = 8) achieved the highest top-1 accuracy (0.878) and the majority (91.8%) of misclassification was due to laterality error. Class activation maps indicated that joint morphology, not side markers or other non-anatomical image regions, drove the model decision. Deep convolutional neural networks can classify equine pre-import radiographs into the 48 standard views including moderate discrimination of laterality, independent of side marker presence.

A New Partitioned Spatial–Temporal Graph Attention Convolution Network for Human Motion Recognition

At present, human action recognition can be used in all walks of life, because the skeleton can transmit intuitive information without being affected by environmental factors. However, it only focuses on local information. In order to solve these problems, we introduce a neural network model for human body recognition in this paper. We propose a model named NEW-STGCN-CA. The model is based on a spatial–temporal graph convolution network (ST-GCN), which contains a new partition strategy and coordination attention (CA) mechanism. By integrating the CA attention mechanism model, we enable the network to focus on input-related information, ignore unnecessary information, and prevent information loss. Second, a new partitioning strategy is proposed for the sampled regions, which is used to enhance the connection between local information and global information. We proved that the Top-1 accuracy of the NEW-STGCN-CA model in the NTU-RGB+D 60 dataset reached 84.86%, which was 1.7% higher than the original model; the accuracy of Top-1 on the Kinetics-Skeleton dataset reached 32.40%, which was 3.17% higher than the original model. The experimental results show that NEW-STGCN-CA can effectively improve the algorithm’s accuracy while also having high robustness and performance.

Pruning-aware Sparse Regularization for Network Pruning

Structural neural network pruning aims to remove the redundant channels in the deep convolutional neural networks (CNNs) by pruning the filters of less importance to the final output accuracy. To reduce the degradation of performance after pruning, many methods utilize the loss with sparse regularization to produce structured sparsity. In this paper, we analyze these sparsity-training-based methods and find that the regularization of unpruned channels is unnecessary. Moreover, it restricts the network’s capacity, which leads to under-fitting. To solve this problem, we propose a novel pruning method, named MaskSparsity, with pruning-aware sparse regularization. MaskSparsity imposes the fine-grained sparse regularization on the specific filters selected by a pruning mask, rather than all the filters of the model. Before the fine-grained sparse regularization of MaskSparity, we can use many methods to get the pruning mask, such as running the global sparse regularization. MaskSparsity achieves a 63.03% float point operations (FLOPs) reduction on ResNet-110 by removing 60.34% of the parameters, with no top-1 accuracy loss on CIFAR-10. On ILSVRC-2012, MaskSparsity reduces more than 51.07% FLOPs on ResNet-50, with only a loss of 0.76% in the top-1 accuracy. The code of this paper is released at https://github.com/CASIA-IVA-Lab/MaskSparsity . We have also integrated the code into a self-developed PyTorch pruning toolkit, named EasyPruner, at https://gitee.com/casia_iva_engineer/easypruner .

Text-based person search via local-relational-global fine grained alignment

The core difficulty of text-based person search is how to achieve fine-grained alignment of visual and linguistic modal data, so as to bridge the gap of modal heterogeneity. Most existing works on this task focus on global and local features extraction and matching, ignoring the importance of relational information. This paper proposes a new text-based person search model, named CM-LRGNet, which extracts Cross-Modal Local-Relational-Global features in an end-to-end manner, and performs fine-grained cross-modal alignment on the above three feature levels. Concretely, we first split the convolutional feature maps to obtain local features of images, and adaptively extract textual local features. Then a relation encoding module is proposed to implicitly learn the relational information implied in the images and texts. Finally, a relation-aware graph attention network is designed to fuse the local and relational features to generate global representations for both images and text queries. Extensive experimental results on benchmark dataset (CUHK-PEDES) show that our approach can achieve state-of-the-art performance (64.18%, 82.97%, 89.85% in terms of Top-1, Top-5, and Top-10 accuracies), by learning and aligning local-relational-global representations from different modalities. Our code has been released in https://github.com/zhangweifeng1218/Text-based-Person-Search.

Development of a machine learning model to identify elbow implants from radiographic images

BackgroundTotal elbow arthroplasty rates continue to rise year after year with up to 7.6% annual growth, yet complication and failure rates have been reported as high as 90%. Proper identification of implants prior to revision is crucial to optimize clinical resources and postoperative outcomes. This study aims to evaluate a machine learning algorithm for quick and accurate identification of elbow implants from radiographic images. MethodsA total of 142 elbow implant radiographs representing 5 unique prostheses were obtained from deidentified patients within the Mount Sinai Health System in New York. Images were split into training (n = 97) and test (n = 45) sets. A modified version of the convolutional neural network DenseNet-121 was used to train and evaluate the data. The model was then assessed for accuracy, sensitivity, positive predictive value, and f-1 scores. ResultsOverall model accuracy was 97.8% and top-3 accuracy was 100% for each of the 5 prostheses. Sensitivity, positive predictive value, and f-1 scores for the model were 0.978, 0.980, and 0.96, respectively. The algorithm took 14.10 seconds to classify the 45 images in the test set, averaging 0.31 seconds per image. ConclusionsThe model demonstrated strong performance in the identification of elbow implants among the sample of 142 images and 5 implant types. While these results suggest promise in providing future clinical utility within surgical workflows, the method requires external validation and further testing with a greater number and wider variety of implants.

Fast Full-Resolution Target-Adaptive CNN-Based Pansharpening Framework

In the last few years, there has been a renewed interest in data fusion techniques, and, in particular, in pansharpening due to a paradigm shift from model-based to data-driven approaches, supported by the recent advances in deep learning. Although a plethora of convolutional neural networks (CNN) for pansharpening have been devised, some fundamental issues still wait for answers. Among these, cross-scale and cross-datasets generalization capabilities are probably the most urgent ones since most of the current networks are trained at a different scale (reduced-resolution), and, in general, they are well-fitted on some datasets but fail on others. A recent attempt to address both these issues leverages on a target-adaptive inference scheme operating with a suitable full-resolution loss. On the downside, such an approach pays an additional computational overhead due to the adaptation phase. In this work, we propose a variant of this method with an effective target-adaptation scheme that allows for the reduction in inference time by a factor of ten, on average, without accuracy loss. A wide set of experiments carried out on three different datasets, GeoEye-1, WorldView-2 and WorldView-3, prove the computational gain obtained while keeping top accuracy scores compared to state-of-the-art methods, both model-based and deep-learning ones. The generality of the proposed solution has also been validated, applying the new adaptation framework to different CNN models.

Cyclic Differentiable Architecture Search.

Differentiable ARchiTecture Search, i.e., DARTS, has drawn great attention in neural architecture search. It tries to find the optimal architecture in a shallow search network and then measures its performance in a deep evaluation network. The independent optimization of the search and evaluation networks, however, leaves a room for potential improvement by allowing interaction between the two networks. To address the problematic optimization issue, we propose new joint optimization objectives and a novel Cyclic Differentiable ARchiTecture Search framework, dubbed CDARTS. Considering the structure difference, CDARTS builds a cyclic feedback mechanism between the search and evaluation networks with introspective distillation. First, the search network generates an initial architecture for evaluation, and the weights of the evaluation network are optimized. Second, the architecture weights in the search network are further optimized by the label supervision in classification, as well as the regularization from the evaluation network through feature distillation. Repeating the above cycle results in a joint optimization of the search and evaluation networks and thus enables the evolution of the architecture to fit the final evaluation network. The experiments and analysis on CIFAR, ImageNet and NATS-Bench [95] demonstrate the effectiveness of the proposed approach over the state-of-the-art ones. Specifically, in the DARTS search space, we achieve 97.52% top-1 accuracy on CIFAR10 and 76.3% top-1 accuracy on ImageNet. In the chain-structured search space, we achieve 78.2% top-1 accuracy on ImageNet, which is 1.1% higher than EfficientNet-B0. Our code and models are publicly available at https://github.com/microsoft/Cream.