Query Sample Research Articles

Few-shot classification with task-adaptive semantic feature learning

Few-shot classification aims to learn a classifier that categorizes objects of unseen classes with limited samples. One general approach is to mine as much information as possible from limited samples. This can be achieved by incorporating data from multiple modalities. However, existing multi-modality methods only use additional modality in support samples while adhering to a single modal in query samples. Such approach could lead to information imbalance between support and query samples, which confounds model generalization from support to query samples. Towards this problem, we propose a task-adaptive semantic feature learning mechanism to incorporate semantic features for both support and query samples. The semantic feature learner is trained episodic-wisely by regressing from the feature vectors of the support samples. It is utilized to predict semantic features for the query samples. Such method maintains a consistent training scheme between support and query samples and enables direct model transfer from support to query data, which significantly improves model generalization. We conduct extensive experiments on four benchmarks in both inductive and transductive settings. Results show that the proposed TasNet outperforms state-of-the-art methods with an improvement of 1% to 5% in classification accuracy, demonstrating the superiority of our method. The exhaustive ablation studies further validate the effectiveness of our framework. The code is available at: https://github.com/pmhDL/TasNet

Unconstrained neighbor selection for minimum reconstruction error-based K-NN classifiers

It is essential to define more convincing and applicable classifiers for small datasets. In this paper, a minimum reconstruction error-based K-nearest neighbors (K-NN) classifier is proposed. We propose a new neighbor selection method. In the proposed neighbor selection method, a subset of data that minimize the reconstruction error is assigned as the neighbors of the query sample. Also, there is not any constraint on the distance of the neighbors from the query sample. An {{l}}_{0}-based sparse representation problem is introduced for selecting the neighbors. These neighbors are assigned as neighbors of the minimum reconstruction error-based K-NN classifiers. Three {{l}}_{0}-based minimum reconstruction error-based K-NN classifiers are introduced. These classifiers are less sensitive to the reconstruction coefficients in minimum reconstruction error-based K-NN classifiers and reconstruct the query sample with less error. The results on UCI machine learning repository, UCR time-series archive datasets, and a small subset (16%) of MNIST handwritten digit database demonstrate the suitable performance of the proposed method. The recognition precision increases up to more than 3% in some evaluations.

Active learning for deep object detection by fully exploiting unlabeled data

Object detection is a challenging task that requires a large amount of labeled data to train high-performance models. However, labeling huge amounts of data is expensive, making it difficult to train a good detector with limited labeled data. Existing approaches mitigate this issue via active learning or semi-supervised learning, but there is still room for improvement. In this paper, we propose a novel active learning method for deep object detection that fully exploits unlabeled data by combining the benefits of active learning and semi-supervised learning. Our method first trains an initial model using limited labeled data, then uses self-training and data augmentation strategies to train a semi-supervised model using labeled and unlabeled data. We then select query samples based on informativeness and representativeness from the unlabeled data to further improve the model through semi-supervised training. Experimental results on commonly used object detection datasets demonstrate the effectiveness of our approach, outperforming state-of-the-art methods.

Quantum support vector machine without iteration

Quantum algorithms can enhance machine learning in different aspects. The quantum support vector machine was proposed to improve the performance, in which the Swap Test plays a crucial role in realizing the classification. However, as the Swap Test is destructive, the quantum support vector machine must be repeated in preparing qubits and manipulating operations. This paper proposes a quantum support vector machine based on the amplitude estimation (AE-QSVM) which gets rid of the constraint of repetitive process and saves the quantum resources. At first, a generalized quantum amplitude estimation is introduced in which the initial state can be arbitrary instead of being |0〉. Then, AE-QSVM is trained by the quantum singular value decomposition and a query sample is classified by the generalized quantum amplitude estimation. In AE-QSVM, a high accuracy can be achieved by adding auxiliary qubits instead of repeating the algorithm. The time and space complexity of AE-QSVM are reduced compared with other algorithms. Finally, we ran experiments on the IBM's quantum computer and experimental results demonstrate that classification with a 95% probability of success only uses 12 qubits.

Deep Metric Learning for Scalable Gait-Based Person Re-Identification Using Force Platform Data

Walking gait data acquired with force platforms may be used for person re-identification (re-ID) in various authentication, surveillance, and forensics applications. Current force platform-based re-ID systems classify a fixed set of identities (IDs), which presents a problem when IDs are added or removed from the database. We formulated force platform-based re-ID as a deep metric learning (DML) task, whereby a deep neural network learns a feature representation that can be compared between inputs using a distance metric. The force platform dataset used in this study is one of the largest and the most comprehensive of its kind, containing 193 IDs with significant variations in clothing, footwear, walking speed, and time between trials. Several DML model architectures were evaluated in a challenging setting where none of the IDs were seen during training (i.e., zero-shot re-ID) and there was only one prior sample per ID to compare with each query sample. The best architecture was 85% accurate in this setting, though an analysis of changes in walking speed and footwear between measurement instances revealed that accuracy was 28% higher on same-speed, same-footwear comparisons, compared to cross-speed, cross-footwear comparisons. These results demonstrate the potential of DML algorithms for zero-shot re-ID using force platform data, and highlight challenging cases.

Self-supervised vision transformer-based few-shot learning for facial expression recognition

Facial expression recognition (FER) is embedded in many real-world human-computer interaction tasks, such as online learning, depression recognition and remote diagnosis. However, FER is often hindered by privacy concerns and low recognition accuracy due to inadequate data transfer restrictions on public clouds, insufficient quantities of effective labeled samples and class imbalance. To address the above challenges, we have developed an automatic privacy-preserving learning state recognition system for supervising the quality of online teaching with the cooperation of edge servers and cloud servers to reduce the risk of privacy exposure. In particular, we propose few-shot facial expression recognition with a self-supervised vision transformer (SSF-ViT) by integrating self-supervised learning (SSL) and few-shot learning (FSL) to train a deep learning model with fewer labeled samples. Specifically, a vision transformer (ViT) is jointly pretrained with four self-supervised pretext tasks, including image denoising and reconstruction, image rotation prediction, jigsaw puzzle and masked patch prediction, to obtain a pretrained ViT encoder. Then, the pretrained ViT encoder is used on a lab-controlled labeled FER dataset to extract the spatiotemporal features and implement the FER task to fine-tune the parameters. Finally, we construct prototypes to verify the few-shot classification method for specific expression recognition. Support and query sets are divided in the wild FER dataset, and few-shot classification episodes are constructed. The fine-tuned ViT encoder is used as the feature extractor to build the prototype for each support set category, and the expression classification results are obtained by computing the Euclidean distance between the query samples and the prototypes. The extensive experimental results show that SSF-ViT can achieve recognition accuracies of 74.95%, 66.04%, 63.69% and 90.98% on the FER2013, AffectNet, SFEW 2.0 and RAF-DB datasets, respectively. In addition, SSF-ViT can improve the recognition performance of specific expression categories on these datasets.

Soft sensor method of multimode BOF steelmaking endpoint carbon content and temperature based on vMF-WSAE dynamic deep learning

Abstract The difficulty of endpoint determination in basic oxygen furnace (BOF) steelmaking lies in achieving accurate real-time measurements of carbon content and temperature. For the characteristics of serious nonlinearity between process data, deep learning can perform excellent nonlinear feature representation for complex structural data. However, there is a process drift phenomenon in BOF steelmaking, and the existing deep learning-based soft sensor models cannot adapt to changes in the characteristics of samples, which may lead to their performance degradation. To deal with this problem, considering the characteristics of multimode distribution of process data, an adaptive updating deep learning model based on von-Mises Fisher (vMF) mixture model and weighted stacked autoencoder is proposed. First, the stacked autoencoder (SAE) and vMF mixture model are constructed for complex structural data, which can initially establish nonlinear mapping relationships and division of different distributions. Second, for each query sample, the basic SAE network will perform online adaptive fine-tuning according to its data with the same distribution to achieve dynamic updating. Moreover, each sample is assigned a weight according to its similarity with the query sample. Through the designed weighted loss function, the updated deep network will better match the working conditions of the query sample. Experimental studies with numerical examples and actual BOF steelmaking process data are provided to demonstrate the effectiveness of the proposed method.

Cycle optimization metric learning for few-shot classification

Metric learning methods are widely used in few-shot learning due to their simplicity and effectiveness. Most existing methods directly predict query labels by comparing the similarity between support and query samples. In this paper, we design a cycle optimization metric network for few-shot classification task that optimizes model performance based on loop-prediction of the labels of query samples and support samples. Specifically, we construct a forward network and reverse network based on a geometric algebra Graph Neural Network (GA-GNN). These two networks form the loop prediction from support samples to query samples and then back to support samples, guided by a cycle-consistency loss. We also introduce an optimization module that is able to correct the predicted results of query samples to further improve the network performance. Our extensive experimental results demonstrate that the proposed cycle optimization metric network outperforms existing state-of-the-art few-shot learning methods on classification tasks.

ShrewdAttack: Low Cost High Accuracy Model Extraction

Machine learning as a service (MLaaS) plays an essential role in the current ecosystem. Enterprises do not need to train models by themselves separately. Instead, they can use well-trained models provided by MLaaS to support business activities. However, such an ecosystem could be threatened by model extraction attacks-an attacker steals the functionality of a trained model provided by MLaaS and builds a substitute model locally. In this paper, we proposed a model extraction method with low query costs and high accuracy. In particular, we use pre-trained models and task-relevant data to decrease the size of query data. We use instance selection to reduce query samples. In addition, we divided query data into two categories, namely low-confidence data and high-confidence data, to reduce the budget and improve accuracy. We then conducted attacks on two models provided by Microsoft Azure as our experiments. The results show that our scheme achieves high accuracy at low cost, with the substitution models achieving 96.10% and 95.24% substitution while querying only 7.32% and 5.30% of their training data on the two models, respectively. This new attack approach creates additional security challenges for models deployed on cloud platforms. It raises the need for novel mitigation strategies to secure the models. In future work, generative adversarial networks and model inversion attacks can be used to generate more diverse data to be applied to the attacks.

Ranked Similarity Weighting and Top-nk Sampling in Deep Metric Learning

Deep metric learning has been widely used in many visual tasks. Its key idea is to increase the similarity of positive samples and decrease the similarity of negative samples through network training. To achieve this purpose, many studies excessively extend the distance between the query sample and hard negative samples. This may compress the distance between similar samples of other classes, causing these samples to cluster together. We call this phenomenon Negative Sample Aggregation. To address this problem, first, we propose a weighting method based on the Ranking Similarity of sample pairs, short for RS. The proposed weighting method can not only enlarge the distance between the query sample and hard negative samples, but also maintain the embedding distribution of proximal negative samples. Second, we propose a Top-nk sampling method, which can dynamically adjust the sampling strategy according to the distribution of a dataset. It solves the problem that the descent direction of the network gradient is inconsistent with the optimization target. The effectiveness of our methods is evaluated by extensive experiments on four public datasets and compared with that of other state-of-the-art methods. The results show that the proposed method obtains excellent performance, reaching 67.8% on CUB-200-2011 and 85.2% on Cars-196 at Recall@1.

Similarity function for one-shot learning to enhance the flexibility of myoelectric interfaces.

This study aims to develop a flexible myoelectric pattern recognition (MPR) method based on one-shot learning, which enables convenient switching across different usage scenarios, thereby reducing the re-training burden. First, a one-shot learning model based on a Siamese neural network was constructed to assess the similarity for any given sample pair. In a new scenario involving a new set of gestural categories and/or a new user, just one sample of each category was required to constitute a support set. This enabled the quick deployment of the classifier suitable for the new scenario, which decided for any unknown query sample by selecting the category whose sample in the support set was quantified to be the most like the query sample. The effectiveness of the proposed method was evaluated by experiments conducting MPR across diverse scenarios. The proposed method achieved high recognition accuracy of over 89% under the cross-scenario conditions, and it significantly outperformed other common one-shot learning methods and conventional MPR methods (p < 0.01). This study demonstrates the feasibility of applying one-shot learning to rapidly deploy myoelectric pattern classifiers in response to scenario change. It provides a valuable way of improving the flexibility of myoelectric interfaces toward intelligent gestural control with extensive applications in medical, industrial, and consumer electronics.

A Dynamic Time Warping Based Locally Weighted LSTM Modeling for Temperature Prediction of Recycled Aluminum Smelting

In the process of recycled aluminum smelting, timely measurement of the temperature of the smelting furnace is very important for the aluminum yield and quality. However, it is sometimes difficult or costly to measure the temperature in a timely manner due to the high temperature and pressure environment in the furnace. To tackle this problem, a soft sensor modeling framework which combines an operating condition classification and a prediction model based on locally sample-weighted long short-term memory (LSTM) neural network is proposed. In the operating condition classification, a hybrid of dynamic time warping (DTW) based fuzzy c-means and convolutional neural network is used to cluster the training samples and to classify the query samples. In the prediction model, the dynamic time warping and locally sample-weighted technique are introduced to LSTM to solve time-varying and strong nonlinear problems of the process. By adopting the method of classifying the operating conditions of the query samples before temperature prediction, the prediction time can be effectively reduced and the prediction accuracy can be maintained. The results of the experiment show that the proposed method can meet the prediction accuracy and time efficiency requirements of the regenerative aluminum smelting furnace.

SMAM: Self and Mutual Adaptive Matching for Skeleton-Based Few-Shot Action Recognition.

This paper focuses on skeleton-based few-shot action recognition. Since skeleton is essentially a sparse representation of human action, the feature maps extracted from it, through a standard encoder network in the few-shot condition, may not be sufficiently discriminative for some action sequences that look partially similar to each other. To address this issue, we propose a self and mutual adaptive matching (SMAM) module to convert such feature maps into more discriminative feature vectors. Our method, named as SMAM-Net, first leverages both the temporal information associated with each individual skeleton joint and the spatial relationship among them for feature extraction. Then, the SMAM module adaptively measures the similarity between labeled and query samples and further carries out feature matching within the query set to distinguish similar skeletons of various action categories. Experimental results show that the SMAM-Net outperforms other baselines on the large-scale NTU RGB+D 120 dataset in the tasks of one-shot and five-shot action recognition. We also report our results on smaller datasets including NTU RGB+D 60, SYSU and PKU-MMD to demonstrate that our method is reliable and generalises well on different datasets. Codes and the pretrained SMAM-Net will be made publicly available.

Query by Example in Remote Sensing Image Archive Using Enhanced Deep Support Vector Data Description

This article studies remote sensing image retrieval using kernel-based support vector data description (SVDD). We exploit deep SVDD, which is a well-known method for one-class classification to recover the most relevant samples from the archive. To this end, a deep neural network (DNN) is jointly trained to map the data into a hypersphere of minimum volume in the latent space. It is expected that similar samples to the query are compressed inside of the hypersphere. The closest embedding to the center of the hypersphere is related to the most relevant sample to query. We enhance deep SVDD by injecting the statistical information of data to the DNN by means of additional terms in the cost function. The first enhancement method takes advantage of covariance regularization of batches of the training set to penalize unnecessary redundancy and minimize the correlation between the different dimensions of the embedding. The second method involves unlocking the hypersphere's predefined center while preventing network divergence during training. Therefore, two parameters are designed to control the importance of the drifting of the center and the importance of a fixed predefined center (convergence), respectively. This has been implemented by considering the average of batches of embedding in each iteration as the updated center. This pushes irrelevant samples away from query samples, making data clustering easier for the DNN. The performance of the proposed methods is evaluated on benchmark datasets.

Deep Meta-Learning With 1D-CNNs for Surface Deterioration Recognition of Overhead Conductors of Electricity Grid

The surface deterioration process of aluminum-stranded conductors in high-voltage alternating current (ac) electricity grids is naturally gradual, dynamic, and slow, which will greatly influence the remaining life of conductors. An effective method for deterioration recognition of conductors is to design an evaluation model with energy-dispersive X-ray spectroscopy (EDS). The small number of labeled samples is commonly available for the deterioration recognition task. The conventionally successful deep learning (DL)-based applications are dependent on large-scale data to train an effective model, which restricts the application of DL in the aging recognition scenario. A deep meta-learning (DML) model with 1-D-convolutional neural networks (1D-CNNs) is proposed for the recognition of deteriorated conductors based on EDS, which learns the representation of each support class by the embedding function. Then, it performs classification tasks by comparing the Euclidean distance between the query sample and this representation. A comprehensive systematic comparison with nonneural network methods, conventional DL, and deep transfer learning shows that the proposed model substantially improves the classification accuracy of aging conductors.

Collaborative Contrastive Refining for Weakly Supervised Person Search.

Weakly supervised person search involves training a model with only bounding box annotations, without human-annotated identities. Clustering algorithms are commonly used to assign pseudo-labels to facilitate this task. However, inaccurate pseudo-labels and imbalanced identity distributions can result in severe label and sample noise. In this work, we propose a novel Collaborative Contrastive Refining (CCR) weakly-supervised framework for person search that jointly refines pseudo-labels and the sample-learning process with different contrastive strategies. Specifically, we adopt a hybrid contrastive strategy that leverages both visual and context clues to refine pseudo-labels, and leverage the sample-mining and noise-contrastive strategy to reduce the negative impact of imbalanced distributions by distinguishing positive samples and noise samples. Our method brings two main advantages: 1) it facilitates better clustering results for refining pseudo-labels by exploring the hybrid similarity; 2) it is better at distinguishing query samples and noise samples for refining the sample-learning process. Extensive experiments demonstrate the superiority of our approach over the state-of-the-art weakly supervised methods by a large margin (more than 3% mAP on CUHK-SYSU). Moreover, by leveraging more diverse unlabeled data, our method achieves comparable or even better performance than the state-of-the-art supervised methods.

Multistage Relation Network With Dual-Metric for Few-Shot Hyperspectral Image Classification

Recently, few-shot learning (FSL) has exhibited great potentials in hyperspectral image (HSI) classification due to its promising performance under few training samples. Although existing FSL methods have achieved great success, some limitations can still be witnessed. On the one hand, current methods mainly rely on the single metric to identify, which cannot effectively represent the class distribution with few labeled samples. On the other hand, existing methods usually only use the last deep feature of feature extractor, which may lead to the under-utilization of scarce labeled samples. To overcome the above issues, a novel multistage relation network with dual-metric (DM-MRN) is proposed for few-shot HSI classification. Firstly, a sample recombination strategy is designed to increase the variety of classification tasks in training period. Secondly, an embedding module is employed to extract deep features of the input image patches. Thirdly, we propose two relation modules: image-to-class (I2C) block and image-to-image (I2I) block. I2C block is designed to compute I2C-level relation score between second-order features, and I2I block is conceived to generate I2I-level relation score between first-order features. Finally, DM-MRN is constructed by integrating one embedding module, two I2C blocks, and one I2I block. In addition, an adaptive weighting strategy is designed to fuse the obtained relation scores, and classification can be achieved by assigning each query sample to the class with the highest value of the fused relation score. Extensive experiments carried out on five popular HSI data sets demonstrate that the proposed method outperforms other traditional and advanced models under few training samples in terms of classification accuracy and generalization performance, i.e., the performance improvement in terms of OA is around 0.30%-27.98% under 10 labeled samples per class.

Few-Shot Segmentation for Prohibited Items Inspection With Patch-Based Self-Supervised Learning and Prototype Reverse Validation

Prohibited items inspection using X-ray screening is essential for maintaining border and transport security and reducing the risk of crime and terrorist attacks. The difficulty in prohibited items inspection lies in automatically and accurately detecting prohibited items in complex X-ray images and limited access to X-ray images containing prohibited items. Few-shot segmentation aims at learning with limited examples and assigning a category label to each image pixel. However, current few-shot methods are mostly full-supervised and less robust to the prohibited items categories that did not appear during training process. Self-Supervised Learning has been developed to learn to perform segmentation from lots of unlabeled examples. In this paper, we propose a method for few-shot prohibited items segmentation tasks which utilize lots of unlabeled data and better leverage the representation of input samples during model training process. Specifically, a patch-based self-supervised embedding network is firstly devised as the base learner to learn an abstract representation of the observation from a lot of unlabeled samples. Then we apply few-shot learning and generate abstract representation related to prohibited items from support sample within the embedding space, which is followed by obtaining the corresponding class-specific prototype representations via masked average pooling. The distance between each pixel of query sample and prototypes are calculated to predict the label of each pixel. Moreover, prototype reverse validation strategy (PRV) is proposed to further exploit the support representation to assist training. Extensive experiments show that our proposed method outperforms the state-of-the-art by delivering a higher accuracy on automated prohibited items inspection and requiring less labeled samples, which demonstrates the potential application of our method in real-world inspections.

A Bayesian Meta-Learning-Based Method for Few-Shot Hyperspectral Image Classification

Few-shot learning provides a new way to solve the problem of insufficient training samples in hyperspectral classification. It can implement reliable classification under several training samples by learning meta-knowledge from similar tasks. However, most existing works perform frequency statistics, which may suffer from the prevalent uncertainty in point estimates (PEs) with limited training samples. To overcome this problem, we reconsider the hyperspectral image few-shot classification (HSI-FSC) task as a hierarchical probabilistic inference from a Bayesian view and provide a careful process of meta-learning probabilistic inference. We introduce a prototype vector for each class as latent variables and adopt distribution estimates (DEs) for them to obtain their posterior distribution. The posterior of the prototype vectors is maximized by updating the parameters in the model via the prior distribution of HSI and labeled samples. The features of the query samples are matched with prototype vectors drawn from the posterior; thus, a posterior predictive distribution over the labels of query samples can be inferred via an amortized Bayesian variational inference approach. Experimental results on four datasets demonstrate the effectiveness of our method. Especially given only three to five labeled samples, the method achieves noticeable upgrades of overall accuracy (OA) against competitive methods.

A Survey of Deep Active Learning for Foundation Models

Active learning (AL) is an effective sample selection approach that annotates only a subset of the training data to address the challenge of data annotation, and deep learning (DL) is data-intensive and reliant on abundant training data. Deep active learning (DeepAL) benefits from the integration of AL and DL, offering an efficient solution that balances model performance and annotation costs. The importance of DeepAL has been increasingly recognized with the emergence of large foundation models that depend heavily on substantial computational resources and extensive training data. This survey endeavors to provide a comprehensive overview of DeepAL. Specifically, we first analyze and summarize various sample query strategies, data querying considerations, model training paradigms, and real-world applications of DeepAL. In addition, we discuss the challenges that arise in the era of foundation models and propose potential directions for future AL research. The survey aims to bridge a gap in the existing literature by organizing and summarizing current approaches, offering insights into DeepAL and highlighting the necessity of developing specialized DeepAL techniques tailored to foundation models. By critically examining the current state of DeepAL, this survey contributes to a more profound understanding of the field and serves as a guide for researchers and practitioners interested in DeepAL techniques.