Pairwise Loss Research Articles

COTR: Efficient Job Task Recognition for Occupational Information Systems with Class-Incremental Learning

Occupation-specific job tasks (OSTs) refer to the duties, responsibilities, and activities associated with a particular occupation, which define the core functions and performance expectations for those engaged in that profession. Efficient recognition and extraction of OSTs from large-scale job description data are essential for establishing a continually updated occupational information system (OIS), such as O*NET, which serves as critical tools for advancing research in work and labor markets. However, this task presents substantial challenges due to its heavy reliance on domain experts for the labor-intensive annotation of job postings, rendering the process time-consuming and difficult to scale for large-scale implementation. To this end, in this paper, we present COTR , a novel data-driven framework designed for the efficient recognition of OSTs from job postings, capable of continually identifying new tasks through class-incremental learning. Specifically, we first employ large language models (LLMs) and prompt learning to develop a three-phase process—“expansion, translation, and generation”—that addresses the critical challenge of the absence of predefined OSTs in non-English labor market data, leveraging O*NET as a foundational reference. Subsequently, we introduce a BERT-based model for OST recognition, incorporating a uniquely designed pair-wise loss function that distills valuable insights from ChatGPT or other LLMs, thereby substantially enhancing recognition performance. In addition, to achieve cost-effective training data annotation, we develop an LLM-based coarse-to-fine candidate OSTs generation algorithm, integrating contrastive active learning to optimize the annotation process through human-machine collaboration. Notably, we design a supervised fine-tuning strategy with a novel encoding technique to optimize LLMs, improving the recall rate of the generated candidate OSTs and achieving up to a 343-fold increase in annotation efficiency compared to traditional manual expert annotation in our experiments. Afterward, we propose an efficient class-incremental learning method that incorporates an out-of-distribution (OOD) detection module for identifying potential novel OSTs and a fine-tuning module to extend the model’s recognition capabilities to include newly discovered tasks. Finally, we construct two real-world datasets using job posting data collected from the labor markets of China and the United States, respectively. Extensive experiments on the real-world datasets, along with two publicly available datasets, have demonstrated the effectiveness of the proposed COTR. Furthermore, several case studies showcase the significant benefits of COTR for various downstream applications in labor market analysis, including analyzing the evolving demand for OSTs, assessing the value of OSTs, and recognizing the relationships between OSTs and associated skills.

Apparatus for producing single strontium atoms in an optical tweezer array

Abstract We outline an experimental setup for efficiently preparing a tweezer array of 88Sr atoms. Our setup uses permanent magnets to maintain a steady-state two-dimensional magneto-optical trap (MOT) which results in a loading rate of up to 108 s−1 at 5 mK for the three-dimensional blue MOT. This enables us to trap 2 × 106 88Sr atoms at 2 μK in a narrow-line red MOT with the 1S0 → 3P1 intercombination transition at 689 nm. With the Sisyphus cooling and pairwise loss processes, single atoms are trapped and imaged in 813 nm optical tweezers, exhibiting a lifetime of 2.5 min. We further investigate the survival fraction of a single atom in the tweezers and characterize the optical tweezer array using a release and recapture technique. Our experimental setup serves as an excellent reference for those engaged in experiments involving optical tweezer arrays, cold atom systems, and similar research.

Imbalanced image classification algorithm based on fine-grained analysis

Fine-grained attribute analysis and data imbalance have always been research hotspots in the field of computer vision. Due to the complexity and diversity of fine-grained attribute images, traditional image classification methods have shortcomings in paying attention to fine-grained attributes of images and perform poorly when dealing with imbalanced data sets. To overcome these problems, this study proposes a fine-grained image threshold classification algorithm based on deep metric learning. By introducing a metric learning method, the focus on fine-grained attributes of images is enhanced. At the same time, by applying pairwise loss and proxy loss, the classification accuracy of the model is improved and the model convergence speed is accelerated. In order to deal with the problem of data imbalance, a classifier based on threshold analysis is designed. The classifier uses threshold analysis technology to achieve multi-level classification of fine-grained images, thereby improving the problem of low classification accuracy of a few categories in imbalanced data sets. Experimental results show that the proposed fine-grained image threshold classification algorithm based on deep metric learning is significantly superior to other methods in terms of classification accuracy.

Towards sharper excess risk bounds for differentially private pairwise learning

Pairwise learning is a vital part of machine learning. It depends on pairs of training instances, and is naturally fit for modeling relationships between samples. However, as a data driven paradigm, it faces huge privacy issues. Differential privacy (DP) is a useful tool to protect the privacy of machine learning, but corresponding excess population risk bounds are loose in existing DP pairwise learning analysis. In this paper, we propose a gradient perturbation algorithm for pairwise learning to get better risk bounds under Polyak–Łojasiewicz condition, including both convex and non-convex cases. Specifically, for the theoretical risk bound in expectation, previous best results are of rates O(pn2ϵ2+1n) and O(pnϵ+1n) under strongly convex condition and convex conditions, respectively. In this paper, we use the on-average stability and achieve an O(min{pn1.5ϵ+p1.5n2.5ϵ3,pn2ϵ2+1n}) bound, significantly improving previous bounds. For the high probability risk bound, previous best results are analyzed by the uniform stability, and O(βnU+pnϵ) excess population risk bounds are achieved under strongly convex or convex conditions, where βnU is the traditional pairwise uniform stability parameter, it is large since it considers the worst case of the loss sensitivity. In this paper, we propose the pairwise locally elastic stability and improve the high probability bound to O(βEn+pnϵ), in which the pairwise locally elastic stability parameter βE≪βnU because it considers the average sensitivity of the pairwise loss function.

Ranking Model Averaging: Ranking Based on Model Averaging

Ranking problems are commonly encountered in practical applications, including order priority ranking, wine quality ranking, and piston slap noise performance ranking. The responses of these ranking applications are often considered as continuous responses, and there is uncertainty on which scoring function is used to model the responses. In this paper, we address the scoring function uncertainty of continuous response ranking problems by proposing a ranking model averaging (RMA) method. With a set of candidate models varied by scoring functions, RMA assigns weights for each model determined by a K-fold crossvalidation criterion based on pairwise loss. We provide two main theoretical properties for RMA. First, we prove that the averaging ranking predictions of RMA are asymptotically optimal in achieving the lowest possible ranking risk. Second, we provide a bound on the difference between the empirical RMA weights and theoretical optimal ones, and we show that RMA weights are consistent. Simulation results validate RMA superiority over competing methods in reducing ranking risk. Moreover, when applied to empirical examples—order priority, wine quality, and piston slap noise—RMA shows its effectiveness in building accurate ranking systems. History: Accepted by Ram Ramesh, Area Editor for Data Science and Machine Learning. Funding: This research was supported by the Beijing Municipal Natural Science Foundation [Grant 1222002], the National Natural Science Foundation of China [Grants 12071457, 12201018, 12301364, 71925007, 72091212, and 72273120], National Quality Infrastructure [Grant 2022YFF0609903], the Chinese Academy of Sciences Project for Young Scientists in Basic Research [Grant YSBR-008], and the Natural Science Foundation of Anhui Province [Grant 2308085QA09]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://doi.org/10.1287/ijoc.2023.0257 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2023.0257 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Code and Data Repository for Ranking Model Averaging: Ranking Based on Model Averaging

Ranking problems are commonly encountered in practical applications, including order priority ranking, wine quality ranking, and piston slap noise performance ranking. The responses of these ranking applications are often considered as continuous responses and there is uncertainty on which scoring function is used to model the responses. In this paper, we address the scoring function uncertainty of continuous response ranking problems by proposing a Ranking Model Averaging (RMA) method. With a set of candidate models varied by scoring functions, RMA assigns weights for each model determined by a K-fold cross-validation criterion based on pairwise loss.

An Efficient Multi-Branch Attention Network for Person Re-Identification

Due to the absence of tailored designs that address challenges such as variations in scale, disparities in illumination, and instances of occlusion, the implementation of current person re-identification techniques remains challenging in practical applications. An Efficient Multi-Branch Attention Network over OSNet (EMANet) is proposed. The structure is composed of three parts, the global branch, relational branch, and global contrastive pooling branch, and corresponding features are obtained from different branches. With the attention mechanism, which focuses on important features, DAS attention evaluates the significance of learned features, awarding higher ratings to those that are deemed crucial and lower ratings to those that are considered distracting. This approach leads to an enhancement in identification accuracy by emphasizing important features while discounting the influence of distracting ones. Identity loss and adaptive sparse pairwise loss are used to efficiently facilitate the information interaction. In experiments on the Market-1501 mainstream dataset, EMANet exhibited high identification accuracies of 96.1% and 89.8% for Rank-1 and mAP, respectively. The results indicate the superiority and effectiveness of the proposed model.

Mask-guided discriminative feature network for occluded person re-identification

In recent years, although research on person re-identification (ReID) has made significant progress, occluded person ReID remains a major challenge. In real-world scenes, persons are often occluded by various obstacles such as vehicles, umbrellas, and other persons. This leads to the noisy interference and the loss of visual information, which result in poor ReID performance of occluded persons. To address this issue, we propose an end-to-end Mask-guided Discriminative Feature Network (MDFNet). First, MDFNet adopts a dual-branch architecture with a shared encoder as the Feature Extraction module for paired images. Each pair of images consist of one image from the training set and its corresponding occluded image generated through an occlusion augmentation strategy. Second, MDFNet utilizes a Mask-guided Discriminative Feature Enhancement and Fusion (MDFEF) module to achieve the fusion and enhancement of global and local features for high-quality person representations. MDFEF module effectively suppresses the interference of occlusion, enriches the representation capacity of person features, and enables the model to focus more on the important features in non-occluded regions. Furthermore, MDFNet introduces a sparse pairwise loss for enabling the model to dynamically adapt to intra-class variations and reduce the negative impact of complex occlusions. The experimental results on four challenging person ReID datasets demonstrate the effectiveness of the proposed method.

Limited Memory Online Gradient Descent for Kernelized Pairwise Learning with Dynamic Averaging

Pairwise learning, an important domain within machine learning, addresses loss functions defined on pairs of training examples, including those in metric learning and AUC maximization. Acknowledging the quadratic growth in computation complexity accompanying pairwise loss as the sample size grows, researchers have turned to online gradient descent (OGD) methods for enhanced scalability. Recently, an OGD algorithm emerged, employing gradient computation involving prior and most recent examples, a step that effectively reduces algorithmic complexity to O(T), with T being the number of received examples. This approach, however, confines itself to linear models while assuming the independence of example arrivals. We introduce a lightweight OGD algorithm that does not require the independence of examples and generalizes to kernel pairwise learning. Our algorithm builds the gradient based on a random example and a moving average representing the past data, which results in a sub-linear regret bound with a complexity of O(T). Furthermore, through the integration of O(√T logT) random Fourier features, the complexity of kernel calculations is effectively minimized. Several experiments with real-world datasets show that the proposed technique outperforms kernel and linear algorithms in offline and online scenarios.

SDGMNet: Statistic-Based Dynamic Gradient Modulation for Local Descriptor Learning

Rescaling the backpropagated gradient of contrastive loss has made significant progress in descriptor learning. However, current gradient modulation strategies have no regard for the varying distribution of global gradients, so they would suffer from changes in training phases or datasets. In this paper, we propose a dynamic gradient modulation, named SDGMNet, for contrastive local descriptor learning. The core of our method is formulating modulation functions with dynamically estimated statistical characteristics. Firstly, we introduce angle for distance measure after deep analysis on backpropagation of pair-wise loss. On this basis, auto-focus modulation is employed to moderate the impact of statistically uncommon individual pairs in stochastic gradient descent optimization; probabilistic margin cuts off the gradients of proportional triplets that have achieved enough optimization; power adjustment balances the total weights of negative pairs and positive pairs. Extensive experiments demonstrate that our novel descriptor surpasses previous state-of-the-art methods in several tasks including patch verification, retrieval, pose estimation, and 3D reconstruction.

On the Effectiveness of Sampled Softmax Loss for Item Recommendation

The learning objective plays a fundamental role to build a recommender system. Most methods routinely adopt either pointwise (e.g., binary cross-entropy) or pairwise (e.g., BPR) loss to train the model parameters, while rarely pay attention to softmax loss, which assumes the probabilities of all classes sum up to 1, due to its computational complexity when scaling up to large datasets or intractability for streaming data where the complete item space is not always available. The sampled softmax (SSM) loss emerges as an efficient substitute for softmax loss. Its special case, InfoNCE loss, has been widely used in self-supervised learning and exhibited remarkable performance for contrastive learning. Nonetheless, limited recommendation work uses the SSM loss as the learning objective. Worse still, none of them explores its properties thoroughly and answers “Does SSM loss suit for item recommendation?” and “What are the conceptual advantages of SSM loss, as compared with the prevalent losses?”, to the best of our knowledge. In this work, we aim at offering a better understanding of SSM for item recommendation. Specifically, we first theoretically reveal three model-agnostic advantages: (1) mitigating popularity bias, which is beneficial to long-tail recommendation; (2) mining hard negative samples, which offers informative gradients to optimize model parameters; and (3) maximizing the ranking metric, which facilitates top- K performance. However, based on our empirical studies, we recognize that the default choice of cosine similarity function in SSM limits its ability in learning the magnitudes of representation vectors. As such, the combinations of SSM with the models that also fall short in adjusting magnitudes (e.g., matrix factorization) may result in poor representations. One step further, we provide mathematical proof that message passing schemes in graph convolution networks can adjust representation magnitude according to node degree, which naturally compensates for the shortcoming of SSM. Extensive experiments on four benchmark datasets justify our analyses, demonstrating the superiority of SSM for item recommendation. Our implementations are available in both TensorFlow 1 and PyTorch. 2

Integrating listwise ranking into pairwise-based image-text retrieval

Image-Text Retrieval (ITR) is essentially a ranking problem. Given a query caption, the goal is to rank candidate images by relevance, from large to small. The current ITR datasets are constructed in a pairwise manner. Image-text pairs are annotated as positive or negative. Correspondingly, ITR models mainly use pairwise losses, such as triplet loss, to learn to rank. Pairwise-based ITR increases positive pair similarity while decreasing negative pair similarity indiscriminately. However, the relevance between dissimilar negative pairs is different. Pairwise annotations cannot reflect this difference in relevance. In the current datasets, pairwise annotations miss many correlations. There are many potential positive pairs among the pairs labeled as negative. Pairwise-based ITR can only rank positive samples before negative samples, but cannot rank negative samples by relevance. In this paper, we integrate listwise ranking into conventional pairwise-based ITR. Listwise ranking optimizes the entire ranking list based on relevance scores. Specifically, we first propose a Relevance Score Calculation (RSC) module to calculate the relevance score of the entire ranked list. Then we choose the ranking metric, Normalised Discounted Cumulative Gain (NDCG), as the optimization objective. We apply a metric smoothing method named Smooth-NDCG (S-NDCG) to ITR, which transforms the non-differentiable NDCG into a differentiable listwise loss. Our listwise ranking approach can be plug-and-play integrated into current pairwise-based ITR models. Experiments on ITR benchmarks show that integrating listwise ranking can improve the performance of current ITR models and provide more user-friendly retrieval results. The code is available at https://github.com/AAA-Zheng/Listwise_ITR.

Improving top-[formula omitted] recommendations using batch approximation for weighted pair-wise loss

In collaborative filtering, matrix factorization and collaborative metric learning are challenged by situations where non-preferred items may appear so close to a user in the feature embedding space that they lead to degrading the recommendation performance. We call such items ‘potential impostor’ risks. Addressing the issues with ‘potential impostor’ is important because it can result in inefficient learning and poor feature extraction. To achieve this, we propose a novel loss function formulation designed to enhance learning efficiency by actively identifying and addressing impostors, leveraging item associations and learning the distribution of negative items. This approach is crucial for models to differentiate between positive and negative items effectively, even when they are closely aligned in the feature space. Here, a loss function is generally an objective optimization function that is defined based on user–item interaction data, through either implicit or explicit feedback. The loss function essentially decides how well a recommendation algorithm performs. In this paper, we introduce and define the concept of ‘potential impostor’, highlighting its impact on learned representation quality and algorithmic efficiency. We tackle the limitations of non-metric methods, like the Weighted Approximate Rank Pairwise Loss (WARP) method, which struggles to capture item–item similarities, by using a ‘similarity propagation’ strategy with a new loss term. Similarly, we address fixed margin inefficiencies in Weighted Collaborative Metric Learning (WCML), through density distribution approximation. This moves potential impostors away from the margin for more robust learning. Additionally, we propose a large-scale batch approximation algorithm for increased detection of impostors, coupled with an active learning strategy for improved top-N recommendation performance. Our extensive empirical analysis across five major and diverse datasets demonstrates the effectiveness and feasibility of our methods, compared to existing techniques with respect to improving AUC, reducing impostor rate, and increasing the average distance metrics. More specifically, our evaluation shows that our two proposed methods outperform the existing state-of-the-art techniques, with an improvement of AUC by 3.5% and 3.7%, NDCG by 1.0% and 9.1% and HR by 1.3% and 3.6%, respectively. Similarly, the impostor rate is decreased by 35% and 18%, and their average distance is increased by 33% and 37%, respectively.

Application of deep metric learning in the verification process of wheel design similarity: Hyundai motor company case

AbstractThe global automobile market experiences quick changes in design preferences. In response to the demand shifts, manufacturers now try to apply new technologies to bring a novel design to market faster. In this paper, we introduce a novel AI application that performs a similarity verification task of wheel designs that aims to solve the real‐world problem. Through the deep metric learning approach, we empirically prove that the cross‐entropy loss does similar tasks as the pairwise losses do in the embedding space. On Jan 2022, we successfully transitioned the verification system to the wheel design process of Hyundai Motor Company's design team and shortened the verification time by 90% to a maximum of 10 min. With a few clicks, the designers at Hyundai Motor could take advantage of our verification system.

Efficient block contrastive learning via parameter-free meta-node approximation

Contrastive learning has recently achieved remarkable success in many domains including graphs. However contrastive loss, especially for graphs, requires a large number of negative samples which is unscalable and computationally prohibitive with a quadratic time complexity. Sub-sampling is not optimal. Incorrect negative sampling leads to sampling bias. In this work, we propose a meta-node based approximation technique that is (a) simple, (b) canproxy all negative combinations (c) in quadratic cluster size time complexity, (d) at graph level, not node level, and (e) exploit graph sparsity. By replacing node-pairs with additive cluster-pairs, we compute the negatives in cluster-time at graph level. The resulting Proxy approximated meta-node Contrastive (PamC) loss, based on simple optimized GPU operations, captures the full set of negatives, yet is efficient with a linear time complexity. By avoiding sampling, we effectively eliminate sample bias. We meet the criterion for larger number of samples, thus achieving block-contrastiveness, which is proven to outperform pair-wise losses. We use learnt soft cluster assignments for the meta-node construction, and avoid possible heterophily and noise added during edge creation. Theoretically, we show that real world graphs easily satisfy conditions necessary for our approximation. Empirically, we show promising accuracy gains over state-of-the-art graph clustering on 6 benchmarks. Importantly, we gain substantially in efficiency; over 2x reduction in training time and over 5x in GPU memory reduction. Additionally, our embeddings, combined with a single learnt linear transformation, is sufficient for node classification; we achieve state-of-the-art on Citeseer classification benchmark. code:https://github.com/gayanku/PAMC

Discriminator-Enhanced Knowledge-Distillation Networks

Query auto-completion (QAC) serves as a critical functionality in contemporary textual search systems by generating real-time query completion suggestions based on a user’s input prefix. Despite the prevalent use of language models (LMs) in QAC candidate generation, LM-based approaches frequently suffer from overcorrection issues during pair-wise loss training and efficiency deficiencies. To address these challenges, this paper presents a novel framework—discriminator-enhanced knowledge distillation (Dis-KD)—for the QAC task. This framework combines three core components: a large-scale pre-trained teacher model, a lightweight student model, and a discriminator for adversarial learning. Specifically, the discriminator aids in discerning generative-level differences between the teacher and the student models. An additional discriminator score loss is amalgamated with the traditional knowledge-distillation loss, resulting in enhanced performance of the student model. Contrary to the stepwise evaluation of each generated word, our approach assesses the entire generation sequence. This method alleviates the prevalent overcorrection issue in the generation process. Consequently, our proposed framework boasts improvements in model accuracy and a reduction in parameter size. Empirical results highlight the superiority of Dis-KD over established baseline methods, with the student model surpassing the teacher model in QAC tasks for sub-word languages.

Developing the Wheel Image Similarity Application with Deep Metric Learning: Hyundai Motor Company Case

The global automobile market experiences quick changes in design preferences. In response to the demand shifts, manufacturers now try to apply new technologies to bring a novel design to market faster. In this paper, we introduce a novel application that performs a similarity verification task of wheel designs using an AI model and cloud computing technology. At Jan 2022, we successfully implemented the application to the wheel design process of Hyundai Motor Company’s design team and shortened the similarity verification time by 90% to a maximum of 10 minutes. We believe that this study is the first to build a wheel image database and empirically prove that the cross-entropy loss does similar tasks as the pairwise losses do in the embedding space. As a result, we successfully automated Hyundai Motor’s verification task of wheel design similarity. With a few clicks, the end-users in Hyundai Motor could take advantage of our application.

Self-training on graph neural networks for recommendation with implicit feedback

Graph Convolutional Networks (GCNs) gain success for recommendation, but still face great challenges of data sparseness and negative sampling in implicit feedback-based recommendation. In particular, they ignore the unique graph structure for information propagation and thus fail to fully explore the potentials of GCN.To tackle the problem, we propose a GCN-based self-training approach, called STL, which exploits the learning results in the training procedure, and the potential relations in the embedding space of GCN. First, to handle the data sparsity, we modify the interaction graph structure by adding edges linking a selected part of users and their potential positive items. Second, we adaptively expand the set of positive samples that are used in the pairwise loss function, which not only supplements the dataset but also avoids sampling noises. Further, similarity of structural neighbors on graph is used to mine hard negative sample for improving the sample quality. Experiments on three representative GCN-based recommenders and four widely used public datasets show that STL alleviates the problem of data sparsity, thereby improving recommendation performance compared to normal training.

Pairwise Metric Learning With Angular Margin for Tag Recommendation

The users on the Internet have been growing exponentially, and tag recommendation can automatically provide users with a selection of tags of interest to meet their personalized needs. Users can utilize these tags to freely annotate their favorite resources, making them efficient and fast in retrieving related resources. Tensor factorization methods are commonly used in tag recommendation at present. These methods model the user × item × tag interactions, transform the latent feature representations of users, items and tags into low-rank matrices and use inner products for prediction. However, the problem of using inner product is that it does not satisfy the triangle inequality, it ignores the distance relationship among entity pairs and cannot capture the fine-grained preference information. Metric learning in recommendation domains focus on using pairwise loss, which assumes that different categories (such as users, items and tags) have fixed margins. Different categories often have different intra-class variations. With fixed margins, it is often difficult to accurately distinguish between positive and negative samples, thus reducing recommendation performance and limiting the expression ability of the model. In this study, the metric learning method is used to explore the distance relationship among user × item × tag triplet, and the existing metric learning based methods (namely LRML, CML, SML) are applied to the tag recommendation. A pairwise metric learning method with angular margin is proposed, named PMLT. The pairwise distance relationship between user-tag and item-tag is modeled for the information of different entities. And an extra angular margin regularizer is added to the original pairwise loss to control the size of angular margin for user-tag and item-tag respectively. The strength of the constrained angular margin regularizer is controlled to dynamically adjust the distance changes of entity. This method constrains the fixed margin and also the angular margin of user-tag and item-tag. Compared with the traditional metric learning method, this method can capture additional relationship structure and has good recommendation performance. Finally, we conducted extensive experiments on two datasets, LastFm and Movielens, and the experimental results showed that the proposed method PMLT outperform the state-of-the-art baseline in the evaluation metrics Recall@N and NDCG@N, and obtain better prediction quality. We also analyze the influence of different parameters and internal components on the performance of the proposed method, which improves the interpretability of the proposed method.

Adversarial Binary Mutual Learning for Semi-Supervised Deep Hashing

Hashing is a popular search algorithm for its compact binary representation and efficient Hamming distance calculation. Benefited from the advance of deep learning, deep hashing methods have achieved promising performance. However, those methods usually learn with expensive labeled data but fail to utilize unlabeled data. Furthermore, the traditional pairwise loss used by those methods cannot explicitly force similar/dissimilar pairs to small/large distances. Both weaknesses limit existing methods' performance. To solve the first problem, we propose a novel semi-supervised deep hashing model named adversarial binary mutual learning (ABML). Specifically, our ABML consists of a generative model GH and a discriminative model DH , where DH learns labeled data in a supervised way and GH learns unlabeled data by synthesizing real images. We adopt an adversarial learning (AL) strategy to transfer the knowledge of unlabeled data to DH by making GH and DH mutually learn from each other. To solve the second problem, we propose a novel Weibull cross-entropy loss (WCE) by using the Weibull distribution, which can distinguish tiny differences of distances and explicitly force similar/dissimilar distances as small/large as possible. Thus, the learned features are more discriminative. Finally, by incorporating ABML with WCE loss, our model can acquire more semantic and discriminative features. Extensive experiments on four common data sets (CIFAR-10, large database of handwritten digits (MNIST), ImageNet-10, and NUS-WIDE) and a large-scale data set ImageNet demonstrate that our approach successfully overcomes the two difficulties above and significantly outperforms state-of-the-art hashing methods.