Matrix Factorization Hashing Research Articles

EDMH: Efficient discrete matrix factorization hashing for multi-modal similarity retrieval

Hashing has been an emerging topic and has recently attracted widespread attention in multi-modal similarity search applications. However, most existing approaches rely on relaxation schemes to generate binary codes, leading to large quantization errors. In addition, amounts of existing approaches embed labels into the pairwise similarity matrix, leading to expensive time and space costs and losing category information. To address these issues, we propose an Efficient Discrete Matrix factorization Hashing (EDMH). Specifically, EDMH first learns the latent subspaces for individual modality through matrix factorization strategy, which preserves the semantic structure representation information of each modality. In particular, we develop a semantic label offset embedding learning strategy, improving the stability of label embedding regression. Furthermore, we design an efficient discrete optimization scheme to generate compact binary codes discretely. Eventually, we present two efficient learning strategies EDMH-L and EDMH-S to pursue high-quality hash functions. Extensive experiments on various widely-used databases verify that the proposed algorithms produce significant performance and outperform some state-of-the-art approaches, with an average improvement of 2.50% (for Wiki), 2.66% (for MIRFlickr) and 2.25% (for NUS-WIDE) over the best available results, respectively.

Discrete matrix factorization cross-modal hashing with multi-similarity consistency

AbstractRecently, matrix factorization-based hashing has gained wide attention because of its strong subspace learning ability and high search efficiency. However, some problems need to be further addressed. First, uniform hash codes can be generated by collective matrix factorization, but they often cause serious loss, degrading the quality of hash codes. Second, most of them preserve the absolute similarity simply in hash codes, failing to capture the inherent semantic affinity among training data. To overcome these obstacles, we propose a Discrete Multi-similarity Consistent Matrix Factorization Hashing (DMCMFH). Specifically, an individual subspace is first learned by matrix factorization and multi-similarity consistency for each modality. Then, the subspaces are aligned by a shared semantic space to generate homogenous hash codes. Finally, an iterative-based discrete optimization scheme is presented to reduce the quantization loss. We conduct quantitative experiments on three datasets, MSCOCO, Mirflickr25K and NUS-WIDE. Compared with supervised baseline methods, DMCMFH achieves increases of $$0.22\%$$ 0.22 % , $$3.00\%$$ 3.00 % and $$0.79\%$$ 0.79 % on the image-query-text tasks for three datasets respectively, and achieves increases of $$0.21\%$$ 0.21 % , $$1.62\%$$ 1.62 % and $$0.50\%$$ 0.50 % on the text-query-image tasks for three datasets respectively.

Semantic Constraints Matrix Factorization Hashing for cross-modal retrieval

Cross-modal hashing methods have attracted considerable attention due to their low memory usage and high query speed in large-scale cross-modal retrieval. During the encoding process, there still remains two crucial bottlenecks: how to equip hash codes with cross-modal semantic information, and how to rapidly obtain hash codes. In this paper, we propose Semantic Constraints Matrix Factorization Hashing (SCMFH) which simultaneously considers modality-specific and cross-modal semantic information for hash codes with closed solution. Specifically, the original representation of each modality is factorized into modality-specific semantic representations, and then the cross-modal semantic similarity matrices is used to constrain correlation of modality-specific representation. Finally, all the modality-specific representations are regressed to the common hash codes, and the fast hash codes with a closed solution is obtained. Extensive experimental have been carried out on public datasets. The results show that the method outperforms many current cross-modal hashing methods in terms of mean average precision (mAP), up to 2.7%.

Robust and discrete matrix factorization hashing for cross-modal retrieval

Hashing based methods have gained great success for cross-modal similarity search, due to its fast query speed and low storage cost. However, there are some challenging problems that need to be further solved: 1) Many approaches are sensitive to noises and outliers, because ℓ2 norm is utilized in the objective function, the error may be amplified. 2) Most existing methods take relaxation or rounding scheme to generate binary codes, causing a large quantization loss. 3) Many supervised cross-media algorithms usually take a large n×n matrix to preserve the similarity relationship, leading to large calculation and making them unscalable. To mitigate these challenges, we develop a novel cross-media search algorithm, i.e., robust and discrete matrix factorization hashing, dubbed RDMH. The method takes a two-step strategy. In the first phase, the ℓ2,1 norm is utilized to improve the robustness, which makes our model not sensitive to noises and outliers. We can learn the hash codes directly by the proposed discrete optimization method instead of relaxation scheme, avoiding the large quantization loss. Moreover, RDMH correlates the hash codes and semantic labels directly instead of manipulating the large similarity matrix. In the second phase, we propose an autoencoder strategy to learn the hash functions, more valuable information can be preserved and making the hash functions more powerful. Comprehensive experiments on several databases demonstrate the superior performance and efficacy of the developed RDMH.

Label Consistent Flexible Matrix Factorization Hashing for Efficient Cross-modal Retrieval

Hashing methods have sparked a great revolution on large-scale cross-media search due to its effectiveness and efficiency. Most existing approaches learn unified hash representation in a common Hamming space to represent all multimodal data. However, the unified hash codes may not characterize the cross-modal data discriminatively, because the data may vary greatly due to its different dimensionalities, physical properties, and statistical information. In addition, most existing supervised cross-modal algorithms preserve the similarity relationship by constructing an n × n pairwise similarity matrix, which requires a large amount of calculation and loses the category information. To mitigate these issues, a novel cross-media hashing approach is proposed in this article, dubbed label flexible matrix factorization hashing (LFMH). Specifically, LFMH jointly learns the modality-specific latent subspace with similar semantic by the flexible matrix factorization. In addition, LFMH guides the hash learning by utilizing the semantic labels directly instead of the large n × n pairwise similarity matrix. LFMH transforms the heterogeneous data into modality-specific latent semantic representation. Therefore, we can obtain the hash codes by quantifying the representations, and the learned hash codes are consistent with the supervised labels of multimodal data. Then, we can obtain the similar binary codes of the corresponding modality, and the binary codes can characterize such samples flexibly. Accordingly, the derived hash codes have more discriminative power for single-modal and cross-modal retrieval tasks. Extensive experiments on eight different databases demonstrate that our model outperforms some competitive approaches.

Cross-modal retrieval via label category supervised matrix factorization hashing

• The proposed model can achieve efficient and accurate cross-modal retrieval task. • The proposed model integrates data similarity and category information to improve performance. • The author validates the effectiveness of the proposed model through experiments on several public data sets. Due to the emergence and development of big data, cross-modal hash retrieval has become progressively more important in large-scale multi-modal retrieval tasks depending on its accuracy and efficiency. It completes the retrieval task in a common low-dimensional space by finding a common semantic space for heterogeneous data of different modalities. Recently, many works have concentrated on supervised cross-modal hashing and achieved higher retrieval accuracy. However, there are still many challenges in how to maintain the local geometric structure of the original space in the public space and how to use the supervision information efficiently. To deal with such issues, this paper proposes a hash retrieval method that incorporates supervised information based on matrix factorization (LCSMFH) by maintain the inter-modal and the intra-modal similarity in the original space and make the most of the label information to improve the retrieval task effect. Through experiments on two benchmark data sets, our method is more effective and outperforms state-of-the-art methods.

Modality-specific matrix factorization hashing for cross-modal retrieval

Cross-modal retrieval has been attracted attentively in the past years. Recently, the collective matrix factorization was proposed to learn the common representations for cross-modal retrieval based on assumption that the pairwise data from different modalities should have the same common semantic representations. However, this unified common representation could inherently sacrifice the modality-specific representations for each modality because the distributions and representations of different modalities are inconsistent. To mitigate this problem, in this paper, we propose Modality-specific Matrix Factorization Hashing (MsMFH) via alignment, which learns the modality-specific semantic representation for each modality and then aligns the representations via the correlation information. Specifically, we factorize the original feature representations into individual latent semantic representations, and then align the distributions of individual latent semantic representations via an orthogonal transformation. Then, we embed the class label into the hash codes learning via latent semantic space, and obtain hash codes directly by an efficient optimization with a closed solution. Extensive experimental results on three public datasets demonstrate that the proposed method outperforms to many existing cross-modal hashing methods up to 3% in term of mean average precision (mAP).

Joint and individual matrix factorization hashing for large-scale cross-modal retrieval

Multimodal hashing methods have gained considerable attention in recent years due to their effectiveness and efficiency for cross-modal similarity searches. Existing multimodal hashing methods either learn unified hash codes for different modalities or learn individual hash codes for each modality and then explore cross-correlations between them. Generally, learning unified hash codes tends to preserve the shared properties of multimodal data and learning individual hash codes tends to preserve the specific properties of each modality. There remains a crucial bottleneck regarding how to learn hash codes that simultaneously preserve the shared properties and specific properties of multimodal data. Therefore, we present a joint and individual matrix factorization hashing (JIMFH) method, which not only learns unified hash codes for multimodal data to preserve their common properties but also learns individual hash codes for each modality to retain its specific properties. The proposed JIMFH learns unified hash codes by joint matrix factorization, which jointly factorizes all modalities into a shared latent semantic space. In addition, JIMFH learns individual hash codes by individual matrix factorization, which separately factorizes each modality into a modal-specific latent semantic space. Finally, unified hash codes and individual hash codes are combined to obtain the final hash codes. In this way, hash codes learned by JIMFH can preserve both the shared properties and specific properties of multimodal data, and therefore the retrieval performance is enhanced. Comprehensive experiments show that the proposed JIMFH performs much better than many state-of-the-art methods on cross-modal retrieval applications.

Supervised Matrix Factorization Hashing With Quantitative Loss for Image-Text Search

Image-text hashing approaches have been widely applied in large-scale similarity search applications due to their efficiency in both search speed and storage efficiency. Most recent supervised hashing approaches learn a hash function by constructing a pairwise similarity matrix or directly learning the hash function and hash code (i.e.,1 or -1) procedure based on class labels. However, the former suffers from high training complexity and storage cost, and the latter ignores the semantic correlation of the original data, both of which prevent discriminative hash codes. To this end, we propose a novel discrete hashing algorithm called supervised matrix factorization hashing with quantitative loss (SMFH-QL). The proposed SMFH-QL first generates hash codes via the class label, avoiding the construction of a pairwise similarity; then, matrix factorization is used to design hash codes from original image-text data, thereby eliminating the impact of class labels and reducing the quantization error. Moreover, we introduce a quantitative loss function term to learn hash codes by incorporating class labels and the original data information, facilitating learning a similarity-preserving hash function in image-text search. Extensive experiments show that SMFH-QL outperforms several existing hashing methods on three representative datasets.

Adaptive Hashing With Sparse Matrix Factorization.

Hashing offers a desirable and effective solution for efficiently retrieving the nearest neighbors from large-scale data because of its low storage and computation costs. One of the most appealing techniques for hashing learning is matrix factorization. However, most hashing methods focus only on building the mapping relationships between the Euclidean and Hamming spaces and, unfortunately, underestimate the naturally sparse structures of the data. In addition, parameter tuning is always a challenging and head-scratching problem for sparse hashing learning. To address these problems, in this article, we propose a novel hashing method termed adaptively sparse matrix factorization hashing (SMFH), which exploits sparse matrix factorization to explore the parsimonious structures of the data. Moreover, SMFH adopts an orthogonal transformation to minimize the quantization loss while deriving the binary codes. The most distinguished property of SMFH is that it is adaptive and parameter-free, that is, SMFH can automatically generate sparse representations and does not require human involvement to tune the regularization parameters for the sparse models. Empirical studies on four publicly available benchmark data sets show that the proposed method can achieve promising performance and is competitive with a variety of state-of-the-art hashing methods.

Semantic-enhanced discrete matrix factorization hashing for heterogeneous modal matching

The simultaneous hash representation of heterogeneous modalities has shown its excellent performance in the multi-modal learning community. However, the binary discrete constraint of hash brings a great challenge to the common hash representation of heterogeneous modal data. To resolve the issue, this paper proposes a novel discrete hash method, referred to as Semantic enhanced Discrete Matrix Factorization Hashing (SDMFH). SDMFH directly extracts the common discrete hash representation of all modalities from the reconstructed semantic intermodal similarity graph, which makes the hash codes more discriminative. Meanwhile, the semantic labels are regressed to the extracted discrete hash book, so as to further strengthen the discriminating ability of the learned discrete hash book. Moreover, a linear embedding from the kernel space of the original data to the Hamming space is explored to generate the common hash codes of each modality to ensure the consistency of the hash codes of heterogeneous modalities. More importantly, we develop an efficient discrete iterative algorithm based on Stiefel manifold that can directly learn the discrete hash book in a closed form, thus both avoiding quantization loss caused by discrete relaxation and reducing computational complexity. Experimental results on three benchmark data sets show that SDMFH performs better than several state-of-the-art methods for heterogeneous modal matching tasks.

Bidirectional Discrete Matrix Factorization Hashing for Image Search.

Unsupervised image hashing has recently gained significant momentum due to the scarcity of reliable supervision knowledge, such as class labels and pairwise relationship. Previous unsupervised methods heavily rely on constructing sufficiently large affinity matrix for exploring the geometric structure of data. Nevertheless, due to lack of adequately preserving the intrinsic information of original visual data, satisfactory performance can hardly be achieved. In this article, we propose a novel approach, called bidirectional discrete matrix factorization hashing (BDMFH), which alternates two mutually promoted processes of 1) learning binary codes from data and 2) recovering data from the binary codes. In particular, we design the inverse factorization model, which enforces the learned binary codes inheriting intrinsic structure from the original visual data. Moreover, we develop an efficient discrete optimization algorithm for the proposed BDMFH. Comprehensive experimental results on three large-scale benchmark datasets show that the proposed BDMFH not only significantly outperforms the state-of-the-arts but also provides the satisfactory computational efficiency.

SCRATCH: A Scalable Discrete Matrix Factorization Hashing Framework for Cross-Modal Retrieval

In this paper, we present a novel supervised cross-modal hashing framework, namely Scalable disCRete mATrix faCtorization Hashing (SCRATCH). First, it utilizes collective matrix factorization on original features together with label semantic embedding, to learn the latent representations in a shared latent space. Thereafter, it generates binary hash codes based on the latent representations. During optimization, it avoids using a large $n\times n$ similarity matrix and generates hash codes discretely. Besides, based on different objective functions, learning strategy, and features, we further present three models in this framework, i.e., SCRATCH-o, SCRATCH-t, and SCRATCH-d. The first one is a one-step method, learning the hash functions and the binary codes in the same optimization problem. The second is a two-step method, which first generates the binary codes and then learns the hash functions based on the learned hash codes. The third one is a deep version of SCRATCH-t, which utilizes deep neural networks as hash functions. The extensive experiments on two widely used benchmark datasets demonstrate that SCRATCH-o and SCRATCH-t outperform some state-of-the-art shallow hashing methods for cross-modal retrieval. The SCRATCH-d also outperforms some state-of-the-art deep hashing models.

Unsupervised cross-modal retrieval via Multi-modal graph regularized Smooth Matrix Factorization Hashing

The existing cross-modal hashing methods often encounter quantization loss which is caused by relaxing discrete hash codes in the process of cross-modal retrieval. To counter this problem, a Multi-modal graph regularized Smooth matrix Factorization Hashing (MSFH) approach is represented for unsupervised cross-modal retrieval. In the proposal framework, a smooth matrix generated by a control parameter is introduced into the matrix decomposition model, which can guarantee the sparsity of the dictionaries learned and the extracted common features at the same time, thus reducing the quantization loss in the hashing process. Furthermore, to preserve the topology of the original data, a multi-modal graph regularization term is drawn into the model, which consists of two parts. One is the intra-modal similarity graph which is used to preserve the geometric structure of each modality. The other is the inter-modal similarity graph reconstructed by the symmetric nonnegative matrix factorization, which is employed to soften the structure difference between modalities. The goal of MSFH is to learn unified hash-codes for multi-modal data in a shared latent semantic space in which the similarity of different modalities can be estimated effectively. And the corresponding experimental results on three benchmark data sets demonstrate the superiority of the proposed approach over several state-of-the-art cross-modality hashing approaches.

Efficient cross-modal retrieval via flexible supervised collective matrix factorization hashing

Cross-modal retrieval has recently drawn much attention in multimedia analysis, and it is still a challenging topic mainly attributes to its heterogeneous nature. In this paper, we propose a flexib...

Label Consistent Matrix Factorization Hashing for Large-Scale Cross-Modal Similarity Search.

Multimodal hashing has attracted much interest for cross-modal similarity search on large-scale multimedia data sets because of its efficiency and effectiveness. Recently, supervised multimodal hashing, which tries to preserve the semantic information obtained from the labels of training data, has received considerable attention for its higher search accuracy compared with unsupervised multimodal hashing. Although these algorithms are promising, they are mainly designed to preserve pairwise similarities. When semantic labels of training data are given, the algorithms often transform the labels into pairwise similarities, which gives rise to the following problems: (1) constructing pairwise similarity matrix requires enormous storage space and a large amount of calculation, making these methods unscalable to large-scale data sets; (2) transforming labels into pairwise similarities loses the category information of the training data. Therefore, these methods do not enable the hash codes to preserve the discriminative information reflected by labels and, hence, the retrieval accuracies of these methods are affected. To address these challenges, this paper introduces a simple yet effective supervised multimodal hashing method, called label consistent matrix factorization hashing (LCMFH), which focuses on directly utilizing semantic labels to guide the hashing learning procedure. Considering that relevant data from different modalities have semantic correlations, LCMFH transforms heterogeneous data into latent semantic spaces in which multimodal data from the same category share the same representation. Therefore, hash codes quantified by the obtained representations are consistent with the semantic labels of the original data and, thus, can have more discriminative power for cross-modal similarity search tasks. Thorough experiments on standard databases show that the proposed algorithm outperforms several state-of-the-art methods.

Efficient cross-modal retrieval via flexible supervised collective matrix factorization hashing

Cross-modal retrieval has recently drawn much attention in multimedia analysis, and it is still a challenging topic mainly attributes to its heterogeneous nature. In this paper, we propose a flexible supervised collective matrix factorization hashing (FS-CMFH) to efficient cross-modal retrieval. First, we exploit a flexible collective matrix factorization framework to jointly learn the individual latent space of similar semantic with respected to each modality. Meanwhile, the label consistency across different modalities is simultaneously exploited to preserve both intra-modal and inter-modal semantics within these similar latent semantic spaces. Accordingly, these two ingredients are formulated as a joint graph regularization term in an overall objective function, through which the similar hash codes of different modalities in an instance can be discriminatively obtained to flexibly characterize such instance. As a result, these derived hash codes incorporating higher discrimination power are able to improve the cross-modal searching accuracy significantly. The extensive experiments tested on three popular benchmark datasets show that the proposed approach performs favorably compared to the state-of-the-art competing approaches.

Large-Scale Cross-Modality Search via Collective Matrix Factorization Hashing.

By transforming data into binary representation, i.e., Hashing, we can perform high-speed search with low storage cost, and thus, Hashing has collected increasing research interest in the recent years. Recently, how to generate Hashcode for multimodal data (e.g., images with textual tags, documents with photos, and so on) for large-scale cross-modality search (e.g., searching semantically related images in database for a document query) is an important research issue because of the fast growth of multimodal data in the Web. To address this issue, a novel framework for multimodal Hashing is proposed, termed as Collective Matrix Factorization Hashing (CMFH). The key idea of CMFH is to learn unified Hashcodes for different modalities of one multimodal instance in the shared latent semantic space in which different modalities can be effectively connected. Therefore, accurate cross-modality search is supported. Based on the general framework, we extend it in the unsupervised scenario where it tries to preserve the Euclidean structure, and in the supervised scenario where it fully exploits the label information of data. The corresponding theoretical analysis and the optimization algorithms are given. We conducted comprehensive experiments on three benchmark data sets for cross-modality search. The experimental results demonstrate that CMFH can significantly outperform several state-of-the-art cross-modality Hashing methods, which validates the effectiveness of the proposed CMFH.

Supervised Matrix Factorization Hashing for Cross-Modal Retrieval

The target of cross-modal hashing is to embed heterogeneous multimedia data into a common low-dimensional Hamming space, which plays a pivotal part in multimedia retrieval due to the emergence of big multimodal data. Recently, matrix factorization has achieved great success in cross-modal hashing. However, how to effectively use label information and local geometric structure is still a challenging problem for these approaches. To address this issue, we propose a cross-modal hashing method based on collective matrix factorization, which considers both the label consistency across different modalities and the local geometric consistency in each modality. These two elements are formulated as a graph Laplacian term in the objective function, leading to a substantial improvement on the discriminative power of latent semantic features obtained by collective matrix factorization. Moreover, the proposed method learns unified hash codes for different modalities of an instance to facilitate cross-modal search, and the objective function is solved using an iterative strategy. The experimental results on two benchmark data sets show the effectiveness of the proposed method and its superiority over state-of-the-art cross-modal hashing methods.

Block-based image hashing with restricted blocking strategy for rotational robustness

Image hashing is a potential solution for image content authentication (a desired image hashing algorithm should be robust to common image processing operations and various geometric distortions). In the literature, researchers pay more attention to block-based image hashing algorithms due to their robustness to common image processing operations (such as lossy compression, low-pass filtering, and additive noise). However, the block-based hashing strategies are sensitive to rotation processing operations. This indicates that the robustness of the block-based hashing methods against rotation operations is an important issue. Towards this direction, in this article we propose a restricted blocking strategy by investigating effect of two rotation operations on an image and its blocks in both theoretical and experimental ways. Furthermore, we apply the proposed blocking strategy for the recently reported non-negative matrix factorization (NMF) hashing. Experimental results have demonstrated the validity of the block-based hashing algorithms with restricted blocking strategy for rotation operations.