Common Feature Space Research Articles

Transfer fault diagnosis based on local maximum mean difference and K-means

Existing feature-based transfer learning methods have achieved great performance in the transfer fault diagnosis with unlabeled data. While most of them are global alignment methods based on maximum mean difference (MMD), which ignore the differences between different faults and pay little attention to the structural information in the unlabeled target samples. This paper proposes a transfer sparse auto-encoder (SAE) based on local maximum mean difference (LMMD) and K-means to solve the above problems. Firstly, we build a deep network based on SAE and LMMD for learning a common latent feature space where source and target subdomains are aligned. Subsequently, to fully explore the target domain information, we put forward the K-means-based method which can obtain final diagnosis results by synthesizing the source and target domain information in the latent feature space. Lastly, a case study is conducted to verify the robustness and effectiveness of the proposed methods. The experimental result demonstrates that the proposed methods outperform the MMD-based methods in the transfer fault diagnosis problem.

Learning to Prompt for Vision-Language Models

Large pre-trained vision-language models like CLIP have shown great potential in learning representations that are transferable across a wide range of downstream tasks. Different from the traditional representation learning that is based mostly on discretized labels, vision-language pre-training aligns images and texts in a common feature space, which allows zero-shot transfer to a downstream task via prompting, i.e., classification weights are synthesized from natural language describing classes of interest. In this work, we show that a major challenge for deploying such models in practice is prompt engineering, which requires domain expertise and is extremely time-consuming -- one needs to spend a significant amount of time on words tuning since a slight change in wording could have a huge impact on performance. Inspired by recent advances in prompt learning research in natural language processing (NLP), we propose Context Optimization (CoOp), a simple approach specifically for adapting CLIP-like vision-language models for downstream image recognition. Concretely, CoOp models a prompt's context words with learnable vectors while the entire pre-trained parameters are kept fixed. To handle different image recognition tasks, we provide two implementations of CoOp: unified context and class-specific context. Through extensive experiments on 11 datasets, we demonstrate that CoOp requires as few as one or two shots to beat hand-crafted prompts with a decent margin and is able to gain significant improvements over prompt engineering with more shots, e.g., with 16 shots the average gain is around 15% (with the highest reaching over 45%). Despite being a learning-based approach, CoOp achieves superb domain generalization performance compared with the zero-shot model using hand-crafted prompts.

Progressively Discriminative Transfer Network for Cross-Corpus Speech Emotion Recognition

Cross-corpus speech emotion recognition (SER) is a challenging task, and its difficulty lies in the mismatch between the feature distributions of the training (source domain) and testing (target domain) data, leading to the performance degradation when the model deals with new domain data. Previous works explore utilizing domain adaptation (DA) to eliminate the domain shift between the source and target domains and have achieved the promising performance in SER. However, these methods mainly treat cross-corpus tasks simply as the DA problem, directly aligning the distributions across domains in a common feature space. In this case, excessively narrowing the domain distance will impair the emotion discrimination of speech features since it is difficult to maintain the completeness of the emotion space only by an emotion classifier. To overcome this issue, we propose a progressively discriminative transfer network (PDTN) for cross-corpus SER in this paper, which can enhance the emotion discrimination ability of speech features while eliminating the mismatch between the source and target corpora. In detail, we design two special losses in the feature layers of PDTN, i.e., emotion discriminant loss and distribution alignment loss . By incorporating prior knowledge of speech emotion into feature learning (i.e., high and low valence speech emotion features have their respective cluster centers), we integrate a valence-aware center loss and an emotion-aware center loss as the to guarantee the discriminative learning of speech emotions except an emotion classifier. Furthermore, a multi-layer distribution alignment loss is adopted to more precisely eliminate the discrepancy of feature distributions between the source and target domains. Finally, through the optimization of PDTN by combining three losses, i.e., cross-entropy loss , , and , we can gradually eliminate the domain mismatch between the source and target corpora while maintaining the emotion discrimination of speech features. Extensive experimental results of six cross-corpus tasks on three datasets, i.e., Emo-DB, eNTERFACE, and CASIA, reveal that our proposed PDTN outperforms the state-of-the-art methods.

Multimodal Biometrics Recognition Using a Deep Convolutional Neural Network with Transfer Learning in Surveillance Videos

Biometric recognition is a critical task in security control systems. Although the face has long been widely accepted as a practical biometric for human recognition, it can be easily stolen and imitated. Moreover, in video surveillance, it is a challenge to obtain reliable facial information from an image taken at a long distance with a low-resolution camera. Gait, on the other hand, has been recently used for human recognition because gait is not easy to replicate, and reliable information can be obtained from a low-resolution camera at a long distance. However, the gait biometric alone still has constraints due to its intrinsic factors. In this paper, we propose a multimodal biometrics system by combining information from both the face and gait. Our proposed system uses a deep convolutional neural network with transfer learning. Our proposed network model learns discriminative spatiotemporal features from gait and facial features from face images. The two extracted features are fused into a common feature space at the feature level. This study conducted experiments on the publicly available CASIA-B gait and Extended Yale-B databases and a dataset of walking videos of 25 users. The proposed model achieves a 97.3 percent classification accuracy with an F1 score of 0.97and an equal error rate (EER) of 0.004.

Common feature learning for brain tumor MRI synthesis by context-aware generative adversarial network

Multi-modal structural Magnetic Resonance Image (MRI) provides complementary information and has been used widely for diagnosis and treatment planning of gliomas. While machine learning is popularly adopted to process and analyze MRI images, most existing tools are based on complete sets of multi-modality images that are costly and sometimes impossible to acquire in real clinical scenarios. In this work, we address the challenge of multi-modality glioma MRI synthesis often with incomplete MRI modalities. We propose 3D Common-feature learning-based Context-aware Generative Adversarial Network (CoCa-GAN) for this purpose. In particular, our proposed CoCa-GAN method adopts the encoder-decoder architecture to map the input modalities into a common feature space by the encoder, from which (1) the missing target modality(-ies) can be synthesized by the decoder, and also (2) the jointly conducted segmentation of the gliomas can help the synthesis task to better focus on the tumor regions. The synthesis and segmentation tasks share the same common feature space, while multi-task learning boosts both their performances. In particular, for the encoder to derive the common feature space, we propose and validate two different models, i.e., (1) early-fusion CoCa-GAN (eCoCa-GAN) and (2) intermediate-fusion CoCa-GAN (iCoCa-GAN). The experimental results demonstrate that the proposed iCoCa-GAN outperforms other state-of-the-art methods in synthesis of missing image modalities. Moreover, our method is flexible to handle the arbitrary combination of input/output image modalities, which makes it feasible to process brain tumor MRI data in real clinical circumstances.

Multisource Deep Transfer Learning Based on Balanced Distribution Adaptation.

The current traditional unsupervised transfer learning assumes that the sample is collected from a single domain. From the aspect of practical application, the sample from a single-source domain is often not enough. In most cases, we usually collect labeled data from multiple domains. In recent years, multisource unsupervised transfer learning with deep learning has focused on aligning in the common feature space and then seeking to minimize the distribution difference between the source and target domains, such as marginal distribution, conditional distribution, or both. Moreover, conditional distribution and marginal distribution are often treated equally, which will lead to poor performance in practical applications. The existing algorithms that consider balanced distribution are often based on a single-source domain. To solve the above-mentioned problems, we propose a multisource transfer learning algorithm based on distribution adaptation. This algorithm considers adjusting the weights of two distributions to solve the problem of distribution adaptation in multisource transfer learning. A large number of experiments have shown that our method MTLBDA has achieved significant results in popular image classification datasets such as Office-31.

Data-driven modeling of multiaxial fatigue in frequency domain

A data-driven approach to modeling multiaxial fatigue in frequency domain is proposed. The paper presents a methodology through which models can be built to predict fatigue damage based on a multiaxial cycle counting method and the associated life assessment approach in frequency domain. Through apt parameterization of spectrum in conjunction with principal component analysis transformation, every biaxial stress state in frequency domain can be represented as a point on a common feature space. The proposed methodology consists of three phases: exploratory study, data generation and model development. Two structural components are used for the demonstration. The performance of the models are evaluated in comparison with time domain approaches, and results are compared. The proposed methodology is able to predict multiaxial fatigue damage in frequency domain within acceptable error limits, allowing for a fast and efficient method to tackle multiaxial fatigue analysis in frequency domain.

Dual Modality Collaborative Learning for Cross-Source Remote Sensing Retrieval

Content-based remote sensing (RS) image retrieval (CBRSIR) is a critical way to organize high-resolution RS (HRRS) images in the current big data era. The increasing volume of HRRS images from different satellites and sensors leads to more attention to the cross-source CSRSIR (CS-CBRSIR) problem. Due to the data drift, one crucial problem in CS-CBRSIR is the modality discrepancy. Most existing methods focus on finding a common feature space for various HRRS images to address this issue. In this space, their similarity relations can be measured directly to obtain the cross-source retrieval results straight. This way is feasible and reasonable, however, the specific information corresponding to HRRS images from different sources is always ignored, limiting retrieval performance. To overcome this limitation, we develop a new model for CS-CBRSIR in this paper named dual modality collaborative learning (DMCL). To fully explore the specific information from diverse HRRS images, DMCL first introduces ResNet50 as the feature extractor. Then, a common space mutual learning module is developed to map the specific features into a common space. Here, the modality discrepancy is reduced from the aspects of features and their distributions. Finally, to supplement the specific knowledge to the common features, we develop modality transformation and the dual-modality feature learning modules. Their function is to transmit the specific knowledge from different sources mutually and fuse the specific and common features adaptively. The comprehensive experiments are conducted on a public dataset. Compared with many existing methods, the behavior of our DMCL is stronger. These encouraging results for a public dataset indicate that the proposed DMCL is useful in CS-CBRSIR tasks.

Visible-Infrared Person Re-Identification With Modality-Specific Memory Network.

Visible-infrared person re-identification (VI-ReID) is challenging due to the large modality discrepancy between visible and infrared images. Existing methods mainly focus on learning modality-shared representations by embedding images from different modalities into a common feature space, in which some discriminative modality information is discarded. Different from these methods, in this paper, we propose a novel Modality-Specific Memory Network (MSMNet) to complete the missing modality information and aggregate visible and infrared modality features into a unified feature space for the VI-ReID task. The proposed model enjoys several merits. First, it can exploit the missing modality information to alleviate the modality discrepancy when only the single-modality input is provided. To the best of our knowledge, this is the first work to exploit the missing modality information completion and alleviate the modality discrepancy with the memory network. Second, to guide the learning process of the memory network, we design three effective learning strategies, including feature consistency, memory representativeness and structural alignment. By incorporating these learning strategies in a unified model, the memory network can be well learned to propagate identity-related information between modalities and boost the VI-ReID performance. Extensive experimental results on two standard benchmarks (SYSU-MM01 and RegDB) demonstrate that the proposed MSMNet performs favorably against state-of-the-art methods.

Unsupervised Change Detection From Heterogeneous Data Based on Image Translation

It is quite an important and challenging problem for change detection (CD) from heterogeneous remote sensing images. The images obtained from different sensors (i.e., synthetic aperture radar (SAR) & optical camera) characterize the distinct properties of objects. Thus, it is impossible to detect changes by direct comparison of heterogeneous images. In this article, a new unsupervised change detection (USCD) method is proposed based on image translation. The cycle-consistent adversarial networks (CycleGANs) are employed to learn the subimage to subimage mapping relation using the given pair (i.e., before and after the event) of heterogeneous images from which the changes will be detected. Then, we can translate one image (e.g., SAR) from its original feature space (e.g., SAR) to another space (e.g., optical). By doing this, the pair of images can be represented in a common feature space (e.g., optical). The pixels with close pattern values in the before-event image may have quite different values in the after-event image if the change happens on some ones. Thus, we can generate the difference map between the translated before-event image and the original after-event image. Then, the difference map is divided into changed and unchanged parts. However, these detection results are not very reliable. We will select some significantly changed and unchanged pixel pairs from the two parts with the clustering technique (i.e., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> -means). These selected pixel pairs are used to learn a binary classifier, and the other pixel pairs will be classified by this classifier to obtain the final CD results. Experimental results on different real datasets demonstrate the effectiveness of the proposed USCD method compared with several other related methods.

A Hybrid Model-Based and Learning-Based Approach for Classification Using Limited Number of Training Samples

The fundamental task of classification given a limited number of training data samples is considered for physical systems with known parametric statistical models. The standalone learning-based and statistical model-based classifiers face major challenges towards the fulfillment of the classification task using a small training set. Specifically, classifiers that solely rely on the physics-based statistical models usually suffer from their inability to properly tune the underlying unobservable parameters, which leads to a mismatched representation of the system's behaviors. Learning-based classifiers, on the other hand, typically rely on a large number of training data from the underlying physical process, which might not be feasible in most practical scenarios. In this paper, a hybrid classification method -- termed HyPhyLearn -- is proposed that exploits both the physics-based statistical models and the learning-based classifiers. The proposed solution is based on the conjecture that HyPhyLearn would alleviate the challenges associated with the individual approaches of learning-based and statistical model-based classifiers by fusing their respective strengths. The proposed hybrid approach first estimates the unobservable model parameters using the available (suboptimal) statistical estimation procedures, and subsequently use the physics-based statistical models to generate synthetic data. Then, the training data samples are incorporated with the synthetic data in a learning-based classifier that is based on domain-adversarial training of neural networks. Specifically, in order to address the mismatch problem, the classifier learns a mapping from the training data and the synthetic data to a common feature space. Simultaneously, the classifier is trained to find discriminative features within this space in order to fulfill the classification task.

A cross-view UAV image localization method based on directional feature alignment and visual transformer

Using satellite images and UAV images to locate the same geographical target can provide new ideas for UAV positioning and navigation. However, the images from the two different remote sensing platforms, UAV and satellite, have a huge difference in appearance, which is a huge challenge for this task. Existing methods are usually limited to convolutional neural networks, leading to a lack of utilization of global information of images, and these methods do not focus on the spatial information of images. To address these issues, this research propose a method that extracts posture information using SFM(Structure-from-Motion) and then uses the posture to align the spatial features of the image, and introduces a visual transformer to focus the network on acquiring the common feature space shared by the viewpoint sources. In this study, a large number of experiments have been carried out on the large datum dataset University-1652. The experimental results show that the method proposed in this paper outperforms the baseline and has the same advantages as other advanced methods.

Siamese Networks Based Deep Fusion Framework for Multi-Source Satellite Imagery

A critical aim of pansharpening is to fuse coherent spatial and spectral features from panchromatic and multispectral images respectively. This study proposes deep siamese network based pansharpening model as a two-stage framework in a multiscale setting. In the first stage, a siamese network learns a common feature space between panchromatic and multispectral bands. The second stage follows by fusing the output feature maps of the siamese network. The parameters of these two stages are shared across scales in order to add spatial information consistently (across scales). The spectral information is preserved by adding appropriate skip connections from input multispectral image. Multi-level network parameters sharing mechanism in pyramidal reconstruction of pansharpened image, better preserves spatial and spectral details simultaneously. Experimental work carried out using deep siamese network in multi-scale setting (to obtain inter-band similarity among different sensor data) outperforms several latest pansharpening methods.

Deep Radiotranscriptomics of Non-Small Cell Lung Carcinoma for Assessing Molecular and Histology Subtypes with a Data-Driven Analysis.

Radiogenomic and radiotranscriptomic studies have the potential to pave the way for a holistic decision support system built on genomics, transcriptomics, radiomics, deep features and clinical parameters to assess treatment evaluation and care planning. The integration of invasive and routine imaging data into a common feature space has the potential to yield robust models for inferring the drivers of underlying biological mechanisms. In this non-small cell lung carcinoma study, a multi-omics representation comprised deep features and transcriptomics was evaluated to further explore the synergetic and complementary properties of these diverse multi-view data sources by utilizing data-driven machine learning models. The proposed deep radiotranscriptomic analysis is a feature-based fusion that significantly enhances sensitivity by up to 0.174 and AUC by up to 0.22, compared to the baseline single source models, across all experiments on the unseen testing set. Additionally, a radiomics-based fusion was also explored as an alternative methodology yielding radiomic signatures that are comparable to several previous publications in the field of radiogenomics. Furthermore, the machine learning multi-omics analysis based on deep features and transcriptomics achieved an AUC performance of up to 0.831 ± 0.09/0.925 ± 0.04 for the examined molecular and histology subtypes analysis, respectively. The clinical impact of such high-performing models can add prognostic value and lead to optimal treatment assessment by targeting specific oncogenes, namely the response of tyrosine kinase inhibitors of EGFR mutated or predicting the chemotherapy resistance of KRAS mutated tumors.

Unsupervised Domain Adaptation With Variational Approximation for Cardiac Segmentation.

Unsupervised domain adaptation is useful in medical image segmentation. Particularly, when ground truths of the target images are not available, domain adaptation can train a target-specific model by utilizing the existing labeled images from other modalities. Most of the reported works mapped images of both the source and target domains into a common latent feature space, and then reduced their discrepancy either implicitly with adversarial training or explicitly by directly minimizing a discrepancy metric. In this work, we propose a new framework, where the latent features of both domains are driven towards a common and parameterized variational form, whose conditional distribution given the image is Gaussian. This is achieved by two networks based on variational auto-encoders (VAEs) and a regularization for this variational approximation. Both of the VAEs, each for one domain, contain a segmentation module, where the source segmentation is trained in a supervised manner, while the target one is trained unsupervisedly. We validated the proposed domain adaptation method using two cardiac segmentation tasks, i.e., the cross-modality (CT and MR) whole heart segmentation and the cross-sequence cardiac MR segmentation. Results show that the proposed method achieved better accuracies compared to two state-of-the-art approaches and demonstrated good potential for cardiac segmentation. Furthermore, the proposed explicit regularization was shown to be effective and efficient in narrowing down the distribution gap between domains, which is useful for unsupervised domain adaptation. The code and data have been released via https://zmiclab.github.io/projects.html.

Supervised Domain Adaptation for Few-Shot Radar-Based Human Activity Recognition

With the application of deep learning (DL) techniques, radar-based human activity recognition (HAR) attracts increasing attention thanks to its high accuracy and good privacy. However, training a DL model requires a large volume of data, and generally the trained model cannot be adapted to a new scenario. In this paper, we propose a supervised few-shot adversarial domain adaptation ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FS-ADA</i> ) method for HAR, where only limited radar training data is collected from a new application scenario. We adopt the domain adaptation method to learn a common feature space between a pre-existing radar dataset and the newly acquired training data. We also design a multi-class discriminator network, which integrates the category classifier and the binary domain discriminator, to employ the supervised label information in the limited radar data for model training. Then, a multitask generative adversarial training mechanism is proposed to optimize <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FS-ADA</i> . In this way, both domain-invariant and category-discriminative features can be extracted for HAR in a new scenario. Experimental results for two few-shot radar-based HAR tasks show that the proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FS-ADA</i> method is effective and outperforms state-of-the-art methods.

Integration of Imaging Genomics Data for the Study of Alzheimer's Disease Using Joint-Connectivity-Based Sparse Nonnegative Matrix Factorization.

Imaging genetics reveals the connection between microscopic genetics and macroscopic imaging, enabling the identification of disease biomarkers. In this work, we make full use of prior knowledge that has significant reference value for investigating the correlation between the brain and genetics to explore more biologically substantial biomarkers. In this paper, we propose joint-connectivity-based sparse nonnegative matrix factorization (JCB-SNMF). The algorithm simultaneously projects structural magnetic resonance imaging (sMRI), single-nucleotide polymorphism sites (SNPs), and gene expression data onto a common feature space, where heterogeneous variables with large coefficients in the same projection direction form a common module. In addition, the connectivity information for each region of the brain and genetic data are added as prior knowledge to identify regions of interest (ROIs), SNPs, and gene-related risks related to Alzheimer's disease (AD) patients. GraphNet regularization increases the anti-noise performance of the algorithm and the biological interpretability of the results. The simulation results show that compared with other NMF-based algorithms (JNMF, JSNMNMF), JCB-SNMF has better anti-noise performance and can identify and predict biomarkers closely related to AD from significant modules. By constructing a protein-protein interaction (PPI) network, we identified SF3B1, RPS20, and RBM14 as potential biomarkers of AD. We also found some significant SNP-ROI and gene-ROI pairs. Among them, two SNPs rs4472239 and rs11918049 and three genes KLHL8, ZC3H11A, and OSGEPL1 may have effects on the gray matter volume of multiple brain regions. This model provides a new way to further integrate multimodal impact genetic data to identify complex disease association patterns.

Integrating multiple genomic imaging data for the study of lung metastasis in sarcomas using multi-dimensional constrained joint non-negative matrix factorization

Integrative analysis of histopathology images and genomic data enables the discovery of potential biomarkers and multimodal association patterns. However, few studies have established effective association models for complex diseases, such as sarcoma, by combining histopathological images with multiple genetic variation data. Here, we present an integrative multiple genomic imaging framework called multi-dimensional constrained joint non-negative matrix factorization (MDJNMF) to identify modules related to lung metastasis of sarcomas based on sample-matched whole-solid image, DNA methylation, and copy number variation features. Three types of feature matrices were projected onto a common feature space, in which heterogeneous variables with large coefficients in the same projected direction form a common module. The correlation between image features and genetic variation features is used as network-regularized constraints to improve the module accuracy. Sparsity and orthogonal constraints are utilized to achieve the modular sparse solution. Multi-level analysis indicates that our method effectively discovers biologically functional modules associated with sarcoma or lung metastasis. The representative module reveals a significant correlation between image features and genetic variation features and excavates potential diagnostic biomarkers. In summary, the proposed method provides new clues for identifying association patterns and biomarkers using multiple types of data sources for other diseases.

FPS-Net: A convolutional fusion network for large-scale LiDAR point cloud segmentation

Scene understanding based on LiDAR point cloud is an essential task for autonomous cars to drive safely, which often employs spherical projection to map 3D point cloud into multi-channel 2D images for semantic segmentation. Most existing methods simply stack different point attributes/modalities (e.g. coordinates, intensity, depth, etc.) as image channels to increase information capacity, but ignore distinct characteristics of point attributes in different image channels. We design FPS-Net, a convolutional fusion network that exploits the uniqueness and discrepancy among the projected image channels for optimal point cloud segmentation. FPS-Net adopts an encoder-decoder structure. Instead of simply stacking multiple channel images as a single input, we group them into different modalities to first learn modality-specific features separately and then map the learnt features into a common high-dimensional feature space for pixel-level fusion and learning. Specifically, we design a residual dense block with multiple receptive fields as a building block in encoder which preserves detailed information in each modality and learns hierarchical modality-specific and fused features effectively. In the FPS-Net decoder, we use a recurrent convolution block likewise to hierarchically decode fused features into output space for pixel-level classification. Extensive experiments conducted on two widely adopted point cloud datasets show that FPS-Net achieves superior semantic segmentation as compared with state-of-the-art projection-based methods. Specifically, FPS-Net outperforms the state-of-the-art in both accuracy (4.9% higher than RangeNet++ and 2.8% higher than PolarNet in mIoU) and computation speed (15.0 FPS faster than SqueezeSegV3) for SemanticKITTI benchmark. For KITTI benchmark, FPS-Net achieves significant accuracy improvement (12.6% higher than RangeNet++ in mIoU) with comparable computation speed. In addition, the proposed modality fusion idea is compatible with typical projection-based methods and can be incorporated into them with consistent performance improvement.

Intra-class low-rank regularization for supervised and semi-supervised cross-modal retrieval

Cross-modal retrieval aims to retrieve related items across different modalities, for example, using an image query to retrieve related text. The existing deep methods ignore both the intra-modal and inter-modal intra-class low-rank structures when fusing various modalities, which decreases the retrieval performance. In this paper, two deep models (denoted as ILCMR and Semi-ILCMR) based on intra-class low-rank regularization are proposed for supervised and semi-supervised cross-modal retrieval, respectively. Specifically, ILCMR integrates the image network and text network into a unified framework to learn a common feature space by imposing three regularization terms to fuse the cross-modal data. First, to align them in the label space, we utilize semantic consistency regularization to convert the data representations to probability distributions over the classes. Second, we introduce an intra-modal low-rank regularization, which encourages the intra-class samples that originate from the same space to be more relevant in the common feature space. Third, an inter-modal low-rank regularization is applied to reduce the cross-modal discrepancy. To enable the low-rank regularization to be optimized using automatic gradients during network back-propagation, we propose the rank-r approximation and specify the explicit gradients for theoretical completeness. In addition to the three regularization terms that rely on label information incorporated by ILCMR, we propose Semi-ILCMR in the semi-supervised regime, which introduces a low-rank constraint before projecting the general representations into the common feature space. Extensive experiments on four public cross-modal datasets demonstrate the superiority of ILCMR and Semi-ILCMR over other state-of-the-art methods.