Retrieval Applications Research Articles

All‐Sky Microwave Radiance Observation Operator Based on Deep Learning With Physical Constraints

AbstractSatellite data assimilation relies on the radiative transfer models (RTMs) to establish the relationships between model state variables and satellite radiances. However, atmospheric radiative transfer calculations are computationally expensive, especially when involving multiple‐scattering calculations in cloudy areas. In recent years, deep learning (DL) models have been increasingly applied to emulate and accelerate physical models. This study, for the first time, explores DL techniques to emulate all‐sky radiative transfer in microwave bands. The FengYun‐3E (FY‐3E) Microwave Humidity Sounder‐2 (MWHS‐2) was selected as the target instrument due to its comprehensive spectral coverage, with the radiative transfer for TOVS scattering module (RTTOV‐SCATT) serving as the reference model. Three DL architectures were trained and compared, including multilayer perceptron (MLP), Bidirectional Long Short‐Term Memory with Attention (BiLSTM‐Attention), and Transformer. The BiLSTM‐Attention architecture demonstrated superior performance in both clear‐sky and cloudy radiance simulations. This may be attributed to its bidirectional recurrent structure resembling physical radiative transfer processes and the attention mechanism's ability to link MWHS‐2 channels with corresponding vertical layers. Although DL models achieve high accuracy in forward prediction, they often struggle with instability in Jacobian calculations. To address this issue, the trained BiLSTM‐Attention model was fine‐tuned using the reference model Jacobians as physical constraints. The fine‐tuned BiLSTM‐Attention model accurately characterized radiance sensitivities to temperature, water vapor, and hydrometeors under different cloud conditions, indicating its potential to serve as a radiance observation operator in data assimilation and physical retrieval applications.

Enhancing retrieval performance of embedding models via fine-tuning on synthetic data in RAG chatbot for Vietnamese military science domain

Retrieval-Augmented Generation (RAG) is a technology that combines information retrieval with large language models, enabling chatbots to provide accurate answers by querying relevant documents from a data repository before generating responses. While RAG chatbot has demonstrated effectiveness in many applications, there are still limitations in specialized Vietnamese data domains, particularly in the military science field. To address this challenge, this paper proposes a framework for fine-tuning embedding models using synthetic datasets generated by ChatGPT to enhance retrieval performance in a Q&A application focused on the history of the Institute of Information Technology (IoIT). Our approach evaluates 11 popular embedding models and shows a significant average improvement of 18.15% in the MAP@K metric. The resulting IoIT history Q&A chatbot, developed with fine-tuned embedding models and the Vietnamese language model Vistral-7B, outperforms chatbots utilizing OpenAI's embedding models and ChatGPT. These findings highlight the potential of RAG chatbot technology for advancing information retrieval applications in specialized fields like military science.

Dual Contrastive Learning for Cross-Domain Named Entity Recognition

Benefiting many information retrieval applications, named entity recognition (NER) has shown impressive progress. Recently, there has been a growing trend to decompose complex NER tasks into two subtasks (e.g., entity span detection (ESD) and entity type classification (ETC), to achieve better performance. Despite the remarkable success, from the perspective of representation, existing methods do not explicitly distinguish non-entities and entities, which may lead to ESD errors. Meanwhile, they do not explicitly distinguish entities with different entity types, which may lead to entity type misclassification. As such, the limited representation abilities may challenge some competitive NER methods, leading to unsatisfactory performance, especially in the low-resource setting (e.g., cross-domain NER). In light of these challenges, we propose to utilize contrastive learning to refine the original chaotic representations and learn the generalized representations for cross-domain NER. In particular, this article proposes a dual contrastive learning model (Dual-CL), which respectively utilizes a token-level contrastive learning module and a sentence-level contrastive learning module to enhance ESD, ETC for cross-domain NER. Empirical results on 10 domain pairs under two different settings show that Dual-CL achieves better performances than compared baselines in terms of several standard metrics. Moreover, we conduct detailed analyses to are presented to better understand each component’s effectiveness.

Tropical Cyclone Wind Direction Retrieval Based on Wind Streaks and Rain Bands in SAR Images

Tropical cyclones (TCs) are associated with severe weather phenomena, making accurate wind field retrieval crucial for TC monitoring. SAR’s high-resolution imaging capability provides detailed information for TC observation, and wind speed calculations require wind direction as prior information. Therefore, utilizing SAR images to retrieve TC wind fields is of significant importance. This study introduces a novel approach for retrieving wind direction from SAR images of TCs through the classification of TC sub-images. The method utilizes a transfer learning-based Inception V3 model to identify wind streaks (WSs) and rain bands in SAR images under TC conditions. For sub-images containing WSs, the Mexican-hat wavelet transform is applied, while for sub-images containing rain bands, an edge detection technique is used to locate the center of the TC eye and subsequently the tangent to the spiral rain bands is employed to determine the wind direction associated with the rain bands. Wind direction retrieval from 10 SAR TC images showed an RMSD of 19.52° and a correlation coefficient of 0.96 when compared with ECMWF and HRD observation wind directions, demonstrating satisfactory consistency and providing highly accurate TC wind directions. These results confirm the method’s potential applications in TC wind direction retrieval.

A novel Deep High-level Concept-mining Jointing Hashing Model for unsupervised cross-modal retrieval

Unsupervised cross-modal hashing has achieved great success in various information retrieval applications owing to its efficient storage usage and fast retrieval speed. Recent studies have primarily focused on training the hash-encoded networks by calculating a sample-based similarity matrix to improve the retrieval performance. However, there are two issues remain to solve: (1) The current sample-based similarity matrix only considers the similarity between image-text pairs, ignoring the different information densities of each modality, which may introduce additional noise and fail to mine key information for retrieval; (2) Most existing unsupervised cross-modal hashing methods only consider alignment between different modalities, while ignoring consistency between each modality, resulting in semantic conflicts. To tackle these challenges, a novel Deep High-level Concept-mining Jointing Hashing (DHCJH) model for unsupervised cross-modal retrieval is proposed in this study. DHCJH is able to capture the essential high-level semantic information from image modalities and integrate into the text modalities to improve the accuracy of guidance information. Additionally, a new hashing loss with a regularization term is introduced to avoid the cross-modal semantic collision and false positive pairs problems. To validate the proposed method, extensive comparison experiments on benchmark datasets are conducted. Experimental findings reveal that DHCJH achieves superior performance in both accuracy and efficiency. The code of DHCJH is available at Github.

Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.

In the complex and multidimensional field of medicine, multimodal data are prevalent and crucial for informed clinical decisions. Multimodal data span a broad spectrum of data types, including medical images (eg, MRI and CT scans), time-series data (eg, sensor data from wearable devices and electronic health records), audio recordings (eg, heart and respiratory sounds and patient interviews), text (eg, clinical notes and research articles), videos (eg, surgical procedures), and omics data (eg, genomics and proteomics). While advancements in large language models (LLMs) have enabled new applications for knowledge retrieval and processing in the medical field, most LLMs remain limited to processing unimodal data, typically text-based content, and often overlook the importance of integrating the diverse data modalities encountered in clinical practice. This paper aims to present a detailed, practical, and solution-oriented perspective on the use of multimodal LLMs (M-LLMs) in the medical field. Our investigation spanned M-LLM foundational principles, current and potential applications, technical and ethical challenges, and future research directions. By connecting these elements, we aimed to provide a comprehensive framework that links diverse aspects of M-LLMs, offering a unified vision for their future in health care. This approach aims to guide both future research and practical implementations of M-LLMs in health care, positioning them as a paradigm shift toward integrated, multimodal data-driven medical practice. We anticipate that this work will spark further discussion and inspire the development of innovative approaches in the next generation of medical M-LLM systems.

Generating grating in cavity magnomechanics

Abstract We investigate the phenomenon of magnomechanically induced grating (MMIG) within a cavity magnomechanical system, comprising magnons (spins in a ferromagnet, such as yttrium iron garnet), cavity microwave photons, and phonons (Li et al 2018 Phys. Rev. Lett. 121 203601). By applying an external standing wave control, we observe modifications in the transmission profile of a probe light beam, signifying the presence of MMIG. Through numerical analysis, we explore the diffraction intensities of the probe field, examining the impact of interactions between cavity magnons, magnon-phonon interactions, standing wave field strength, and interaction length. MMIG systems leverage the unique properties of magnons, and collective spin excitations with attributes like long coherence times and spin-wave propagation. These distinctive features can be harnessed in MMIG systems for innovative applications in information storage, retrieval, and quantum memories, offering various orders of diffraction grating.

Precipitable water vapor retrieval for rainfall forecasting using BDS-3 PPP-B2b signal in the coastal region of China

Abstract Currently, rainfall cannot be accurately forecast because of poor network communication at the ocean. The advantage of the BeiDou Global Navigation Satellite System (BDS-3) precise point positioning (PPP-B2b) signal, which does not rely on network communication to receive data, is that it can provide precipitable water vapor (PWV) retrieval application services for the open seas in eastern China, where the communication system presents difficulties. In this study, data from stations in the coastal region of China were used to establish a rainfall forecasting method for monitoring extreme weather on the sea. First, the service performance of PPP-B2b was explored. Then, based on 17 Chinese coastal stations, the PWV accuracy was evaluated. Finally, based on an analysis of the relationship between PWV and actual rainfall, a threshold rainfall forecasting method based on a sliding window was constructed. The experimental results show that the PWV accuracy varies slightly depending on the geographic location, in which the mean absolute error in the North Sea region is the smallest (2.1 mm), that of the South China Sea region is the largest (2.60 mm), and that of the East China Sea region is in the middle (2.48 mm). The optimal predictors of the constructed 12 h sliding-window threshold rainfall prediction method are a PWV maximum of 49 mm, PWV increase of 5 mm, and PWV increase rate of 1.2 mm h−1. The prediction results can reach a critical success index value of more than 45%, indicating high prediction accuracy and applicability to the coastal region of China during the same period.

A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers

Relation extraction (RE) involves identifying the relations between entities from underlying content. RE serves as the foundation for many natural language processing (NLP) and information retrieval applications, such as knowledge graph completion and question answering. In recent years, deep neural networks have dominated the field of RE and made noticeable progress. Subsequently, the large pre-trained language models (PLMs) have taken the state-of-the-art RE to a new level. This survey provides a comprehensive review of existing deep learning techniques for RE. First, we introduce RE resources, including datasets and evaluation metrics. Second, we propose a new taxonomy to categorize existing works from three perspectives, i.e., text representation, context encoding, and triplet prediction. Third, we discuss several important challenges faced by RE and summarize potential techniques to tackle these challenges. Finally, we outline some promising future directions and prospects in this field. This survey is expected to facilitate researchers’ collaborative efforts to address the challenges of real-world RE systems.

Chinese Title Generation for Short Videos: Dataset, Metric and Algorithm.

Previous work for video captioning aims to objectively describe the video content but the captions lack human interest and attractiveness, limiting its practical application scenarios. The intention of video title generation (video titling) is to produce attractive titles, but there is a lack of benchmarks. This work offers CREATE, the first large-scale Chinese shoRt vidEo retrievAl and Title gEneration dataset, to assist research and applications in video titling, video captioning, and video retrieval in Chinese. CREATE comprises a high-quality labeled 210 K dataset and two web-scale 3 M and 10 M pre-training datasets, covering 51 categories, 50K+ tags, 537K+ manually annotated titles and captions, and 10M+ short videos with original video information. This work presents ACTEr, a unique Attractiveness-Consensus-based Title Evaluation, to objectively evaluate the quality of video title generation. This metric measures the semantic correlation between the candidate (model-generated title) and references (manual-labeled titles) and introduces attractive consensus weights to assess the attractiveness and relevance of the video title. Accordingly, this work proposes a novel multi-modal ALignment WIth Generation model, ALWIG, as one strong baseline to aid future model development. With the help of a tag-driven video-text alignment module and a GPT-based generation module, this model achieves video titling, captioning, and retrieval simultaneously. We believe that the release of the CREATE dataset, ACTEr metric, and ALWIG model will encourage in-depth research on the analysis and creation of Chinese short videos.

Evaluating LLMs on document-based QA: Exact answer selection and numerical extraction using CogTale dataset

Document-based Question-Answering (QA) tasks are crucial for precise information retrieval. While some existing work focus on evaluating large language model’s (LLMs) performance on retrieving and answering questions from documents, assessing the LLMs performance on QA types that require exact answer selection from predefined options and numerical extraction is yet to be fully assessed. In this paper, we specifically focus on this underexplored context and conduct empirical analysis of LLMs (GPT-4 and GPT-3.5) on question types, including single-choice, yes–no, multiple-choice, and number extraction questions from documents. We use the CogTale dataset for evaluation, which provide human expert-tagged responses, offering a robust benchmark for precision and factual grounding. We found that LLMs, particularly GPT-4, can precisely answer many single-choice and yes–no questions given relevant context, demonstrating their efficacy in information retrieval tasks. However, their performance diminishes when confronted with multiple-choice and number extraction formats, lowering the overall performance of the models on this task, indicating that these models may not yet be sufficiently reliable for the task. This limits the applications of LLMs on applications demanding precise information extraction and inference from documents, such as meta-analysis tasks. Our work offers a framework for ongoing dataset evaluation, ensuring that LLM applications for information retrieval and document analysis continue to meet evolving standards.

Eliminating Negative Word Similarities for Measuring Document Distances: A Thoroughly Empirical Study on Word Mover's Distance.

Document distance is a fundamental yet significant research topic in the information retrieval community, and its accuracy dominates the performance of many text retrieval applications. Beyond the Bag-of-Words (BoW) model, the Word Mover's Distance (WMD) semantically defines the distance between documents as the minimum cost (i.e., measured by word similarities of embeddings) required to transport the words from one document to another, and it has been proven to be superior by k -nearest neighbor classification. In this article, we thoroughly study the characteristics of WMD and its relaxed versions, e.g., Relaxed WMD (RWMD) and Iterative Constrained Transfers (ICT), in various scenarios. Specifically, we concentrate on the problem of negative word similarity: the WMD family leverages all word similarities, however, most of them are meaningless, resulting in negative effects for measuring document distances. To remedy this problem, we propose Informative Similarity Filter (ISF), which retains a very small percentage of top word similarities and fixes the others as the same lower similarity. Built on it, we propose a greedy optimization (GOM) for WMD, an accurate approximation to WMD. We theoretically analyze that ISF-GOM is more applicable for relatively longer documents. Extensive experiments have been conducted to validate: 1) the problem of RWMD; 2) the effectiveness of ISF-GOM; and 3) the consistence of our analysis of ISF-GOM. Our codes and datasets are available at https://github.com/BoCheng-96/ISF-GOM.

The digital lab manager: Automating research support

Laboratory management automation is essential for achieving interoperability in the domain of experimental research and accelerating scientific discovery. The integration of resources and the sharing of knowledge across organisations enable scientific discoveries to be accelerated by increasing the productivity of laboratories, optimising funding efficiency, and addressing emerging global challenges. This paper presents a novel framework for digitalising and automating the administration of research laboratories through The World Avatar, an all-encompassing dynamic knowledge graph. This Digital Laboratory Framework serves as a flexible tool, enabling users to efficiently leverage data from diverse systems and formats without being confined to a specific software or protocol. Establishing dedicated ontologies and agents and combining them with technologies such as QR codes, RFID tags, and mobile apps, enabled us to develop modular applications that tackle some key challenges related to lab management. Here, we showcase an automated tracking and intervention system for explosive chemicals as well as an easy-to-use mobile application for asset management and information retrieval. Implementing these, we have achieved semantic linking of BIM and BMS data with laboratory inventory and chemical knowledge. Our approach can capture the crucial data points and reduce inventory processing time. All data provenance is recorded following the FAIR principles, ensuring its accessibility and interoperability.

Text Summarization Based on Semantic Similarity

In the contemporary information age, the sheer volume of textual data poses a significant challenge for efficient comprehension and utilization. This project endeavors to address this challenge by developing a Text Summarization System grounded in semantic similarities. The primary goal is to create a robust and intuitive tool that extracts key information from large textual datasets, offering users a concise and meaningful summary. The proposed system employs advanced Natural Language Processing (NLP) techniques to analyze the semantic relationships within the text. Rather than relying solely on syntactic structures, the model identifies and leverages semantic similarities, such as shared concepts, themes, and contextual relationships, to distill the essential content. This approach enhances the summarization process by ensuring that the generated summaries reflect a deeper understanding of the underlying semantics, thereby capturing the core meaning of the text. Throughout the development of this project, the B.Tech student will delve into the intricacies of semantic analysis, exploring techniques to recognize and prioritize key concepts. The system's effectiveness will be evaluated through rigorous testing on diverse textual datasets, assessing its ability to generate coherent and relevant summaries across various domains. This project not only contributes to the field of NLP but also has practical applications in information retrieval, document summarization, and content curation. By providing an innovative solution to the challenges of information overload, the Text Summarization System based on semantic similarities offers a valuable tool for enhancing efficiency in information processing and decision-making. Index terms Text Summarization, Semantic Similarities, Natural Language Processing (NLP), Semantic Analysis, Information Retrieval, Document Summarization, Content Curation, Information Overload, Decision Making, Textual Data Analysis, Key Concept Recognition, Conceptual Relationships, Syntactic Structures, Semantic Understanding, Textual Datasets Evalution.

Greenhouse gas retrievals for the CO2M mission using the FOCAL method: first performance estimates

Abstract. The Anthropogenic Carbon Dioxide Monitoring (CO2M) mission is a constellation of satellites currently planned to be launched in 2026. CO2M is planned to be a core component of a Monitoring and Verification Support (MVS) service capacity under development as part of the Copernicus Atmosphere Monitoring Service (CAMS). The CO2M radiance measurements will be used to retrieve column-averaged dry-air mole fractions of atmospheric carbon dioxide (XCO2), methane (XCH4) and total columns of nitrogen dioxide (NO2). Using appropriate inverse modelling, the atmospheric greenhouse gas (GHG) observations will be used to derive United Nations Framework Convention on Climate Change (UNFCCC) COP 21 Paris Agreement relevant information on GHG sources and sinks. This challenging application requires highly accurate XCO2 and XCH4 retrievals. Three different retrieval algorithms to derive XCO2 and XCH4 are currently under development for the operational processing system at EUMETSAT. One of these algorithms uses the heritage of the FOCAL (Fast atmOspheric traCe gAs retrievaL) method, which has already successfully been applied to measurements from other satellites. Here, we show recent results generated using the CO2M version of FOCAL, called FOCAL-CO2M. To assess the quality of the FOCAL-CO2M retrievals, a large set of representative simulated radiance spectra has been generated using the radiative transfer model SCIATRAN. These simulations consider the planned viewing geometry of the CO2 instrument and corresponding geophysical scene data (including different types of aerosols and varying surface properties), which were taken from model data for the year 2015. We consider instrument noise and systematic errors caused by the retrieval method but have not considered additional error sources due to, for example, instrumental issues, spectroscopy or meteorology. On the other hand, we have also not taken advantage in this study of CO2M's MAP (multi-angle polarimeter) instrument, which will provide additional information on aerosols and cirrus clouds. By application of the FOCAL retrieval to these simulated data, confidence is gained that the FOCAL method is able to fulfil the challenging requirements for systematic errors for the CO2M mission (spatio-temporal bias ≤ 0.5 ppm for XCO2 and ≤ 5 ppb for XCH4).

DIML: Deep Interpretable Metric Learning via Structural Matching.

In this paper, we present a new framework named DIML to achieve more interpretable deep metric learning. Unlike traditional deep metric learning method that simply produces a global similarity given two images, DIML computes the overall similarity through the weighted sum of multiple local part-wise similarities, making it easier for human to understand the mechanism of how the model distinguish two images. Specifically, we propose a structural matching strategy that explicitly aligns the spatial embeddings by computing an optimal matching flow between feature maps of the two images. We also devise a multi-scale matching strategy, which considers both global and local similarities and can significantly reduce the computational costs in the application of image retrieval. To handle the view variance in some complicated scenarios, we propose to use cross-correlation as the marginal distribution of the optimal transport to leverage semantic information to locate the important region in the images. Our framework is model-agnostic, which can be applied to off-the-shelf backbone networks and metric learning methods. To extend our DIML to more advanced architectures like vision Transformers (ViTs), we further propose truncated attention rollout and partial similarity to overcome the lack of locality in ViTs. We evaluate our method on three major benchmarks of deep metric learning including CUB200-2011, Cars196, and Stanford Online Products, and achieve substantial improvements over popular metric learning methods with better interpretability.

Neural Bookmarks: Information Retrieval with Deep Learning and EEG Data

In neural memory decoding, a concept being mentally recalled is identified using brain data. Recently, the feasibility of neural memory decoding with EEG data has been demonstrated. Here we propose a new application – neural information retrieval – that uses neural memory decoding to allow a document to be retrieved merely by thinking about it. In this paper we describe neural memory decoding, define the application of neural information retrieval, present experimental results related to the practicality of the application, and discuss issues of deployment and data privacy.

Realizing Efficient On-Device Language-based Image Retrieval

Advances in deep learning have enabled accurate language-based search and retrieval, e.g., over user photos, in the cloud. Many users prefer to store their photos in the home due to privacy concerns. As such, a need arises for models that can perform cross-modal search on resource-limited devices. State-of-the-art cross-modal retrieval models achieve high accuracy through learning entangled representations that enable fine-grained similarity calculation between a language query and an image, but at the expense of having a prohibitively high retrieval latency. Alternatively, there is a new class of methods that exhibits good performance with low latency, but requires a lot more computational resources, and an order of magnitude more training data (i.e. large web-scraped datasets consisting of millions of image-caption pairs) making them infeasible to use in a commercial context. From a pragmatic perspective, none of the existing methods are suitable for developing commercial applications for low-latency cross-modal retrieval on low-resource devices. We propose CrispSearch, a cascaded approach that greatly reduces the retrieval latency with minimal loss in ranking accuracy for on-device language-based image retrieval. The idea behind our approach is to combine a light-weight and runtime-efficient coarse model with a fine re-ranking stage. Given a language query, the coarse model effectively filters out many of the irrelevant image candidates. After this filtering, only a handful of strong candidates will be selected and sent to a fine model for re-ranking. Extensive experimental results with two SOTA models for the fine re-ranking stage, on standard benchmark datasets show that CrispSearch results in a speedup of up to 38 times over the SOTA fine methods with negligible performance degradation. Moreover, our method does not require millions of training instances, making it a pragmatic solution to on-device search and retrieval.

Deep Neighborhood-aware Proxy Hashing with Uniform Distribution Constraint for Cross-modal Retrieval

Cross-modal retrieval methods based on hashing have gained significant attention in both academic and industrial research. Deep learning techniques have played a crucial role in advancing supervised cross-modal hashing methods, leading to significant practical improvements. Despite these achievements, current deep cross-modal hashing still encounters some underexplored limitations. Specifically, most of the available deep hashing usually utilizes pair-wise or triplet-wise strategies to promote the separation of the inter-classes by calculating the relative similarities between samples, weakening the compactness of intra-class data from different modalities, which could generate ambiguous neighborhoods. In this article, the Deep Neighborhood-aware Proxy Hashing (DNPH) framework is proposed to learn a discriminative embedding space with the original neighborhood relation preserved. By introducing learnable shared category proxies, the neighborhood-aware proxy loss is proposed to project the heterogeneous data into a unified common embedding, in which the sample is pulled closer to the corresponding category proxy and is pushed away from other proxies, capturing small within-class scatter and big between-class scatter. To enhance the quality of the obtained binary codes, the uniform distribution constraint is developed to make each hash bit independently obey the discrete uniform distribution. In addition, the discrimination loss is designed to preserve modality-specific semantic information of samples. Extensive experiments are performed on three benchmark datasets to prove that our proposed DNPH framework achieves comparable or even better performance compared with the state-of-the-art cross-modal retrieval applications. The corresponding code implementation of our DNPH framework is as follows: https://github.com/QinLab-WFU/OUR-DNPH .

Structure-aware contrastive hashing for unsupervised cross-modal retrieval

Cross-modal hashing has attracted a lot of attention and achieved remarkable success in large-scale cross-media similarity retrieval applications because of its superior computational efficiency and low storage overhead. However, constructing similarity relationship among samples in cross-modal unsupervised hashing is challenging because of the lack of manual annotation. Most existing unsupervised methods directly use the representations extracted from the backbone of their respective modality to construct instance similarity matrices, leading to inaccurate similarity matrices and resulting in suboptimal hash codes. To address this issue, a novel unsupervised hashing model, named Structure-aware Contrastive Hashing for Unsupervised Cross-modal Retrieval (SACH), is proposed in this paper. Specifically, we concurrently employ both high-dimensional representations and discriminative representations learned by the network to construct a more informative semantic correlative matrix across modalities. Moreover, we design a multimodal structure-aware alignment network to minimize heterogeneous gap in the high-order semantic space of each modality, effectively reducing disparities within heterogeneous data sources and enhancing the consistency of semantic information across modalities. Extensive experimental results on two widely utilized datasets demonstrate the superiority of our proposed SACH method in cross-modal retrieval tasks over existing state-of-the-art methods.