Retrieval Framework Research Articles

Correcting Exoplanet Transmission Spectra for Stellar Activity with an Optimized Retrieval Framework

The chromatic contamination that arises from photospheric heterogeneities, e.g., spots and faculae on the host star presents a significant noise source for exoplanet transmission spectra. If this contamination is not corrected for, it can introduce substantial bias in our analysis of the planetary atmosphere. We utilize two stellar models of differing complexity, StARPA (Stellar Activity Removal for Planetary Atmospheres) and ASteRA (Active Stellar Retrieval Algorithm), to explore the biases introduced by stellar contamination in retrieval under differing degrees of stellar activity. We use the retrieval framework TauREx3 and a grid of 27 synthetic, spot-contaminated transmission spectra to investigate potential biases and to determine how complex our stellar models must be in order to accurately extract the planetary parameters from transmission spectra. The input observation is generated using the more complex model (StARPA), in which the spot latitude is an additional, fixable parameter. This observation is then fed into a combined stellar-planetary retrieval, which contains a simplified stellar model (ASteRA). Our results confirm that the inclusion of stellar activity parameters in retrieval minimizes bias under all activity regimes considered. ASteRA performs very well under low-to-moderate activity conditions, retrieving the planetary parameters with a high degree of accuracy. For the most active cases, characterized by larger, higher-temperature contrast spots, some minor residual bias remains due to ASteRA neglecting the interplay between the spot and the limb-darkening effect. As a result of this, we find larger errors in retrieved planetary parameters for central spots (0°) and those found close to the limb (60°) than those at intermediate latitudes (30°).

A novel blockchain and fractional DCT based image storage and retrieval framework

Securing medical images are a challenging task for researchers as they are prone to cyber- attacks. A potential solution to this problem can be blockchain, which provide secure and unchangeable storage. Fractional Discrete Cosine Transform (frDCT) has the capability to minimizes the amount of data necessary for expressing an image in a secure way. This paper presents a novel framework for compressing as well as securely storing and retrieving medical images by extraction of feature maps from medical images by frDCT, followed by encoding and storing the feature map on decentralized cloud and linking them on blockchain using fractional angle α. The proposed framework provides numerous benefits such as improved data sharing and collaboration, enhanced security, compression as well as efficient retrieval and processing of medical image data which in turn decreases the computational energy requirement providing sustainability. The performance of proposed framework has been evaluated by employing image quality metric such as MSE, PSNR, SSIM and multi-SSIM by comparing it with correct and incorrect α values on six different values of frDCT parameter α.

Fabric image retrieval based on decoupling of texture and color feature

Fabric image retrieval, a form of content based image retrieval, is a high value research with the potential to be applied in many fields, such as e-commerce and inventory management. However, this research hotspot is plagued by two major challenges, namely the high requirements for retrieval results and the peculiarities of fabric images. Unlike general image retrieval, fabric image retrieval systems have to pay more attention to texture and color features. To address these challenges, we propose a novel framework for fabric retrieval by using self-supervised and deep hashing techniques. The framework consists of two modules for feature learning and hashing learning. During the feature learning phase, the color and texture information in the image is decoupled under the drive of augmented based pretext tasks. In hashing learning, Bi-half layer is introduced to generate high-quality hash codes. The visualization results indicate that the proposed method performs well for the representation of fabric images. And the experimental results show that the proposed retrieval framework can achieve a good performance (best mAP 0.903) and outperforms other methods, including several deep hashing methods and our previous work.

GRLoR: A Unified Global Retrieval and Local Reranking Framework for 3-D Place Recognition

GRLoR: A Unified Global Retrieval and Local Reranking Framework for 3-D Place Recognition

Atmospheric Metallicity and C/O of HD 189733 b from High-resolution Spectroscopy

We present high-resolution K-band emission spectra of the quintessential hot Jupiter HD 189733 b from the Keck Planet Imager and Characterizer. Using a Bayesian retrieval framework, we fit the dayside pressure–temperature profile, orbital kinematics, mass-mixing ratios of H2O, CO, CH4, NH3, HCN, and H2S, and the 13CO/12CO ratio. We measure mass fractions of and , and place upper limits on the remaining species. Notably, we find logCH4 < −4.5 at 99% confidence, despite its anticipated presence at the equilibrium temperature of HD 189733 b assuming local thermal equilibrium. We make a tentative (∼3σ) detection of 13CO, and the retrieved posteriors suggest a 12C/13C ratio similar to or substantially less than the local interstellar value. The possible 13C enrichment would be consistent with accretion of fractionated material in ices or in the protoplanetary disk midplane. The retrieved abundances correspond to a substantially substellar atmospheric C/O = 0.3 ± 0.1, while the carbon and oxygen abundances are stellar to slightly superstellar, consistent with core-accretion models which predict an inverse correlation between C/O and metallicity. The specific combination of low C/O and high metallicity suggests significant accretion of solid material may have occurred late in the formation process of HD 189733 b.

Dual Modality Reverse Re-Ranking (DM-RR) based image retrieval framework

Dual Modality Reverse Re-Ranking (DM-RR) based image retrieval framework

GSSF: Generalized Structural Sparse Function for Deep Cross-Modal Metric Learning.

Cross-modal metric learning is a prominent research topic that bridges the semantic heterogeneity between vision and language. Existing methods frequently utilize simple cosine or complex distance metrics to transform the pairwise features into a similarity score, which suffers from an inadequate or inefficient capability for distance measurements. Consequently, we propose a Generalized Structural Sparse Function to dynamically capture thorough and powerful relationships across modalities for pair-wise similarity learning while remaining concise but efficient. Specifically, the distance metric delicately encapsulates two formats of diagonal and block-diagonal terms, automatically distinguishing and highlighting the cross-channel relevancy and dependency inside a structured and organized topology. Hence, it thereby empowers itself to adapt to the optimal matching patterns between the paired features and reaches a sweet spot between model complexity and capability. Extensive experiments on cross-modal and two extra uni-modal retrieval tasks (image-text retrieval, person re-identification, fine-grained image retrieval) have validated its superiority and flexibility over various popular retrieval frameworks. More importantly, we further discover that it can be seamlessly incorporated into multiple application scenarios, and demonstrates promising prospects from Attention Mechanism to Knowledge Distillation in a plug-and-play manner.

Developing a Base Domain Ontology from Geoscience Report Collection to Aid in Information Retrieval towards Spatiotemporal and Topic Association

The efficient and precise retrieval of desired information from extensive geological databases is a prominent and pivotal focus within the realm of geological information services. Conventional information retrieval methods primarily rely on keyword matching approaches, which often overlook the contextual and semantic aspects of the keywords, consequently impeding the retrieval system’s ability to accurately comprehend user query requirements. To tackle this challenge, this study proposes an ontology-driven information-retrieval framework for geological data that integrates spatiotemporal and topic associations. The framework encompasses the development of a geological domain ontology, extraction of key information, establishment of a multi-feature association and retrieval framework, and validation through a comprehensive case study. By employing the proposed framework, users are empowered to actively and automatically retrieve pertinent information, simplifying the information access process, mitigating the burden of comprehending information organization and software application models, and ultimately enhancing retrieval efficiency.

SODRet: Instance retrieval using salient object detection for self-service shopping

Self-service shopping technologies have become commonplace in modern society. Although various innovative solutions have been adopted, there is still a gap in providing efficient services to consumers. Recent developments in mobile application technologies and internet-of-things devices promote information and knowledge dissemination by integrating innovative services to meet users’ needs. We argue that object retrieval applications can be used to provide effective online or self-service shopping. Therefore, to fill this technological void, this study aims to propose an object retrieval system using a fusion-based salient object detection (SOD) method. The SOD has attracted significant attention, and recently many heuristic computational models have been developed for object detection. It has been widely used in object detection and retrieval applications. This work proposes an instance retrieval system based on the SOD to find the objects from the commodity datasets. A prediction about the object’s position is made using the saliency detection system through a saliency model, and the proposed SOD-based retrieval (SODRet) framework uses saliency maps for retrieving the searched items. The method proposed in this work is evaluated on INSTRE and Flickr32 datasets. Our proposed work outperforms state-of-the-art object retrieval methods and can further be employed for large-scale self-service shopping-based points of sales.

Improving First-stage Retrieval of Point-of-interest Search by Pre-training Models

Point-of-interest (POI) search is important for location-based services, such as navigation and online ride-hailing service. The goal of POI search is to find the most relevant destinations from a large-scale POI database given a text query. To improve the effectiveness and efficiency of POI search, most existing approaches are based on a multi-stage pipeline that consists of an efficiency-oriented retrieval stage and one or more effectiveness-oriented re-rank stages. In this article, we focus on the first efficiency-oriented retrieval stage of the POI search. We first identify the limitations of existing first-stage POI retrieval models in capturing the semantic-geography relationship and modeling the fine-grained geographical context information. Then, we propose a Geo-Enhanced Dense Retrieval framework for POI search to alleviate the above problems. Specifically, the proposed framework leverages the capacity of pre-trained language models (e.g., BERT) and designs a pre-training approach to better model the semantic match between the query prefix and POIs. With the POI collection, we first perform a token-level pre-training task based on a geographical-sensitive masked language prediction and design two retrieval-oriented pre-training tasks that link the address of each POI to its name and geo-location. With the user behavior logs collected from an online POI search system, we design two additional pre-training tasks based on users’ query reformulation behavior and the transitions between POIs. We also utilize a late-interaction network structure to model the fine-grained interactions between the text and geographical context information within an acceptable query latency. Extensive experiments on the real-world datasets collected from the Didichuxing application demonstrate that the proposed framework can achieve superior retrieval performance over existing first-stage POI retrieval methods.

A multi-modal lecture video indexing and retrieval framework with multi-scale residual attention network and multi-similarity computation

A multi-modal lecture video indexing and retrieval framework with multi-scale residual attention network and multi-similarity computation

Combining Satellite Imagery and a Deep Learning Algorithm to Retrieve the Water Levels of Small Reservoirs

There are an estimated 800,000 small reservoirs globally with a range of uses. Given the collective importance of these reservoirs to water resource management and wider society, it is essential that we can monitor and understand the hydrological dynamics of ungauged reservoirs, particularly in a changing climate. However, unlike large reservoirs, continuous and systematic hydrological observations of small reservoirs are often unavailable. In response, this study has developed a retrieval framework for water levels of small reservoirs using a deep learning algorithm and remotely sensed satellite data. Demonstrated at four reservoirs in California, satellite imagery from both Sentinel-1 and Sentinel-2 along with corresponding water level field measurements was collected. Post-processed images were fed into a water level inversion convolutional neural network model for water level inversion, while different combinations of these satellite images, sampling approaches for training/testing data, and attention modules were used to train the model and evaluated for accuracy. The results show that random sampling of training data coupled with Sentinel-2 satellite imagery was generally the most accurate initially. Performance is improved by incorporating a channel attention mechanism, with the average R2 increasing by 8.6% and the average RMSE and MAE decreasing by 15.5% and 36.4%, respectively. The proposed framework was further validated on three additional reservoirs in different regions. In conclusion, the retrieval framework proposed in this study provides a stable and accurate methodology for water level estimation of small reservoirs and can be a powerful tool for small reservoir monitoring over large spatial scales.

Efficient Retrieval of Images with Irregular Patterns Using Morphological Image Analysis: Applications to Industrial and Healthcare Datasets.

Image retrieval is the process of searching and retrieving images from a datastore based on their visual content and features. Recently, much attention has been directed towards the retrieval of irregular patterns within industrial or healthcare images by extracting features from the images, such as deep features, colour-based features, shape-based features, and local features. This has applications across a spectrum of industries, including fault inspection, disease diagnosis, and maintenance prediction. This paper proposes an image retrieval framework to search for images containing similar irregular patterns by extracting a set of morphological features (DefChars) from images. The datasets employed in this paper contain wind turbine blade images with defects, chest computerised tomography scans with COVID-19 infections, heatsink images with defects, and lake ice images. The proposed framework was evaluated with different feature extraction methods (DefChars, resized raw image, local binary pattern, and scale-invariant feature transforms) and distance metrics to determine the most efficient parameters in terms of retrieval performance across datasets. The retrieval results show that the proposed framework using the DefChars and the Manhattan distance metric achieves a mean average precision of 80% and a low standard deviation of ±0.09 across classes of irregular patterns, outperforming alternative feature-metric combinations across all datasets. Our proposed ImR framework performed better (by 8.71%) than Super Global, a state-of-the-art deep-learning-based image retrieval approach across all datasets.

BiC-Net: Learning Efficient Spatio-temporal Relation for Text-Video Retrieval

The task of text-video retrieval aims to understand the correspondence between language and vision and has gained increasing attention in recent years. Recent works have demonstrated the superiority of local spatio-temporal relation learning with graph-based models. However, most existing graph-based models are handcrafted and depend heavily on expert knowledge and empirical feedback, which may be unable to mine the high-level fine-grained visual relations effectively. These limitations result in their inability to distinguish videos with the same visual components but different relations. To solve this problem, we propose a novel cross-modal retrieval framework, Bi-Branch Complementary Network (BiC-Net), which modifies Transformer architecture to effectively bridge text-video modalities in a complementary manner via combining local spatio-temporal relation and global temporal information. Specifically, local video representations are encoded using multiple Transformer blocks and additional residual blocks to learn fine-grained spatio-temporal relations and long-term temporal dependency, calling the module a Fine-grained Spatio-temporal Transformer (FST). Global video representations are encoded using a multi-layer Transformer block to learn global temporal features. Finally, we align the spatio-temporal relation and global temporal features with the text feature on two embedding spaces for cross-modal text-video retrieval. Extensive experiments are conducted on MSR-VTT, MSVD, and YouCook2 datasets. The results demonstrate the effectiveness of our proposed model. Our code is public at https://github.com/lionel-hing/BiC-Net .

Privacy-Preserving Image Retrieval with Multi-Modal Query

Abstract The ever-growing multi-modal images pose great challenges to local image storage and retrieval systems. Cloud computing provides a solution to large-scale image data storage but suffers from privacy issues and lacks the support for multi-modal image retrieval. To address these, a searchable encryption-empowered privacy-preserving multi-modal image retrieval method is proposed. First, we design a hybrid image retrieval framework that fuses visual features and textual features at a decision level and further supports similar image retrieval and multi-keyword image retrieval. Second, we construct a new hybrid inverted index structure to distinguish high-frequency terms from low-frequency terms and index them through hierarchical index trees and data blocks, respectively, which greatly improves query efficiency. Third, we design a prime encoding-based multi-keyword query method that converts mapping operations in bloom filters into inner product calculations, and further implements secure multi-keyword image query. Experiments against the Baseline schemes are conducted to verify the performance of the scheme in terms of high efficiency.

Data harmonization across eight publicly available longitudinal studies of aging, neurodegeneration and dementia

AbstractBackgroundLarge‐scale multi‐study analyses are required to ensure reproducibility, reliability and generalizability in mapping neurodegeneration and risk for ADRD. However, the heterogeneity in data collection paradigms can complicate and confound data pooling; data harmonization is essential. Longitudinal studies add to the complexity of harmonization as a variety of follow‐up time points and time encoding schemes are used. Here, we pool data from 8 publicly available longitudinal neuroimaging studies on neurodegeneration and dementia to highlight differences in: 1) diagnostic categorization of controls, and people with mild cognitive impairment, and dementia; 2) the extent of follow‐up visits across neuroimaging and clinical assessments; and 3) encodings of various meta‐data elements including scanner manufacturer, sex, and handedness. To allow a systematic approach to multi‐study dementia research, we propose an initial ontological framework for longitudinal data archival and retrieval, capable of capturing similar data elements within given themes while ensuring that unique differences across studies are retained.MethodAIBL, ADNI‐1, ADNI2/GO, ADNI‐3, OASIS‐2, OASIS‐3, PREVENT‐AD, and MIRIAD, were accessed. Study designs, inclusion and exclusion criteria, and downloaded data elements were used to describe diagnostic criteria, imaging data, demographic features, and longitudinal data collection schemes. We create common terms to map data elements across studies, and a consistent naming scheme for longitudinal data.ResultFigure 1 shows the breakdown of diagnostic labels per cohort, and highlights how different cognitive labels may be assigned to people with the same performance scores, for example, on the Mini Mental State Examination (MMSE). Figure 2 highlights the differences in longitudinal data collection schemes across cohorts, and in labels of commonly collected elements. Using common terms provided an easy way to simultaneously query data across all studies, while retaining a map back to the original terms. Figure 3 showcases our proposed naming scheme for capturing longitudinal data elements using a relational ontological framework. This framework successfully harmonized data across study demographics, timepoints, and imaging properties.ConclusionMulti‐study data analyses are becoming common practice thanks to large scale efforts such as ENIGMA and GAAIN. Harmonization of meta‐data across studies will allow for efficient pooling of data for tracking disease progression and related risk over time.

Letter from the Chair

Greetings SIGIR Members! SIGIR in Taipei was a success. A big thank you to the general chairs Hsin-Hsi Chen, Wei-Jou (Edward) Duh , and Hen-Hsen Huang , for putting on a great conference, in a beautiful venue. Thank you also to the Program Chairs Makoto Kato, Josiane Mothe and Barbara Poblete for the excellent technical programming! We would like to congratulate the Best Paper winners Maik Fröbe, Jan Heinrich Reimer, Sean MacAvaney, Niklas Deckers, Simon Reich, Janek Bevendorff, Benno Stein, Matthias Hagen , and Martin Potthast for their paper The Information Retrieval Experiment Platform. We would also like to congratulate Alireza Salemi, Juan Altmayer Pizzorno , and Hamed Zamani whose paper A Symmetric Dual Encoding Dense Retrieval Framework for Knowledge-Intensive Visual Question Answering won the Best Student Paper Award at SIGIR this year. Preparations for SIGIR 2024 in Washington DC are well underway, as preparations for SIGIR 2025 in Padua are beginning.

Fine-scale spatial variability of marine acoustic environment corresponds with habitat utilization of Indo-Pacific humpback dolphins in Hong Kong waters

Acoustic properties of the underwater environment are important in maintaining biological processes of various marine organisms. However, with the increasing level of underwater noise in the global ocean, there is a growing need to better understand how marine animals use soundscape cues in their habitat selection. Indo-Pacific humpback dolphins (Sousa chinensis) inhabiting the Pearl River Estuary, southeast China, live in one of world’s most developed and noisiest coastal environment and are subjected to many sources of anthropogenic noise. To investigate whether spatial variability of underwater soundscape corresponds with their habitat utilization, we collected daytime underwater recordings in western Hong Kong waters from mid-2016 to mid-2018, and quantified the spatial pattern of marine acoustic environment and its differing characteristics in a fine spatial scale. We developed a framework of soundscape information retrieval to investigate spectral features that may facilitate identification of dolphins’ core habitats. Our findings reveal that a spectral feature, which peaks at 2 kHz, is a reliable predictor of humpback dolphin core habitat. Further modelling of spatial and seasonal variations of underwater soundscape demonstrates that the relative strength of this spectral feature is positively correlated with the sighting rates of humpback dolphins throughout the year. Although the source of the 2 kHz feature remains unknown, it is likely associated with humpback dolphins’ prey. We suggest that underwater acoustic environment represents an important component in evaluating the quality and suitability of coastal habitats for the daily needs of this threatened dolphin species. Local and regional conservation authorities should include habitat-specific baseline soundscape data when developing conservation management strategies.

NLP-based approach for automated safety requirements information retrieval from project documents

Numerous project documents generated during the construction phase pertain to the actual safety requirements (SRs) specified by project managers (PMs), and these concealed requirements can aid in making safety decisions if accurately identified and utilized. However, PMs’ requirements are frequently recorded informally in project documents, necessitating extensive manual analysis to guide safety management practices. To address this limitation, this study developed a natural language processing (NLP)-based framework of requirement retrieval and document association (RRDA) to mine requirements and retrieve requirement-related documents. In particular, requirement-document (RD) association rules are designed to retrieve the requirement-related documents. The results demonstrate that our framework can retrieve the PMs’ requirements with a maximum ontology relevance of 91.37% and emotional preference of requirement with a maximum semantic tendency intensity of 0.91. The presented algorithm shows satisfactory performance in the number of iterations and threshold, and there are outstanding advantages in model training comparison. Additionally, there is a significant degree of matching between the retrieved documents and the requirements, which has significant managerial implications for requirement-oriented construction safety management.

Assessment of Artificial Intelligence Language Models and Information Retrieval Strategies for QA in Hematology

Introduction Large language models (LLMs) have gained popularity due to their natural language generation and interpretation capabilities. Integrating these models in medicine enables multiple tasks like summarizing medical histories, synthesizing literature, and suggesting diagnoses. Models like ChatGPT, GPT-4, and Med-PaLM2 (Singhal et al., 2023) have demonstrated their proficiency by achieving high scores in medical tests like the United States Medical Licensing Examination (USMLE) (Kung et al., 2023). However, LLMs may sometimes be inaccurate, providing unverified and erroneous information. In this study, we investigate the potential uses of LLMs in hematology, assessing their knowledge through hematology questions from the USMLE. Additionally, we propose augmenting LLMs with retrieval capabilities for medical guidelines in order to eliminate incorrect information. By extracting relevant information from specified medical documents, this approach holds the potential to streamline decision-making processes. Methods For comparative purposes, all experiments were conducted using both GPT 3.5-turbo and GPT-4 models. In a first step, we evaluated the general knowledge and performance of LLM in the field of hematology by testing it in a collected dataset of 127 question-answer pairs from the hematology section (covering various aspects of the field) of the USMLE. In a second step, we evaluated the proposed information retrieval framework using a set of 120 multiple-choice questions. These questions were specifically focused on the 4th revision of the World Health Organization classification of myeloid neoplasms and acute leukemia guidelines (subsequently called WHO 2017). By testing the framework on this domain-specific dataset, we aimed to assess its ability to extract specific clinical context and relevant information from complex clinical guidelines. Each question from the WHO 2017 guideline dataset was subjected to a comprehensive evaluation using two techniques. First, the questions were assessed using a zero-shot approach (the question together with the different options are directly posed to the model) to assess the LLM's capability to respond based on its own knowledge. Second, we employed our proposed retrieval information approach, enabling the system to conduct in-depth searches throughout the external documents (WHO 2017 guideline) to identify relevant (and similar) extracts about each question. Subsequently, the system provided answers based on the retrieved contexts from the document, thus facilitating more accurate and contextually informed responses. To achieve this, we created an embedding space containing the document's content and conducted a cosine-similarity search between a given question and all the content extracts from the document. The top three relevant extracts, based on similarity to the given question, were used as context for the LLM. Results In the evaluation of 127 hematology questions from the USMLE, GPT-3.5 in zero-shot mode achieved 63% accuracy, while GPT-4 demonstrated a higher accuracy rate of 82%. The evaluation of the WHO 2017 questions dataset revealed that the zero-shot approach achieved accuracy rates of 51% for GPT-3.5 and 71% for GPT-4. Incorporating information retrieval, retrieving the three most relevant extracts from the guidelines, substantially improved performance, with GPT-3.5 achieving 86% accuracy and GPT-4 demonstrating 97% accuracy. Conclusions LLMs have great potential, with current models showcasing substantial knowledge in hematology. However, ensuring their consistency and safety in responses is critical for their reliable application in medical settings (Thirunavukarasu et al., 2023). To address this, we demonstrated the benefits of information retrieval for question-answering in the field of hematology, significantly improving response reliability and accuracy by empowering LLMs to deliver more informed and contextually appropriate answers. The concept was effectively validated using the WHO 2017 guideline, and it can be effortlessly adapted to answer questions based on any set of hematology-related documents. Leveraging LLMs has the potential to significantly enhance the efficiency and effectiveness of clinical, educational, and research work in hematology.