Query Representation Research Articles

Using query semantic and feature transfer fusion to enhance cardinality estimating of property graph queries

With the increasing complexity and diversity of query tasks, cardinality estimation has become one of the most challenging problems in query optimization. In this study, we propose an efficient and accurate cardinality estimation method to address the cardinality estimation problem in property graph queries, particularly in response to the current research gap regarding the neglect of contextual semantic features. We first propose formal representations of the property graph query and define its cardinality estimation problem. Then, through the query featurization, we transform the query into a vector representation that can be learned by the estimation model, and enrich the feature vector representation by the context semantic information of the query. We finally propose an estimation model for property graph queries, specifically introducing a feature information transfer module to dynamically control the information flow meanwhile achieving the model’s feature fusion and inference. Experimental results on three datasets show that the estimation model can accurately and efficiently estimate the cardinality of property graph queries, the mean Q_error and RMSE are reduced by about 30% and 25% than the state-of-art estimation models. The context semantics features of queries can improve the model’s estimation accuracy, the mean Q_error result is reduced by about 20% and the RMSE result is about 5%.

Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval

Abstract Retrieval-augmented language models (LMs) have received much attention recently. However, typically the retriever is not trained jointly as a native component of the LM, but added post-hoc to an already-pretrained LM, which limits the ability of the LM and the retriever to adapt to one another. In this work, we propose the Retrieval-Pretrained Transformer (RPT), an architecture and training procedure for jointly training a retrieval-augmented LM from scratch and applying it to the task of modeling long texts. Given a recently generated text chunk in a long document, the LM computes query representations, which are then used to retrieve earlier chunks in the document, located potentially tens of thousands of tokens before. Information from retrieved chunks is fused into the LM representations to predict the next target chunk. We train the retriever component with a semantic objective, where the goal is to retrieve chunks that increase the probability of the next chunk, according to a reference LM. We evaluate RPT on four long-range language modeling tasks, spanning books, code, and mathematical writing, and demonstrate that RPT improves retrieval quality and subsequently perplexity across the board compared to strong baselines.

3D printing application for orthopedic pediatric surgery – a systematic review

PurposeThis paper aims to present a systematic review of the latest scientific literature, in the context of pediatric orthopedics, on the development by additive manufacturing of anatomical models, orthoses, surgical guides and prostheses and their clinical applications.Design/methodology/approachFollowing the current guidelines for systematic reviews, three databases (Elsevier Scopus®, Clarivate Web of ScienceTM and USA National Library of Medicine PubMed®) were screened using a representative query to find pertinent documents within the timeframe 2016–2023. Among the information, collected across the reviewed documents, the work focused on the 3D printing workflow involving acquisition, elaboration and fabrication stages.FindingsFollowing the inclusion and exclusion criteria, the authors found 20 studies that fitted the defined criteria. The reviewed studies mostly highlighted the positive impact of additive manufacturing in pediatric orthopedic surgery, particularly in orthotic applications where lightweight, ventilated and cost-effective 3D-printed devices demonstrate efficacy comparable to traditional methods, but also underlined the limitations such as printing errors and high printing times. Among the reviewed studies, material extrusion was the most chosen 3D printing technology to manufacture the typical device, particularly with acrylonitrile butadiene styrene.Originality/valueTo the best of the authors’ knowledge, this is the first systematic review which annotates, from a more engineering point of view, the latest literature on the admittance of the clinical application of additive manufacturing (and its effects) within typical pediatric orthopedic treatments workflows.

Comparative Evaluation of Pre-Trained Language Models for Biomedical Information Retrieval.

Finding relevant information in the biomedical literature increasingly depends on efficient information retrieval (IR) algorithms. Cross-Encoders, SentenceBERT, and ColBERT are algorithms based on pre-trained language models that use nuanced but computable vector representations of search queries and documents for IR applications. Here we investigate how well these vectorization algorithms estimate relevance labels of biomedical documents for search queries using the OHSUMED dataset. For our evaluation, we compared computed scores to provided labels by using boxplots and Spearman's rank correlations. According to these metrics, we found that Sentence-BERT moderately outperformed the alternative vectorization algorithms and that additional fine-tuning based on a subset of OHSUMED labels yielded little additional benefit. Future research might aim to develop a larger dedicated dataset in order to optimize such methods more systematically, and to evaluate the corresponding functions in IR tools with end-users.

Training-free approach to constructing ensemble of local experts

Conventional machine learning methods assume that a central server holds the entire training dataset for training a global model. However, in many real-world situations, datasets are collected from individual local nodes in distributed environments. Gathering these datasets often poses challenges owing to various practical constraints such as data security and privacy issues, storage limits, and transmission costs. When data distributions differ among local nodes, local experts trained on individual local nodes exhibit poor generalization performance for new instances. In this study, we propose a training-free method for constructing an ensemble of local experts without accessing datasets. For a query instance, we apply uncertainty quantification and out-of-distribution (OOD) detection to calculate the uncertainty and OOD scores of the predictions from individual models. We then determine the weights of the models by assigning a lower weight to those with higher uncertainty and OOD scores. Using these weights, the predictions of the individual models are aggregated to obtain a final prediction. In contrast to existing methods that impose specific application requirements, the proposed method can be applied in situations in which only individual predictions from local models are available for use when making a prediction for a query instance. We evaluate the effectiveness of the proposed method using image classification benchmark datasets.

Listen as you wish: Fusion of audio and text for cross-modal event detection in smart cities

In the era of smart cities, the advent of the Internet of Things technology has catalyzed the proliferation of multimodal sensor data, presenting new challenges in cross-modal event detection, particularly in audio event detection via textual queries. This paper focuses on the novel task of text-to-audio grounding (TAG), aiming to precisely localize sound segments that correspond to events described in textual queries within an untrimmed audio. This challenging new task requires multi-modal (acoustic and linguistic) information fusion as well as the reasoning for the cross-modal semantic matching between the given audio and textual query. Unlike conventional methods that often overlook the nuanced interactions between and within modalities, we introduce the Cross-modal Graph Interaction (CGI) model. This innovative approach leverages a language graph to model complex semantic relationships between query words, enhancing the understanding of textual queries. Additionally, a cross-modal attention mechanism generates snippet-specific query representations, facilitating fine-grained semantic matching between audio segments and textual descriptions. A cross-gating module further refines this process by emphasizing relevant features across modalities and suppressing irrelevant information, optimizing multimodal information fusion. Our comprehensive evaluation on the Audiogrounding benchmark dataset not only demonstrates the CGI model’s superior performance over existing methods, but also underscores the significance of sophisticated multimodal interaction in improving the efficacy of TAG in smart cities.

Modified LDA vector and feedback analysis for short query Information Retrieval systems

Abstract Information Retrieval systems benefit from the use of long queries containing a large volume of search-relevant information. This situation is not common, as users of such systems tend to use very short and precise queries with few keywords. In this work we propose a modification of the Latent Dirichlet Allocation (LDA) technique using data from the document collection and its vocabulary for a better representation of short queries. Additionally, a study is carried out on how the modification of the proposed LDA weighted vectors increase the performance using relevant documents as feedback. The work shown in this paper is tested using three biomedical corpora (TREC Genomics 2004, TREC Genomics 2005 and OHSUMED) and one legal corpus (FIRE 2017). Results prove that the application of the proposed representation technique, as well as the feedback adjustment, clearly outperforms the baseline methods (BM25 and non-modified LDA).

Score distillation for anomaly detection

Recently, significant performance improvements have been achieved in deep learning-based anomaly detection methods by introducing large neural network architectures and complex anomaly scoring functions. However, the computational cost and memory usage required in the inference phase have also increased significantly, thereby limiting their use in real-time applications. In this paper, we propose a score distillation method that adopts the concept of knowledge distillation. An existing high-performance anomaly detection method is used as the teacher. A small neural network is then trained as the student to mimic the scoring function of the teacher. In the inference phase, the anomaly score for a query instance is obtained by a single forward pass through the student network without requiring any complicated computation processes. We demonstrate that the proposed method makes anomaly detection faster and more efficient while maintaining high performance.

Not All Instances Contribute Equally: Instance-Adaptive Class Representation Learning for Few-Shot Visual Recognition.

Few-shot visual recognition refers to recognize novel visual concepts from a few labeled instances. Many few-shot visual recognition methods adopt the metric-based meta-learning paradigm by comparing the query representation with class representations to predict the category of query instance. However, the current metric-based methods generally treat all instances equally and consequently often obtain biased class representation, considering not all instances are equally significant when summarizing the instance-level representations for the class-level representation. For example, some instances may contain unrepresentative information, such as too much background and information of unrelated concepts, which skew the results. To address the above issues, we propose a novel metric-based meta-learning framework termed instance-adaptive class representation learning network (ICRL-Net) for few-shot visual recognition. Specifically, we develop an adaptive instance revaluing network (AIRN) with the capability to address the biased representation issue when generating the class representation, by learning and assigning adaptive weights for different instances according to their relative significance in the support set of corresponding class. In addition, we design an improved bilinear instance representation and incorporate two novel structural losses, i.e., intraclass instance clustering loss and interclass representation distinguishing loss, to further regulate the instance revaluation process and refine the class representation. We conduct extensive experiments on four commonly adopted few-shot benchmarks: miniImageNet, tieredImageNet, CIFAR-FS, and FC100 datasets. The experimental results compared with the state-of-the-art approaches demonstrate the superiority of our ICRL-Net.

Semi-supervised Open-World Object Detection

Conventional open-world object detection (OWOD) problem setting first distinguishes known and unknown classes and then later incrementally learns the unknown objects when introduced with labels in the subsequent tasks. However, the current OWOD formulation heavily relies on the external human oracle for knowledge input during the incremental learning stages. Such reliance on run-time makes this formulation less realistic in a real-world deployment. To address this, we introduce a more realistic formulation, named semi-supervised open-world detection (SS-OWOD), that reduces the annotation cost by casting the incremental learning stages of OWOD in a semi-supervised manner. We demonstrate that the performance of the state-of-the-art OWOD detector dramatically deteriorates in the proposed SS-OWOD setting. Therefore, we introduce a novel SS-OWOD detector, named SS-OWFormer, that utilizes a feature-alignment scheme to better align the object query representations between the original and augmented images to leverage the large unlabeled and few labeled data. We further introduce a pseudo-labeling scheme for unknown detection that exploits the inherent capability of decoder object queries to capture object-specific information. On the COCO dataset, our SS-OWFormer using only 50% of the labeled data achieves detection performance that is on par with the state-of-the-art (SOTA) OWOD detector using all the 100% of labeled data. Further, our SS-OWFormer achieves an absolute gain of 4.8% in unknown recall over the SOTA OWOD detector. Lastly, we demonstrate the effectiveness of our SS-OWOD problem setting and approach for remote sensing object detection, proposing carefully curated splits and baseline performance evaluations. Our experiments on 4 datasets including MS COCO, PASCAL, Objects365 and DOTA demonstrate the effectiveness of our approach. Our source code, models and splits are available here https://github.com/sahalshajim/SS-OWFormer

SRFormer: Text Detection Transformer with Incorporated Segmentation and Regression

Existing techniques for text detection can be broadly classified into two primary groups: segmentation-based and regression-based methods. Segmentation models offer enhanced robustness to font variations but require intricate post-processing, leading to high computational overhead. Regression-based methods undertake instance-aware prediction but face limitations in robustness and data efficiency due to their reliance on high-level representations. In our academic pursuit, we propose SRFormer, a unified DETR-based model with amalgamated Segmentation and Regression, aiming at the synergistic harnessing of the inherent robustness in segmentation representations, along with the straightforward post-processing of instance-level regression. Our empirical analysis indicates that favorable segmentation predictions can be obtained at the initial decoder layers. In light of this, we constrain the incorporation of segmentation branches to the first few decoder layers and employ progressive regression refinement in subsequent layers, achieving performance gains while minimizing computational load from the mask. Furthermore, we propose a Mask-informed Query Enhancement module. We take the segmentation result as a natural soft-ROI to pool and extract robust pixel representations, which are then employed to enhance and diversify instance queries. Extensive experimentation across multiple benchmarks has yielded compelling findings, highlighting our method's exceptional robustness, superior training and data efficiency, as well as its state-of-the-art performance. Our code is available at https://github.com/retsuh-bqw/SRFormer-Text-Det.

Graph contrastive learning with consistency regularization

Contrastive learning has actively been used for unsupervised graph representation learning owing to its success in computer vision. Most graph contrastive learning methods use instance discrimination. It treats each instance as a distinct class against a query instance as the pretext task. However, such methods inevitably cause a class collision problem because some instances may belong to the same class as the query. Thus, the similarity shared through instances from the same class cannot be reflected in the pre-training stage. To address this problem, we propose graph contrastive learning with consistency regularization (GCCR), which introduces a consistency regularization term to graph contrastive learning. Unlike existing methods, GCCR can obtain a graph representation that reflects intra-class similarity by introducing a consistency regularization term. To verify the effectiveness of the proposed method, we performed extensive experiments and demonstrated that GCCR improved the quality of graph representations for most datasets. Notably, experimental results in various settings show that the proposed method can learn effective graph representations with better robustness against transformations than other state-of-the-art methods.

On The Reasonable Effectiveness of Relational Diagrams: Explaining Relational Query Patterns and the Pattern Expressiveness of Relational Languages

Comparing relational languages by their logical expressiveness is well understood. Less well understood is how to compare relational languages by their ability to represent relational query patterns. Indeed, what are query patterns other than "a certain way of writing a query"? And how can query patterns be defined across procedural and declarative languages, irrespective of their syntax? To the best of our knowledge, we provide the first semantic definition of relational query patterns by using a variant of structure-preserving mappings between the relational tables of queries. This formalism allows us to analyze the relative pattern expressiveness of relational language fragments and create a hierarchy of languages with equal logical expressiveness yet different pattern expressiveness. Notably, for the non-disjunctive language fragment, we show that relational calculus can express a larger class of patterns than the basic operators of relational algebra. Our language-independent definition of query patterns opens novel paths for assisting database users. For example, these patterns could be leveraged to create visual query representations that faithfully represent query patterns, speed up interpretation, and provide visual feedback during query editing. As a concrete example, we propose Relational Diagrams, a complete and sound diagrammatic representation of safe relational calculus that is provably (i) unambiguous, (ii) relationally complete, and (iii) able to represent all query patterns for unions of non-disjunctive queries. Among all diagrammatic representations for relational queries that we are aware of, ours is the only one with these three properties. Furthermore, our anonymously preregistered user study shows that Relational Diagrams allow users to recognize patterns meaningfully faster and more accurately than SQL.

Multi-Content Interaction Network for Few-Shot Segmentation

Few-Shot Segmentation (FSS) poses significant challenges due to limited support images and large intra-class appearance discrepancies. Most existing approaches focus on aligning the support-query correlations from the same layer of the frozen backbone while neglecting the bias between different tasks and different layers. In this article, we propose a Multi-Content Interaction Network (MCINet) to remedy these issues by fully exploiting and interacting with the different contextual information contained in distinct branches. Specifically, MCINet improves FSS from three perspectives: (1) boosting the query representations through incorporating the independent information from another learnable branch into the features from the frozen backbone, (2) enhancing the support-query correlations by exploiting both the same-layer and adjacent-layer features, and (3) refining the predicted results with a multi-scale mask prediction strategy. Experiments on three benchmarks demonstrate that our approach reaches state-of-the-art performances and outperforms the best competitors with many desirable advantages, especially on the challenging COCO dataset. Code will be released on GitHub ( https://github.com/chenhao-zju/mcinet ).

Dynamic ensemble of regression neural networks based on predictive uncertainty

In a data-decentralized environment where data are distributed across individual local nodes, inter-node data sharing is often restricted due to security concerns and communication costs. Under these restrictions, using entire data to train a global predictive model is challenging. A viable strategy is to leverage local predictive models trained independently on local nodes. However, this adversely affect generalization performance, particularly when the data distribution varies between local nodes. To address this issue in regression problems, we propose a dynamic ensemble method that combines the predictions of locally trained neural networks without the need to access data from local nodes. For a query instance, the prediction and its uncertainty from each neural network are computed by applying Monte Carlo dropout. Subsequently, smaller/larger weights are assigned to neural networks with higher/lower predictive uncertainty. The final prediction result is derived using the weighted average of the individual predictions. The proposed method is applicable to a data-decentralized environment where data are inaccessible and thus cannot be shared across local nodes. Transmitting query instances to local nodes and receiving predictions and weights from the local nodes are the only requirements, thereby ensuring low communication costs. Experimental results on simulated data-decentralized environments using benchmark regression datasets validated the superior generalization performance of the proposed method compared with baseline methods. The proposed method was found to be more effective for regression neural networks that demonstrate higher performance at their respective local nodes.

Dense Text Retrieval Based on Pretrained Language Models: A Survey

Text retrieval is a long-standing research topic on information seeking, where a system is required to return relevant information resources to user’s queries in natural language. From heuristic-based retrieval methods to learning-based ranking functions, the underlying retrieval models have been continually evolved with the ever-lasting technical innovation. To design effective retrieval models, a key point lies in how to learn text representations and model the relevance matching. The recent success of pretrained language models (PLM) sheds light on developing more capable text-retrieval approaches by leveraging the excellent modeling capacity of PLMs. With powerful PLMs, we can effectively learn the semantic representations of queries and texts in the latent representation space, and further construct the semantic matching function between the dense vectors for relevance modeling. Such a retrieval approach is calleddense retrieval, since it employs dense vectors to represent the texts. Considering the rapid progress on dense retrieval, this survey systematically reviews the recent progress on PLM-based dense retrieval. Different from previous surveys on dense retrieval, we take a new perspective to organize the related studies by four major aspects, including architecture, training, indexing and integration, and thoroughly summarize the mainstream techniques for each aspect. We extensively collect the recent advances on this topic, and include 300+ reference papers. To support our survey, we create a website for providing useful resources, and release a code repository for dense retrieval. This survey aims to provide a comprehensive, practical reference focused on the major progress for dense text retrieval.

Personalized and Diversified: Ranking Search Results in an Integrated Way

Ambiguity in queries is a common problem in information retrieval. There are currently two solutions: search result personalization and diversification. The former aims to tailor results for different users based on their preferences, but the limitations are redundant results and incomplete capture of user intents. The goal of the latter is to return results that cover as many aspects related to the query as possible. It improves diversity yet loses personality and cannot return the exact results the user wants. Intuitively, such two solutions can complement each other and bring more satisfactory reranking results. In this article, we propose a novel framework, namely, PnD , to integrate personalization and diversification reasonably. We employ the degree of refinding to determine the weight of personalization dynamically. Moreover, to improve the diversity and relevance of reranked results simultaneously, we design a reset RNN structure (RRNN) with the “reset gate” to measure the influence of the newly selected document on novelty. Besides, we devise a “subtopic learning layer” to learn the virtual subtopics, which can yield fine-grained representations of queries, documents, and user profiles. Experimental results illustrate that our model can significantly outperform existing search result personalization and diversification methods.

Query-oriented two-stage attention-based model for code search

Applying code search models to search through a large-scale codebase can significantly contribute to developers finding and reusing existing code. Researchers have applied deep learning (DL) techniques to code search models, which first compute deeper semantics representation for query and candidate code snippets, and then rank code snippets. However, these models do not well deeply analyze the semantics gap (i.e., the difference and correlation between queries written in natural language and code in programming languages), or suitably apply the correlation to the code search task. Moreover, most DL-based models use complex networks, slowing down code search tasks.To build the correlation of two languages, and apply the correlation well to code search task, we propose a query-oriented code search model named QobCS. QobCS leverage two attention-based stages, which are simple and quick, and the cooperation of the two stages bridges the semantic gap between code and query. Stage1 learns deeper semantics representation for code and query. Stage2 applies their deeper semantic correlation and query’s intention to learn better code representation.We evaluated QobCS on two datasets. On dataset1/dataset2 with 485k/542k code snippets, QobCS achieves the MRRs of 0.701/0.595, outperforming DL-based code search models DeepCS, CARLCS-CNN, UNIF, and our prior study TabCS. For efficiency, our model shows desirable performances on both datasets compared to DL-based models.

STFormer: Spatial-Temporal-Aware Transformer for Video Instance Segmentation.

Video instance segmentation (VIS) is a challenging task, requiring handling object classification, segmentation, and tracking in videos. Existing Transformer-based VIS approaches have shown remarkable success, combining encoded features and instance queries as decoder inputs. However, their decoder inputs are low-resolution due to computational cost, resulting in a loss of fine-grained information, sensitivity to background interference, and poor handling of small objects. Moreover, the queries are randomly initialized without location information, hindering convergence efficiency and accurate object instance localization. To address these issues, we propose a novel VIS approach, STFormer, with a spatial-temporal feature aggregation (STFA) module and spatial-temporal-aware Transformer (STT). Specifically, STFA obtains robust high-resolution masked features efficiently for the decoder, while STT's location-guided instance query (LGIQ) improves initial instance queries. STFormer preserves more fine-grained information, improves convergence efficiency, and localizes object instance features accurately. Extensive experiments on YouTube-VIS 2019, YouTube-VIS 2021, and OVIS datasets show that STFormer outperforms mainstream VIS methods.

Deep Learning to Hash with Application to Cross-View Nearest Neighbor Search

Learning hash functions for approximate nearest neighbor search of high-dimensional data has received a surge of interests in recent years. Most existing methods are often concerned with learning hash functions for nearest neighbor search on high-dimensional data from a single source. In many real-world applications, data can be collected from diverse sources or represented using different feature descriptors. This raises an open challenge, i.e., the Cross-View Nearest Neighbor Search (CVNNS), where the representation of a query instance can be different from that of target instances to be retrieved in database. The key challenge of cross-view search is to learn an effective shared representation which can effectively connect the query instance and the target instances to be retrieved. In this paper, we present a new cross-view nearest neighbor search scheme by applying the emerging deep learning to hash techniques. In particular, we investigate two different architectures of deep Restricted Boltzmann Machines (RBMs) for learning to hash toward cross-view nearest neighbor search, and conduct extensive experiments to examine their empirical performance on diverse settings of cross-view image retrieval tasks. The encouraging results show that our technique outperforms the state-of-the-art approaches.