High Retrieval Performance Research Articles

Relation-Aggregated Cross-Graph Correlation Learning for Fine-Grained Image-Text Retrieval.

Fine-grained image-text retrieval has been a hot research topic to bridge the vision and languages, and its main challenge is how to learn the semantic correspondence across different modalities. The existing methods mainly focus on learning the global semantic correspondence or intramodal relation correspondence in separate data representations, but which rarely consider the intermodal relation that interactively provide complementary hints for fine-grained semantic correlation learning. To address this issue, we propose a relation-aggregated cross-graph (RACG) model to explicitly learn the fine-grained semantic correspondence by aggregating both intramodal and intermodal relations, which can be well utilized to guide the feature correspondence learning process. More specifically, we first build semantic-embedded graph to explore both fine-grained objects and their relations of different media types, which aim not only to characterize the object appearance in each modality, but also to capture the intrinsic relation information to differentiate intramodal discrepancies. Then, a cross-graph relation encoder is newly designed to explore the intermodal relation across different modalities, which can mutually boost the cross-modal correlations to learn more precise intermodal dependencies. Besides, the feature reconstruction module and multihead similarity alignment are efficiently leveraged to optimize the node-level semantic correspondence, whereby the relation-aggregated cross-modal embeddings between image and text are discriminatively obtained to benefit various image-text retrieval tasks with high retrieval performance. Extensive experiments evaluated on benchmark datasets quantitatively and qualitatively verify the advantages of the proposed framework for fine-grained image-text retrieval and show its competitive performance with the state of the arts.

Exploiting Cloud Object Storage for High-Performance Analytics

Elasticity of compute and storage is crucial for analytical cloud database systems. All cloud vendors provide disaggregated object stores, which can be used as storage backend for analytical query engines. Until recently, local storage was unavoidable to process large tables efficiently due to the bandwidth limitations of the network infrastructure in public clouds. However, the gap between remote network and local NVMe bandwidth is closing, making cloud storage more attractive. This paper presents a blueprint for performing efficient analytics directly on cloud object stores. We derive cost- and performance-optimal retrieval configurations for cloud object stores with the first in-depth study of this foundational service in the context of analytical query processing. For achieving high retrieval performance, we present AnyBlob , a novel download manager for query engines that optimizes throughput while minimizing CPU usage. We discuss the integration of high-performance data retrieval in query engines and demonstrate it by incorporating AnyBlob in our database system Umbra. Our experiments show that even without caching, Umbra with integrated AnyBlob achieves similar performance to state-of-the-art cloud data warehouses that cache data on local SSDs while improving resource elasticity.

Continuous cross-modal hashing

Generally, multimodal data with new classes arrive continuously in the real world. While advanced cross-modal hashing (CMH) focuses primarily on batch-based data with previously observed classes (ASCs), it disregards the effect of newly arriving classes (ANCs) on hash-code conflicts. In addition, class-level continuous hashing scenarios do not suit themselves well with the generic CMH configuration. To solve the aforementioned issues, we propose a novel framework, called CT-CMH, for the new task of continuous cross-modal hashing. For dealing with ANCs, CMH models require the ability of continuous learning, i.e. they can preserve the knowledge of previously observed data and, more crucially, they can be adapted to unseen data with ANCs. Specifically, we introduce the adaptive weight importance updating (AWIU) mechanism to alleviate the catastrophic forgetting problem of CMH and a new hash-code divergence (HCD) method to eliminate hash-code conflicts between ASCs and ANCs. When CT-CMH is equipped with both AWIU and HCD, it can consistently achieve high retrieval performance. The experiment results and visualization analyses validate the effectiveness of our approach. To the best of our knowledge, we are the first to introduce and implement the task of CCMH for ANCs.

An optimized deep supervised hashing model for fast image retrieval

As multimedia data grows exponentially, searching for and retrieving a relevant image is becoming a challenge for researchers. Hashing is a widely adopted method because of its high performance in image retrieval with deep neural networks and multiple convolutional layers. Even so, most hashing methods ignore the computational cost and memory storage consumption. When the deep hashing model size is large, it leads to a slowdown in the response time of the model compared to the small model. Addressing these issues, a novel optimized deep supervised hashing based on a teacher-student approach for swift and precise image retrieval is proposed in this paper. In this work, the small student model is trained using the knowledge distillation from the large teacher model and the information from the one-hot labels. Therefore, a weight allocation loss function based on the teacher and student models is defined. Meanwhile, we apply model pruning to decrease the amount of the student model further to increase the response time. Therefore, knowledge distillation is performed on the pruned model. After that, the remaining weights are quantized to reach the smaller size of the model. Extensive experimental outcomes on two widely used datasets prove the outstanding efficiency of our proposed method.

Dimensionality reduction to improve search time and memory footprint in content-retrieval tasks: Application to semiconductor inspection images

Quality control in semiconductors is a crucial step to produce high quality microchips. During the last years, advances in artificial vision have significantly improved image quality control techniques. In the semiconductor industry, automated visual inspection is fundamental to avoid human intervention and keep the pipeline sanitized. Different types of images are collected during this process, feeding image databases that continually grow and cannot be labelled by humans in an exhaustive manner. Advances in image retrieval search methods are fundamental to develop more efficient techniques that meet user requirements.In this work we propose a dimensionality reduction approach on the feature vectors computed by a classifying deep learning model, while keeping a high retrieval performance. To validate this technique, we evaluate four well-known reduction algorithms on a subset of the full database: Principal Component Analysis (PCA), Sparse Random Projection (SRP), Isomap, Locally Linear Embedding (LLE), in combination with three similarity metrics: Euclidian (L2), cosine and inner product. As the number of components of the vectors is reduced, the performance of the image retrieval is measured by recall, time to search, and memory footprint of the database.PCA offers the best results, allowing a significant reduction in search time and memory usage, while SRP becomes an option only when the cosine distance is used. With PCA, we were able to divide the memory footprint by a factor of 16, the search time by 6, while maintaining an average recall of 0.96.

DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrieval

In this paper, we address the problem of high performance and computationally efficient content-based video retrieval in large-scale datasets. Current methods typically propose either: (i) fine-grained approaches employing spatio-temporal representations and similarity calculations, achieving high performance at a high computational cost or (ii) coarse-grained approaches representing/indexing videos as global vectors, where the spatio-temporal structure is lost, providing low performance but also having low computational cost. In this work, we propose a Knowledge Distillation framework, called Distill-and-Select (DnS), that starting from a well-performing fine-grained Teacher Network learns: (a) Student Networks at different retrieval performance and computational efficiency trade-offs and (b) a Selector Network that at test time rapidly directs samples to the appropriate student to maintain both high retrieval performance and high computational efficiency. We train several students with different architectures and arrive at different trade-offs of performance and efficiency, i.e., speed and storage requirements, including fine-grained students that store/index videos using binary representations. Importantly, the proposed scheme allows Knowledge Distillation in large, unlabelled datasets—this leads to good students. We evaluate DnS on five public datasets on three different video retrieval tasks and demonstrate (a) that our students achieve state-of-the-art performance in several cases and (b) that the DnS framework provides an excellent trade-off between retrieval performance, computational speed, and storage space. In specific configurations, the proposed method achieves similar mAP with the teacher but is 20 times faster and requires 240 times less storage space. The collected dataset and implementation are publicly available: https://github.com/mever-team/distill-and-select.

Weighted top-k dominating queries on highly incomplete data

Top-k dominating (TKD) query retrieves the top k items that dominate other objects in the dataset. This is a key decision-making tool for any organization since it allows data analysts to discover dominant objects that can be used for recommendation. Incomplete data is a regular occurrence in real-world applications which occurs in many ways such as system failure, privacy protection, data loss, unavailability of data, and other issues. In this paper, we introduce a new approach for answering the top-k dominating queries over incomplete data. In many scenarios, the dominating object is one which has very high average rating but the number of rating is very low. We apply a weighted factor to calculate the score for dominating object. Hence realistic recommendation is possible. The idea of data bucketing is used to prune the non-candidate objects. The buckets are built using the B+ tree that makes the processing faster for high retrieval performance. In terms of top-k dominating query performance with incomplete data, the proposed model outperforms previous methods.

Content-Based Image Retrieval Using Angles Across Scales

This letter proposes a content-based image retrieval technique using novel dense angle descriptor and dictionary learning (DL). The histogram of oriented gradients (HOG) descriptor fails to obtain rotation invariance and well-defined rotation behavior, and therefore, a dense angle-based HOG descriptor has been presented to address the image rotation invariance. The technique computes angles across multiple scales and uses bag-of-visual features at different scales for DL. Experiments conducted on building and remote sensing datasets show that the proposed technique achieves high retrieval performance.

Transluminal intrahepatic bile duct stone removal using a novel spiral basket catheter via the endoscopic ultrasound-guided hepaticogastrostomy route (with video).

Recently, endoscopic ultrasound (EUS)-guided biliary drainage has been developed as an alternative biliary drainage technique for failed endoscopic retrograde cholangiopancreatography (ERCP) or inaccessible papilla. Among the various EUS-guided biliary drainage procedures, EUS-guided hepaticogastrostomy (HGS) can be performed in patients with surgically altered anatomy. More recently, various transluminal treatments have been described after EUS-HGS, such as antegrade stone removal. In patients with hepaticojejunostomy strictures, stone extraction into the intestine might be challenging even after performing hepaticojejunostomy stricture dilation using a balloon catheter. In such cases, transluminal stone removal is considered an alternative method. With transluminal stone removal, a small stone that escapes from the conventional basket or from a balloon catheter could migrate into the branch bile ducts. The novel spiral basket catheter available in Japan. The wires form a helix shape, wherein each wire is wound counterclockwise, and the winding pitch becomes gradually tighter from the proximal portion to the tip. As the winding pitch is smaller and the wires are in closer contact with the bile duct wall as compared with conventional basket catheter, small bile duct stones can be easily captured by simply pulling back the catheter while the basket is open. In addition, even during withdrawal inside the bile duct, a high retrieval performance is assured by the special design that maintains the opening width on the top end. Therefore, transluminal stone removal using this novel basket catheter might be clinically useful, although further prospective evaluation of a larger number of cases is needed.

Noise Resilient Local Gradient Orientation for Content-Based Image Retrieval

In a world of multimedia information, where users seek accurate results against search query and demand relevant multimedia content retrieval, developing an accurate content-based image retrieval (CBIR) system is difficult due to the presence of noise in the image. The performance of the CBIR system is impaired by this noise. To estimate the distance between the query and database images, CBIR systems use image feature representation. The noise or artifacts present within the visual data might confuse the CBIR when retrieving relevant results. Therefore, we propose Noise Resilient Local Gradient Orientation (NRLGO) feature representation that overcomes the noise factor within the visual information and strengthens the CBIR to retrieve accurate and relevant results. The proposed NRLGO consists of three steps: estimation and removal of noise to protect the local visual structure; extraction of color, texture, and local contrast features; and, at the end, generation of microstructure for visual representation. The Manhattan distance between the query image and the database image is used to measure their similarity. The proposed technique was tested using the Corel dataset, which contains 10000 images from 100 different categories. The outcomes of the experiment signify that the proposed NRLGO has higher retrieval performance in comparison with state-of-the-art techniques.

DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrieval

In this paper, we address the problem of high performance and computationally efficient content-based video retrieval in large-scale datasets. Current methods typically propose either: (i) fine-grained approaches employing spatio-temporal representations and similarity calculations, achieving high performance at a high computational cost or (ii) coarse-grained approaches representing/indexing videos as global vectors, where the spatio-temporal structure is lost, providing low performance but also having low computational cost. In this work, we propose a Knowledge Distillation framework, which we call Distill-and-Select (DnS), that starting from a well-performing fine-grained Teacher Network learns: a) Student Networks at different retrieval performance and computational efficiency trade-offs and b) a Selection Network that at test time rapidly directs samples to the appropriate student to maintain both high retrieval performance and high computational efficiency. We train several students with different architectures and arrive at different trade-offs of performance and efficiency, i.e., speed and storage requirements, including fine-grained students that store index videos using binary representations. Importantly, the proposed scheme allows Knowledge Distillation in large, unlabelled datasets -- this leads to good students. We evaluate DnS on five public datasets on three different video retrieval tasks and demonstrate a) that our students achieve state-of-the-art performance in several cases and b) that our DnS framework provides an excellent trade-off between retrieval performance, computational speed, and storage space. In specific configurations, our method achieves similar mAP with the teacher but is 20 times faster and requires 240 times less storage space. Our collected dataset and implementation are publicly available: this https URL.

RSIMS: Large-Scale Heterogeneous Remote Sensing Images Management System

With the remarkable development and progress of earth-observation techniques, remote sensing data keep growing rapidly and their volume has reached exabyte scale. However, it’s still a big challenge to manage and process such huge amounts of remote sensing data with complex and diverse structures. This paper designs and realizes a distributed storage system for large-scale remote sensing data storage, access, and retrieval, called RSIMS (remote sensing images management system), which is composed of three sub-modules: RSIAPI, RSIMeta, RSIData. Structured text metadata of different remote sensing images are all stored in RSIMeta based on a set of uniform models, and then indexed by the distributed multi-level Hilbert grids for high spatiotemporal retrieval performance. Unstructured binary image files are stored in RSIData, which provides large scalable storage capacity and efficient GDAL (Geospatial Data Abstraction Library) compatible I/O interfaces. Popular GIS software and tools (e.g., QGIS, ArcGIS, rasterio) can access data stored in RSIData directly. RSIAPI provides users a set of uniform interfaces for data access and retrieval, hiding the complex inner structures of RSIMS. The test results show that RSIMS can store and manage large amounts of remote sensing images from various sources with high and stable performance, and is easy to deploy and use.

Triplet-object loss for large scale deep image retrieval

Deep hashing has been widely applied in large scale image retrieval due to its high computation efficiency and retrieval performance. Recently, training deep hashing networks with a triplet ranking loss become a common framework. However, most of the triplet ranking loss based deep hashing methods cannot obtain satisfactory retrieval performance due to their ignoring the relative similarities among the objects. In this paper, we propose a method to learn the discriminative object features and utilize these features to compute the adaptive margins of the proposed loss for learning powerful hash codes. Experimental results show that our learned hash codes can yield state-of-the-art retrieval performance on three challenging datasets

A comparative analysis of system features used in the TREC-COVID information retrieval challenge

The COVID-19 pandemic has resulted in a rapidly growing quantity of scientific publications from journal articles, preprints, and other sources. The TREC-COVID Challenge was created to evaluate information retrieval (IR) methods and systems for this quickly expanding corpus. Using the COVID-19 Open Research Dataset (CORD-19), several dozen research teams participated in over 5 rounds of the TREC-COVID Challenge. While previous work has compared IR techniques used on other test collections, there are no studies that have analyzed the methods used by participants in the TREC-COVID Challenge. We manually reviewed team run reports from Rounds 2 and 5, extracted features from the documented methodologies, and used a univariate and multivariate regression-based analysis to identify features associated with higher retrieval performance. We observed that fine-tuning datasets with relevance judgments, MS-MARCO, and CORD-19 document vectors was associated with improved performance in Round 2 but not in Round 5. Though the relatively decreased heterogeneity of runs in Round 5 may explain the lack of significance in that round, fine-tuning has been found to improve search performance in previous challenge evaluations by improving a system’s ability to map relevant queries and phrases to documents. Furthermore, term expansion was associated with improvement in system performance, and the use of the narrative field in the TREC-COVID topics was associated with decreased system performance in both rounds. These findings emphasize the need for clear queries in search. While our study has some limitations in its generalizability and scope of techniques analyzed, we identified some IR techniques that may be useful in building search systems for COVID-19 using the TREC-COVID test collections.

Cosine metric supervised deep hashing with balanced similarity

Deep supervised hashing takes prominent advantages of low storage cost, high computational efficiency and good retrieval performance, which draws attention in the field of large-scale image retrieval. However, similarity-preserving, quantization errors and imbalanced data are still great challenges in deep supervised hashing. This paper proposes a pairwise similarity-preserving deep hashing scheme to handle the aforementioned problems in a unified framework, termed as Cosine Metric Supervised Deep Hashing with Balanced Similarity (BCMDH). BCMDH integrates contrastive cosine similarity and Cosine distance entropy quantization to preserve the original semantic distribution and reduce the quantization errors simultaneously. Furthermore, a weighted similarity measure with cosine metric entropy is designed to reduce the impact of imbalanced data, which adaptively assigns weights according to sample attributes (pos/neg and easy/hard) in the embedding process of similarity-preserving. The experimental results on four widely-used datasets demonstrate that the proposed method is capable of generating hash codes of high quality and improve large-scale image retrieval performance.

Relevance Feedback For Image Retrieval Using Transfer Learning and Improved MQHOA

Image retrieval is a challenging technology in multimedia applications where meeting the users’ subjective retrieval needs while achieving high retrieval performance is insufficient for existing methods. In this work, a related feedback image retrieval algorithm based on deep learning and optimization algorithm (CAMQHOA-RF) is proposed. Transfer learning based on the deep convolutional neural network is applied to extract deeper image features to reduce the semantic gap. The multi-scale quantum harmonic oscillator algorithm improved by the idea of “aggregation” is introduced to search the feature space effectively. The covariance matrix is used to strengthen the relationship between feature points at different scales to guide feature points to approach ideal query points faster. Moreover, the query point is reselected based on the feedback information to explore more potential users’ interest areas. Experiments have shown that compared with other algorithms, the proposed algorithm has fewer parameters that need to be set, but higher retrieval accuracy, faster retrieval speed, and stronger robustness are obtained, which can meet users better.

IOTA-Based Data Encryption Storage and Retrieval Method

At present, the traditional blockchain for data storage and retrieval reflects the characteristics of slow data uploading speed, high cost, and transparency, and there are a lot of corresponding problems, such as not supporting private data storage, large data operation costs, and not supporting Data field query. This paper proposes a method of data encryption storage and retrieval based on the IOTA distributed ledger, combined with the fast transaction processing speed and zero-value transactions of the IOTA blockchain, through the Masked Authenticated Messaging technology, so that the data is encrypted in the data stream. The form is stored in the distributed ledger, quickly retrieved through the field index mechanism established by the data form, and the data operation is carried out on the chain. Experimental results show that this system has high storage, encryption and retrieval performance, and good practicability.

ST-Trie: A Novel Indexing Scheme for Efficiently Querying Heterogeneous, Spatiotemporal IoT Data

Recently, various environmental data, such as microdust pollution, temperature, humidity, etc., have been continuously collected by widely deployed Internet of Things (IoT) sensors. Although these data can provide great insight into developing sustainable application services, it is challenging to rapidly retrieve such data, due to their multidimensional properties and huge growth in volume over time. Existing indexing methods for efficiently locating those data expose several problems, such as high administrative cost, spatial overhead, and slow retrieval performance. To mitigate these problems, we propose a novel indexing scheme termed ST-Trie, for efficient retrieval over spatiotemporal IoT environment data. Given IoT sensor data with latitude, longitude, and time, the proposed scheme first converts the three-dimensional attributes to one-dimensional index keys. The scheme then builds a trie-based index, consisting of internal nodes inserted by the converted keys and leaf nodes containing the keys and pointers to actual IoT data. We leverage this index to process various types of queries. In our experiments with three real-world datasets, we show that the proposed ST-Trie index outperforms existing approaches by a substantial margin regarding response time. Furthermore, we show that the query processing performance via ST-Trie also scales very well with an increasing time interval. Finally, we demonstrate that when compressed, the ST-Trie index can significantly reduce its space overhead by approximately a factor of seven.

Diffusion Parameters Analysis in a Content-Based Image Retrieval Task for Mobile Vision.

Most recent computer vision tasks take into account the distribution of image features to obtain more powerful models and better performance. One of the most commonly used techniques to this purpose is the diffusion algorithm, which fuses manifold data and k-Nearest Neighbors (kNN) graphs. In this paper, we describe how we optimized diffusion in an image retrieval task aimed at mobile vision applications, in order to obtain a good trade-off between computation load and performance. From a computational efficiency viewpoint, the high complexity of the exhaustive creation of a full kNN graph for a large database renders such a process unfeasible on mobile devices. From a retrieval performance viewpoint, the diffusion parameters are strongly task-dependent and affect significantly the algorithm performance. In the method we describe herein, we tackle the first issue by using approximate algorithms in building the kNN tree. The main contribution of this work is the optimization of diffusion parameters using a genetic algorithm (GA), which allows us to guarantee high retrieval performance in spite of such a simplification. The results we have obtained confirm that the global search for the optimal diffusion parameters performed by a genetic algorithm is equivalent to a massive analysis of the diffusion parameter space for which an exhaustive search would be totally unfeasible. We show that even a grid search could often be less efficient (and effective) than the GA, i.e., that the genetic algorithm most often produces better diffusion settings when equal computing resources are available to the two approaches. Our method has been tested on several publicly-available datasets: Oxford5k, Oxford5k, Paris6k, Paris6k, and Oxford105k, and compared to other mainstream approaches.

Neuromorphic Computing in Autoassociative Memory with a Regular Spiking Neuron Model

Digital Spiking Silicon Neuron (DSSN) model is a qualitative neuron model specifically designed for efficient digital circuit implementation which exhibits high biological plausibility. In this study we analyzed the behavior of an autoassociative memory composed of 3-variable DSSN model which has a slow negative feedback variable that models the effect of slow ionic currents responsible for Spike Frequency Adaptation (SFA). We observed the network dynamics by altering the strength of SFA which is known to be dependent on Acetylcholine volume, together with the magnitude of neuronal interaction. By altering these parameters, we obtained various pattern retrieval dynamics, such as chaotic transitions within stored patterns or stable and high retrieval performance. In the end, we discuss potential applications of the obtained results for neuromorphic computing.