Content-based Similarity Search Research Articles

Diversity-Robust Acoustic Feature Signatures Based on Multiscale Fractal Dimension for Similarity Search of Environmental Sounds

This paper proposes new acoustic feature signatures based on the multiscale fractal dimension (MFD), which are robust against the diversity of environmental sounds, for the content-based similarity search. The diversity of sound sources and acoustic compositions is a typical feature of environmental sounds. Several acoustic features have been proposed for environmental sounds. Among them is the widely-used Mel-Frequency Cepstral Coefficients (MFCCs), which describes frequency-domain features. However, in addition to these features in the frequency domain, environmental sounds have other important features in the time domain with various time scales. In our previous paper, we proposed enhanced multiscale fractal dimension signature (EMFD) for environmental sounds. This paper extends EMFD by using the kernel density estimation method, which results in better performance of the similarity search tasks. Furthermore, it newly proposes another acoustic feature signature based on MFD, namely very-long-range multiscale fractal dimension signature (MFD-VL). The MFD-VL signature describes several features of the time-varying envelope for long periods of time. The MFD-VL signature has stability and robustness against background noise and small fluctuations in the parameters of sound sources, which are produced in field recordings. We discuss the effectiveness of these signatures in the similarity sound search by comparing with acoustic features proposed in the DCASE 2018 challenges. Due to the unique descriptiveness of our proposed signatures, we confirmed the signatures are effective when they are used with other acoustic features.

Liquid Silicon: A Nonvolatile Fully Programmable Processing-in-Memory Processor With Monolithically Integrated ReRAM

The slowdown of the CMOS technology scaling, and the trade-off between efficiency and flexibility have fueled the exploration into novel architectures with emerging post-CMOS technology [e.g., resistive-RAM (RRAM)]. In this article, a nonvolatile fully programmable processing-in-memory (PIM) processor named Liquid Silicon is demonstrated, which combines the superior programmability of general-purpose computing devices [e.g., field-programmable gate array (FPGA)] and the high efficiency of domain-specific accelerators. Besides the general computing applications, Liquid Silicon is particularly well suited for artificial intelligence (AI)/machine learning and big data applications, which not only poses high computational/memory demand but also evolves rapidly. To fabricate the Liquid Silicon chip, the HfO2 RRAM is monolithically integrated on top of the commercial 130 nm CMOS. Our measurement confirms that Liquid Silicon chip can operate reliably at a low voltage of 650 mV. It achieves 60.9 TOPS/W in performing neural network (NN) inferences, and 480 GOPS/W in performing content-based similarity search (a key big data application) at a nominal voltage supply of 1.2 V, showing $3\times $ and $100\times $ improvement over the state-of-the-art domain-specific CMOS-/RRAM-based accelerators. In addition, it outperforms the latest nonvolatile FPGA in energy efficiency by $3\times $ in general computing applications.

Query Expansion for Content-Based Similarity Search Using Local and Global Features

This article presents an efficient and totally unsupervised content-based similarity search method for multimedia data objects represented by high-dimensional feature vectors. The assumption is that the similarity measure is applicable to feature vectors of arbitrary length. During the offline process, different sets of features are selected by a generalized version of the Laplacian Score in an unsupervised way for individual data objects in the database. Online retrieval is performed by ranking the query object in the feature spaces of candidate objects. Those candidates for which the query object is ranked highly are selected as the query results. The ranking scheme is incorporated into an automated query expansion framework to further improve the semantic quality of the search result. Extensive experiments were conducted on several datasets to show the capability of the proposed method in boosting effectiveness without losing efficiency.

MDPV: metric distance permutation vocabulary

Sub-image content-based similarity search forms an important operation in current image archives since it provides users with images that contain a query image as their part. Such a search can conveniently be implemented using the bag-of-features model. Its integral part is a construction of visual vocabulary. Most existing algorithms to create a visual vocabulary suffer from high computational (e.g. k-means) or supervisor-guidance (e.g. visual-bit classifier, or sparse coding) requirements. In this paper, we propose a novel approach to visual vocabulary construction called metric distance permutation vocabulary. It is based on permutations of metric distances to create compact visual words. Its major advantage over prior techniques is time and space efficiency of vocabulary construction and quantization process during querying, while achieving comparable or even better effectiveness (query result quality). Moreover, this basic concept is extended to combine more independent permutations. Both the proposals are experimented on well-known real-world data-sets and compared to other state-of-the-art techniques.

Analyzing and dynamically indexing the query set

Most of the current metric indexes focus on indexing the collection of reference. In this work we study the problem of indexing the query set by exploiting some property that query objects may have. Thereafter, we present the Snake Table, which is an index structure designed for supporting streams of k-NN searches within a content-based similarity search framework. The index is created and updated in the online phase while resolving the queries, thus it does not need a preprocessing step. This index is intended to be used when the stream of query objects fits a snake distribution, that is, when the distance between two consecutive query objects is small. In particular, this kind of distribution is present in content-based video retrieval systems, image classification based on local descriptors, rotation-invariant shape matching, and others. We show that the Snake Table improves the efficiency of k-NN searches in these systems, avoiding the building of a static index in the offline phase.

Similarity caching in large-scale image retrieval

Feature-rich data, such as audio-video recordings, digital images, and results of scientific experiments, nowadays constitute the largest fraction of the massive data sets produced daily in the e-society. Content-based similarity search systems working on such data collections are rapidly growing in importance. Unfortunately, similarity search is in general very expensive and hardly scalable.In this paper we study the case of content-based image retrieval (CBIR) systems, and focus on the problem of increasing the throughput of a large-scale CBIR system that indexes a very large collection of digital images. By analyzing the query log of a real CBIR system available on the Web, we characterize the behavior of users who experience a novel search paradigm, where content-based similarity queries and text-based ones can easily be interleaved. We show that locality and self-similarity is present even in the stream of queries submitted to such a CBIR system. According to these results, we propose an effective way to exploit this locality, by means of a similarity caching system, which stores the results of recently/frequently submitted queries and associated results. Unlike traditional caching, the proposed cache can manage not only exact hits, but also approximate ones that are solved by similarity with respect to the result sets of past queries present in the cache. We evaluate extensively the proposed solution by using the real query stream recorded in the log and a collection of 100 millions of digital photographs. The high hit ratios and small average approximation error figures obtained demonstrate the effectiveness of the approach.

Investigating Music Collections at Different Scales with AudioDB

Content-based search of music collections presents differing challenges at different scales and according to the task at hand. In this paper, we consider a number of different use cases for content-based similarity search, at scales ranging between a detailed investigation of a single track to searching for fragments of a track against a collection of millions of media items. We pay particular attention to the varying tradeoff between precision and recall in these contexts, both from the point of view of system evaluation and from the point of view of a user of a system searching an unknown collection. We present the audioDB software for content-based search, and describe how it has been used to address use cases across these different collection sizes; in addition we show that the interpretation of similarity as a distance which can be modelled statistically, initially motivated by our desire to achieve sublinear retrieval time on large databases, can be used to improve the precision of searches over small and medium-sized collections.

KpyrRec: a recursive multidimensional indexing structure

The emergence of digital technologies in the audio–visual sector requires the use of powerful tools for fast data access. We focus on high multidimensional indexing structures for content-based similarity search in very large audiovisual databases of companies specialized in business films. This paper summarizes our study on analytical and experimental results for two new indexing structures we propose. These structures are integrated and tested in a search tool intended for professional use in large company film databases.

Ferret

Building content-based search tools for feature-rich data has been a challenging problem because feature-rich data such as audio recordings, digital images, and sensor data are inherently noisy and high dimensional. Comparing noisy data requires comparisons based on similarity instead of exact matches, and thus searching for noisy data requires similarity search instead of exact search.The Ferret toolkit is designed to help system builders quickly construct content-based similarity search systems for feature-rich data types. The key component of the toolkit is a content-based similarity search engine for generic, multi-feature object representations. To solve the similarity search problem in high-dimensional spaces, we have developed approximation methods inspired by recent theoretical results on dimension reduction. The search engine constructs sketches from feature vectors as highly compact data structures for matching, filtering and ranking data objects. The toolkit also includes several other components to help system builders address search system infrastructure issues. We have implemented the toolkit and used it to successfully construct content-based similarity search systems for four data types: audio recordings, digital photos, 3D shape models and genomic microarray data.

An experimental effectiveness comparison of methods for 3D similarity search

Methods for content-based similarity search are fundamental for managing large multimedia repositories, as they make it possible to conduct queries for similar content, and to organize the repositories into classes of similar objects. 3D objects are an important type of multimedia data with many promising application possibilities. Defining the aspects that constitute the similarity among 3D objects, and designing algorithms that implement such similarity definitions is a difficult problem. Over the last few years, a strong interest in 3D similarity search has arisen, and a growing number of competing algorithms for the retrieval of 3D objects have been proposed. The contributions of this paper are to survey a body of recently proposed methods for 3D similarity search, to organize them along a descriptor extraction process model, and to present an extensive experimental effectiveness and efficiency evaluation of these methods, using several 3D databases.

Cluster of re-configurable nodes for scanning large genomic banks

Genomic data are growing exponentially and are daily scanned by thousands of biologists. To reduce the scan time, efficient parallelism can be exploited by dispatching data among a cluster of processing units able to scan locally and independently their own data. If PC clusters are well suited to support this type of parallelism, we propose to substitute PCs by re-configurable hardware closely connected to a hard disk. We show that low cost FPGA nodes interconnected through a standard Ethernet network may advantageously compete against high performance clusters. A prototype of 48 re-configurable processing nodes has been experimented on two genomic applications: a content-based similarity search and a pattern search.

I 2Cnet : Content-based similarity search in geographically distributed repositories of medical images

The retrieval of images by content is rapidly gaining acceptance as an important function of image database systems. This paper discusses the architecture of I 2Cnet , a network of servers which provide content-based query services through a WWW browser. In I 2Cnet , algorithms for the representation, storage, and retrieval of medical images based on different descriptions of image content are implemented using description types. AttributeMatch, a description type supported by I 2Cnet , aims to capture the knowledge of medical experts in queries by using a similarity criterion which can be tailored to user preferences. We present results showing the query response time of AttributeMatch, obtained with image classes of various sizes, and the degree of similarity of retrieved images to the query image under different similarity criteria.