2DKD: a toolkit for content-based local image search

Daisuke Kihara,Julian S Deville,Atilla Sit

doi:10.1186/s13029-020-0077-1

Abstract

BackgroundDirect comparison of 2D images is computationally inefficient due to the need for translation, rotation, and scaling of the images to evaluate their similarity. In many biological applications, such as digital pathology and cryo-EM, often identifying specific local regions of images is of particular interest. Therefore, finding invariant descriptors that can efficiently retrieve local image patches or subimages becomes necessary.ResultsWe present a software package called Two-Dimensional Krawtchouk Descriptors that allows to perform local subimage search in 2D images. The new toolkit uses only a small number of invariant descriptors per image for efficient local image retrieval. This enables querying an image and comparing similar patterns locally across a potentially large database. We show that these descriptors appear to be useful for searching local patterns or small particles in images and demonstrate some test cases that can be helpful for both assembly software developers and their users.ConclusionsLocal image comparison and subimage search can prove cumbersome in both computational complexity and runtime, due to factors such as the rotation, scaling, and translation of the object in question. By using the 2DKD toolkit, relatively few descriptors are developed to describe a given image, and this can be achieved with minimal memory usage.

Highlights

Direct comparison of 2D images is computationally inefficient due to the need for translation, rotation, and scaling of the images to evaluate their similarity
The mathematical formulation of Moment-based approaches are very useful for repre- Two-dimensional Krawtchouk descriptors (2DKD) was already established in [1], which brings in senting biological and medical images as they are pix- the following advantages: 1) Krawtchouk polynomials are elized [1] or voxelized data [2,3,4]
For instance, redundancy is critical in their discriminative performance

Summary

Results

We present some experimental results and evaluate the discriminative power of 2DKD. Experiment II we test the local search performance of 2DKD on a more realistic problem, particle selection in 2D projection images of cryo-EM. From these images, individual particles need to be selected by hand or by automated algorithms. DbIndex, as in Experiment I to produce descriptors for all subimages in a 1024 × 1024 projection image (a section of which is shown in Fig. 5a) so a query can be compared with them. As justified by the figure, most of the retrievals from global image visually match the query except only three of them: the eleventh, thirteenth, and fourteenth In this experiment, we only search within one image, but the code can be adapted to handle a database with multiple projection images. Assuming that the descriptors were precomputed and stored, the search can be performed in real-time, which makes the software promising for larger datasets

Conclusions