Query Patch Research Articles

Unpaired Image-to-Image Translation Using Negative Learning for Noisy Patches

Unpaired image-to-image translation finds a mapping between two domains that do not have paired data. One approach is patchwise contrastive learning, a one-sided translation that maximizes mutual information between corresponding input and output patches. Noncorresponding patches are treated as negatives. Previous approaches randomly select noncorresponding patches, resulting in semantically similar patches incorrectly labeled as negatives. Inspired by negative learning, we propose the novel patchwise negative learning loss to address this issue. We do not naively minimize mutual information between all noncorresponding ones, unlike prior methods. Instead, we choose one noncorresponding patch and maximize dissimilarity with the query patch. The selected noncorresponding patch reduces the chance of choosing false negatives that contain high mutual information. By further maximizing dissimilarity with that single negative, we discourage our model from fitting on noisy negative patches. We demonstrate the capabilities of our model against other prominent image translation methods on the selfie2anime, horse2zebra, and cat2dog datasets.

Augmentative contrastive learning for one-shot object detection

We present an Augmentative Contrastive Learning for One-Shot Object Detection method that is inspired by the co-attention and co-excitation (CoAE) method. In the One-shot Object Detection (OSOD) task, the target image contains rich background information that influences the outcomes of different categories of objects more significantly. As a result, the network frequently suggests objects that do not fall under this category, leading to false detections. To learn the similarity between the target object and the query patch, we built the data augmentation module of object switch by similarity, encourage and restrain module. It primarily consists of the following three components: 1) To improve the contrast between the foreground and background, we propose an object switch module based on data augmentation and feature similarity; 2) To decrease the likelihood of false detections, we propose an encourage and restrain module that allows the network to encourage the recommendation of similar objects while restricting the recommendation of different categories of objects; and 3) To increase classification results, we use the nonlinear projector. We perform tests on the PASCAL VOC and MS COCO datasets to confirm the efficacy of our technique.

Unsupervised Pre-Training for Detection Transformers.

DEtection TRansformer (DETR) for object detection reaches competitive performance compared with Faster R-CNN via a transformer encoder-decoder architecture. However, trained with scratch transformers, DETR needs large-scale training data and an extreme long training schedule even on COCO dataset. Inspired by the great success of pre-training transformers in natural language processing, we propose a novel pretext task named random query patch detection in Unsupervised Pre-training DETR (UP-DETR). Specifically, we randomly crop patches from the given image and then feed them as queries to the decoder. The model is pre-trained to detect these query patches from the input image. During the pre-training, we address two critical issues: multi-task learning and multi-query localization. (1) To trade off classification and localization preferences in the pretext task, we find that freezing the CNN backbone is the prerequisite for the success of pre-training transformers. (2) To perform multi-query localization, we develop UP-DETR with multi-query patch detection with attention mask. Besides, UP-DETR also provides a unified perspective for fine-tuning object detection and one-shot detection tasks. In our experiments, UP-DETR significantly boosts the performance of DETR with faster convergence and higher average precision on object detection, one-shot detection and panoptic segmentation. Code and pre-training models: https://github.com/dddzg/up-detr.

Three-Dimensional Block Matching Using Orthonormal Tree-Structured Haar Transform for Multichannel Images.

Multichannel images, i.e., images of the same object or scene taken in different spectral bands or with different imaging modalities/settings, are common in many applications. For example, multispectral images contain several wavelength bands and hence, have richer information than color images. Multichannel magnetic resonance imaging and multichannel computed tomography images are common in medical imaging diagnostics, and multimodal images are also routinely used in art investigation. All the methods for grayscale images can be applied to multichannel images by processing each channel/band separately. However, it requires vast computational time, especially for the task of searching for overlapping patches similar to a given query patch. To address this problem, we propose a three-dimensional orthonormal tree-structured Haar transform (3D-OTSHT) targeting fast full search equivalent for three-dimensional block matching in multichannel images. The use of a three-dimensional integral image significantly saves time to obtain the 3D-OTSHT coefficients. We demonstrate superior performance of the proposed block matching.

Patch Matching for Image Denoising Using Neighborhood-Based Collaborative Filtering

We consider patch matching as a recommendation system problem and introduce a new patch-matching approach using nearest neighbor-based collaborative filtering (NN-CF). Our approach involves recommending similar patches to a query patch with the help of other similar patches in a noisy image or an external database. Using user-oriented and item-oriented formulations of NN-CF, we present two variations of CF-based patch-matching criterion. To demonstrate the superior matches found with our method, we apply the new patch-matching scheme to patch-based image denoising and evaluate its effect on the denoising performance. We test the methods on two data sets with varying background and image complexities and under different levels of noise. The proposed method not only improves robustness to patch matching but also provides a new formulation to seamlessly combine internal and external denoising.

Mixed Noise Removal via Robust Constrained Sparse Representation

In recent years, the sparse coding-based techniques have been widely used for image denoising. However, most of the sparse coding-based mixed noise reduction methods fail to take full advantage of the geometric structure of data samples. In other words, they neglect the common information shared by the similar patches in sparse coding. To address this concern, in this paper, we propose a robust constrained sparse representation (RCSR) method to remove mixed noise. By using the center coefficient of similar patches as the guider which is approximated by the coefficient of query patch in sparse coding, the geometric structure of data can be well preserved. Moreover, different from most existing two-stage mixed noise reduction methods that use explicit detectors to restrain impulse noise, the proposed RCSR adaptively adjusts the contribution of each pixel in the loss function to eliminate the influences of outliers. Experiments on the reconstruction of synthetic data and the removal of mixed noise in real images demonstrate the effectiveness of our proposed method.

Edge Propagation KD-Trees: Computing Approximate Nearest Neighbor Fields

Propagation-assisted kd-tree is a state-of-the-art method for computing approximate nearest neighbor (ANN) fields. In this method, each query patch needs descending search in the kd-tree and propagation search in the nearby patches. We observed that the query patches in the edge region need descending search, while other query patches only need propagation search. This can be an opportunity to save plenty of search time. In this letter, we propose edge propagation kd-trees to quickly compute ANNs. Our method can distinguish between edge patches and propagation patches in choosing the proper search. Experiments on public data set VidPairs show that our search method is 2-3 times faster than the propagation-assisted kd-tree search method at nearly the same accuracy.

Scalable analysis of Big pathology image data cohorts using efficient methods and high-performance computing strategies.

BackgroundWe describe a suite of tools and methods that form a core set of capabilities for researchers and clinical investigators to evaluate multiple analytical pipelines and quantify sensitivity and variability of the results while conducting large-scale studies in investigative pathology and oncology. The overarching objective of the current investigation is to address the challenges of large data sizes and high computational demands.ResultsThe proposed tools and methods take advantage of state-of-the-art parallel machines and efficient content-based image searching strategies. The content based image retrieval (CBIR) algorithms can quickly detect and retrieve image patches similar to a query patch using a hierarchical analysis approach. The analysis component based on high performance computing can carry out consensus clustering on 500,000 data points using a large shared memory system.ConclusionsOur work demonstrates efficient CBIR algorithms and high performance computing can be leveraged for efficient analysis of large microscopy images to meet the challenges of clinically salient applications in pathology. These technologies enable researchers and clinical investigators to make more effective use of the rich informational content contained within digitized microscopy specimens.

COLOR CHILD: a novel color image local descriptor for texture classification and segmentation

Designing a robust image local descriptor for the purpose of image segmentation, analysis, recognition and classification has been an active area of research to date. In this paper, a robust and computationally efficient image local descriptor named "COLOR CHILD" has been proposed. COLOR CHILD addresses the weaknesses of Weber Local Descriptor (WLD) by considering Laplacian of Gaussian (LoG) for its differential excitation component and Tiansi fractional derivative-based filter for its orientation component. For any given image, these two components are then used to construct a concatenated histogram and with the addition of color moments up to third order the capabilities of the proposed descriptor COLOR CHILD has been extended to handle textures in color space. COLOR CHILD is shown to outperform all of the known state-of-the-art image local descriptors of parametric and non-parametric types on a variety of benchmark texture databases such as KTH-TIPS2-a, KTH-TIPS2-b, and CUReT under varying degrees of noise while performing texture classification task. Further, the response profile of the COLOR CHILD in terms of Wasserstein distance measures (obtained by sliding a query patch across the image to be segmented) is found to be better suited as initial image for active contour-based image and texture segmentation algorithms. The efficacy of the COLOR CHILD for segmentation task is amply demonstrated on synthetic color images under varying degrees of noise and on real-world texture images.

Pan-sharpening based on geometric clustered neighbor embedding

Utilizing an implicit nonparametric learning framework, a neighbor-embedding-based method is proposed to solve the remote-sensing pan-sharpening problem. First, the original high-resolution (HR) and downsampled panchromatic (Pan) images are used to train the high/low-resolution (LR) patch pair dictionaries. Based on the perspective of locally linear embedding, patches in LR and HR images form manifolds with similar local intrinsic structure in the corresponding feature space. Every patch in each multispectral (MS) image band is modeled by its K nearest neighbors in the patch set generated from the LR Pan image, and this model can be generalized to the HR condition. Then, the desired HR MS patch is reconstructed from the corresponding neighbors in the HR Pan patch set. Finally, HR MS images are recovered by stitching these patches together. Recognizing that the K nearest neighbors should have local geometric structures similar to the input query patch based on clustering, we employ a dominant orientation algorithm to perform such clustering. The K nearest neighbors of each input LR MS patch are adaptively chosen from the associate subdictionary. Four datasets of images acquired by QuickBird and IKONOS satellites are used to test the performance of the proposed method. Experimental results show that the proposed method performs well in preserving spectral information as well as spatial details.

Parallel content-based sub-image retrieval using hierarchical searching

The capacity to systematically search through large image collections and ensembles and detect regions exhibiting similar morphological characteristics is central to pathology diagnosis. Unfortunately, the primary methods used to search digitized, whole-slide histopathology specimens are slow and prone to inter- and intra-observer variability. The central objective of this research was to design, develop, and evaluate a content-based image retrieval system to assist doctors for quick and reliable content-based comparative search of similar prostate image patches. Given a representative image patch (sub-image), the algorithm will return a ranked ensemble of image patches throughout the entire whole-slide histology section which exhibits the most similar morphologic characteristics. This is accomplished by first performing hierarchical searching based on a newly developed hierarchical annular histogram (HAH). The set of candidates is then further refined in the second stage of processing by computing a color histogram from eight equally divided segments within each square annular bin defined in the original HAH. A demand-driven master-worker parallelization approach is employed to speed up the searching procedure. Using this strategy, the query patch is broadcasted to all worker processes. Each worker process is dynamically assigned an image by the master process to search for and return a ranked list of similar patches in the image. The algorithm was tested using digitized hematoxylin and eosin (H&E) stained prostate cancer specimens. We have achieved an excellent image retrieval performance. The recall rate within the first 40 rank retrieved image patches is ∼90%. Both the testing data and source code can be downloaded from http://pleiad.umdnj.edu/CBII/Bioinformatics/.