Space Of Image Patches Research Articles

A topological loss function for image Denoising on a new BVI-lowlight dataset

Although image denoising algorithms have attracted significant research attention, surprisingly few have been proposed for, or evaluated on, noise from imagery acquired under real low-light conditions. Moreover, noise characteristics are often assumed to be spatially invariant, leading to edges and textures being distorted after denoising. Here, we introduce a novel topological loss function which is based on persistent homology. The method performs in the space of image patches, where topological invariants are calculated and represented in persistent diagrams. The loss function is a combination of ℓ1 or ℓ2 losses with the new persistence-based topological loss. We compare its performance across popular denoising architectures and loss functions, training the networks on our new comprehensive dataset of natural images captured in low-light conditions – BVI-LOWLIGHT. Analysis reveals that this approach outperforms existing methods, adapting well to complex structures and suppressing common artifacts.

Fast image interpolation via random forests.

This paper proposes a two-stage framework for fast image interpolation via random forests (FIRF). The proposed FIRF method gives high accuracy, as well as requires low computation. The underlying idea of this proposed work is to apply random forests to classify the natural image patch space into numerous subspaces and learn a linear regression model for each subspace to map the low-resolution image patch to high-resolution image patch. The FIRF framework consists of two stages. Stage 1 of the framework removes most of the ringing and aliasing artifacts in the initial bicubic interpolated image, while Stage 2 further refines the Stage 1 interpolated image. By varying the number of decision trees in the random forests and the number of stages applied, the proposed FIRF method can realize computationally scalable image interpolation. Extensive experimental results show that the proposed FIRF(3, 2) method achieves more than 0.3 dB improvement in peak signal-to-noise ratio over the state-of-the-art nonlocal autoregressive modeling (NARM) method. Moreover, the proposed FIRF(1, 1) obtains similar or better results as NARM while only takes its 0.3% computational time.

Multiatlas segmentation as nonparametric regression.

This paper proposes a novel theoretical framework to model and analyze the statistical characteristics of a wide range of segmentation methods that incorporate a database of label maps or atlases; such methods are termed as label fusion or multiatlas segmentation. We model these multiatlas segmentation problems as nonparametric regression problems in the high-dimensional space of image patches. We analyze the nonparametric estimator's convergence behavior that characterizes expected segmentation error as a function of the size of the multiatlas database. We show that this error has an analytic form involving several parameters that are fundamental to the specific segmentation problem (determined by the chosen anatomical structure, imaging modality, registration algorithm, and label-fusion algorithm). We describe how to estimate these parameters and show that several human anatomical structures exhibit the trends modeled analytically. We use these parameter estimates to optimize the regression estimator. We show that the expected error for large database sizes is well predicted by models learned on small databases. Thus, a few expert segmentations can help predict the database sizes required to keep the expected error below a specified tolerance level. Such cost-benefit analysis is crucial for deploying clinical multiatlas segmentation systems.

Efficient single image super-resolution via graph-constrained least squares regression

We explore in this paper an efficient algorithmic solution to single image super-resolution (SR). We propose the gCLSR, namely graph-Constrained Least Squares Regression, to super-resolve a high-resolution (HR) image from a single low-resolution (LR) observation. The basic idea of gCLSR is to learn a projection matrix mapping the LR image patch to the HR image patch space while preserving the intrinsic geometric structure of the original HR image patch manifold. Even if gCLSR resembles other manifold learning-based SR methods in preserving the local geometric structure of HR and LR image patch manifolds, the innovation of gCLSR lies in that it preserves the intrinsic geometric structure of the original HR image patch manifold rather than the LR image patch manifold, which may be contaminated by image degeneration (e.g., blurring, down-sampling and noise). Upon acquiring the projection matrix, the target HR image can be simply super-resolved from a single LR image without the need of HR-LR training pairs, which favors resource-limited applications. Experiments on images from the public database show that gCLSR method can achieve competitive quality as state-of-the-art methods, while gCLSR is much more efficient in computation than some state-of-the-art methods.

Coupled Dictionary Training for Image Super-Resolution

In this paper, we propose a novel coupled dictionary training method for single image super-resolution based on patchwise sparse recovery, where the learned couple dictionaries relate the low- and high-resolution image patch spaces via sparse representation. The learning process enforces that the sparse representation of a low-resolution image patch in terms of the low-resolution dictionary can well reconstruct its underlying high-resolution image patch with the dictionary in the highresolution image patch space. We model the learning problem as a bilevel optimization problem, where the optimization includes an 1-norm minimization problem in its constraints. Implicit differentiation is employed to calculate the desired gradient for stochastic gradient descent. We demonstrate that our coupled dictionary learning method can outperform the existing joint dictionary training method both quantitatively and qualitatively. Furthermore, for real applications, we speed up the algorithm approximately 10 times by learning a neural network model for fast sparse inference and selectively processing only those visually salient regions. Extensive experimental comparisons with stateof- the-art super-resolution algorithms validate the effectiveness of our proposed approach.

Representing and recognizing objects with massive local image patches

Natural image patches are fundamental elements for visual pattern modeling and recognition. By studying the intrinsic manifold structures in the space of image patches, this paper proposes an approach for representing and recognizing objects with a massive number of local image patches (e.g. 17x17 pixels). Given a large collection (>10^4) of proto image patches extracted from objects, we map them into two types of manifolds with different metrics: explicit manifolds of low dimensions for structural primitives, and implicit manifolds of high dimensions for stochastic textures. We define these manifolds grown from patches as the ''@e-balls'', where @e corresponds to the perception residual or fluctuation. Using these @e-balls as features, we present a novel generative learning algorithm by the information projection principle. This algorithm greedily stepwise pursues the object models by selecting sparse and independent @e-balls (say 10^3 for each category). During the detection and classification phase, only a small number (say 20) of features are activated by a fast KD-tree indexing technique. The proposed method owns two characters. (1) Automatically generating features (@e-balls) from local image patches rather than designing marginal feature carefully and category-specifically. (2) Unlike the weak classifiers in the boosting models, these selected @e-ball features are used to explain object in a generative way and are mutually independent. The advantage and performance of our approach is evaluated on several challenging datasets with the task of localizing objects against appearance variance, occlusion and background clutter.