Scale-space Properties Research Articles

Multi-Scale Self-Calibrated Dual-Attention Lightweight Residual Dense Deraining Network Based on Monogenic Wavelets

Image rain removal is an essential problem of common concern in the fields of image processing and computer vision. Existing methods have resorted to deep learning techniques to separate rain streaks from the background by leveraging some prior knowledge. However, the mismatch between the size of the rain streaks during the training and testing phases, especially when large rain streaks are present, frequently leads to unsatisfactory deraining results. To address this issue, we propose a multi-scale self-calibrated dual attention lightweight residual dense deraining network (MDARNet) for better deraining performance. Specifically, the network consists of monogenic wavelet transform-like hierarchy and self-calibrated dual attention mechanism. With the help of scale-space properties of the monogenic wavelet transform, key features at different scales can be extracted at the same location, which makes it easier to match structural features across scales. The self-calibrated double attention mechanism was used as a basic model for enhancing the channel dependence and spatial correlation between each layer component of the monogenic wavelet transform. Thus, the network can establish long-range dependencies and take advantage of rich contextual information and multi-scale redundancy to accommodate rain streaks of different shapes and sizes. Experiments on synthetic and real image datasets show that the method outperforms many of the latest single-image deraining methods in terms of visual and quantitative metrics. The source code can be obtained from https://github.com/smart-hzw/MDARNet..

Corner Detection via Scale-Space Behavior-Guided Trajectory Tracing

Existing <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">curvature scale-space</i> (CSS) methods detect corners by tracing the CSS trajectories from a determined high scale toward the lowest one. For those images with sophisticated details, such approach could often yield unsatisfactory corner detection results; i.e., miss-detected <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">true</i> corners (false negatives) and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">round</i> corners (false positives). In this letter, these two fundamental problems are investigated. To tackle them, a novel CSS-based corner detector is proposed by incorporating our mathematically derived scale-space properties of the planar curves and corner points into the developed trajectory tracing algorithm, called the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">scale-space behavior-guided trajectory tracing</i> (SBTT). In view of lacking a benchmark dataset with the ground truth, another contribution from our work is on the establishment of an augmented test image dataset, containing 147 test images with manually-labelled ground truth and their augmented images up to 62,328 images in total. Based on the ground truth, four commonly-used metrics are exploited to conduct corner detection performance evaluation. The obtained simulation results show that our proposed corner detector yields the highest F-score, when compared with that of nine state-of-the-art methods.

Unconstrained texture classification using efficient jet texton learning

This paper proposes a simple and effective texture recognition method that uses a new class of jet texton learning. In this approach, first a Jet space representation of the image is derived from a set of derivative of Gaussian (DtGs) filter responses upto 2nd order (R6), so called local jet vector (Ljv), which satisfies the scale space properties, where the combinations of local jets preserve the intrinsic local structure of the image in a hierarchical way and are invariant to image translation, rotation and scaling. Next, the jet textons dictionary is learned using K-means clustering algorithm from DtGs responses, followed by a contrast Weber law normalization pre-processing step. Finally, the feature distribution of jet texton is considered as a model which is utilized to classify texture using a non-parametric nearest regularized subspace (Nrs) classifier. Extensive experiments on three large and well-known benchmark database for texture classification like KTH-TIPS, Brodatz and CUReT show that the proposed method achieves state-of-the-art performance, especially when the number of available training samples is limited. The source code of complete system is made publicly available at https://github.com/swalpa/JetTexton.

Provably Scale-Covariant Continuous Hierarchical Networks Based on Scale-Normalized Differential Expressions Coupled in Cascade

This article presents a theory for constructing hierarchical networks in such a way that the networks are guaranteed to be provably scale covariant. We first present a general sufficiency argument for obtaining scale covariance, which holds for a wide class of networks defined from linear and nonlinear differential expressions expressed in terms of scale-normalized scale-space derivatives. Then, we present a more detailed development of one example of such a network constructed from a combination of mathematically derived models of receptive fields and biologically inspired computations. Based on a functional model of complex cells in terms of an oriented quasi quadrature combination of first- and second-order directional Gaussian derivatives, we couple such primitive computations in cascade over combinatorial expansions over image orientations. Scale-space properties of the computational primitives are analysed, and we give explicit proofs of how the resulting representation allows for scale and rotation covariance. A prototype application to texture analysis is developed, and it is demonstrated that a simplified mean-reduced representation of the resulting QuasiQuadNet leads to promising experimental results on three texture datasets.

Local jet pattern: a robust descriptor for texture classification

Methods based on locally encoded image features have recently become popular for texture classification tasks, particularly in the existence of large intra-class variation due to changes in illumination, scale, and viewpoint. Inspired by the theories of image structure analysis, this work proposes an efficient, simple, yet robust descriptor namely local jet pattern (Ljp) for texture classification. In this approach, a jet space representation of a texture image is computed from a set of derivatives of Gaussian (DtGs) filter responses up to second order, so-called local jet vectors (Ljv), which also satisfy the Scale Space properties. The Ljp is obtained by using the relation of center pixel with its’ local neighborhoods in jet space. Finally, the feature vector of a texture image is formed by concatenating the histogram of Ljp for all elements of Ljv. All DtGs responses up to second order together preserves the intrinsic local image structure, and achieves invariance to scale, rotation, and reflection. This allows us to design a discriminative and robust framework for texture classification. Extensive experiments on five standard texture image databases, employing nearest subspace classifier (Nsc), the proposed descriptor achieves 100%, 99.92%, 99.75%, 99.16%, and 99.65% accuracy for Outex_TC10, Outex_TC12, KTH-TIPS, Brodatz, CUReT, respectively, which are better compared to state-of-the-art methods.

Ultimate levelings

This work presents a new class of residual operators called ultimate levelings which are powerful image operators based on numerical residues. Within a multi-scale framework, these operators analyze a given image under a series of levelings. Thus, contrasted objects can be detected if a relevant residue is generated when they are filtered out by one of these levelings. Our approach consists of, firstly, (i) representing the input image as a morphological tree; then, (ii) showing that a certain operation on this tree results in a leveling operator; and finally (iii) demonstrating that a sequential application of this operation on the tree is able to produce a family of levelings that satisfies scale-space properties. Besides, other contributions of this paper include: (i) the statement of properties of ultimate levelings, (ii) the presentation of an efficient algorithm for their computation, (iii) the provision of strategies for choosing families of primitives, (iv) the presentation of strategies for filtering undesirable residues, and (v) the provision of some illustrative examples of application of ultimate levelings. Furthermore, ultimate levelings are computationally efficient and their performance evaluations are comparable to the state of art methods for filtering and image segmentation.

Cross-Diffusion Systems for Image Processing: I. The Linear Case

The use of cross-diffusion systems as mathematical models of different image processes is investigated. The present paper is concerned with linear filtering. First, those systems satisfying the most important scale-space properties are identified. Then a numerical study on the effect of cross-diffusion in filtering problems is presented.

Cross-Diffusion Systems for Image Processing: II. The Nonlinear Case

In this paper the use of nonlinear cross-diffu\-sion systems to model image restoration is investigated, theoretically and numerically. In the first case, well-posedness, scale-space properties and long time behaviour are analyzed. From a numerical point of view, a computational study of the performance of the models is carried out, suggesting their diversity and potentialities to treat image filtering problems. The present paper is a continuation of a previous work of the same authors, devoted to linear cross-diffusion models. \keywords{Cross-diffusion \and Complex diffusion \and Image restoration}

A SCALE-SPACE TOGGLE OPERATOR FOR IMAGE TRANSFORMATIONS

The analysis of different representation levels has been largely used in several image analysis tasks to handle the multiscale nature of image data, allowing the extraction of specific features that become explicit at each scale. In this work, we explore the scale-space properties of a self-dual toggle operator defined on a scaled morphological framework. These properties conduce to a well-controlled image simplification where its maxima and minima interact at the same time during pixels' transformation, in contrast to other approaches that consider these extrema separately. In such a way, it is possible to identify significant image extrema information to be used in several high level tasks. To assess the robustness of our approach, we carry out tests on images of several classes and subjected to different lighting conditions for various applications, including segmentation and binarization.

Semiautomatic White Blood Cell Segmentation Based on Multiscale Analysis

This paper approaches novel methods to segment the nucleus and cytoplasm of white blood cells (WBC). This information is the basis to perform higher level tasks such as automatic differential counting, which plays an important role in the diagnosis of different diseases. We explore the image simplification and contour regularization resulting from the application of the Self-Dual Multiscale Morphological Toggle (SMMT), an operator with scale-space properties. To segment the nucleus, the image preprocessing with SMMT has shown to be essential to ensure the accuracy of two well-known image segmentations techniques, namely, watershed transform and Level Set methods. To identify the cytoplasm region, we propose two different schemes, based on granulometric analysis and on morphological transformations. The proposed methods have been successfully applied to a large number of images, showing promising segmentation and classification results for varying cell appearance and image quality, encouraging future works.

Scale-Space Analysis of Discrete Filtering over Arbitrary Triangulated Surfaces

Discrete filtering of information over triangulated surfaces has proved very useful in computer graphics applications. This technique is based on diffusion equations and has been extensively applied to image processing, harmonic map regularization and texture generating, etc. [C. L. Bajaj and G. Xu, ACM Trans. Graph., 22 (2003), pp. 4-32], [C. Wu, J. Deng, and F. Chen, IEEE Trans. Vis. Comput. Graph., 14 (2008), pp. 666-679]. However, little has been done on analysis (especially quantitative analysis) of the behavior of these filtering procedures. Since in applications mesh surfaces can be of arbitrary topology and the filtering can be nonlinear and even anisotropic, the analysis of the quantitative behavior is a very difficult issue. In this paper, we first present the discrete linear, nonlinear, and anisotropic filtering schemes via discretizing diffusion equations with appropriately defined differential operators on triangulated surfaces, and then use concepts of discrete scale-spaces to describe these filtering procedures and analyze their properties respectively. Scale-space properties such as existence and uniqueness, continuous dependence on initial value, discrete semigroup property, grey level shift invariance and conservation of total grey level, information reduction (also known as topology simplification), and constant limit behavior have been proved. In particular, the information reduction property is analyzed by eigenvalue and eigenvector analysis of matrices. Different from the direct observation of the local filtering to the diffusion equations and other interpretation methods based on wholly global quantities such as energy and entropy, this viewpoint helps us understand the filtering both globally (information reduction as image components shrink) and locally (how the image component contributes to its shrink rate). With careful consideration of the correspondence between eigenvalues and eigenvectors and their features, differences between linear and nonlinear filtering, as well as between isotropic and anisotropic filtering, are discussed. We also get some stability results of the filtering schemes. Several examples are provided to illustrate the properties.

The Uncertainty Principle: Group Theoretic Approach, Possible Minimizers and Scale-Space Properties

The uncertainty principle is a fundamental concept in the context of signal and image processing, just as much as it has been in the framework of physics and more recently in harmonic analysis. Uncertainty principles can be derived by using a group theoretic approach. This approach yields also a formalism for finding functions which are the minimizers of the uncertainty principles. A general theorem which associates an uncertainty principle with a pair of self-adjoint operators is used in finding the minimizers of the uncertainty related to various groups. This study is concerned with the uncertainty principle in the context of the Weyl-Heisenberg, the SIM(2), the Affine and the Affine-Weyl-Heisenberg groups. We explore the relationship between the two-dimensional affine group and the SIM (2) group in terms of the uncertainty minimizers. The uncertainty principle is also extended to the Affine-Weyl-Heisenberg group in one dimension. Possible minimizers related to these groups are also presented and the scale-space properties of some of the minimizers are explored.

Scale-Space Properties of Nonstationary Iterative Regularization Methods

Most scale-space concepts have been expressed as parabolic or hyperbolic partial differential equations (PDEs). In this paper we extend our work on scale-space properties of elliptic PDEs arising from regularization methods: we study linear and nonlinear regularization methods that are applied iteratively and with different regularization parameters. For these so-called nonstationary iterative regularization techniques we clarify their relations to both isotropic diffusion filters with a scalar-valued diffusivity and anisotropic diffusion filters with a diffusion tensor. We establish scale-space properties for iterative regularization methods that are in complete accordance with those for diffusion filtering. In particular, we show that nonstationary iterative regularization satisfies a causality property in terms of a maximum–minimum principle, possesses a large class of Lyapunov functionals, and converges to a constant image as the regularization parameters tend to infinity. We also establish continuous dependence of the result with respect to the sequence of regularization parameters. Numerical experiments in two and three space dimensions are presented that illustrate the scale-space behavior of regularization methods.

Nonlinear Scale-Space Representation with Morphological Levelings

In this paper we present a nonlinear scale-space representation based on a general class of morphological strong filters, the levelings, which include the openings and closings by reconstruction. These filters are very useful for image simplification and segmentation. From one scale to the next, details vanish, but the contours of the remaining objects are preserved sharp and perfectly localized. Both the lattice algebraic and the scale-space properties of levelings are analyzed and illustrated. We also develop a nonlinear partial differential equation that models the generation of levelings as the limit of a controlled growth starting from an initial seed signal. Finally, we outline the use of levelings in improving the Gaussian scale-space by using the latter as an initial seed to generate multiscale levelings that have a superior preservation of image edges.

CORNER ENHANCEMENT IN CURVATURE SCALE SPACE

Corner detection is very important for pattern recognition, computer vision and other works. However, successfully extracting corners from a planar curve is difficult because we do not know how to select the appropriate scale for the extraction. In this paper, the problem of corner enhancement via the curve scale space is addressed. The main idea of the approach is to deform the original curve (or construct the curve scale space) and the corners on the deformed curve become distinguished from other structures which eases the process of corner detection. Started from the general geometric heat flow (GGHF), we study under what conditions the GGHF satisfies the scale space causality criteria. This is very important because many different evolution approaches, which have regular scale space properties and other specific properties, can then be found from these conditions. The criteria of corner enhancement are also proposed. Having all these constraints, a new curve evolution scheme which can enhance strong corners suppress noise and satisfies the scale space criteria is presented.

Nonlinear Multiscale Representations for Image Segmentation

In order to segment an image the use of information at multiple scales is invaluable. The hyperstack, a linking-model-based segmentation technique, uses intensity to link points in adjacent levels of a scale space stack. This approach has been successfully applied to linear multiscale representations. Multiscale representions which satisfy two scale space properties, viz. a causality criterion and a semigroup property in differential form, are valid inputs as well. In this paper we consider linear scale space, gradient-dependent diffusion, and the Euclidean shortening flow. Since no global scale parameter is available in the latter two approaches we compare scale levels based on evolution time, information theoretic measures, and by counting the number of objects. The multiscale representations are compared with respect to their performance in image segmentation tasks on test and MR images. The hyperstack proves to be rather insensitive to the underlying multiscale representation although the nonlinear representations reduced the number of post processing steps.

A Subdivision Scheme for Continuous-Scale B-Splinesand Affine-Invariant Progressive Smoothing

Multiscale representations and progressive smoothing constitute an important topic in different fields as computer vision, CAGD, and image processing. In this work, a multiscale representation of planar shapes is first described. The approach is based on computing classical B-splines of increasing orders, and therefore is automatically affine invariant. The resulting representation satisfies basic scale-space properties at least in a qualitative form, and is simple to implement. The representation obtained in this way is discrete in scale, since classical B-splines are functions in {\bf C}^{k-2}, where k is an integer bigger or equal than two. We present a subdivision scheme for the computation of B-splines of finite support at continuous scales. With this scheme, B-splines representations in {\bf C}^r are obtained for any real r in [0, \infty) , and the multiscale representation is extended to continuous scale. The proposed progressive smoothing receives a discrete set of points as initial shape, while the smoothed curves are represented by continuous (analytical) functions, allowing a straightforward computation of geometric characteristics of the shape.

Scale-space properties of quadratic feature detectors

We consider the scale-space properties of quadratic feature detectors and, in particular, investigate whether, like linear detectors, they permit a scale selection scheme with the property, which guarantees that features are never created as the scale is coarsened. We concentrate on the design of one dimensional detectors with two constituent filters, with the scale selection implemented as convolution and a scaling function. We consider two special cases of interest: the constituent filter pairs related by the Hilbert transform, and by the first spatial derivative. We show that, under reasonable assumptions, Hilbert-pair quadratic detectors cannot have the causality property. In the case of derivative-pair detectors, we describe a family of scaling functions related to fractional derivatives of the Gaussian that are necessary and sufficient for causality. In addition, we report experiments that show the effects of these properties in practice. We thus demonstrate that at least one class of quadratic feature detectors has the same desirable scaling property as the more familiar detectors based on linear filtering.

Area and length preserving geometric invariant scale-spaces

In this paper, area preserving multi-scale representations of planar curves are described. This allows smoothing without shrinkage at the same time preserving all the scale-space properties. The representations are obtained deforming the curve via geometric heat flows while simultaneously magnifying the plane by a homethety which keeps the enclosed area constant. When the Euclidean geometric heat flow is used, the resulting representation is Euclidean invariant, and similarly it is affine invariant when the affine one is used. The flows are geometrically intrinsic to the curve, and exactly satisfy all the basic requirements of scale-space representations. In the case of the Euclidean heat flow, it is completely local as well. The same approach is used to define length preserving geometric flows. A similarity (scale) invariant geometric heat flow is studied as well in this work. >

Discrete derivative approximations with scale-space properties: A basis for low-level feature extraction

This article shows how discrete derivative approximations can be defined so thatscale-space properties hold exactly also in the discrete domain. Starting from a set of natural requirements on the first processing stages of a visual system,the visual front end, it gives an axiomatic derivation of how a multiscale representation of derivative approximations can be constructed from a discrete signal, so that it possesses analgebraic structure similar to that possessed by the derivatives of the traditional scale-space representation in the continuous domain. A family of kernels is derived that constitutediscrete analogues to the continuous Gaussian derivatives. The representation has theoretical advantages over other discretizations of the scale-space theory in the sense that operators that commute before discretizationcommute after discretization. Some computational implications of this are that derivative approximations can be computeddirectly from smoothed data and that this will giveexactly the same result as convolution with the corresponding derivative approximation kernel. Moreover, a number ofnormalization conditions are automatically satisfied. The proposed methodology leads to a scheme of computations of multiscale low-level feature extraction that is conceptually very simple and consists of four basic steps: (i)large support convolution smoothing, (ii)small support difference computations, (iii)point operations for computing differential geometric entities, and (iv)nearest-neighbour operations for feature detection. Applications demonstrate how the proposed scheme can be used for edge detection and junction detection based on derivatives up to order three.