Nonlinear Image Transformation Research Articles

Schlieren measurements of shock train flow fields in a supersonic cylindrical isolator at Mach 2

In a supersonic cylindrical isolator at Mach 2, the structures and frequency characteristics of shock train flow fields were experimentally studied by the schlieren measurement method. According to the design principle of parallel light through schlieren windows in a cylindrical duct, a high-precision conformal optical window pair was designed and integratively processed before. Based on a self-built pipeline structure with conformal windows in a direct-connect wind tunnel under adjustable back-pressure conditions, the shock surfaces in a cylindrical isolator at Mach 2 were first captured by the schlieren method. Then, the schlieren photographs were corrected by a nonlinear image transformation algorithm for the restoration of real shock train structures, and the experimental results were compared with numerical simulation results quantitatively. Finally, the shock train positions were calculated by an image recognition algorithm to analyze the self-excited oscillation frequency characteristics of shock train structures. The methods and experiments in this study enriched optical observation methods of supersonic flows through non-rectangular cross-section isolators in scramjet.Graphical abstract

Quantifying U-Net uncertainty in multi-parametric MRI-based glioma segmentation by spherical image projection.

Uncertainty quantification in deep learning is an important research topic. For medical image segmentation, the uncertainty measurements are usually reported as the likelihood that each pixel belongs to the predicted segmentation region. In potential clinical applications, the uncertainty result reflects the algorithm's robustness and supports the confidence and trust of the segmentation result when the ground-truth result is absent. For commonly studied deep learning models, novel methods for quantifying segmentation uncertainty are in demand. To develop a U-Net segmentation uncertainty quantification method based on spherical image projection of multi-parametric MRI (MP-MRI) in glioma segmentation. The projection of planar MRI data onto a spherical surface is equivalent to a nonlinear image transformation that retains global anatomical information. By incorporating this image transformation process in our proposed spherical projection-based U-Net (SPU-Net) segmentation model design, multiple independent segmentation predictions can be obtained from a single MRI. The final segmentation is the average of all available results, and the variation can be visualized as a pixel-wise uncertainty map. An uncertainty score was introduced to evaluate and compare the performance of uncertainty measurements. The proposed SPU-Net model was implemented on the basis of 369 glioma patients with MP-MRI scans (T1, T1-Ce, T2, and FLAIR). Three SPU-Net models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. The SPU-Net model was compared with (1) the classic U-Net model with test-time augmentation (TTA) and (2) linear scaling-based U-Net (LSU-Net) segmentation models in terms of both segmentation accuracy (Dice coefficient, sensitivity, specificity, and accuracy) and segmentation uncertainty (uncertainty map and uncertainty score). The developed SPU-Net model successfully achieved low uncertainty for correct segmentation predictions (e.g., tumor interior or healthy tissue interior) and high uncertainty for incorrect results (e.g., tumor boundaries). This model could allow the identification of missed tumor targets or segmentation errors in U-Net. Quantitatively, the SPU-Net model achieved the highest uncertainty scores for three segmentation targets (ET/TC/WT): 0.826/0.848/0.936, compared to 0.784/0.643/0.872 using the U-Net with TTA and 0.743/0.702/0.876 with the LSU-Net (scaling factor=2). The SPU-Net also achieved statistically significantly higher Dice coefficients, underscoring the improved segmentation accuracy. The SPU-Net model offers a powerful tool to quantify glioma segmentation uncertainty while improving segmentation accuracy. The proposed method can be generalized to other medical image-related deep-learning applications for uncertainty evaluation.

SIMULATION OF CELLS FOR SIGNALS INTENSITY TRANSFORMATION IN MIXED IMAGE PROCESSORS AND ACTIVATION FUNCTIONS OF NEURONS IN NEURAL NETWORKS

The paper considers results of design, simulation of continuously logical pixel cells (CLPC) based on current mirrors (CM) with functions of preliminary analogue processing for image intensity transformation and coding for construction of mixed image processors (IP) and neural networks (NN). The methodology and principles of construction of such cells are based on the use of piecewise-linear approximation of functions for nonlinear transformation of analog signals. It is shown that for the realization of generalized arbitrary functions by such gamma correctors, it is possible to apply basic step functions with controlled parameters. To implement the basic step functions, it is proposed to use nodes that perform a continuous-logical operation of a limited current difference and are quite simply implemented on current reflectors (VDS). The design and modeling of continuous-logical pixel cells (CLPC) based on VDS in different modes and for different conversion functions. Such CLC has a number of advantages: high speed and reliability, simplicity, small power consumption, high integration level for linear and matrix structures. We show design of CLC variants for photocurrents transformation and their simulations. The basic element of such cells is a scheme that implements the operation of a bounded difference of continuous logic. Using a set of circuits implemented on CMOS technology, we consider generalized methods for designing cells for nonlinear conversion of the photocurrent intensity. Selection of the appropriate parameters, which can be specified as constructive constants or as parameters for external control, allows changing type of synthesized functions. Possibilities of synthesis by such cells of functions with descending sections and different types are shown: sigmoid, lambda and others. Such CLPCs consist of several dozen CMOS transistors, have low power supply voltage (1.8 ÷ 3.3V), the range of an input photocurrent is 0.1÷24μA, the transformation time is less than 1 μs, low power consumption (microwatts). The circuits and the simulation results of their design with OrCAD are shown. Examples of nonlinear image transformations are given.

Preserving Low-Quality Video through Deep Learning

Lossy video stream compression is performed to reduce the bandwidth and storage requirements. Moreover also image compression is a need that arises in many circumstances.It is often the case that older archive are stored at low resolution and with a compression rate suitable for the technology available at the time the video was created. Unfortunately, lossy compression algorithms cause artifact. Such artifacts, usually damage higher frequency details also adding noise or novel image patterns. There are several issues with this phenomenon. Low-quality images can be less pleasant to persons. Object detectors algorithms may have their performance reduced. As a result, given a perturbed version of it, we aim at removing such artifacts to recover the original image. To obtain that, one should reverse the compression process through a complicated non-linear image transformation. We propose a deep neural network able to improve image quality. We show that this model can be optimized either traditionally, directly optimizing an image similarity loss (SSIM), or using a generative adversarial approach (GAN). Our restored images have more photorealistic details with respect to traditional image enhancement networks. Our training procedure based on sub-patches is novel. Moreover, we propose novel testing protocol to evaluate restored images quantitatively. Differently from previously proposed approaches we are able to remove artifacts generated at any quality by inferring the image quality directly from data. Human evaluation and quantitative experiments in object detection show that our GAN generates images with finer consistent details and these details make a difference both for machines and humans.

Deep Universal Generative Adversarial Compression Artifact Removal

Image compression is a need that arises in many circumstances. Unfortunately, whenever a lossy compression algorithm is used, artifacts will manifest. Image artifacts, caused by compression tend to eliminate higher frequency details and, in certain cases, may add noise or small image structures. There are two main drawbacks of this phenomenon. First, images appear much less pleasant to the human eye. Second, computer vision algorithms, such as object detectors, may be hindered and their performance reduced. Removing such artifacts means recovering the original image from a perturbed version of it. This means that one ideally should invert the compression process through a complicated nonlinear image transformation. We propose an image transformation approach based on a feedforward fully convolutional residual network model. We show that this model can be optimized either traditionally, directly optimizing an image similarity loss (SSIM), or using a generative adversarial approach (GAN). Our GAN is able to produce images with more photorealistic details than SSIM-based networks. We describe a novel training procedure based on subpatches and devise a novel testing protocol to evaluate restored images quantitatively. We show that our approach can be used as a preprocessing step for different computer vision tasks in case images are degraded by compression to a point that state-of-the art algorithms fail. In this case, our GAN-based approach obtains better performance than MSE or SSIM trained networks. Different from previously proposed approaches, we are able to remove artifacts generated at any QF by inferring the image quality directly from data.

Anatomical connectivity changes in bipolar disorder and schizophrenia investigated using whole-brain tract-based spatial statistics and machine learning approaches

Schizophrenia and bipolar disorder have similar clinical features. Their differential diagnosis is crucial because each has different prognostic and therapeutic characteristics. Earlier studies have used numerous methods, including magnetic resonance investigation, in an effort to differentiate these two disorders. Research has consistently shown that there is reduced white matter density in the fronto-temporal and fronto-thalamic pathways in both patients with bipolar disorder and schizophrenia; however, the sensitivity of the methods used is limited. Tract-based spatial statistics is a method of whole-brain analysis that relies on voxel-based comparison, and uses nonlinear image transformation and permutation tests with correction for multiple comparisons. The primary aim of the present study was to investigate anatomical connectivity changes in patients with bipolar disorder and schizophrenia using tract-based spatial statistics, to classify the patients according to white matter integrity patterns using machine learning, and to identify features that represent the key differences between the disorders. Whole-brain images of 41 bipolar disorder patients, 39 schizophrenia patients, and 23 controls were acquired using a 1.5 T magnetic resonance investigation scanner. As compared to the controls, the schizophrenia and bipolar disorder patients had reduced fractional anisotropy in similar white matter tracts. In addition, the imaging method employed differentiated the schizophrenia and bipolar disorder patients with 81.25% accuracy. Although the bipolar disorder and schizophrenia patients exhibited similar anatomical connectivity changes, as compared to the controls, the connectivity reductions in the right hemisphere in the bipolar disorder patients differentiated them from the schizophrenia patients. The present findings improve our understanding of the etiology and pathogenesis of bipolar disorder and schizophrenia, and can potentially be used as a biomarker for the diagnosis and treatment of both disorders.

Adaptive pseudo-p-norm regularization based De-speckling of SAR images

ABSTRACTA novel variational model is proposed to de-speckle the multiplication noise in the synthetic aperture radar (SAR) image to provide a high-quality interpretation of SAR data. The model consists of the data fidelity term and the adaptive pseudo-p-norm regularization term. The data fidelity term ensures the convexity of the model and avoids the nonlinear image transformation. With an improved ratio of exponentially weighted averages (ROEWA) operator and the edge detection image, the exponent value of the pseudo-p-norm regularization term varies with the intensity of the edge adaptively, which results in a better retention of the image geometric structure while removing the speckle noise. Compared with the total variation (TV) regularization, the staircase-like artifacts can be restrained more effectively. Experiments demonstrate that the proposed method are more competitive both visually and quantitatively than that from the existing nonlocal mean-based and variational-based methods.

Speckle reduction model for synthetic aperture radar images based on Beltrami regularization

A variational model is proposed to despeckle the multiplication noise in synthetic aperture radar (SAR) images to provide a high-quality interpretation of SAR data. The model consists of the data fidelity term and Beltrami regularization term. The former ensures the convexity of the model and avoids the nonlinear image transformation. The latter restrains the staircase-like artifacts caused by total variation regularization and effectively preserves the geometric structure of the image. The despeckled image is formulated as an optimization solution of the energy functional and the primal-dual algorithm is adopted to efficiently find the global minimum. Experiments on both synthetic and real SAR data demonstrate that, compared with five state-of-the-art methods, the proposed method achieves competitive results, both visually and quantitatively.

Robust feature matching via Gaussian field criterion for remote sensing image registration

Feature matching, which refers to establishing reliable feature correspondences between two images of the same scene, is a critical prerequisite in a wide range of remote sensing tasks including environment monitoring, multispectral image fusion, image mosaic, change detection, map updating. In this paper, we propose a method for robust feature matching and apply it to the problem of remote sensing image registration. We start by creating a set of putative feature matches which can contain a number of unknown false matches, and then focus on mismatch removal. This is formulated as a robust regression problem, and we customize a robust estimator, namely the Gaussian field criterion, to solve it. The robust criterion can handle both linear and nonlinear image transformations. In the linear case, we use a general homography to model the transformation, while in the nonlinear case, the non-rigid functions located in a reproducing kernel Hilbert space are considered, and a regularization term is added to the objective function to ensure its well-posedness. Moreover, we apply a sparse approximation to the non-rigid transformation and reduce the computational complexity from cubic to linear. Extensive experiments on various natural and remote sensing images show the effectiveness of our approach, which is able to yield superior results compared to other state-of-the-art methods.

Colors of the Sublunar.

Generic red, green, and blue images can be regarded as data sources of coarse (three bins) local spectra, typical data volumes are 104 to 107 spectra. Image data bases often yield hundreds or thousands of images, yielding data sources of 109 to 1010 spectra. There is usually no calibration, and there often are various nonlinear image transformations involved. However, we argue that sheer numbers make up for such ambiguity. We propose a model of spectral data mining that applies to the sublunar realm, spectra due to the scattering of daylight by objects from the generic terrestrial environment. The model involves colorimetry and ecological physics. Whereas the colorimetry is readily dealt with, one needs to handle the ecological physics with heuristic methods. The results suggest evolutionary causes of the human visual system. We also suggest effective methods to generate red, green, and blue color gamuts for various terrains.

Fisheye correction by CGA non‐linear transformation

We provide a non‐linear image transformation by means of conformal geometric algebra (CGA) elements and operations. We show the correspondence to the fisheye correction algorithms; more precisely, we prove the proportionality of classical models with CGA algorithm and provide exact formula in terms of CGA. Consequently, we show that the geometric construction allows to determine the inverse model quite universally.

An efficient approach for differentiating Alzheimer's disease from normal elderly based on multicenter MRI using gray-level invariant features.

Machine learning techniques, along with imaging markers extracted from structural magnetic resonance images, have been shown to increase the accuracy to differentiate patients with Alzheimer's disease (AD) from normal elderly controls. Several forms of anatomical features, such as cortical volume, shape, and thickness, have demonstrated discriminative capability. These approaches rely on accurate non-linear image transformation, which could invite several nuisance factors, such as dependency on transformation parameters and the degree of anatomical abnormality, and an unpredictable influence of residual registration errors. In this study, we tested a simple method to extract disease-related anatomical features, which is suitable for initial stratification of the heterogeneous patient populations often encountered in clinical data. The method employed gray-level invariant features, which were extracted from linearly transformed images, to characterize AD-specific anatomical features. The intensity information from a disease-specific spatial masking, which was linearly registered to each patient, was used to capture the anatomical features. We implemented a two-step feature selection for anatomic recognition. First, a statistic-based feature selection was implemented to extract AD-related anatomical features while excluding non-significant features. Then, seven knowledge-based ROIs were used to capture the local discriminative powers of selected voxels within areas that were sensitive to AD or mild cognitive impairment (MCI). The discriminative capability of the proposed feature was measured by its performance in differentiating AD or MCI from normal elderly controls (NC) using a support vector machine. The statistic-based feature selection, together with the knowledge-based masks, provided a promising solution for capturing anatomical features of the brain efficiently. For the analysis of clinical populations, which are inherently heterogeneous, this approach could stratify the large amount of data rapidly and could be combined with more detailed subsequent analyses based on non-linear transformation.

Leveraging polynomial approximation for non-linear image transformations in real time

Applying non-linear image transformations in real time remains a challenge for cost-sensitive embedded systems today. This paper offers a method to perform such transformations efficiently using polynomial approximations. It is based on a previously published method, but is not limited to transformations with little vertical variation, such as lens distortion removal. Instead, a wider range of image transformations is supported.Thus, a new, improved hardware architecture is presented and discussed. It can be configured to achieve virtually any required accuracy for practical transformation tasks at low hardware costs. As a proof of concept, the architecture has been integrated into a gesture recognition and gaze analysis system as an intelligent stereo camera delivering undistorted and rectified images.

Mutual Information-Based Visual Servoing

In this paper, we propose a new information theoretic approach to achieve visual servoing directly utilizing the information (as defined by Shannon) contained in the images. A metric derived from information theory, i.e., mutual information, is considered. Mutual information is widely used in multimodal image registration since it is insensitive to changes in the lighting condition and to a wide class of nonlinear image transformations. In this paper, mutual information is used as a new visual feature for visual servoing, which allows us to build a new control law that can control the six degrees of freedom (DOF) of a robot. Among various advantages, this approach requires no matching or tracking step, is robust to large illumination variations, and allows the consideration of different image modalities within the same task. Experiments on a real robot demonstrate the efficiency of the proposed visual-servoing approach.

Nonlinear signal processing techniques in ultrasonic tomography of concrete structures

Questions of using various techniques for processing echo signals in ultrasonic tomography instruments for extended complexly structured concrete articles are considered. Various algorithms are discussed for constructing tomograms of tested articles, techniques for radio engineering processing of ultrasonic signals, and different variants of nonlinear image transformations intended for increasing the signal-to-structural noise ratio and improving image quality.

Load distribution of articular cartilage from MR-images by neural nets

Artificial neural nets were used to determine the Young's modulus and spatial load distribution in articular cartilage by means of T2-weighted MR imaging. MR images were obtained in vitro (ex vivo?) from the joints of sheep of different ages (3 months, 9 months, 15 months, 1.5 years, 5 years, 5.5 years) and pigs (4 and 6 months) with a Bruker AMX 300 (7 T) spectrometer equipped with a micro-imaging unit. The knee of a 29-year-old male volunteer was studied in vivo under mechanical load using a clinical Siemens Vision MRT (1.5 T). The load of the cartilage is understood as a non-linear image transformation of loaded versus unloaded images. The artificial neural net was used to recognize given reference pixels of the unloaded cartilage within the image of the loaded cartilage. The Young's modulus was calculated from the local strain and the external pressure using the Hooke's law. With this method, the average Young's modulus was obtained in relationship to the biological age of the cartilage. The investigated age interval showed a progressive increase of 0.5 ± 0.3 MPa per year. These results are consistent with published results. As shown in this pilot study, the method of neural nets allows the visualization of the spatial load distribution within the articular cartilage.

Morphing diagnostic ultrasound images for perfusion assessment

An ultrasound image is morphed for perfusion assessment. Various images within a sequence of images or movie clip are mapped to one frame of reference using local warping. Nonlinear local image transformations or other warping based on local estimates of motion are used to interpolate different images to the common reference. The elastic stretching and compression of local image data results in a sequence of images where the same spatial location in each image represents the substantially same spatial location of the imaged tissue. The organ, tissue or region of interest is stationary throughout the sequence of images.

Self-dual morphological operators and filters

The median operator is a nonlinear image transformation celebrated for its noise cleaning capacities. It treats the foreground and background of an image identically, i.e., it is self-dual. Unfortunately, the median operator has one major drawback: it is not idempotent. Even worse, subsequent iterations of a given image may lead to oscillations. This paper describes a general method for the construction of morphological operators which are self-dual. This construction is based upon the concept of a switch operator. Subsequently, the paper treats a class of operators, the so-called activity-extensive operators, which have the intriguing property that every sequence of iterates of a given image is pointwise monotone and therefore convergent. The underlying concept is that of the activity ordering. Every increasing, self-dual operator can be modified in such a way that it becomes activity-extensive. The sequence of iterates of this modification converges to a self-dual morphological filter.

Nonlinear image transformations implemented with spatial light modulators

A technique for optically implementing nonlinear gray-level transformations on images via a spatial light modulator (SLM) coupled to a computer image-processing system is presented. Both the exponential and the logarithmic functions are implemented. The only restriction in the choice of nonlinear transformations is that they must lie below the SLM's linear intensity output versus intensity input curve obtained with the SLM set to a full transmittance.