Image Warping Research Articles

Image retargeting with a 3D saliency model

Image retargeting is an attractive topic since variety devices are with different resolutions. So far, there are two main strategies for this problem. One is removing or adding appropriate pixels in image directly, such as the seam carving approach, and the other is warping the mesh of image and then mapping the texture. However, existing approaches may encounter difficulties on different types of images since most of them only consider 2D features. In this paper, we propose an adaptive 3D saliency model based on the 3D structure of images to incorporate 3D information into both seam carving and mesh warping for image retargeting. Considering the characteristics of the two strategies, the 3D information are used in different ways. In seam caving, the adaptive 3D saliency is combined with L1 norm of gradient to generate an energy map for searching the least important seam. In warping, depth information is explored to improve the detection of important edges in mesh generation. Experimental results demonstrate the advantages of our method in both the seam carving and the image warping algorithms.

Model-free prediction of atmospheric warp based on artificial neural network.

This paper presents the application of artificial neural network for predicting the warping of images of remote objects or scenes ahead of time. The algorithm is based on estimating the pattern of warping of previously captured short-exposure frames through a generalized regression neural network (GRNN) and then predicting the warping of the upcoming frame. A high-accuracy optical flow technique is employed to estimate the dense motion fields of the captured frames, which are considered as training data for the GRNN. The proposed approach is independent of the pixel-oscillatory model unlike the state-of-the-art Kalman filter (KF) approach. Simulation experiments on synthetic and real-world turbulence degraded videos show that the proposed GRNN-based approach performs better than the KF approach in atmospheric warp prediction.

Automating Image Morphing Using Structural Similarity on a Halfway Domain

The main challenge in achieving good image morphs is to create a map that aligns corresponding image elements. Our aim is to help automate this often tedious task. We compute the map by optimizing the compatibility of corresponding warped image neighborhoods using an adaptation of structural similarity. The optimization is regularized by a thin-plate spline and may be guided by a few user-drawn points. We parameterize the map over a halfway domain and show that this representation offers many benefits. The map is able to treat the image pair symmetrically, model simple occlusions continuously, span partially overlapping images, and define extrapolated correspondences. Moreover, it enables direct evaluation of the morph in a pixel shader without mesh rasterization. We improve the morphs by optimizing quadratic motion paths and by seamlessly extending content beyond the image boundaries. We parallelize the algorithm on a GPU to achieve a responsive interface and demonstrate challenging morphs obtained with little effort.

LittleQuickWarp: An ultrafast image warping tool

Warping images into a standard coordinate space is critical for many image computing related tasks. However, for multi-dimensional and high-resolution images, an accurate warping operation itself is often very expensive in terms of computer memory and computational time. For high-throughput image analysis studies such as brain mapping projects, it is desirable to have high performance image warping tools that are compatible with common image analysis pipelines. In this article, we present LittleQuickWarp, a swift and memory efficient tool that boosts 3D image warping performance dramatically and at the same time has high warping quality similar to the widely used thin plate spline (TPS) warping. Compared to the TPS, LittleQuickWarp can improve the warping speed 2–5 times and reduce the memory consumption 6–20 times. We have implemented LittleQuickWarp as an Open Source plug-in program on top of the Vaa3D system (http://vaa3d.org). The source code and a brief tutorial can be found in the Vaa3D plugin source code repository.

An Automatic Method of Stereoscopic Video Synthesis from Monoscopic Video

This paper presents a novel depth-image-based rendering system for the generation of stereoscopic video from a monoscopic video sequence. The key point of the proposed system is that all the conversion procedures are accomplished automatically. First, we use Global Motion Estimation and an Adaptive Gaussian Mixture Model to segment the foreground and background. Second, different depth values are assigned to the foreground and background. Finally, image warping according to the depth map and stereoscopic video is synthesized. Experimental results show that the proposed method can effectively segment the foreground from the background and synthesize a satisfactory 3D effect, so the proposed approach is successful.

Simple and efficient approach for restoration of non-uniformly warped images.

A high accuracy image dewarping method is proposed to restore images from non-uniformly warped video sequences degraded by atmospheric turbulence. This approach contains three major steps. First, a non-rigid image registration technique is employed to register all the frames in the sequence to a reference frame and estimate the motion fields. Second, an iterative First Register Then Average And Subtract (iFRTAAS) method is applied to correct the geometric deformations of the warped frames. The third step involves applying a non-local means filter for the compensation of noise and to improve the signal-to-noise ratio (SNR) of the restored reference frame. Simulations are carried out by applying the method to synthetic and real-life turbulence degraded videos and by determining various quality metrics. A performance comparison is presented between the proposed method and two earlier methods, which verifies that the proposed method provides significant improvement on the image restoration accuracy.

An augmented-reality edge enhancement application for Google Glass.

Google Glass provides a platform that can be easily extended to include a vision enhancement tool. We have implemented an augmented vision system on Glass, which overlays enhanced edge information over the wearer's real-world view, to provide contrast-improved central vision to the Glass wearers. The enhanced central vision can be naturally integrated with scanning. Google Glass' camera lens distortions were corrected by using an image warping. Because the camera and virtual display are horizontally separated by 16 mm, and the camera aiming and virtual display projection angle are off by 10°, the warped camera image had to go through a series of three-dimensional transformations to minimize parallax errors before the final projection to the Glass' see-through virtual display. All image processes were implemented to achieve near real-time performance. The impacts of the contrast enhancements were measured for three normal-vision subjects, with and without a diffuser film to simulate vision loss. For all three subjects, significantly improved contrast sensitivity was achieved when the subjects used the edge enhancements with a diffuser film. The performance boost is limited by the Glass camera's performance. The authors assume that this accounts for why performance improvements were observed only with the diffuser filter condition (simulating low vision). Improvements were measured with simulated visual impairments. With the benefit of see-through augmented reality edge enhancement, natural visual scanning process is possible and suggests that the device may provide better visual function in a cosmetically and ergonomically attractive format for patients with macular degeneration.

A new method to measure higher visual functions in an immersive environment.

BackgroundHigher visual functions can be defined as cognitive processes responsible for object recognition, color and shape perception, and motion detection. People with impaired higher visual functions after unilateral brain lesion are often tested with paper pencil tests, but such tests do not assess the degree of interaction between the healthy brain hemisphere and the impaired one. Hence, visual functions are not tested separately in the contralesional and ipsilesional visual hemifields.MethodsA new measurement setup, that involves real-time comparisons of shape and size of objects, orientation of lines, speed and direction of moving patterns, in the right or left visual hemifield, has been developed. The setup was implemented in an immersive environment like a hemisphere to take into account the effects of peripheral and central vision, and eventual visual field losses. Due to the non-flat screen of the hemisphere, a distortion algorithm was needed to adapt the projected images to the surface. Several approaches were studied and, based on a comparison between projected images and original ones, the best one was used for the implementation of the test. Fifty-seven healthy volunteers were then tested in a pilot study. A Satisfaction Questionnaire was used to assess the usability of the new measurement setup.ResultsThe results of the distortion algorithm showed a structural similarity between the warped images and the original ones higher than 97%. The results of the pilot study showed an accuracy in comparing images in the two visual hemifields of 0.18 visual degrees and 0.19 visual degrees for size and shape discrimination, respectively, 2.56° for line orientation, 0.33 visual degrees/s for speed perception and 7.41° for recognition of motion direction. The outcome of the Satisfaction Questionnaire showed a high acceptance of the battery by the participants.ConclusionsA new method to measure higher visual functions in an immersive environment was presented. The study focused on the usability of the developed battery rather than the performance at the visual tasks. A battery of five subtasks to study the perception of size, shape, orientation, speed and motion direction was developed. The test setup is now ready to be tested in neurological patients.

Atrophy of the cholinergic basal forebrain in dementia with Lewy bodies and Alzheimer's disease dementia.

Similar to Alzheimer's disease (AD), dementia with Lewy bodies (DLB) is characterized by a profound degeneration of cortically-projecting cholinergic neurons of the basal forebrain (BF) and associated depletion of cortical cholinergic activity. We aimed to investigate subregional atrophy of the BF in DLB in vivo and compare it to the pattern of BF atrophy in AD. Structural MRI scans of 11 patients with DLB, 11 patients with Alzheimer's disease, and 22 healthy controls were analysed using a recently developed technique for automated BF morphometry based on high-dimensional image warping and cytoarchitectonic maps of BF cholinergic nuclei. For comparison, hippocampus volume was assessed within the same morphometric framework using recently published consensus criteria for the definition of hippocampus outlines on MRI. The DLB group demonstrated pronounced and subregion-specific atrophy of the BF which was comparable to BF atrophy in AD: volume of the nucleus basalis Meynert was significantly reduced by 20-25%, whereas rostral BF nuclei were only marginally affected. By contrast, hippocampus volume was markedly less affected in DLB compared to AD. Global cognition as determined by MMSE score was associated with BF volume in AD, but not in DLB, whereas visuoperceptual function as determined by the trail making test was associated with BF volume in DLB, but not in AD. DLB may be characterized by a more selective degeneration of the cholinergic BF compared to AD, which may be related to the differential cognitive profiles in both conditions.

Optimized Generation of Stereoscopic CGI Films by 3D Image Warping

AbstractThe generation of a stereoscopic animation film requires doubling the rendering times and hence the cost. In this paper, we address this problem and propose an automatic system for generating a stereo pair from a given image and its depth map. Although several solutions exist in the literature, the high standards of image quality required in the context of a professional animation studio forced us to develop specially crafted algorithms that avoid artefacts caused by occlusions, anti‐aliasing filters, etc. This paper describes all the algorithms involved in our system and provides their GPU implementation. The proposed system has been tested with real‐life working scenarios. Our experiments show that the second view of the stereoscopic pair can be computed with as little as 15% of the effort of the original image while guaranteeing a similar quality.

Stereo reconstruction using high-order likelihoods

Under the popular Markov random field (MRF) model, low-level vision problems are usually formulated by prior and likelihood models. In recent years, the priors have been formulated from high-order cliques and have demonstrated their robustness in many problems. However, the likelihoods have remained zeroth-order clique potentials. This zeroth-order clique assumption causes inaccurate solution and gives rise to undesirable fattening effect especially when window-based matching costs are employed. In this paper, we investigate high-order likelihood modeling for the stereo matching problem which advocates the dissimilarity measure between the whole reference image and the warped non-reference image. If the dissimilarity measure is evaluated between filtered stereo images, the matching cost can be modeled as high-order clique potentials. When linear filters and nonparametric census filter are used, it is shown that the high-order clique potentials can be reduced to pairwise energy functions. Consequently, a global optimization is possible by employing efficient graph cuts algorithm. Experimental results show that the proposed high-order likelihood models produce significantly better results than the conventional zeroth-order models qualitatively as well as quantitatively.

Using image warping for time-lapse image domain wavefield tomography

Time-lapse seismic data are widely used for monitoring subsurface changes. A quantitative assessment of how reservoir properties have changed allows for better interpretation of fluid substitution and fluid migration during processes such as oil and gas production and carbon sequestration. Full-waveform inversion (FWI) has been proposed as a way to retrieve quantitative estimates of subsurface properties through seismic waveform fitting. However, for some monitoring systems, the offset range versus depth of interest is not large enough to provide information about the low-wavenumber component of the velocity model. We evaluated an image domain wavefield tomography (IDWT) method using the local warping between baseline and monitor images as the cost function. This cost function is sensitive to volumetric velocity anomalies, and it is capable of handling large velocity changes with very limited acquisition apertures, where traditional FWI fails. We described the theory and workflow of our method. Layered model examples were used to investigate the performance of the algorithm and its robustness to velocity errors and acquisition geometry perturbations. The Marmousi model was used to simulate a realistic situation in which IDWT successfully recovers time-lapse velocity changes.

Object-Coherence Warping for Stereoscopic Image Retargeting

This paper addresses the topic of content-aware stereoscopic image retargeting. The key to this topic is consistently adapting a stereoscopic image to fit displays with various aspect ratios and sizes while preserving visually salient content. Most methods focus on preserving the disparities and shapes of visually salient objects through nonlinear image warping, in which distortions caused by warping are propagated to homogenous and low-significance regions. However, disregarding the consistency of object deformation sometimes results in apparent distortions in both the disparities and shapes of objects. An object-coherence warping scheme is proposed to reduce this unwanted distortion. The basic idea is to utilize the information of matched objects rather than that of matched pixels in warping. Such information implies object correspondences in a stereoscopic image pair, which allows the generation of an object significance map and the consistent preservation of objects. This strategy enables our method to consistently preserve both the disparities and shapes of visually salient objects, leading to good content-aware retargeting. In the experiments, qualitative and quantitative analyses of various stereoscopic images show that our results are better than those generated by related methods in terms of consistency of object preservation.

3-D from 2-D Using Warping Transformations .

Shown in this paper are methods on how to find the third dimension of a single image or from the two views of the image taking in a different angle using the method more accurate and faster to get to the third dimension in the following cases: One image of the same scene. Two views of the same scene from two different perspectives. Pictures of parts of the same scene. Set of pictures for different views of the work of the subject Panorama. This method is known Image Warping, which falls below a set of transfers such as (Affine - Bilinear - Projective - Mosaic – Similarity transformation) was compared to the work of transfers between the previous and this will be applied to more pictures. The idea is based on building code software is built on the programming language Visual C + + with the library for drawing an OpenGL program Matlab, which way can build a model of the following conversions, which fall under the so-called image warping of the conversion linear Bilinear Mapping and conversion Affine Mapping and conversion imagery Projective Mapping . shown in this paper are methods on how to correct camera exposure changes and how to blend the stitching line between the images. We will show panorama photos generated from both still image.

삼각형 반복분할에 의한 영상 보간법을 활용한 2D 얼굴 영상의 변형

ABSTRACT Image warping is a technology to transform input images to be suitable for given conditions and has been recently utilized in changing face shape of characters in the field of movies or animation. Mesh warping which is one of warping methods that change shapes based on the features of face forms warping images by forming rectangular mesh groups around the eyes, nose, and mouth and matching them 1:1. This method has a problem in the resultant images are distorted in the segments of boundaries between meshes when there are errors in mesh control points or when meshes have been formed as many small area meshes. This study proposes a triangle based image interpolation technique to minimize the occurrence of errors in the process of forming natural warping images of face and process accurate results with a small amount of arithmetic operation and a short time. First, feature points that represent the face are found and these points are connected to form basic triangle meshes. The fact that the proposed method can reduce errors occurring in the process of warping while reducing the amount of arithmetic operation and time is shown through experiments. Keywords : Facial image warping(얼굴 영상 워핑), Feature-based method(특징기반 방법), Triangle subdivision(삼각형 반복분할), Image interpolation(영상보간법), Mesh warping(메쉬워핑)

Linearized wave-equation migration velocity analysis by image warping

Seismic imaging produces images of contrasts in physical parameters in the subsurface, e.g., velocity or impedance. To build such images, a background model describing the wave kinematics in the earth is necessary. In practice, the structural image and background velocity model are unknown and have to be estimated from the acquired data. Migration velocity analysis deals with estimation of the background model in the framework of seismic migration and relies on two main elements: data redundancy and invariance of the structures with respect to different seismic experiments. Because all the experiments probe the same model, the reflectors must be invariant in suitable domains (e.g., shots or reflection angle); the semblance principle is the tool used to measure the invariance of a set of multiple images. We measure the similarity of the structural features between pairs of single-shot migrated images obtained from adjacent experiments. By using the estimated warping vector field between two migrated images, we construct an image perturbation which describes the difference in reflectivity observed by two shots. We derive an expression for the image perturbation that drives a migration velocity analysis procedure based on a linearization of the wave-equation with respect to the model parameters. Synthetic 2D examples show promising results in retrieving errors in the velocity model. This methodology can be directly applied to 3D.

Topology-Preserving Rigid Transformation of 2D Digital Images.

We provide conditions under which 2D digital images preserve their topological properties under rigid transformations. We consider the two most common digital topology models, namely dual adjacency and well-composedness. This paper leads to the proposal of optimal preprocessing strategies that ensure the topological invariance of images under arbitrary rigid transformations. These results and methods are proved to be valid for various kinds of images (binary, gray-level, label), thus providing generic and efficient tools, which can be used in particular in the context of image registration and warping.

EVALUATION OF THE WARP‐TURBO SPIN ECHO SEQUENCE FOR 3 TESLA MAGNETIC RESONANCE IMAGING OF STIFLE JOINTS IN DOGS WITH STAINLESS STEEL TIBIAL PLATEAU LEVELING OSTEOTOMY IMPLANTS

Susceptibility artifacts caused by ferromagnetic implants compromise magnetic resonance imaging (MRI) of the canine stifle after tibial plateau leveling osteotomy (TPLO) procedures. The WARP-turbo spin echo sequence is being developed to mitigate artifacts and utilizes slice encoding for metal artifact reduction. The aim of the current study was to evaluate the WARP-turbo spin echo sequence for imaging post TPLO canine stifle joints. Proton density weighted images of 19 canine cadaver limbs were made post TPLO using a 3 Tesla MRI scanner. Susceptibility artifact sizes were recorded and compared for WARP vs. conventional turbo spin echo sequences. Three evaluators graded depiction quality for the tibial tuberosity, medial and lateral menisci, tibial osteotomy, and caudal cruciate ligament as sufficient or insufficient to make a diagnosis. Artifacts were subjectively smaller and local structures were better depicted in WARP-turbo spin echo images. Signal void area was also reduced by 75% (sagittal) and 49% (dorsal) in WARP vs. conventional turbo spin echo images. Evaluators were significantly more likely to grade local anatomy depiction as adequate for making a diagnosis in WARP-turbo spin echo images in the sagittal but not dorsal plane. The proportion of image sets with anatomic structure depiction graded adequate to make a diagnosis ranged from 28 to 68% in sagittal WARP-turbo spin echo images compared to 0-19% in turbo spin echo images. Findings indicated that the WARP-turbo spin echo sequence reduces the severity of susceptibility artifacts in canine stifle joints post TPLO. However, variable depiction of local anatomy warrants further refinement of the technique.

Detection of Time-Varying Structures by Large Deformation Diffeomorphic Metric Mapping to Aid Reading of High-Resolution CT Images of the Lung

ObjectivesTo evaluate the accuracy of advanced non-linear registration of serial lung Computed Tomography (CT) images using Large Deformation Diffeomorphic Metric Mapping (LDDMM).MethodsFifteen cases of lung cancer with serial lung CT images (interval: 62.2±26.9 days) were used. After affine transformation, three dimensional, non-linear volume registration was conducted using LDDMM with or without cascading elasticity control. Registration accuracy was evaluated by measuring the displacement of landmarks placed on vessel bifurcations for each lung segment. Subtraction images and Jacobian color maps, calculated from the transformation matrix derived from image warping, were generated, which were used to evaluate time-course changes of the tumors.ResultsThe average displacement of landmarks was 0.02±0.16 mm and 0.12±0.60 mm for proximal and distal landmarks after LDDMM transformation with cascading elasticity control, which was significantly smaller than 3.11±2.47 mm and 3.99±3.05 mm, respectively, after affine transformation. Emerged or vanished nodules were visualized on subtraction images, and enlarging or shrinking nodules were displayed on Jacobian maps enabled by highly accurate registration of the nodules using LDDMM. However, some residual misalignments were observed, even with non-linear transformation when substantial changes existed between the image pairs.ConclusionsLDDMM provides accurate registration of serial lung CT images, and temporal subtraction images with Jacobian maps help radiologists to find changes in pulmonary nodules.

Pattern recognition methods and multivariate image analysis in HPTLC fingerprinting of propolis extracts

High‐performance thin‐layer chromatography (HPTLC) combined with image analysis and pattern recognition methods were used for fingerprinting and classification of 52 propolis samples collected from Serbia and one sample from Croatia. Modern thin‐layer chromatography equipment in combination with software for image processing and warping was applied for fingerprinting and data acquisition. The three mostly used chemometric techniques for classification, principal component analysis, cluster analysis and partial least square‐discriminant analysis, in combination with simple and fast HPTLC method for fingerprint analysis of propolis, were performed in order to favor and encourage their use in planar chromatography. HPTLC fingerprint analysis of propolis was for the first time performed on amino silica plates. All studied propolis samples have been classified in two major types, orange and blue, supporting the idea of existence of two types of European propolis. Signals at specific RF values responsible for classification of studied extracts have also been isolated and underlying compounds targeted for further investigation. Copyright © 2014 John Wiley & Sons, Ltd.