Warping Model Research Articles

Image stitching by line-guided local warping with global similarity constraint

Low-textured image stitching remains a challenging problem. It is difficult to achieve good alignment of images and it is easy to break images structures are often broken due to insufficient and unreliable point correspondences. Moreover, because of the viewpoint variations between multiple images, the stitched images suffer from projective distortions. To solve these problems, this paper presents a line-guided local warping method with a global similarity constraint for image stitching. Line features which serve well for geometric descriptions and scene constraints, are employed to guide image stitching accurately. On one hand, the line features are integrated into a local warping model through a designed weight function. On the other hand, line features are adopted to impose strong geometric constraints, including line correspondence and line colinearity, to improve the stitching performance through mesh optimization. To mitigate projective distortions, we adopt a global similarity constraint, which is integrated with the projective warps via a designed weight strategy. This constraint causes the final warp to slowly change from a projective to a similarity transformation across the image. Finally, the images undergo a two-stage alignment scheme that provides accurate alignment and reduces projective distortion. We evaluate our method on a series of images and compare it with several other methods. The experimental results demonstrate that the proposed method provides a convincing stitching performance and that it outperforms other state-of-the-art methods.

An efficient intensity-based ready-to-use X-ray image stitcher.

The limited field of view of the X-ray image intensifier makes it difficult to cover a large target area with a single X-ray image. X-ray image stitching techniques have been proposed to produce a panoramic X-ray image. This paper presents an efficient intensity-based X-ray image stitcher, which does not rely on accurate C-arm motion control or auxiliary devices and hence is ready to use in clinic. The stitcher consumes sequentially captured X-ray images with overlap areas and automatically produces a panoramic image. The gradient information for optimization of image alignment is obtained using a back-propagation scheme so that it is convenient to adopt various image warping models. The proposed stitcher has the following advantages over existing methods: (1)no additional hardware modification or auxiliary markers are needed; (2)it is robust against feature-based approaches; (3)arbitrary warping models and shapes of the region of interest are supported; (4)seamless stitching is achieved using multi-band blending. Experiments have been performed to confirm the effectiveness of the proposed method. The proposed X-ray image stitcher is efficient, accurate and ready to use in clinic.

Improvement of time alignment of the speech signals to be used in voice conversion

One of the main applications of time alignment is parallel corpus based voice conversion. In the literature, various methods such as dynamic time warping (DTW) and hidden Markov model have been suggested for time alignment of two speech signals. In this paper, we introduce some modifications to DTW in order to decrease the time alignment error. These modifications are refinement, which is done by exerting a threshold, normalization, and comparisons between the preceding and the following frames to make sound correspondence between two different parallel corpus-based speakers’ speeches. Evaluation of this approach which has been done on some corpus sentences indicates a significant improvement of time alignment. At least about 4% and in some cases 15% decrease of error in comparison with DTW has been achieved.

Stitching for Multi-View Videos With Large Parallax Based on Adaptive Pixel Warping

Conventional stitching techniques for images and videos are based on smooth warping models, and therefore, they often fail to work on multi-view images and videos with large parallax captured by cameras with wide baselines. In this paper, we propose a novel video stitching algorithm for such challenging multi-view videos. We estimate the parameters of ground plane homography, fundamental matrix, and vertical vanishing points reliably, using both of the appearance and activity-based feature matches validated by geometric constraints. We alleviate the parallax artifacts in stitching by adaptively warping the off-plane pixels into geometrically accurate matching positions through their ground plane pixels based on the epipolar geometry. We also exploit the inter-view and inter-frame correspondence matching information together to estimate the ground plane pixels reliably, which are then refined by energy minimization. Experimental results show that the proposed algorithm provides geometrically accurate stitching results of multi-view videos with large parallax and outperforms the state-of-the-art stitching methods qualitatively and quantitatively.

Whispered speech recognition based on gammatone filterbank cepstral coefficients

This paper presents the results on whispered speech recognition using gammatone filterbank cepstral coefficients for speaker dependent mode. The isolated words used for this experiment are taken from the Whi-Spe database. Whispered speech recognition is based on dynamic time warping and hidden Markov models methods. The experiments are focused on the following modes: normal speech, whispered speech and their combinations (normal/whispered and whispered/normal). The results demonstrated an important improvement in recognition after application of cepstral mean subtraction, especially in mixed train/test scenarios.

Design and analysis of an industrial steel building. Limit states, stability check

This work is focused on the design of a structural steel industrial building of double-T type section located in an industrial area of Valladolid (Spain). Firstly, a linear analysis is carried out to dimension the structure and check that it resists according to the established regulations, the Technical Building Code (CTE), the national adaptation of the Eurocodes (ECs). For the analysis of the structure, the 3D model of slender bar bending (Navier-Bernoulli bending theory) and the non-uniform twist or constrained warping model (Vlasov’s torsion theory) are considered. In the second place, a nonlinear analysis is performed with the objective of verifying the Ultimate Limit State of stability of the structural elements and of the industrial building as a whole. All this without the need to estimate buckling coefficients of each of the beams that make up the structure, and without the need for important simplifications. Finally, a linear analysis is once more performed to check the Service Limit State for the frequent combination of actions. The solution of the problem results from an iterative process of searching for the best solution, that is, the one that, with the least amount of steel used, fulfills all the requirements: resistance, stability and displacement limits established by the current regulation.

Stereoscopic Image Stitching Based on a Hybrid Warping Model

Traditional image editing techniques cannot be directly used to process stereoscopic media, as extra constraints are required to ensure consistent changes between the left and right images. In this paper, we propose a hybrid warping model for stereoscopic image stitching by combining projective and content-preserving warping. First, a uniform homography algorithm is proposed to prewarp the left and right images, and thus ensure consistent changes. Second, a content-preserving warping is introduced to locally refine alignment and reduce vertical disparities. Finally, a seam-cutting-based algorithm is used to find a blending seam, and the multiband blending algorithm is used to produce the final stitched image. Experimental results show that the proposed method can effectively stitch stereoscopic images, which not only avoids local distortions, but also reduces vertical disparities reasonably.

An efficient warping model for elastoplastic torsional analysis of composite beams

In this paper, we present an efficient warping model for nonlinear elastoplastic torsional analysis of composite beams developed based on Benscoter warping theory. A major challenge here is how to account for the evolution of warping functions efficiently as materials yield at various locations at different rates. Here, we propose to describe the warping displacement using a linear combination of two asymptotic warping functions with corresponding warping degrees of freedom. The asymptotic warping functions are calculated only once initially by solving the extended St. Venant equations under two material conditions: purely elastic condition and fully plastic condition when no material point in the cross-section remains elastic. Only the warping degrees of freedom are updated incrementally and iteratively in analysis without evaluating the warping functions again. The proposed warping model demonstrates an excellent performance in several numerical examples despite its simplicity.

Region division of China's carbon market based on the provincial/municipal carbon intensity

The difference of carbon trading prices among Emission Trading Scheme pilots may hinder the unified carbon market that China's government plans to establish in 2017. To achieve this goal, this study develops a regional carbon market mode based on the similarity of thirty provinces/municipalities' carbon intensity by the aid of Logarithmic Mean Divisia Index, Autoregressive, and Dynamic Time Warping models. According to the dynamic clustering result, we divide these provinces/municipalities into three sub-markets that are the Western, Eastern, and Central Markets. Furthermore, we propose the carbon emission modes corresponding to each sub-market, namely, the energy-dependence mode (Western Market), sustainable-economy mode (Eastern Market), and industrial-upgrading mode (Central Market). Briefly, we suggest that the unified market should be firstly divided into some sub-markets. Finally, we propose some critical routes as well as corresponding managerial implications for establishing such unified market.

Evaluation of the Warping Model for Analysis of Polystyrene Concrete Slabs with Profiled Steel Sheeting

In this article the possibility of applying the warping model offered by V. G. Piskunov, A. V. Goryk and V. N. Cherednikov for analyzing polystyrene concrete slabs with profiled steel sheeting is considered. The adequacy of the offered model was verified by comparing the experimental and calculated deflections of the slab. Test specimens of polystyrene concrete slabs with profiled steel sheeting (PCSPSS) were made for experimental studies and then mechanical properties of the slab components – the polystyrene concrete and the profiled steel sheeting – were determined. Then the behaviour of slabs under a load was studied. The calculation of the warping model of PCSPSS was done at the same time. The obtained results have proved the possibility of applying of the offered warping model for analysis of PCSPSS. It is proposed to use the considered model for further study of the stress-strained state of inhomogeneous slabs.

Person re-identification by unsupervised video matching

Most existing person re-identification (ReID) methods rely only on the spatial appearance information from either one or multiple person images, whilst ignore the space-time cues readily available in video or image-sequence data. Moreover, they often assume the availability of exhaustively labelled cross-view pairwise data for every camera pair, making them non-scalable to ReID applications in real-world large scale camera networks. In this work, we introduce a novel video based person ReID method capable of accurately matching people across views from arbitrary unaligned image-sequences without any labelled pairwise data. Specifically, we introduce a new space-time person representation by encoding multiple granularities of spatio-temporal dynamics in form of time series. Moreover, a Time Shift Dynamic Time Warping (TS-DTW) model is derived for performing automatically alignment whilst achieving data selection and matching between inherently inaccurate and incomplete sequences in a unified way. We further extend the TS-DTW model for accommodating multiple feature-sequences of an image-sequence in order to fuse information from different descriptions. Crucially, this model does not require pairwise labelled training data (i.e. unsupervised) therefore readily scalable to large scale camera networks of arbitrary camera pairs without the need for exhaustive data annotation for every camera pair. We show the effectiveness and advantages of the proposed method by extensive comparisons with related state-of-the-art approaches using two benchmarking ReID datasets, PRID2011 and iLIDS-VID.

Content-aware disparity adjustment for different stereo displays

In this paper, we present an effective disparity mapping method for binocular stereoscopic image. It is inspired by the observation that its displayed depth would change, when a stereoscopic image is displayed on different size screens. The phenomenon may bring an uncomfortable experience for viewers. To make a comfortable stereoscopic image for viewers, moreover to adapt a stereoscopic image to a target display screen, we propose a content-aware disparity adjustment method. Firstly, the disparity mapping is established to control and retarget the depth of a stereoscopic scene. Then, the relationship between the disparity editing and image content editing is established to guide the proposed warping model. At last, to implement the disparity mapping operator, we propose a content-aware stereoscopic mesh warping model, which can simultaneously avoid the salient region distortion and adjust disparity to a target range by establishing the relationship. Experimental results show that the proposed method can effectively adjust disparity of stereoscopic image, which not only avoids the salient region distortion and adjusts disparity to a target range.

Synthetic data generation models and algorithms for training image recognition algorithms using the Viola-Jones framework

The paper describes mathematical models and algorithms of warping grid functions with discrete parameters. For images, three warping models, applied to the generation of extra training data to build face recognition algorithms, are examined: the one based on harmonic functions, the one based on offsetting user-specified control point coordinates, and the one based on the computation of the optical flow between entropy-filtered images. For the training sets, both initial and those synthetically generated using the above three models, learning of face detection algorithms based on the Viola-Jones framework was performed and corresponding detection rates were compared. It is shown that this approach is applicable for synthetic data generation when training image recognition algorithms for recognition of objects characterized by inherent structure.

Geometry-based Dynamic Hand Gesture Recognition

Hand gestures play an important role in communication in the hard-of-hearing community. They are used to convey information instead of words. Besides, a system which is developed to identify gestures can be also used for human-computer interaction. In this paper, we propose a vision-based approach for recognizing dynamic hand gestures. Our processing consists of three main stages: pre-processing, feature extraction and recognition. The first stage involves two sub-stages: segmentation which locates the hand and extracts the corresponding silhouette using color information; separation that removes the arm based on geometrical properties. Some characteristics which describe the hand posture are then extracted. Finally, the recognition is performed using two popular algorithms, which are Dynamic Time Warping and Hidden Markov Model. The experiment is conducted on SKIG dataset with a comparison of classification accuracies corresponding to the two mentioned methods.

Analysis of warping during flat rolling of bimetallic slabs

The current study analyzes the warping behavior of bimetallic slabs during the flat rolling process. An analytical model that predicts the degree of warping during the rolling of bimetallic slabs is developed, based on differential bending, which occurs due to the mismatch in longitudinal elongation. The warping model focuses on the mismatch of the lengthwise elongation, due to differences in the deformation resistances of constituent materials in the bimetallic slab. With this analytical model, warping can be predicted for various processing parameters, such as initial thickness ratios and rolling reductions. To validate the model, flat rolling experiments using Al1050/Al6061 bimetallic slabs, produced by continuous clad casting, were conducted. The deformation characteristics of the bimetallic slabs during rolling were also assessed by a finite element analysis using the commercial code, FORGE™. The results from the current study show that the warping behavior of bimetallic slabs during flat rolling can be characterized with a high degree of reliability.

Motion-Based Gesture Recognition Algorithms for Robot Manipulation

The prevailing trend of integrating inertial sensors in consumer electronics devices has inspired research on new forms of human-computer interaction utilizing hand gestures, which may be set-up on mobile devices themselves. At present, motion gesture recognition is intensely studied, with various recognition techniques being employed and tested. This paper provides an in-depth, unbiased comparison of different algorithms used to recognize gestures based primarily on the single 3D accelerometer recordings. The study takes two of the most popular and arguably the best recognition methods currently in use - dynamic time warping and hidden Markov model - and sets them against a relatively novel approach founded on distance metric learning. The three selected algorithms are evaluated in terms of their overall performance, accuracy, training time, execution time and storage efficacy. The optimal algorithm is further implemented in a prototype user application, aimed to serve as an interface for controlling the motion of a toy robot via gestures made with a smartphone.

Linear Estimation of Time-Warped Signals

We introduce a novel methodology for estimating the time-axis deformation between two observations on a time-warped signal. Since the problem of estimating the warping function is nonlinear, existing methods iteratively minimize some metric between the observation and a hypothesized deformed template. Assuming the family of possible deformations the signal may undergo admits a finite-dimensional representation, we show that there is a nonlinear mapping from the space of observations to a low-dimensional linear space, such that in this space the problem of estimating the parametric model of the warping function is solved by a linear system of equations. We call the family of estimators derived based on this representation, linear warping estimators (LWE). The new representation of the problem enables an analytic analysis of the behavior of the solution in the presence of model mismatches, which is prohibitive when iterative methods are employed. The ability to achieve this major simplification both in the solution and in analyzing its performance results from the representation of the problem in a new coordinate system which is natural to the properties of the problem, instead of representing it in the standard coordinate system imposed by the sampling mechanism. The proposed solution is unique and exact, as it provides a closed-form expression for evaluating each of the parameters of the warping model using only measurements of the amplitude information of the observed and reference signals. The solution is applicable to any elastic warping regardless of its magnitude. We analyze the behavior of the LWE in the presence of noise and obtain a minimum variance unbiased estimator for the model parameters, by finding an optimal set of nonlinear operators for mapping the original problem into a low-dimensional linear space.

Robust depth-based estimation in the time warping model

In functional data analysis, the time warping model aims at representing a set of curves exhibiting phase and amplitude variation with respect to a common continuous process. Many biological processes, when observed across the time among different individuals, fit into this concept. The observed curves are modeled as the composition of an "amplitude process," which governs the common behavior, and a "warping process" that induces time distortion among the individuals. We aim at characterizing the first one. Because of the phase variation present among the curves, classical sample statistics computed on the observed sample provide poor representations of the amplitude process. Existing methods to estimate the mean behavior of the amplitude process consist on aligning the curves, that is, eliminating time variation, before estimation. However, since they rely on the use of sample means, they are very sensitive to the presence of outliers. In this article, we propose the use of a functional depth-based median as a robust estimator of the central behavior of the amplitude process. We investigate its properties in the time warping model, and we evaluate its performance in different simulation studies where we compare it to existing estimators, and we show its robustness against atypical observations. Finally, we illustrate its use with real data on a yeast time course microarray data set.

New Warping and Differential Drying Shrinkage Models for Jointed Plain Concrete Pavements Derived with Nonlinear Shrinkage Distribution

Warping can be quantified by the equivalent temperature difference required to deform a theoretically flat slab to the same shape as the actual slab. In the Mechanistic–Empirical Pavement Design Guide (MEPDG), the current warping model for jointed concrete pavements uses a triangular shrinkage distribution within the shrinkage zone near the surface of the slab as well as modification factors to account for ambient relative humidity (RH) and the amount of total shrinkage that is expected to be reversible. The current warping model was evaluated and found to predict improbable values. A new warping model based on an assumed nonlinear shrinkage distribution that followed the curve described by one-quarter of an ellipse was proposed. The elliptical shrinkage distribution was found to agree well with predicted moisture distributions through the depth of the slab at various locations around the United States. A new factor accounted for the fact that only drying shrinkage contributed to warping, and changes were made to the RH adjustment factor. The proposed model was derived from plate theory, rather than beam theory, to account for the two-way bending behavior of a slab. Only parameters that were currently inputs in the MEPDG were required for the new proposed warping model. With slight modification, the model could be used to predict differential drying shrinkage levels in jointed concrete pavements.

Fast super-resolution with affine motion using an adaptive Wiener filter and its application to airborne imaging

Fast nonuniform interpolation based super-resolution (SR) has traditionally been limited to applications with translational interframe motion. This is in part because such methods are based on an underlying assumption that the warping and blurring components in the observation model commute. For translational motion this is the case, but it is not true in general. This presents a problem for applications such as airborne imaging where translation may be insufficient. Here we present a new Fourier domain analysis to show that, for many image systems, an affine warping model with limited zoom and shear approximately commutes with the point spread function when diffraction effects are modeled. Based on this important result, we present a new fast adaptive Wiener filter (AWF) SR algorithm for non-translational motion and study its performance with affine motion. The fast AWF SR method employs a new smart observation window that allows us to precompute all the needed filter weights for any type of motion without sacrificing much of the full performance of the AWF. We evaluate the proposed algorithm using simulated data and real infrared airborne imagery that contains a thermal resolution target allowing for objective resolution analysis.