Warping Model Research Articles

INTELLIGENT SPEECH RECOGNITION USING FRACTAL AMENDED GRASSHOPPER OPTIMIZATION ALGORITHM WITH DEEP LEARNING APPROACH

Humans prefer to convey information through speech utilizing similar language. Speech detection is the capability to recognize the spoken words of the speaking person. Recent work demonstrates the increased attention among researcher workers in this field specially in Brain-Like computing applications and emphasizes the real-world usability of speech for speaker recognition across different applications. Automatic speech recognition (ASR) is the method of identifying human speech and converting it into text. This study has gained much popularity in recent times. It is a crucial area of research for human-to-machine interaction. Pioneer methods are concerned with manual feature extraction and classical algorithms including Hidden Markov Models (HMM), Gaussian Mixture Model (GMM), and the Dynamic Time Warping (DTW) model. In recent years, neural networks, namely convolutional neural networks (CNN), recurrent neural networks (RNN), and Transformers, have been utilized in the context of ASR and reached outstanding performance over the past few years. This study introduces Intelligent Speech Recognition using the Fractal Amended Grasshopper Optimization Algorithm with Deep Learning (ISR-AGODL) approach. The presented ISR-AGODL technique correctly identifies and recognizes speech signals. In the ISR-AGODL technique, the speech signals are transformed into spectrograms. Besides, the features are derived using the deep convolutional neural networks (DCNN) model. Followed by the Fractals AGO technique is utilized for the choosing of hyperparameters. Finally, the recognition of speech signals is achieved using the extreme gradient boosting (XGBoost) model. The simulation outcomes of the ISR-AGODL method can be validated using a benchmark dataset. The experimental results of the ISR-AGODL method portrayed a superior accuracy outcome of 96.34% over other models.

Seam estimation based on dense matching for parallax-tolerant image stitching

Image stitching with large parallax poses a significant challenge in the field of computer vision. Existing seam-based approaches attempt to address parallax artifacts by stitching images along seams. However, issues such as object mismatches, disappearances, and duplications still arise occasionally, primarily due to inaccurate alignment of dense pixels or inappropriate seam estimation methods. In this paper, we propose a robust seam-based parallax-tolerant image stitching method that leverages dense flow estimation from state-of-the-art approaches. Firstly, we develop a seam estimation method that does not require pre-estimation of image warping model. Instead, it directly estimates the seam by measuring the local smoothness of the optical flow field and incorporating a penalty term for duplications. Subsequently, we design an iterative algorithm that utilizes the location of estimated seam to solve a spatial smooth warping model and eliminate outlier corresponding pairs. By employing this approach, we effectively address the intertwined challenges of estimating the warping model and seam. Experiment on real-world images shows that our proposed method achieves superior local alignment accuracy near the stitching seam and outperforms other state-of-the-art techniques on visual stitching result. Code is available at https://github.com/zhihao0512/dense-matching-image-stitching.

Accurate image alignment based on multi-warp optimization for large parallax

Image alignment with parallax is a challenging computer vision problem. While existing methods employing local-varying smooth warps have enhanced alignment accuracy for complex spacial deformations compared to global homography estimation, they fall short in adequately representing the discontinuous transformations evident in images with large parallax. In this paper, we propose an image alignment method grounded in the estimation and fusion of multiple warping models. To effectively tackle the challenges posed by discontinuous deformations in different regions, our method comprises two key stages: principal region alignment and fine region refinement. In both stages, multiple warping models are initially estimated using feature correspondences and are then concurrently optimized by minimizing pixel-level photometric loss. For each pixel, we select the optimal model that minimizes warping error. Additionally, we introduce a feature grouping method based on Delaunay triangulation. Experiments on real-world images demonstrate the superior alignment accuracy achieved by our proposed method compared to other state-of-the-arts.

A novel garment transfer method supervised by distilled knowledge of virtual try-on model

Garment transfer can wear the garment of the model image onto the personal image. As garment transfer leverages wild and cheap garment input, it has attracted tremendous attention in the community and has a huge commercial potential. Since the ground truth of garment transfer is almost unavailable in reality, previous studies have treated garment transfer as either pose transfer or garment-pose disentanglement, and trained garment transfer in self-supervised learning, However, these implementation methods do not cover garment transfer intentions completely and face the robustness issue in the testing phase. Notably, virtual try-on technology has exhibited superior performance using self-supervised learning, we propose to supervise the garment transfer training via knowledge distillation from virtual try-on. Specifically, the overall pipeline is first to infer a garment transfer parsing, and to use it to guide downstream warping and inpainting tasks. The transfer parsing reasoning model learns the response and feature knowledge from the try-on parsing reasoning model and absorbs the hard knowledge from the ground truth. The progressive flow warping model learns the content knowledge from virtual try-on for a reasonable and precise garment warping. To enhance transfer realism, we propose an arm regrowth task to infer exposed skin. Experiments demonstrate that our method has state-of-the-art performance in transferring garments between persons compared with other virtual try-on and garment transfer methods.

Multimodel fore-/background alignment for seam-based parallax-tolerant image stitching

Image stitching with large parallax is a challenging computer vision problem. Although existing seam-based approaches were proposed to achieve pleasing results, issues like object dislocation, disappearance, and duplication can still occur. In this paper, to alleviate these problems, we propose a novel seam-based parallax-tolerant image stitching method, which relies on accurately aligning background and foreground regions using multiple warping models. To estimate various spatially smooth models based on feature correspondences from depth-varying objects, we introduce an iterative algorithm that selects inliers and solves the mesh warping model by assigning weights to data. Additionally, we construct matching confidences of foreground pixels based on selecting and grouping unaligned feature pairs, thus penalizing the duplication of seam cuts. To further improve alignment, we refine the models by minimizing pixel-level errors. We then choose the best seam among multiple candidate alignment and seam finding solutions. Finally, we re-estimate the warping model by sampling and weighting points near the seam to achieve a natural-looking stitching result. Experimental results on real-world images demonstrate the effectiveness and superiority of our proposed method over other state-of-the-arts.

Large deformation Kirchhoff–Love shell hierarchically enriched with warping: Isogeometric formulation and modeling of alternating stiff/soft layups

This paper presents a large deformation Kirchhoff–Love shell model hierarchically enhanced with through-the-thickness warping functions, arbitrarily chosen by the user. Two unknowns are introduced for each of them, representing its amplitudes in two directions tangent to the shell surface. NURBS are used to approximate reference surface displacement and warping amplitudes in the weak form. The transverse shear strains are linear functions of the warping parameters only and naturally free from locking. A patch-wise reduced integration avoids membrane locking and improves efficiency. Particular attention is paid to the modeling of composites made up of multiple stiff layers coupled with soft interlayers. The alternating layup with high stiffness ratios induces a significant sectional warping with transverse shear strains concentrated in the soft layers. Two warping models are investigated: (WI) all stiff layers maintain the same director orthogonal to the deformed surface with independent transverse shear deformations of the soft layers; (WZ) a single zigzag function linking these deformations. The numerical tests confirm the great accuracy of the hierarchic shell model in reproducing the solid solution with a small number of discrete parameters, provided that the correct warping model is chosen. WI is reliable for all alternating layups. WZ reduces the unknowns to five per surface point, regardless of the number of layers, and is accurate for uniform soft layers.

A hierarchical methodology for vessel traffic flow prediction using Bayesian tensor decomposition and similarity grouping

Accurate vessel traffic flow (VTF) prediction can enhance navigation safety and economic efficiency. To address the challenge of the inherently complex and dynamic growth of the VTF time series, a new hierarchical methodology for VTF prediction is proposed. Firstly, the original VTF data is reconfigured as a three-dimensional tensor by a modified Bayesian Gaussian CANDECOMP/PARAFAC (BGCP) tensor decomposition model. Secondly, the VTF matrix (hour ✕ day) of each week is decomposed into high- and low-frequency matrices using a Bidimensional Empirical Mode Decomposition (BEMD) model to address the non-stationary signals affecting prediction results. Thirdly, the self-similarities between VTF matrices of each week within the high-frequency tensor are utilised to rearrange the matrices as different one-dimensional time series to solve the weak mathematical regularity in the high-frequency matrix. Then, a Dynamic Time Warping (DTW) model is employed to identify grouped segments with high similarities to generate more suitable high-frequency tensors. The experimental results verify that the proposed methodology outperforms the state-of-the-art VTF prediction methods using real Automatic Identification System (AIS) datasets collected from two areas. The methodology can potentially optimise relation operations and manage vessel traffic, benefiting stakeholders such as port authorities, ship operators, and freight forwarders.

Image stitching method based on feature optimization and circular function weighted fusion

To address the issues of ghosting and chromatic gaps in parallax image stitching, this paper proposes a novel method based on feature optimization and circular function weighted fusion. The method employs the Scale-Invariant Feature Transform (SIFT) algorithm with dimensionality reduction optimization to extract point features, while the Grid-based Motion Statistics (GMS) algorithm is utilized to eliminate mismatched points. To further enhance the number of features, line features are also introduced. A local warping model is employed to guide the deformation of the grid image. Finally, a nonlinear fading-in and fading-out fusion model with circular function weighting is proposed for better image transition. Experimental results demonstrate that the proposed method is effective in dealing with ghosting and chromatic gaps compared to several existing methods.

Intelligent Diagnosis of Abnormal Charging for Electric Bicycles Based on Improved Dynamic Time Warping

The widespread penetration of electric bicycles (E-bicycles) raises numerous charging safety concerns. However, online diagnosis of charging safety for E-bicycles remains challenging due to the limited data and involvement of multiple factors, such as battery, charger, charging mode, and user behavior. To overcome this difficulty and promote charging safety, this article proposes a nonintrusive charging safety intelligent diagnosis scheme on the inputted power grid side. First, more than 150 000 charging records are collected from the grid side, and various charging current patterns are formally identified according to the working principles of different batteries, charging modes, and user behaviors. Then, on the basis of longest similar substring (LSS), an improved dynamic time warping (DTW) model, referred to as LSS-DTW, is established to efficiently identify the charging current profile similarities and meanwhile restrict the overregularization of DTW. By this manner, the abnormal charging processes can be accurately identified. Experimental results reveal that the built LSS-DTW model can distinguish the unsafe charging processes online, and achieve the average identification precision, recall, and F1-score of 94%. Furthermore, the proposed algorithm can be extended to similar charging safety identifications in electric vehicles and other battery-powered systems and provides early warnings to avoid catastrophic consequences.

Feature Correspondences Increase and Hybrid Terms Optimization Warp for Image Stitching.

Feature detection and correct matching are the basis of the image stitching process. Whether the matching is correct and the number of matches directly affect the quality of the final stitching results. At present, almost all image stitching methods use SIFT+RANSAC pattern to extract and match feature points. However, it is difficult to obtain sufficient correct matching points in low-textured or repetitively-textured regions, resulting in insufficient matching points in the overlapping region, and this further leads to the warping model being estimated erroneously. In this paper, we propose a novel and flexible approach by increasing feature correspondences and optimizing hybrid terms. It can obtain sufficient correct feature correspondences in the overlapping region with low-textured or repetitively-textured areas to eliminate misalignment. When a weak texture and large parallax coexist in the overlapping region, the alignment and distortion often restrict each other and are difficult to balance. Accurate alignment is often accompanied by projection distortion and perspective distortion. Regarding this, we propose hybrid terms optimization warp, which combines global similarity transformations on the basis of initial global homography and estimates the optimal warping by adjusting various term parameters. By doing this, we can mitigate projection distortion and perspective distortion, while effectively balancing alignment and distortion. The experimental results demonstrate that the proposed method outperforms the state-of-the-art in accurate alignment on images with low-textured areas in the overlapping region, and the stitching results have less perspective and projection distortion.

Full beam formulation for the lateral torsional buckling analysis of elastic frames by considering the structural detail of beam-to-column joint

The total potential energy formulation plays an essential role in investigating the flexural–torsional instability of elastic plane frames composed of thin-walled members. However, the work conjugateness between the internal moments and rotation variables of typical frame member cannot be restored completely when applying the conventional potential energy approaches. This paper aims to fully consider the virtual work done by the induced conservative and nonconservative moments in the planar frame prone to lateral buckling. The proposed full beam formulation can help identify those neglected moments that are necessary for fulfilling the equilibrium at the critical state no matter which potential energy formulation is concerned. It has been found that the relative significance of the virtual work done by the neglected moments with respect to the first variation of total potential energy is served as an indicator as whether the conventional potential energy equation can be used for investigating the lateral torsional buckling behavior of the frame. In the out-of-plane buckling investigation of benchmark elastic frames, it has been demonstrated in cases where the neglection of non-zero virtual work done by the nonconservative moment or an approximate account of the induced bimoment through the kinematic warping model may result in incorrect critical load. The lateral torsional buckling strength of the elastic frames can only be correctly determined by taking into account the structural configuration of beam-to-column joint.

Parametric chamfer alignment based on mesh deformation

Alignment for natural images in unconstrained environment is a challenging task. Despite the success for complex deformation, existing feature-based methods may be confused in low-textured regions where features are insufficient, while pixel-based approaches may fail when color changes. In this paper, a parametric chamfer alignment method based on mesh warping model is proposed. Warped positions of mesh vertices are considered as parameters and estimated by optimizing an object function, which measures the chamfer distance of edges and the smoothness of warping. To distinguish the sharpness of pixels, edges are detected through K-means cluster and weights are attached to different levels of edge points. In addition, after the warping model is initialized by feature-based alignment, a growing technique for registering the vertices is presented. Experiment shows that the proposed method outperforms some state-of-the-arts on real data and can be applied in stitching ceramic sanitary ware images.

Structure Preservation and Seam Optimization for Parallax-Tolerant Image Stitching

Parallax processing has long been a significant and challenging task in image stitching. In this paper, we study a new hybrid warping model based on multi-homography and structure preservation to achieve accurate alignment of regions at different depths while preserving local and global image structures. The homographies of different depth regions are estimated by dividing matching feature pairs into multiple layers. Then, layered warping is performed by determining the spatial relationships between the image mesh and these multi-homography, and then refining the local and global structural distortions through mesh optimization. Four constraints are considered during the local optimization process, including the local alignment error, global alignment error, and similarity error. In addition, we explore and introduce collinear structures into an objective function as a constraint for mesh optimization warping, which can preserve salient line structures while alleviating distortions in nonoverlapping areas. Furthermore, we develop an optimal seam search method based on seam error evaluation to improve the quality of the seams. Experimental results demonstrate that compared to existing methods, the proposed algorithm presents more accurate stitching results for images with large parallax and preserves salient image structures, and outperforming the existing methods both qualitatively and quantitatively.

Short and Medium-Term Prediction of Winter Wheat NDVI Based on the DTW–LSTM Combination Method and MODIS Time Series Data

The normalized difference vegetation index (NDVI) is an important agricultural parameter that is closely correlated with crop growth. In this study, a novel method combining the dynamic time warping (DTW) model and the long short-term memory (LSTM) deep recurrent neural network model was developed to predict the short and medium-term winter wheat NDVI. LSTM is well-suited for modelling long-term dependencies, but this method may be susceptible to overfitting. In contrast, DTW possesses good predictive ability and is less susceptible to overfitting. Therefore, by utilizing the combination of these two models, the prediction error caused by overfitting is reduced, thus improving the final prediction accuracy. The combined method proposed here utilizes the historical MODIS time series data with an 8-day time resolution from 2015 to 2020. First, fast Fourier transform (FFT) is used to decompose the time series into two parts. The first part reflects the inter-annual and seasonal variation characteristics of winter wheat NDVI, and the DTW model is applied for prediction. The second part reflects the short-term change characteristics of winter wheat NDVI, and the LSTM model is applied for prediction. Next, the results from both models are combined to produce a final prediction. A case study in Hebei Province that predicts the NDVI of winter wheat at five prediction horizons in the future indicates that the DTW–LSTM model proposed here outperforms the LSTM model according to multiple evaluation indicators. The results of this study suggest that the DTW–LSTM model is highly promising for short and medium-term NDVI prediction.

Dynamic mechanical analysis with torsional rectangular geometry: A critical assessment of constrained warping models

Dynamic mechanical oscillatory shear measurements with torsional rectangular geometry are widely carried out in order to determine the mechanical properties of soft solid materials in a quick and practical way. This technique has the advantage of avoiding slip effects, thus being particularly attractive for testing stiff elastomers such as vulcanized rubber compounds. However, one of its drawbacks is the clamping system required to keep the specimen edges in place. Since it imposes a constraint to warping deformations (i.e., out-of-plane cross-section distortions about the torsional axis), a certain increase of dynamic moduli with respect to their values in simple shear is observed and considered as an experimental artifact (i.e., de Saint-Venant's assumption of primary torsion of the specimen is no longer valid). The increase of dynamic moduli in torsion depends on the specimen's cross-section geometry and relative dimensions. We test here the capability of different torsion models to describe the constrained warping effect for an industrial rubber with rectangular specimen cross section using a wide range of different geometric parameters (length-to-width p ratio and width-to-thickness u ratio). We compare two theoretical models (Vlasov's model and Timoshenko's approach) and a phenomenological model (based on finite element simulations by Diani and Gilormini) with our experimental data set. We propose a slight modification of Vlasov's model to obtain realistic predictions of torsion over a wider parameter range. It is shown to be particularly useful in soft rubber testing, and the limitations in the choice of specimen geometry to obtain sufficient torque signal are overcome.

D-LSTM: Short-Term Road Traffic Speed Prediction Model Based on GPS Positioning Data

Short-term road traffic speed prediction is a long-standing topic in the area of Intelligent Transportation System. Apparently, effective prediction of the traffic speed on the road can not only provide timely details for the navigation system concerned and help the drivers to make better path selection, but also greatly improve the road supervision efficiency of the traffic department. At present, some researches on speed prediction based on GPS data, by adding weather and other auxiliary information, using graph convolutional neural network to capture the temporal and spatial characteristics, have achieved excellent results. In this paper, the problem of short-term traffic speed prediction based on GPS positioning data is further studied. For the processing of time series, we innovatively introduce Dynamic Time Warping algorithm into the problem and propose a Long Short-Term Memory with Dynamic Time Warping (D-LSTM) model. D-LSTM model, which integrates Dynamic Time Warping algorithm, can fine-tune the time feature, thus adjusting the current data distribution to be close to the historical data. More importantly, the fine-tuned data can still get a distinct improvement without special treatment of holidays. Meanwhile, considering that the data under different feature distributions have different effects on the prediction results, attention mechanism is also introduced in the model. Our experiments show that our proposed model D-LSTM performs better than other basic models in many kinds of traffic speed prediction problems with different time intervals, and especially significant in the traffic speed prediction on weekends.

A theoretical justification of warping generation for dewarping using CNN

Dewarping is a necessary preprocessing step to recognize text from a distorted camera captured document image. According to recent literature, deep learning-based approaches perform with higher accuracy in similar domains. The deep learning-based neural networks are not yet fully explored in the domain of dewarping. To fill this gap, we propose a dewarping approach based on the convolutional neural network. A large number of images are required to train such networks. However, it is a tedious job to capture such a large number of images. Hence, it is required to generate synthetic warped images for the training phase of the deep learning-based neural network. The existing synthetic warped image generation methods are heuristic-based. In this paper, we propose a novel mathematical model for the generation of warped images. The proposed model takes some parameters such as depth of the surface, camera angle, and camera position and generates the corresponding warped image. These parameters are the ground truth for that particular warped image. We use a Convolutional Neural Network (CNN) based model to estimate the warping parameters from a 2D warped image for dewarping. In the training phase of CNN based model, the synthetic images and their corresponding ground truth are used. Next, the trained model is used to dewarp the unknown warped images. The performance of the proposed warping model is analyzed. Finally, the proposed dewarping method is compared with existing approaches. In both cases, the results are encouraging.

Roundness-Preserving Warping for Aesthetic Enhancement-Based Stereoscopic Image Editing

Image editing is an effective solution to adapt contents for different applications. In this paper, we present a roundness-preserving warping model for stereoscopic image editing, in which energy constraints from image quality energy, aesthetics energy and depth adaptation energy are involved in the framework to solve the optimization. Specifically, to preserve object roundness during warping, the relationship between object's shape and disparity is established and is applied for depth adaptation. Different from the existing stereoscopic image editing methods, the main innovations of our method are to achieve a tradeoff in balancing information loss and reducing semantic distortion while providing a novel death adaptation model for recomposition and retargeting applications. Experimental results demonstrate the effectiveness of our method in enhancing the aesthetics of stereoscopic images.

A Novel Projective-Consistent Plane Based Image Stitching Method

When different target surfaces, in three-dimensional space, are mapped onto an image plane, they have different projections. These projections vary with the viewpoint. These local differences have influence on the accuracy of image stitching. Most of the existing image stitching methods divide an input image into a number of fixed-size cells, and the pixels within the same cell are then warped using the same local transformation model for the alignment. These methods are based on the hypothesis that the transformation models in one cell are consistent. However, this hypothesis does not hold in general. In this paper, we propose a novel projective-consistent plane based image stitching method (termed PCPS). It divides the overlapping regions of an input image into some projective-consistent planes according to the normal vectors’ orientations of local regions and the reprojection errors of aligned images. The local projective transformation model is estimated for each projective-consistent plane. And then, a hybrid warping model is estimated. For the pixels in overlapping regions, the local projective transformation models are adopted to achieve a better alignment. While for the pixels in non-overlapping regions, a global projective transformation model is estimated by using the inliers uniformly distributed in the projective-consistent planes to avoid distortion. Compared with the state-of-the-art image stitching methods, the experimental results on a number of challenging image sequences show that the projective transformation model estimated by the proposed PCPS method for each projective-consistent plane is more accurate, and the achieved stitching results have less seams and projective distortion.

Complex subduction beneath the Tibetan plateau: A slab warping model

Morphology of the subducting Indian lithosphere beneath Tibet is the key to understanding the tectonic evolution of the Tibetan plateau. In the present study we determine three-dimensional P-wave velocity images of the mantle under the entire Tibetan plateau and its adjacent areas by inverting ~200,000 teleseismic relative travel-time residuals recorded at 893 stations belonging to >32 portable seismic networks deployed in and around the plateau. An east-west varying high-velocity anomaly with a large dip angle down to the mantle transition zone is revealed clearly, which represents the subducting Indian slab. On the basis of the tomographic results, we propose a slab warping model to describe the complex subduction of the Indian plate beneath the plateau, which is characterized by lateral bending in addition to downward subduction. The complex geometry of the present Indian slab is possibly caused by non-uniform pulling of an earlier subducted slab in a process with the Indian slab detachment propagating laterally. As a result, a warping arc has formed beneath the Himalaya block to keep the Indian lithosphere intact. This slab warping model agrees well with many observations, such as magmatism and surface deformations in the western and southern Tibetan plateau.