Interpolation Tasks Research Articles

Neighborhood Geometric Structure-Preserving Variational Autoencoder for Smooth and Bounded Data Sources.

Many data sources, such as human poses, lie on low-dimensional manifolds that are smooth and bounded. Learning low-dimensional representations for such data is an important problem. One typical solution is to utilize encoder-decoder networks. However, due to the lack of effective regularization in latent space, the learned representations usually do not preserve the essential data relations. For example, adjacent video frames in a sequence may be encoded into very different zones across the latent space with holes in between. This is problematic for many tasks such as denoising because slightly perturbed data have the risk of being encoded into very different latent variables, leaving output unpredictable. To resolve this problem, we first propose a neighborhood geometric structure-preserving variational autoencoder (SP-VAE), which not only maximizes the evidence lower bound but also encourages latent variables to preserve their structures as in ambient space. Then, we learn a set of small surfaces to approximately bound the learned manifold to deal with holes in latent space. We extensively validate the properties of our approach by reconstruction, denoising, and random image generation experiments on a number of data sources, including synthetic Swiss roll, human pose sequences, and facial expression images. The experimental results show that our approach learns more smooth manifolds than the baselines. We also apply our approach to the tasks of human pose refinement and facial expression image interpolation where it gets better results than the baselines.

Adapting the residual dense network for seismic data denoising and upscaling

In seismic field acquisition, some of the measurements that are intended to be recorded are lost due to excessive noise contamination or because we are unable to occupy the required source or receiver locations. When these issues are severe, the missing data can cause images to be suboptimal. To enhance seismic data, deep learning has been used to perform denoising and data interpolation tasks, but it has rarely been used for both tasks in a unified framework. We have developed a modified residual dense network (RDN), a deep neural network for simultaneously removing noise and interpolating missing data. In addition, we augment the training data by varying source wavelets and constructing negative copied samples. After training, the modified RDN provides a one-stop solution for denoising and interpolation. To validate our approach, we test the modified RDN for random/coherent noise removal and regularly/irregularly subsampled seismic data interpolation. Compared with the benchmark methods, our approach can significantly improve performance in synthetic data (at least 2.41 in signal-to-noise ratio) and reduce undesired artifacts in marine field data.

PriorVAE: encoding spatial priors with variational autoencoders for small-area estimation.

Gaussian processes (GPs), implemented through multivariate Gaussian distributions for a finite collection of data, are the most popular approach in small-area spatial statistical modelling. In this context, they are used to encode correlation structures over space and can generalize well in interpolation tasks. Despite their flexibility, off-the-shelf GPs present serious computational challenges which limit their scalability and practical usefulness in applied settings. Here, we propose a novel, deep generative modelling approach to tackle this challenge, termed PriorVAE: for a particular spatial setting, we approximate a class of GP priors through prior sampling and subsequent fitting of a variational autoencoder (VAE). Given a trained VAE, the resultant decoder allows spatial inference to become incredibly efficient due to the low dimensional, independently distributed latent Gaussian space representation of the VAE. Once trained, inference using the VAE decoder replaces the GP within a Bayesian sampling framework. This approach provides tractable and easy-to-implement means of approximately encoding spatial priors and facilitates efficient statistical inference. We demonstrate the utility of our VAE two-stage approach on Bayesian, small-area estimation tasks.

Bayesian Deep Learning for Spatial Interpolation in the Presence of Auxiliary Information

Earth scientists increasingly deal with ‘big data’. For spatial interpolation tasks, variants of kriging have long been regarded as the established geostatistical methods. However, kriging and its variants (such as regression kriging, in which auxiliary variables or derivatives of these are included as covariates) are relatively restrictive models and lack capabilities provided by deep neural networks. Principal among these is feature learning: the ability to learn filters to recognise task-relevant patterns in gridded data such as images. Here, we demonstrate the power of feature learning in a geostatistical context by showing how deep neural networks can automatically learn the complex high-order patterns by which point-sampled target variables relate to gridded auxiliary variables (such as those provided by remote sensing) and in doing so produce detailed maps. In order to cater for the needs of decision makers who require well-calibrated probabilities, we also demonstrate how both aleatoric and epistemic uncertainty can be quantified in our deep learning approach via a Bayesian approximation known as Monte Carlo dropout. In our example, we produce a national-scale probabilistic geochemical map from point-sampled observations with auxiliary data provided by a terrain elevation grid. By combining location information with automatically learned terrain derivatives, our deep learning approach achieves an excellent coefficient of determination (R^{2} = 0.74) and near-perfect probabilistic calibration on held-out test data. Our results indicate the suitability of Bayesian deep learning and its feature-learning capabilities for large-scale geostatistical applications where uncertainty matters.Graphic

INTERPOLATION OF TABULAR FUNCTIONS FROM ONE INDEPENDENT VARIABLE USING THE TAYLOR POLYNOMIAL

A method of local interpolation of tabular functions from one independent variable using the Taylor polynomial of the nth degree in arbitrarily located interpolation nodes has been developed. This makes it possible to calculate intermediate values of tabular functions between interpolation nodes. The conducted analysis of the latest research and publications in the field of interpolation of tabular functions showed that the main part of the research is a strict theory of interpolation, i.e. clarification of its fundamental mathematical provisions. Some features of the interpolation of tabular functions from one independent variable using the Taylor polynomial of the nth degree are considered, namely: the solution algorithm and mathematical formulation of the interpolation problem are given; its formalized notation is given, as well as the matrix notation of interpolation procedures for certain values of the argument. A scalar algorithm for solving the problem of interpolation of tabular functions from one independent variable using the Taylor polynomial of the 2nd, 3rd, and 4th degrees has been developed. The simplicity and clarity of this algorithm is one of its advantages, but the algorithm is inconvenient for software implementation. The mathematical formulation of the problem of interpolation of tabular functions in terms of matrix algebra is given. The interpolation task is reduced to performing the following actions: based on the values of nodal points known from the table, it is necessary to calculate the Taylor matrix of the nth degree; based on the function values specified in the table a column vector of interpolation nodes should be formed; solve a linear system of algebraic equations, the root of which is the numerical coefficients of the Taylor polynomial of the nth degree. A method of calculating the coefficients of the interpolant, given by the Taylor polynomial of the nth degree for one independent variable has been developed. The essence of the method reduces to the product of the matrix, inverse of the Taylor matrix, which is determined by the nodal points of the tabular function, by a column vector containing the values of the interpolation nodes. Specific examples demonstrate the peculiarities of calculating the interpolant coefficients of the 2nd, 3rd and 4th degrees for one independent variable, and for each of them the interpolated value of the function at a given point is calculated. Calculations were performed in the Excel environment, which by analogy can be successfully implemented in any other computing environment.

Consecutively Missing Seismic Data Interpolation Based on Coordinate Attention Unet

Missing traces interpolation is a basic step in the seismic data processing workflow. Recently, many seismic data interpolation methods based on different neural networks have been proposed. The existing research shows that when the seismic data are consecutively missing, the larger gap for missing traces, the more difficult task of interpolation, due to convolution operation in the neural network can only capture local relations. In this letter, we incorporate the coordinate attention block to the Unet for 2-D successive missing traces interpolation. The hybrid loss function combined with structural similarity (SSIM) and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text {L}_{ {1}}$ </tex-math></inline-formula> norm is used as the loss function to further improve the interpolation performance of the designed network. Comparison experiments on 2-D synthetic and field seismic data show that the interpolation results obtained by the proposed method are more accurate and reasonable compared with Unet and Unets equipped state-of-the-art similar modules.

Progressive Motion Context Refine Network for Efficient Video Frame Interpolation

Recently, flow-based frame interpolation methods have achieved great success by first modeling optical flow between target and input frames, and then building synthesis network for target frame generation. However, above cascaded architecture can lead to large model size and inference delay, hindering them from mobile and real-time applications. To solve this problem, we propose a novel Progressive Motion Context Refine Network (PMCRNet) to predict motion fields and image context jointly for higher efficiency. Different from others that directly synthesize target frame from deep feature, we explore to simplify frame interpolation task by borrowing existing texture from adjacent input frames, which means that decoder in each pyramid level of our PMCRNet only needs to update easier intermediate optical flow, occlusion merge mask and image residual. Moreover, we introduce a new annealed multi-scale reconstruction loss to better guide the learning process of this efficient PMCRNet. Experiments on multiple benchmarks show that proposed approaches not only achieve favorable quantitative and qualitative results but also reduces current model size and running time significantly.

Predicting subsurface sonar observations with satellite-derived ocean surface data in the California Current Ecosystem.

Vessel-based sonar systems that focus on the water column provide valuable information on the distribution of underwater marine organisms, but such data are expensive to collect and limited in their spatiotemporal coverage. Satellite data, however, are widely available across large regions and provide information on surface ocean conditions. If satellite data can be linked to subsurface sonar measurements, it may be possible to predict marine life over broader spatial regions with higher frequency using satellite observations. Here, we use random forest models to evaluate the potential for predicting a sonar-derived proxy for subsurface biomass as a function of satellite imagery in the California Current Ecosystem. We find that satellite data may be useful for prediction under some circumstances, but across a range of sonar frequencies and depths, overall model performance was low. Performance in spatial interpolation tasks exceeded performance in spatial and temporal extrapolation, suggesting that this approach is not yet reliable for forecasting or spatial extrapolation. We conclude with some potential limitations and extensions of this work.

Coastal Depth Extraction from Satellites Images

The problems in bathymetry measurement often have gaps or ‘holes’ within the data. As a result, hydrographic surveyors often have sparse data, and even though the data is dense and equal distances, there is still a gap in time. This paper present coastal depth extraction from satellite images. The problem encountered during the bathymetry derivation process and the problem related to the space, distribution and quantity of the Single-beam echo sounder (SBES) data. Therefore, the idea of using spatial interpolation could be a suitable approach in solving the problems. This study intends to produce Satellite-Derived Bathymetry (SDB) from Landsat 8 images at Pantai Tok Jembal, Terengganu, Malaysia. The proposed method by first interpolating the SBES point in the calibration data using spatial predictors, i.e. Inverse Distance Weightage, Thin-Plate Spline, Spline with Tension, Universal Kriging, Natural Neighbor, and Topo to Raster. Second, the raster output created from the interpolation process then converts into the point shapefile. Third, intersect function use to eliminate the point whereby not in the domain. Finally, the newly generated SBES points in calibration data ready to apply at the SDB computation process, generating SDB. In continuation, a comparative analysis conducted between six SDB results generated using each different newly generated calibration data. The result indicates SDB utilizes with Universal Kriging-newly generated calibration data (RMSE: 0.718 m) was the best result. To summarise, this study has successfully attained the research objectives by utilizing the newly generated calibration data in generating SDB. The task of spatial interpolation recreates the SBES data from irregular space and short data to uniform space and long data, which facilitate in pixel to point value extraction and help refine the bathymetry derivation process. Furthermore, the proposed method suitable to be used when the data are not applicable or limited.

Filling gaps in Landsat ETM+ SLC-off images with Sentinel-2 MSI images

On May 31, 2003, the scan-line corrector (SLC) of Landsat 7 ETM+ failed permanently. The resulting ETM+ SLC-off images contain 22% un-scanned gap pixels, thus, severely limiting their utility. In this paper, we propose a new scheme to fill gaps in SLC-off images by identifying a new source of auxiliary or known image. Specifically, Sentinel-2 MSI images are proposed as known images for gap filling, rather than the common strategy of using Landsat series data. The Sentinel-2 MSI data have the same map projection and similar band wavelengths as the Landsat 7 ETM+ data, and the Sentinel-2A and −2B MSI data together provide global coverage with a 5-day revisit period. The shorter revisit period of Sentinel-2 provides greater opportunities to acquire effective known images that are temporally close to the ETM+ SLC-off images for reliable gap filling. To render the Sentinel-2 MSI images suitable for gap filling, it is necessary to deal with their incompatible spatial resolutions relative to Landsat 7. Here, we used a downscaling-then-upscaling methodology involving Area-to-Point Regression Kriging (ATPRK) for the downscaling step. For the gap filling interpolation task we used a recently developed spatial-spectral radial basis function (SSRBF) method, which makes use of the available spectral as well as spatial information. Experiments were undertaken in which the SSRBF method was applied for gap filling using a range of different known images including Sentinel-2 MSI known images and Landsat 8 known images temporally close to and distant from the predicted image. The experimental results demonstrate that the Sentinel-2 MSI known images produced by downscaling-then-upscaling are a valuable source of information for gap filling, especially when the existing Landsat series images (e.g., the OLI images) are temporally distant to the SLC-off images.

13‐3: Invited Paper: Video Frame Interpolation via Structure Motion based Iterative Feature Fusion

Video Frame Interpolation synthesizes non‐existent images between adjacent frames, with the aim of providing a smooth and consistent visual experience. Two approaches for solving this challenging task are optical flow based and kernel‐based methods. In existing works, optical flow based methods can provide accurate point‐to‐point motion description, however, they lack constraints on object structure. On the contrary, kernel‐based methods focus on structural alignment, which relies on semantic and apparent features, but tends to blur results. Based on these observations, we propose a structure‐motion based iterative fusion method. The framework is an end‐to‐end learnable structure with two stages. First, interpolated frames are synthesized by structure‐based and motion‐based learning branches respectively, then, an iterative refinement module is established via spatial and temporal feature integration. Inspired by the observation that audiences have different visual preferences on foreground and background objects, we for the first time propose to use saliency masks in the evaluation processes of the task of video frame interpolation. Experimental results on three typical benchmarks show that the proposed method achieves superior performance on all evaluation metrics over the state‐of‐the‐art methods, even when our models are trained with only one‐tenth of the data other methods use.

Fast amortized multi-point evaluation

The efficient evaluation of multivariate polynomials at many points is an important operation for polynomial system solving. Kedlaya and Umans have recently devised a theoretically efficient algorithm for this task when the coefficients are integers or when they lie in a finite field. In this paper, we assume that the set of points where we need to evaluate is fixed and “sufficiently generic”. Under these restrictions, we present a quasi-optimal algorithm for multi-point evaluation over general fields. We also present a quasi-optimal algorithm for the opposite interpolation task.

Improved accuracy and transferability of molecular-orbital-based machine learning: Organics, transition-metal complexes, non-covalent interactions, and transition states.

Molecular-orbital-based machine learning (MOB-ML) provides a general framework for the prediction of accurate correlation energies at the cost of obtaining molecular orbitals. The application of Nesbet's theorem makes it possible to recast a typical extrapolation task, training on correlation energies for small molecules and predicting correlation energies for large molecules, into an interpolation task based on the properties of orbital pairs. We demonstrate the importance of preserving physical constraints, including invariance conditions and size consistency, when generating the input for the machine learning model. Numerical improvements are demonstrated for different datasets covering total and relative energies for thermally accessible organic and transition-metal containing molecules, non-covalent interactions, and transition-state energies. MOB-ML requires training data from only 1% of the QM7b-T dataset (i.e., only 70 organic molecules with seven and fewer heavy atoms) to predict the total energy of the remaining 99% of this dataset with sub-kcal/mol accuracy. This MOB-ML model is significantly more accurate than other methods when transferred to a dataset comprising of 13 heavy atom molecules, exhibiting no loss of accuracy on a size intensive (i.e., per-electron) basis. It is shown that MOB-ML also works well for extrapolating to transition-state structures, predicting the barrier region for malonaldehyde intramolecular proton-transfer to within 0.35 kcal/mol when only trained on reactant/product-like structures. Finally, the use of the Gaussian process variance enables an active learning strategy for extending the MOB-ML model to new regions of chemical space with minimal effort. We demonstrate this active learning strategy by extending a QM7b-T model to describe non-covalent interactions in the protein backbone-backbone interaction dataset to an accuracy of 0.28 kcal/mol.

Interpolation Accuracy of Hybrid Soft Computing Techniques in Estimating Discharge Capacity of Triangular Labyrinth Weir

Soft Computing Techniques (SCT) are extensively used to estimate Labyrinth Weir's (LW) flow-rate. Due to the multiplicity of these techniques, identifying the most competent SCT is indispensable. This study aims to estimate the flow-rate of a sharp-crest triangular LW as a function of its side leg angle α and total head ratio (H/P) through several SCTs such as Adaptive Neuro-Fuzzy Inference System (ANFIS), Multi-Layer Perceptron (MLP), Support Vector Regression, and Radial Basis Function Neural Network. Additionally, these SCTs' potential combinations with Firefly Optimization Algorithm (FA) and Particle Swarm Optimization (PSO) are also investigated and used for estimation. The conducted experimental studies on LW examined a wide range of H/P in some limited α values. Correspondingly, all the proposed models and techniques are incapable of estimating the flow rate for intermediate α values without interpolation. Therefore, SCT's Interpolation accuracy is of the utmost importance. Besides the standard evaluation in the testing stage, a novel approach is utilized to evaluate the SCT's accuracy in the interpolation task. The SCTs are evaluated based on several statistical criteria, the Taylor diagram, Kruskal-Wallis, and Mann-Whitney tests. It is concluded that the competence of an SCT in the testing stage cannot guarantee its accuracy in the interpolation task. Subsequently, ANFIS-PSO and MLP-FA show the highest accuracy in the testing stage and interpolation task, respectively. Eventually, according to a systematic investigation in the implemented diagnostic test results, two rankings are presented for the applied SCTs based on their performance in the testing stage and their interpolation accuracy.

DRVI: Dual Refinement for Video Interpolation

The quality of a video clip is considered to be poor if the resolution or the frame rate is low. Video interpolation is thus introduced to enhance video quality and provide a better viewing experience to users. However, there are still some challenges, like the blur caused by motion changes. In this paper, we introduce a dual refinement technique for video interpolation (DRVI). It has three main steps, namely flow refinement, frame synthesis, and Haar refinement. The flow refinement can generate accurate bi-directional flows, which are more suitable for frame interpolation tasks. The Haar refinement uses the Discrete Wavelet Transform (DWT). It can preserve information in different frequency domains and also speed up the learning process. We also add an arbitrary time approximation module to allow multi-frame generation. The number of learnable parameters in our model is much less than existing methods; still, it has excellent performance. Our method is trained on Vimeo90K (Xue et al. , 2019) and tested on three well-known datasets to demonstrate its effectiveness.

Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision

Pseudo-LiDAR point cloud interpolation is a novel and challenging task in autonomous driving, which aims to address the frequency mismatching problem between a camera and a LiDAR. Previous works represent the 3D spatial motion relationship with a coarse 2D optical flow, and the quality of interpolated point clouds only depends on the supervision of depth maps. As a result, the generated point clouds suffer from inferior global distributions and local appearances. To solve the above problems, we propose a Pseudo-LiDAR point cloud interpolation network to generate temporally and spatially high-quality point cloud sequences. By exploiting the scene flow from point clouds, the proposed network is able to learn a more accurate representation of the 3D spatial motion relationship. For a more comprehensive perception of the distribution of a point cloud, we design a novel reconstruction loss function with the chamfer distance to supervise the generation of Pseudo-LiDAR point clouds in 3D space. In addition, we introduce a multi-modal deep aggregation module to facilitate the efficient fusion of texture and depth features. As the benefits of the improved motion representation, training loss function, and model structure, our approach gains significant improvements on the Pseudo-LiDAR point cloud interpolation task. The experimental results evaluated on KITTI dataset demonstrate the state-of-the-art quantitative and qualitative performance of the proposed network.

Static2Dynamic: Video Inference From a Deep Glimpse

In this article, we address a novel and challenging task of video inference, which aims to infer video sequences from given non-consecutive video frames. Taking such frames as the anchor inputs, our focus is to recover possible video sequence outputs based on the observed anchor frames at the associated time. With the proposed Stochastic and Recurrent Conditional GAN (SR-cGAN), we are able to preserve visual content across video frames with additional ability to handle possible temporal ambiguity. In the experiments, we show that our SR-cGAN not only produces preferable video inference results, it can also be applied to relevant tasks of video generation, video interpolation, video inpainting, and video prediction.

Osculating conic biarcs

A circular biarc can be defined by using two points K and L with their (oriented) tangents gK and gL as input. It is well-known that one can determine a one parametric set of circular arc pairs k,ℓ such that k starts at K with tangent gK, ℓ ends at L with tangent gL and k and ℓ meet with a common tangent in an intermediate point P. In this paper we investigate a similar construction where we replace the circle biarcs by pairs of conic arcs. It turns out that in this case we can prescribe a conic k0 with a point K on it, another conic ℓ0 with a point L on it, and moreover an intermediate point P to obtain a unique pair k,ℓ of conics such that k osculates k0 in K, ℓ osculates ℓ0 in L and k and ℓ osculate each other in P. This also confirms a result of H. Pottmann from 1991. We use our method to solve an interpolation task of Hermite type whose input consists of a series of points with their curvature circles and another series of intermediate points. The output is a GC2 spline curve with conic arc segments.

Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks

Generative Adversarial Networks (GANs) are becoming popular machine learning choices for training generators. At the same time there is a concerted effort in the machine learning community to expand the range of tasks in which learning can be applied as well as to utilize methods from other disciplines to accelerate learning. With this in mind, in the current work we suggest ways to enforce given constraints in the output of a GAN generator both for interpolation and extrapolation (prediction). For the case of dynamical systems, given a time series, we wish to train GAN generators that can be used to predict trajectories starting from a given initial condition. In this setting, the constraints can be in algebraic and/or differential form. Even though we are predominantly interested in the case of extrapolation, we will see that the tasks of interpolation and extrapolation are related. However, they need to be treated differently.For the case of interpolation, the incorporation of constraints is built into the training of the GAN. The incorporation of the constraints respects the primary game-theoretic setup of a GAN so it can be combined with existing algorithms. However, it can exacerbate the problem of instability during training that is well-known for GANs. We suggest adding small noise to the constraints as a simple remedy that has performed well in our numerical experiments.The case of extrapolation (prediction) is more involved. During training, the GAN generator learns to interpolate a noisy version of the data and we enforce the constraints. This approach has connections with model reduction that we can utilize to improve the efficiency and accuracy of the training. Depending on the form of the constraints, we may enforce them also during prediction through a projection step. We provide examples of linear and nonlinear systems of differential equations to illustrate the various constructions.

Multi-Antenna Channel Interpolation via Tucker Decomposed Extreme Learning Machine

Channel interpolation is an essential technique for providing high-accuracy estimation of the channel state information (CSI) for wireless systems design where the frequency-space structural correlations of multi-antenna channel are typically hidden in matrix or tensor forms. In this letter, a modified extreme learning machine (ELM) that can process tensorial data, or ELM model with tensorial inputs (TELM), is proposed to handle the channel interpolation task. The TELM inherits many good properties from ELMs. Based on the TELM, the Tucker decomposed extreme learning machine (TDELM) is proposed for further improving the performance. Furthermore, we establish a theoretical argument to measure the interpolation capability of the proposed learning machines. Experimental results verify that our proposed learning machines can achieve comparable mean squared error (MSE) performance against the traditional ELMs but with 15% shorter running time, and outperform the other methods for a 20% margin measured in MSE for channel interpolation.