Arbitrary Scale Factor Research Articles

S[formula omitted]U-PVNet: Arbitrary-scale point cloud upsampling via Point-Voxel Network based on Siamese Self-Supervised Learning

Recently, benefiting from the rapid development of deep learning, many research on point cloud upsampling task has been successfully proposed with remarkable performance. The quality of the upsampled point clouds is also vital for the quality of the reconstructed meshes in 3D surface reconstruction task. However, most existing methods either independently train a specific upsampling network for each scale factor or heavily rely on the paired training data as the supervision information. To address these limitations which are both inefficient and impractical for storage and computation in real applications, we present a Point-Voxel Network based on Siamese Self-supervised learning for arbitrary-Scale point cloud Upsampling (S3U-PVNet), which mainly includes a Down–Up pipeline and an Up–Down pipeline, to support self-supervised point cloud upsampling with arbitrary scale factors by a single network. The core of our network is a series of stacked point-voxel feature fusion (PVFF) modules, which can effectively and efficiently extract and fuse multi-granularity features from input point clouds. Each module includes two branches, a point-based branch that represents the sparse inputs in points to adaptively learn the spatial relationship among points, and a voxel-based branch that extract features in voxels to reduce the problem of inaccurate feature extraction caused by the irregularity and sparsity of point clouds. Furthermore, we also propose an end-to-end training objective for our siamese self-supervised network encapsulating reconstruction loss and similarity loss, which considers both global shape constraint and local geometric constraint. We achieve new state-of-the-art unsupervised upsampling results on various synthetic datasets and demonstrate the generalization of the proposed method on challenging real-world data.

An Image Arbitrary-Scale Super-Resolution Network Using Frequency-domain Information

Image super-resolution (SR) is a technique to recover lost high-frequency information in low-resolution (LR) images. Since spatial-domain information has been widely exploited, there is a new trend to involve frequency-domain information in SR tasks. Besides, image SR is typically application-oriented and various computer vision tasks call for image arbitrary magnification. Therefore, in this article, we study image features in the frequency domain to design a novel image arbitrary-scale SR network. First, we statistically analyze LR-HR image pairs of several datasets under different scale factors and find that the high-frequency spectra of different images under different scale factors suffer from different degrees of degradation, but the valid low-frequency spectra tend to be retained within a certain distribution range. Then, based on this finding, we devise an adaptive scale-aware feature division mechanism using deep reinforcement learning, which can accurately and adaptively divide the frequency spectrum into the low-frequency part to be retained and the high-frequency one to be recovered. Finally, we design a scale-aware feature recovery module to capture and fuse multi-level features for reconstructing the high-frequency spectrum at arbitrary scale factors. Extensive experiments on public datasets show the superiority of our method compared with state-of-the-art methods.

Scale-Arbitrary Invertible Image Downscaling.

Conventional social media platforms usually downscale high-resolution (HR) images to restrict their resolution to a specific size for saving transmission/storage cost, which makes those visual details inaccessible to other users. To bypass this obstacle, recent invertible image downscaling methods jointly model the downscaling/upscaling problems and achieve impressive performance. However, they only consider fixed integer scale factors and may be inapplicable to generic downscaling tasks towards resolution restriction as posed by social media platforms. In this paper, we propose an effective and universal Scale-Arbitrary Invertible Image Downscaling Network (AIDN), to downscale HR images with arbitrary scale factors in an invertible manner. Particularly, the HR information is embedded in the downscaled low-resolution (LR) counterparts in a nearly imperceptible form such that our AIDN can further restore the original HR images solely from the LR images. The key to supporting arbitrary scale factors is our proposed Conditional Resampling Module (CRM) that conditions the downscaling/upscaling kernels and sampling locations on both scale factors and image content. Extensive experimental results demonstrate that our AIDN achieves top performance for invertible downscaling with both arbitrary integer and non-integer scale factors. Also, both quantitative and qualitative evaluations show our AIDN is robust to the lossy image compression standard. The source code and trained models will be publicly available upon publication.

Multi-Scale Factor Image Super-Resolution Algorithm with Information Distillation Network

Deep convolutional neural networks with strong expressive ability have achieved impressive performances in single-image super-resolution algorithms. However, excessive convolutions usually consume high computational cost, which limits the application of super-resolution technology in low computing power devices. Besides, super-resolution of arbitrary scale factor has been ignored for a long time. Most previous researchers have trained a specific network model separately for each factor, and taken the super-resolution of several integer scale factors into consideration. In this paper, we put forward a multi-scale factor network (MFN), which dynamically predicts the weights of the upscale filter by taking the scale factor as input, and generates HR images with corresponding scale factors from the weights. This method is suitable for arbitrary scale factors (integer or non-integer). In addition, we use an information distillation structure to gradually extract multi-scale spatial features. Extensive experiments suggest that the proposed method performs favorably against the state-of-the-art SR algorithms in term of visual quality, PSNR/SSIM evaluation indicators, and model parameters.

Fractional Super-Resolution of Voxelized Point Clouds.

We present a method to super-resolve voxelized point clouds downsampled by a fractional factor, using lookup-tables (LUT) constructed from self-similarities from their own downsampled neighborhoods. The proposed method was developed to densify and to increase the precision of voxelized point clouds, and can be used, for example, as improve compression and rendering. We super-resolve the geometry, but we also interpolate texture by averaging colors from adjacent neighbors, for completeness. Our technique, as we understand, is the first specifically developed for intra-frame super-resolution of voxelized point clouds, for arbitrary resampling scale factors. We present extensive test results over different point clouds, showing the effectiveness of the proposed approach against baseline methods.

Depth Completion and Super-Resolution with Arbitrary Scale Factors for Indoor Scenes.

Depth sensing has improved rapidly in recent years, which allows for structural information to be utilized in various applications, such as virtual reality, scene and object recognition, view synthesis, and 3D reconstruction. Due to the limitations of the current generation of depth sensors, the resolution of depth maps is often still much lower than the resolution of color images. This hinders applications, such as view synthesis or 3D reconstruction, from providing high-quality results. Therefore, super-resolution, which allows for the upscaling of depth maps while still retaining sharpness, has recently drawn much attention in the deep learning community. However, state-of-the-art deep learning methods are typically designed and trained to handle a fixed set of integer-scale factors. Moreover, the raw depth map collected by the depth sensor usually has many depth data missing or misestimated values along the edges and corners of observed objects. In this work, we propose a novel deep learning network for both depth completion and depth super-resolution with arbitrary scale factors. The experimental results on the Middlebury stereo, NYUv2, and Matterport3D datasets demonstrate that the proposed method can outperform state-of-the-art methods.

Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud.

Point cloud upsampling is vital for the quality of the mesh in three-dimensional reconstruction. Recent research on point cloud upsampling has achieved great success due to the development of deep learning. However, the existing methods regard point cloud upsampling of different scale factors as independent tasks. Thus, the methods need to train a specific model for each scale factor, which is both inefficient and impractical for storage and computation in real applications. To address this limitation, in this article, we propose a novel method called "Meta-PU" to first support point cloud upsampling of arbitrary scale factors with a single model. In the Meta-PU method, besides the backbone network consisting of residual graph convolution (RGC) blocks, a meta-subnetwork is learned to adjust the weights of the RGC blocks dynamically, and a farthest sampling block is adopted to sample different numbers of points. Together, these two blocks enable our Meta-PU to continuously upsample the point cloud with arbitrary scale factors by using only a single model. In addition, the experiments reveal that training on multiple scales simultaneously is beneficial to each other. Thus, Meta-PU even outperforms the existing methods trained for a specific scale factor only.

Sensing chiral magnetic noise via quantum impurity relaxometry

We present a theory for quantum impurity relaxometry of magnons in thin films, exhibiting quantitative agreement with recent experiments without needing arbitrary scale factors used in theoretical models thus far. Our theory reveals that chiral coupling between prototypical spin>1/2 quantum impurities and magnons plays a central role in determining impurity relaxation, which is further corroborated by our experiments on nickel films interfaced with nitrogen-vacancy centers. Along with advancing magnonics and understanding decoherence in hybrid quantum platforms with magnets, the ability of a quantum impurity spin to sense chiral magnetic noise presents an opportunity to probe chiral phenomena in condensed matter.

Residual scale attention network for arbitrary scale image super-resolution

Research on super-resolution has achieved great success on synthetic data with deep convolutional neural networks. Some recent works tend to apply super-resolution to practical scenarios. Learning an accurate and flexible model for super-resolution of arbitrary scale factor is important for realistic applications, while most existing works only focus on integer scale factor. In this work, we present a residual scale attention network for super-resolution of arbitrary scale factor. Specifically, we design a scale attention module to learn discriminative features of low-resolution images by introducing the scale factor as prior knowledge. Then, we utilize quadratic polynomial of the coordinate information and scale factor to predict pixel-wise reconstruction kernels and achieve super-resolution of arbitrary scale factor. Besides, we use the predicted reconstruction kernels in image domain to interpolate low-resolution image and obtain coarse high-resolution image first, then make our main network learn high-frequency residual image from feature domain. Extensive experiments on both synthetic and real data show that the proposed method outperforms state-of-the-art super-resolution methods of arbitrary scale factor in terms of both objective metrics and subjective visual quality.

Multi-scale skip-connection network for image super-resolution

A skip-connection learning framework-based convolution neural network (CNN) has recently achieved great success in image super-resolution (SR). However, most CNN models based on the skip-connection learning framework do not fully make use of potential multi-scale features of images. In this paper, we propose a multi-scale skip-connection network (MSN) to improve the visual quality of the image SR. First, convolution kernels with different sizes are exploited to capture the multi-scale features of LR images. All the feature-maps captured by convolution kernels of the same size are direct input into a multi-scale hybrid group (MHG); second, the convolution layers of each MHG are composed of dilated convolutions and standard convolutions. The hybrid convolutions can fully train feature details obtained from preceding and current scale convolution layers; three, the output of each hybrid convolution layer is fed into subsequent hybrid convolution layers by skip-connections, thus producing dense connections; lastly, the meta-upscale module is used as the upscale module, which can magnify the trained feature maps arbitrary scale factors. By being evaluated on a wide variety of images, the proposed MSN network achieves an advantage over the state-of-the-art methods in terms of both numerical results and visual quality.

Arbitrary Back-Projection Networks for Image Super-Resolution

Recently, a method called Meta-SR has solved the problem of super-resolution of arbitrary scale factor with only one single model. However, it has a limited reconstruction accuracy compared with RDN[Formula: see text] and EDSR[Formula: see text]. Inspired by Meta-SR, we noticed that by combining the core idea of Meta-SR and D-DBPN, we might construct a network that has as good image reconstruction accuracy as D-DBPN’s, at the same time, keeps arbitrary scaling function. According to Meta-SR’s Meta-Upscale Module, we designed a different structure called Meta-Downscale Module. By using these two different modules and back-projection structure, we construct an arbitrary back-projection network, which has the ability to enlarge images with arbitrary scale factor by using only one single model, meanwhile, obtains state-of-the-art reconstruction results. Through extensive experiments, our proposed method performs better reconstruction effect than Meta-SR and more efficient than D-DBPN. Besides that, we also evaluated the proposed method on widely used benchmark dataset on single image super-resolution. The experimental results show the superiority of our model compared to RDN+ and EDSR+.

Meta-USR: A Unified Super-Resolution Network for Multiple Degradation Parameters.

Recent research on single image super-resolution (SISR) has achieved great success due to the development of deep convolutional neural networks. However, most existing SISR methods merely focus on super-resolution of a single fixed integer scale factor. This simplified assumption does not meet the complex conditions for real-world images which often suffer from various blur kernels or various levels of noise. More importantly, previous methods lack the ability to cope with arbitrary degradation parameters (scale factors, blur kernels, and noise levels) with a single model. A few methods can handle multiple degradation factors, e.g., noninteger scale factors, blurring, and noise, simultaneously within a single SISR model. In this work, we propose a simple yet powerful method termed meta-USR which is the first unified super-resolution network for arbitrary degradation parameters with meta-learning. In Meta-USR, a meta-restoration module (MRM) is proposed to enhance the traditional upscale module with the capability to adaptively predict the weights of the convolution filters for various combinations of degradation parameters. Thus, the MRM can not only upscale the feature maps with arbitrary scale factors but also restore the SR image with different blur kernels and noise levels. Moreover, the lightweight MRM can be placed at the end of the network, which makes it very efficient for iteratively/repeatedly searching the various degradation factors. We evaluate the proposed method through extensive experiments on several widely used benchmark data sets on SISR. The qualitative and quantitative experimental results show the superiority of our Meta-USR.

A universally efficient algorithm and precision assessment for seamless 3D similarity transformation

Three-dimensional (3D) similarity transformation is popularly applied for measurement datum transformation. In this study, a seamless partial errors-in-variables (EIV) model with equality constraints is established to describe the universal 3D similarity transformation problem (with arbitrary rotation angles and scale factor). Unlike the traditional transformation model, all of the random errors in the measured coordinates for common and non-common points, and their variance–covariance information can be considered. To obtain the least squares solution of this model, the constrained total least squares prediction (CTLSP) algorithm is derived using Gauss–Newton iteration and the Euler–Lagrange approach. Unnecessary matrix calculations in the application of the CTLSP algorithm for 3D datum transformation are avoided and the efficient iterative formulae are derived. Compared with the existing generalized total least squares prediction (GTLSP) algorithm, in which the transformation model is nonlinear with respect to the parameters, the CTLSP algorithm avoids the complex calculations related to the rotation matrix expressed by the trigonometric functions, and allows us to use the simple linear least squares to obtain a satisfactory initial value of the parameter vector. In addition, the linearly approximate cofactor propagation law is employed to assess the precision of the transformed coordinates of non-common points based on the CTLSP algorithm. Finally, the superiorities of CTLSP in transformation accuracy and computational efficiency are verified using an experiment. It should be noted that the new algorithm along with the precision evaluation formulae can easily be extended to the 2D/3D affine and rigid transformation cases as well, such as the map rectification, the point clouds registration, and the image matching.

Spatial general autoregressive model-based image interpolation accommodates arbitrary scale factors.

This paper proposed a novel image interpolation algorithm with an arbitrary upscaling factor based on the spatial general autoregressive model. First, to accommodate arbitrary scale factors, a non-integer mapping method was modulated into the spatial general autoregressive model, which was employed to model the piecewise stationary pattern with a higher description capacity than autoregressive models. A gradient angle guided extension method was utilized to extend the spatial general autoregressive model, and more pixels in the neighborhood were included to estimate the parameters of the spatial general autoregressive model. To realize the high-accuracy estimation of the model parameters, a regularization method via an elastic network was adopted to maintain the complexity of the object function in a reasonable state and address the overfitting problem. We also introduced an iterative curvature method to refine the interpolation result of those image blocks with large variances of gray intensities. Experiments on 25 images were conducted with integer and non-integer magnification factors to systematically verify the objective and subjective measures of the proposed method. The visual artifacts were effectively suppressed by the proposed method, and a flexible interpolation method for arbitrary scale factors was implemented.

Anomalous current due to Weyl anomaly for conformal field theory

Recently it is found that Weyl anomaly leads to new anomalous currents in an external electromagnetic field in a curved spacetime. For simplicity, the initial works mainly focus on weak gravitational fields and the anomalous current is obtained for conformally flat spaces with small scale factors. In this paper, we generalize the results to the case with arbitrary scale factors. Firstly, we derive the transformation law of current under Weyl transformation, from which one can read off the anomalous current in general conformally flat spaces. Secondly, by using the Weyl transformation of currents we provide a new derivation of the Weyl-anomaly-induced current near the boundary. Thus we have obtained the two kinds of anomalous currents in the literature from a unified formula. Finally, we extend the discussions to n-form fields and find similar anomalous currents.

Nonsingular Brans-Dicke-Λcosmology

We discuss a Brans-Dicke model with a cosmological constant, a negative value of the $w$ parameter, and an arbitrary (in general nonvanishing) scale factor at the big bang. The Friedmann equations for a flat universe are considered. The current observational values for Hubble constant ${H}_{0}$ and deceleration parameter ${q}_{0}$ play the role of initial conditions. We follow the approach of Uehara and Kim in order to solve field equations analytically. In K. Uehara and C. W. Kim [Phys. Rev. D 26, 2575 (1982)] only positive values of $w$ were considered; we extend the study to a complete set of possible $w$ values. Our main result is that the scale factor (during its evolution back in time direction) may not vanish, unlike in the standard $\ensuremath{\Lambda}\mathrm{CDM}$ case. In other words, the considered model demonstrates a cosmological bounce instead of the initial singularity. The famous formula (24), that leads to the bounce, is valid only for the dust-filled universe with $p=0$ and, therefore, is not adequate for the early universe hot stage when the bounce happens. So, our results are qualitative in nature and must be used to obtain initial values for the hot stage of the Universe.

Holographic cosmological backgrounds, Wilson loop (de)confinement and dilaton singularities

AbstractWe review a construction of holographic geometries dual to ℳ︁ = 4 SYM theory on a Friedmann‐Robertson‐Walker background and in the presence or absence of a gluon condensate and instanton density. We find the most general solution with arbitrary scale factor and show that it is diffeomorphic to topological black holes. We introduce a time‐dependent boundary cosmological constant λ(t) and show energy‐momentum conservation in this background. For constant λ, the deconfinement properties of the temporal Wilson loop are analysed. In most cases the Wilson loop confines throughout cosmological evolution. However, there is an exceptional case which shows a transition from deconfinement at early times to confinement at late times. We classify the presence or absence of horizons, with important implications for the Wilson loop.

The BTZ black hole with a time-dependent boundary

The non-rotating BTZ solution is expressed in terms of coordinates that allow for an arbitrary time-dependent scale factor in the boundary metric. We provide explicit expressions for the coordinate transformation that generates this form of the metric, and determine the regions of the complete Penrose diagram that are covered by our parametrization. This construction is utilized in order to compute the stress–energy tensor of the dual conformal field theory (CFT) on a time-dependent background. We study in detail the expansion of radial null geodesic congruences in the BTZ background, for various forms of the scale factor of the boundary metric. We also discuss the relevance of our construction for the holographic calculation of the entanglement entropy of the dual CFT on time-dependent backgrounds.

Ultraviolet divergences in cosmological correlations

A method is developed for dealing with ultraviolet divergences in calculations of cosmological correlations, which does not depend on dimensional regularization. An extended version of the WKB approximation is used to analyze the divergences in these calculations, and these divergences are controlled by the introduction of Pauli--Villars regulator fields. This approach is illustrated in the theory of a scalar field with arbitrary self-interactions in a fixed flat-space Robertson--Walker metric with arbitrary scale factor $a(t)$. Explicit formulas are given for the counterterms needed to cancel all dependence on the regulator properties, and an explicit prescription is given for calculating finite regulator-independent correlation functions. The possibility of infrared divergences in this theory is briefly considered.

New Gauge Field from Extension of Space Time Parallel Transport of Vector Spaces to the Underlying Number Systems

One way of describing gauge theories in physics is to assign a vector space \(\bar{V}_{x}\) to each space time point x. For each x the field ψ takes values ψ(x) in \(\bar{V}_{x}\). The freedom to choose a basis in each \(\bar{V}_{x}\) introduces gauge group operators and their Lie algebra representations to define parallel transformations between vector spaces. This paper is an exploration of the extension of these ideas to include the underlying scalar complex number fields. Here a Hilbert space, \(\bar{H}_{x}\), as an example of \(\bar{V}_{x}\), and a complex number field, \(\bar{C}_{x}\), are associated with each space time point. The freedom to choose a basis in \(\bar{H}_{x}\) is expanded to include the freedom to choose complex number fields. This expansion is based on the discovery that there exist representations of complex (and other) number systems that differ by arbitrary scale factors. Compensating changes must be made in the basic field operations so that the relevant axioms are satisfied. This results in the presence of a new real valued gauge field A(x). Inclusion of A(x) into covariant derivatives in Lagrangians results in the description of A(x) as a gauge boson for which mass is optional. The great accuracy of QED suggests that the coupling constant of A(x) to matter fields is very small compared to the fine structure constant. Other physical properties of A(x) are not known at present.