Angular Superresolution Research Articles

Light field angular super-resolution by view-specific queries

Light field angular super-resolution by view-specific queries

Spatial–angular–epipolar transformer for light field spatial and angular super-resolution

Transformer-based light field (LF) super-resolution (SR) methods have recently achieved significant performance improvements due to global feature modeling by self-attention mechanisms. However, as a method designed for natural language processing, 4D LFs are reshaped into 1D sequences with an immense set of tokens, which results in a quadratic computational complexity cost. In this paper, a spatial–angular–epipolar swin transformer (SAEST) is proposed for spatial and angular SR (SASR), which sufficiently extracts SR information in the spatial, angular, and epipolar domains using local self-attention with shifted windows. Specifically, in SAEST, a spatial swin transformer and an angular standard transformer are firstly cascaded to extract spatial and angular SR features, separately. Then, the extracted SR feature is reshaped into the epipolar plane image pattern and fed into an epipolar swin transformer to extract the spatial–angular correlation information. Finally, several SAEST blocks are cascaded in a Unet framework to extract multi-scale SR features for SASR. Experiment results indicate that SAEST is a fast transformer-based SASR method with less running time and GPU consumption and has outstanding performance on simulated and real-world public datasets.

Light field angular super resolution based on residual channel attention and classification up-sampling

Light field angular super resolution based on residual channel attention and classification up-sampling

GLGNet: light field angular superresolution with arbitrary interpolation rates

Acquiring high-resolution light fields (LFs) is expensive. LF angular superresolution aims to synthesize the required number of views from a given sparse set of spatially high-resolution images. Existing methods struggle with sparsely sampled LFs captured with large baselines. Some methods rely on depth estimation and view reprojection, and are sensitive to textureless and occluded regions. Other non-depth based methods suffer from aliasing or blurring effects due to the large disparity. In addition, most methods require specific models for different interpolation rates, which reduces their flexibility in practice. In this paper, we propose a learning framework that overcomes these challenges by exploiting the global and local structures of LFs. Our framework includes aggregation across both the angular and spatial dimensions to fully exploit the input data and a novel bilateral upsampling module that upsamples each epipolar plane image while better preserving its local parallax structure. Furthermore, our method predicts the weights of the interpolation filters based on both subpixel offset and range difference, allowing angular superresolution at different rates with a single model. We show that our non-depth based method outperforms the state-of-the-art methods in terms of handling large disparities and flexibility on both real-world and synthetic LF images.

A modified algebraic method of mathematical signal processing in radar problems

A new method for approximate solution of ill-posed inverse problems of reconstructing images of objects with angular resolution exceeding the Rayleigh criterion is proposed and justified, i.e. with super resolution. Angular super-resolution allows you to obtain images of objects with increased clarity and distinguish previously invisible details of images of complex objects. In addition, on this basis, the probability of correct solutions to problems of object recognition and identification increases. Mathematically, the problem is reduced to solving the linear Fredholm integral equation of the first kind of convolution type. Solutions are sought with additional conditions in the form of restrictions on the location and size of the desired radiation source, which makes it possible to regularize the problem. The method is a development of one of the parameterization methods the algebraic method. The solution is sought in the form of a representation of the desired function in the area where the source is located in the form of a series expansion over the input sequence of orthogonal functions with unknown coefficients. Thus, the inverse problem is parameterized and reduced to searching for expansion coefficients. The presented method is based on the use of a priori information about the localization area of the radiation source, or on an estimate of the location and size of this area obtained by scanning the viewing sector with a goniometer system. Using zero values of the function describing the source outside this region, for systems based on antenna arrays it is possible to find tens and even hundreds of expansion coefficients of the desired function in a Fourier series. The solution is constructed in the form of an iterative process with a consistent increase in the number of functions used in the expansion until the solution remains stable. The adequacy and stability of the solutions was verified during numerical experiments using a mathematical model. The results of numerical studies show that the presented methods of digital processing of received signals make it possible to achieve an effective angular resolution 3–10 times higher than the Rayleigh criterion. The proposed method makes it possible to miniaturize the antenna system without degrading its characteristics. Compared to known ones, it is relatively simple, which allows it to be used by systems in real time.

Semi-supervised angular super-resolution method for autostereoscopic 3D surface measurement.

Autostereoscopic 3D measuring systems are an accurate, rapid, and portable method for in situ measurements. These systems use a micro-lens array to record 3D information based on the light-field theory. However, the spatial-angular-resolution trade-off curtails their performance. Although learning models were developed for super-resolution, the scarcity of data hinders efficient training. To address this issue, a novel, to the best of our knowledge, semi-supervised learning paradigm for angular super-resolution is proposed for data-efficient training, benefiting both autostereoscopic and light-field devices. A convolutional neural network using motion estimation is developed for a view synthesis. Subsequently, a high-angular-resolution autostereoscopic system is presented for an accurate profile reconstruction. Experiments show that the semi-supervision enhances view reconstruction quality, while the amount of training data required is reduced by over 69%.

Angular Superresolution for Forward-Looking Scanning Radar With Pulse Interference Using Cross-Domain Low-Rank and Sparse Optimization

Angular Superresolution for Forward-Looking Scanning Radar With Pulse Interference Using Cross-Domain Low-Rank and Sparse Optimization

Disparity Enhancement-based Light Field Angular Super-Resolution

Disparity Enhancement-based Light Field Angular Super-Resolution

Real Aperture Radar Angular Super-Resolution Imaging Using Modified Smoothed L0 Norm with a Regularization Strategy

Restricted by the ill-posed antenna measurement matrix, the conventional smoothed L0 norm algorithm (SL0) fails to enable direct real aperture radar angular super-resolution imaging. This paper proposes a modified smoothed L0 norm (MSL0) algorithm to address this issue. First, as the pseudo-inverse of the ill-posed antenna measurement matrix is required to set the initial values and calculate the gradient projection, a regularization strategy is employed to relax the ill-posedness. Based on the regularization strategy, the proposed MSL0 algorithm can avoid noise amplification when faced with the ill-posed antenna measurement matrix of real aperture radar. Additionally, to prevent local minima problems, we introduce a hard thresholding operator, based on which the proposed MSL0 algorithm can accurately reconstruct sparse targets. Simulations and experimental results verify the performance of the proposed MSL0 algorithm.

Adaptive pixel aggregation for joint spatial and angular super-resolution of light field images

Recent advancements in Light Field (LF) super-resolution (SR) have significantly enhanced the spatial size and angular sampling rate in LF imagery. Typically, most methods treat spatial and angular SR as separate tasks. However, the two tasks are closely related and can be jointly optimized by leveraging the intrinsic high-dimensional property of LF images. Existing methods for joint spatial and angular SR of LF images also suffer from the limitations of disparity-based warping operations. To address these challenges, we propose a novel one-stage approach for directly synthesizing densely-sampled high-resolution LF images from sparsely-sampled low-resolution observations. Specifically, we propose to adaptively aggregate advantageous pixels from different sub-aperture images of input LF for each novel view interpolation. This adaptive strategy enables our method to effectively incorporate the spatial and angular correlations of input views. Additionally, to enhance the parallax structure of reconstructed LF images, we propose a feature separation and interaction module to refine the intermediate features. Experimental results on public datasets demonstrate that our proposed method can achieve state-of-the-art performance both numerically and visually compared with previous methods. Our codes are available at https://github.com/GaoshengLiu/LFSR.

An Algorithm of Angular Superresolution Using the Cholesky Decomposition and Its Implementation Based on Parallel Computing Technology

An Algorithm of Angular Superresolution Using the Cholesky Decomposition and Its Implementation Based on Parallel Computing Technology

Two-Dimensional Angular Super-Resolution for Airborne Real Aperture Radar by Fast Conjugate Gradient Iterative Adaptive Approach

Two-Dimensional Angular Super-Resolution for Airborne Real Aperture Radar by Fast Conjugate Gradient Iterative Adaptive Approach

Light field angular super-resolution based on intrinsic and geometric information

Light field imaging can encode abundant scene information, including the intensity and direction of light rays, into 4D light field images. However, the limited number of sensors in commercial light field cameras leads to a trade-off between spatial and angular resolutions. In this paper, an angular super-resolution framework is proposed to synthesize new views and overcome hardware restrictions. First, light field intrinsic feature convolution is proposed to extract intrinsic information, i.e., scene content, complete view correlations, and epipolar structures. Consequently, spatial, angular, and cross-domain information can be preserved in the extracted features. Second, the spatial–angular and depth streams are proposed based on the light field intrinsic feature convolution to synthesize high angular resolution light fields. The spatial–angular stream utilizes the light field intrinsic information to improve the angular resolution, whereas the depth stream disentangles the geometric information from the extracted light field intrinsic features, which is used to warp the given sub-aperture images to the new view positions. Both streams can synthesize high-quality intermediate results, where the intrinsic and geometric information are utilized separately. Finally, a confidence-based stream fusion module is proposed to fuse the outputs from the two streams, achieving the joint employment of light field intrinsic and geometric information and solving the problem of insufficient information exploration in the current methods. We conduct a series of experiments to validate the effectiveness of each component in the framework and demonstrate that our method can achieve state-of-the-art performance in various scenes.

Two-Dimensional Problems of Achieving Super-Resolution in Radar and Remote Sensing

A signal processing technique is proposed that allows one to reconstruct two-dimensional radio images of objects with angular super-resolution. The method of double orthogonalization is justified, which allows one to increase the level of the achieved super-resolution. The limiting possibilities of obtaining adequatesolutions to inverse problems in the presence of a priori information about the solution are shown.

Geometry-assisted multi-representation view reconstruction network for Light Field image angular super-resolution

Light Field (LF) imaging enables many attractive applications since scene angular and spatial information can be captured simultaneously. However, the limited angular resolution hinders its development. To alleviate this problem, in this paper, we put forward a geometry-assisted multi-representation view reconstruction network for LF angular super-resolution. There are two stages in our method, namely multi-representation view reconstruction and geometry-assisted refinement. The former fully utilizes the LF lenslet image, sub-aperture image array, and pseudo video sequence representations to synthesize a high-angular-resolution dense LF image stack by adequately exploring the structural characteristics of such three representations. The latter aims to exploit the intra-LF spatial–angular information and inter-LF geometry information contained in the derived dense LF image stack to further promote the reconstruction performance. Specifically, a bidirectional view stack structure is constituted first, which is then fed into the proposed geometry-aware refinement network to allow adequate interactions between the intra-LF spatial–angular information and inter-LF geometry information. Comprehensive angular reconstruction experiments on both real-world and synthetic LF scenes validate the effectiveness of the proposed method. Furthermore, extensible applications on scene depth estimation also demonstrate that the proposed method can recover more texture details.

Algebraic Methods for Achieving Super-Resolution by Digital Antenna Arrays

The actual modern problem of developing and improving measurement and observation systems (including robotic ones) is to increase the volume and quality of the information received. Increasing the angle resolution to values significantly exceeding the Rayleigh criterion, i.e. achieving super-resolution is one of important ways to solve the problem. Angular super-resolution which makes it possible to detail images of research objects and their individual fragments, improves the quality of solutions to detection, recognition and identification problems, increases the range of such systems. In many papers methods developed by authors to achieve a super-resolution based on approximate solutions of inverse problems in the form of Fredholm integral equation of the first kind of convolution type called algebraic are presented. The methods used, as well as their varieties, make it possible to reduce solutions of inverse problems posed to solving sets of linear algebraic equations (SLAE). This paper presents results of further improvement of algebraic methods based on intelligent analysis of received signals. It is shown that their use in systems based on digital antenna arrays makes it possible to increase the achieved degree of exceeding the Rayleigh criterion. In the course of numerical experiments with a mathematical model, the stability of the solutions obtained and their adequacy were confirmed. The numerical results obtained open the following possibilities: (1) obtaining images of studied objects with a resolution exceeding the Rayleigh criterion by 4 to 10 times, (2) determining the angular coordinates of individual small-sized objects as part of multi-element complex objects (group targets), (3) clarifying boundaries of extended objects and their individual elements, (4) localizing individual bright objects on a smoothly inhomogeneous reflective background. Applying presented new methods does not require a significant computing power, what allows you to work in a real time mode using relatively simple and inexpensive computing devices. The ways of further improvement of presented algebraic methods for solving applied inverse problems are described.

Angular Super-Resolution of Multi-Channel APAR in Interference Environments

Aiming to resolve azimuth-dense targets in interference environments, the radar needs to have the ability of single snap echo angular super-resolution with anti-interference. To solve the problem, the angular super-resolution algorithm based on single snap echo while anti-interference with blocking matrix method is studied for active phased array radar (APAR) in this paper. Since the super-resolution ability of the conventional MUSIC algorithm and iterative adaptive algorithm (IAA) algorithm are limited in single snap echo, the iterative re-weighted least squares (IRLS) algorithm under p-norm constraint is proposed. Further, the near main lobe interference suppression ability is enhanced by the adaptive diagonal loading method. The performance differences of IAA and IRLS algorithms for single-target detection and double-target angular super-resolution are analyzed in detail by numerical simulation on three scenes of no interference, side-lobe interference, and near-main-lobe interference. The simulation results show that the proposed algorithm can effectively solve the problem of target angle estimation and super-resolution based on single sample echo in an interference environment, including near main-lobe interference.

Efficient Light Field Angular Super-Resolution With Sub-Aperture Feature Learning and Macro-Pixel Upsampling

The acquisition of densely-sampled light field (LF) images is costly, which hampers the applications of LF imaging technology in 3D reconstruction, digital refocusing, virtual reality, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etc</i> . To mitigate the obstacle, various approaches have been proposed to reconstruct densely-sampled LF images from sparsely-sampled ones. However, most existing methods still suffer from the non-Lambertian effect and large disparity issue. In this paper, we embrace the challenges by introducing a new paradigm for LF angular super-resolution (SR), which first explores the multi-scale spatial-angular correlations on the sparse sub-aperture images (SAIs) and then performs angular SR on macro-pixel features. In this way, we propose an efficient LF angular SR network, termed as EASR, with simple 3D (2D) CNNs and reshaping operations. The proposed EASR can extract effective feature representations on SAIs and can handle large disparities well by performing angular SR on macro-pixel features. Extensive comparisons with state-of-the-art methods demonstrate that our method achieves superior performance visually and quantitatively. Furthermore, our method achieves efficient angular SR by providing an excellent tradeoff between reconstruction performance and inference time. Our code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/GaoshengLiu/LF-EASR</uri>

Disentangling Light Fields for Super-Resolution and Disparity Estimation.

Light field (LF) cameras record both intensity and directions of light rays, and encode 3D scenes into 4D LF images. Recently, many convolutional neural networks (CNNs) have been proposed for various LF image processing tasks. However, it is challenging for CNNs to effectively process LF images since the spatial and angular information are highly inter-twined with varying disparities. In this paper, we propose a generic mechanism to disentangle these coupled information for LF image processing. Specifically, we first design a class of domain-specific convolutions to disentangle LFs from different dimensions, and then leverage these disentangled features by designing task-specific modules. Our disentangling mechanism can well incorporate the LF structure prior and effectively handle 4D LF data. Based on the proposed mechanism, we develop three networks (i.e., DistgSSR, DistgASR and DistgDisp) for spatial super-resolution, angular super-resolution and disparity estimation. Experimental results show that our networks achieve state-of-the-art performance on all these three tasks, which demonstrates the effectiveness, efficiency, and generality of our disentangling mechanism. Project page: https://yingqianwang.github.io/DistgLF/.

Angular Superresolution of Real Aperture Radar for Target Scale Measurement Using a Generalized Hybrid Regularization Approach

Angular Superresolution of Real Aperture Radar for Target Scale Measurement Using a Generalized Hybrid Regularization Approach