High-resolution Ones Research Articles

Wavelet structure-texture-aware super-resolution for pedestrian detection

This study aims to tackle the challenge of detecting pedestrians in low-resolution (LR) images by using super-resolution techniques. The proposed Wavelet Structure-Texture-Aware Super-Resolution (WSTa-SR) method is a novel end-to-end solution that enlarges LR images into high-resolution ones and employs Yolov7 for detection, effectively solving the problems of low detection performance. The LR image is first decomposed into low and high-frequency sub-images with stationary wavelet transform (SWT), which are then processed by different sub-networks to more accurately distinguish pedestrian from background by emphasizing pedestrian features. Additionally, a high-to-low information delivery mechanism (H2LID mechanism) is proposed to transfer the information of high-frequency details to enhance the reconstruction of low-frequency structures. A novel loss function is also introduced that exploits wavelet decomposition properties to further enhance the network’s performance on both image structure reconstruction and pedestrian detection. Experimental results show that the proposed WSTa-SR method can effectively improve pedestrian detection.

Combining optical flow and Swin Transformer for Space-Time video super-resolution

Space–time video super-resolution is a task that aims to interpolate low frame rate, low resolution videos to high frame rate, high resolution ones. While existing Transformer-based methods have achieved results comparable to convolutional neural networks-based methods, the computational cost of Transformer limits its performance with constrained computational resources. Moreover, Swin Transformer may fail to fully exploit the spatio-temporal information of video frames due to the limitation of window size, impeding its effectiveness in handling large motions. To address these limitations, we propose an end-to-end space–time video super-resolution architecture based on optical flow alignment and Swin Transformer. The alignment module is introduced to extract spatio-temporal information from adjacent frames without significantly increasing the computational burden. Additionally, we design a residual convolution layer to enhance the translational invariance of the features extracted by Swin Transformer and introduces additional nonlinear transformations. Experimental results demonstrate that our proposed method achieves superior performance on various benchmark datasets compared to state-of-the-art methods. In terms of Peak Signal-to-Noise Ratio, our method outperforms the state-of-the-art methods by at least 0.15 dB on Vimeo-Medium dataset.

LAMA: LAMOST Medium-Resolution Spectral Analysis Pipeline

The Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST) has obtained more than 23 million spectra, opening an unprecedented opportunity to study stellar physics, as well as the formation and evolution of our Milky Way. In order to obtain the accurate stellar parameters, we develop a LAMOST Medium-Resolution Spectral Analysis Pipeline (LAMA), which estimates the stellar parameters from the LAMOST medium-resolution spectra, including the effective temperature (T eff), surface gravity ( logg ), metallicity ([Fe/H]), radial velocity, and rotational velocity ( vsini ). LAMA estimates these parameters utilizing the template-matching method. The comparison between our results and those from the high-resolution ones, including APOGEE, GALAH, and PASTEL, shows no obvious bias, indicating the reliability of our results. The accuracy of T eff and [Fe/H] can reach 75 K and 0.12 dex, respectively, for the LAMOST Medium-Resolution Spectroscopic Survey (MRS) spectra with a signal-to-noise ratio higher than 10. For dwarfs, the uncertainty of logg is around 0.17 dex, while, for giants, it ranges from 0.18 to 0.30 dex, with the errors decreasing as logg increases. Using LAMA for the LAMOST-MRS spectra, we estimate the stellar parameters of 497,412 stars. This sample will be very helpful for investigating the formation and evolution of our Galaxy.

Low-dose, standard, and high-resolution cone beam computed tomography for alveolar bone measurements related to implant planning: An exvivo study in human specimens.

To evaluate the performance of low-dose cone beam computed tomography (CBCT) protocols with regard to linear bone measurements in the posterior mandible for implant planning compared with higher dose protocols. Forty-two edentulous posterior sites in human cadaveric mandibles were imaged in three CBCT scanners using three or four protocols with varying exposure parameters to achieve lower dose. Co-registration was performed to generate sagittal and cross-sectional image sections representative of the implant site. Three observers measured bone height, from the alveolar crest to the mandibular canal, and width, three mm from the top of the alveolar crest. Intra- and interobserver reproducibility were assessed for the cases rated as nonmeasurable as well as for completed measurements. The measurements were analyzed using paired t-tests for differences among the CBCT protocols and the frequency distribution of nonmeasurable cases with a Pearson Chi-square test. Reproducibility for registering nonmeasurable cases varied among observers; however, no consistent significant differences were found in the frequency distribution of these cases among observers, units, and protocols. Intraclass correlation coefficients (ICC) were >0.9 for all measurements of bone height and width. Mean differences of <0.5 mm were found regardless of protocol; however, one observer did in some cases produce larger differences. Linear bone measurements did not differ significantly and could be performed with excellent reliability, using low-dose CBCT protocols compared with standard and high-resolution ones. Varying approaches for rating nonmeasurable cases were found, indicating differences in diagnostic strategies related to implant planning among observers.

The association between physiological and eye-tracking metrics and cognitive load in drivers: A meta-analysis

Driving performance can be impaired by a high cognitive load of drivers. Thus, it is important to estimate drivers’ cognitive load. Although physiological and eye-tracking metrics have been widely used in many studies to assess cognitive load while driving, conflicts still exist regarding the association between physiological and eye-tracking metrics and different levels of cognitive load. Through a meta-analysis, our study aims to quantify the association between physiological, eye-tracking metrics and cognitive load induced by n-back tasks. A total of 18 articles met the inclusion criteria for the meta-analysis. The results indicate four types of metrics, including the sensitive-to-low ones that can only differentiate the low to medium level of cognitive load (i.e., the power spectrum of θ wave of electroencephalogram at Fp1 channel); high-resolution ones that can differentiate all levels of cognitive load (including pupil size, heart rate, and skin conductance); and low-resolution ones that can only differentiate low and high cognitive load (including the total power spectrum of electrocardiogram, eye blink rate, and respiration rate) and others (the power spectrum of θ wave of electroencephalogram at Fp2 channel). Furthermore, the association between metrics and cognitive load can be modulated by the n-back version, modality of n-back task, automation level, and percentage of male participants. In summary, this study contributes to the literature by quantifying associations between physiological and eye-tracking metrics and different cognitive load levels. Practically, we provide evidence for the selection of physiological and eye-tracking metrics for future driving cognitive load monitoring system design.

Image interpolation with spiking neural network based pixel similarity

Image interpolation is an important topic in the field of image processing. It is defined as the process of transforming low-resolution images into high-resolution ones using image processing methods. Recent studies on interpolation have shown that researchers are focusing on successful interpolation techniques that preserve edge information. Therefore, the edge detection phase plays a vital role in interpolation studies. However, these approaches typically rely on gradient-based linear computations for edge detection. On the other hand, non-linear structures that effectively simulate the human visual system have gained attention. In this study, a non-linear method was developed to detect edge information using a pixel similarity approach. Pixel similarity-based edge detection approach offers both lower computational complexity and more successful interpolation results compared to gradient-based approaches. 1D cubic interpolation was applied to the pixels identified as edges based on pixel similarity, while bicubic interpolation was applied to the remaining pixels. The algorithm was tested on 12 commonly used images and compared with various interpolation techniques. The results were evaluated using metrics such as SSIM and PSNR, as well as visual assessment. The experimental findings clearly demonstrated that the proposed method outperformed other approaches. Additionally, the method offers significant advantages, such as not requiring any parameters and having competitive computational cost.

Super-resolution reconstruction of ultrasound image using a modified diffusion model

Objective. This study aims to perform super-resolution (SR) reconstruction of ultrasound images using a modified diffusion model, designated as the diffusion model for ultrasound image super-resolution (DMUISR). SR involves converting low-resolution images to high-resolution ones, and the proposed model is designed to enhance the suitability of diffusion models for this task in the context of ultrasound imaging. Approach. DMUISR incorporates a multi-layer self-attention (MLSA) mechanism and a wavelet-transform based low-resolution image (WTLR) encoder to enhance its suitability for ultrasound image SR tasks. The model takes interpolated and magnified images as input and outputs high-quality, detailed SR images. The study utilized 1,334 ultrasound images from the public fetal head-circumference dataset (HC18) for evaluation. Main results. Experiments were conducted at 2× , 4× , and 8× magnification factors. DMUISR outperformed mainstream ultrasound SR methods (Bicubic, VDSR, DECUSR, DRCN, REDNet, SRGAN) across all scales, providing high-quality images with clear structures and rich detailed textures in both hard and soft tissue regions. DMUISR successfully accomplished multiscale SR reconstruction while suppressing over-smoothing and mode collapse problems. Quantitative results showed that DMUISR achieved the best performance in terms of learned perceptual image patch similarity, with a significant decrease of over 50% at all three magnification factors (2× , 4× , and 8× ), as well as improvements in peak signal-to-noise ratio and structural similarity index measure. Ablation experiments validated the effectiveness of the MLSA mechanism and WTLR encoder in improving DMUISR’s SR performance. Furthermore, by reducing the number of diffusion steps, the computational time of DMUISR was shortened to nearly one-tenth of its original while maintaining image quality without significant degradation. Significance. This study demonstrates that the modified diffusion model, DMUISR, provides superior performance for SR reconstruction of ultrasound images and has potential in improving imaging quality in the medical ultrasound field.

Evaluation of Essential Dynamics and Fixed-Length Coarse Graining for Multidomain Proteins.

For multiscale modeling of biomolecules, reliable coarse-grained (CG) models can offer great potential to simulate larger temporal and spatial scales than traditional all-atom (AA) models. In this study, we explore the essential dynamics coarse graining (EDCG) and fixed-length coarse graining (FLCG) approaches for constructing highly coarse-grained models for multidomain proteins (MDPs), with 1 to 10 amino acid residues per CG site. In the studies of 13 MDPs, our data indicate that both EDCG and FLCG can preserve the protein dynamics of MDPs. FLCG, which restricts an equal number of residues in each CG site, represents an excellent approximation to EDCG and a straightforward approach for coarse-graining MDPs. Furthermore, FLCG is tested with a class B G-protein-coupled receptor protein, and the agreement with prior experiments suggests its general application to various MDPs in different environments or conditions. Finally, we demonstrate another application of FLCG through progressive backmapping, showcasing the ability to recover from lower-resolution CG models (6 residues/CG site) to higher-resolution ones (1 residue/CG site). These promising outcomes underscore the broad applicability of FLCG to construct highly or ultra-coarse-grained models of complex biomolecules for multiscale simulations.

Correlation Matching Transformation Transformers for UHD Image Restoration

This paper proposes UHDformer, a general Transformer for Ultra-High-Definition (UHD) image restoration. UHDformer contains two learning spaces: (a) learning in high-resolution space and (b) learning in low-resolution space. The former learns multi-level high-resolution features and fuses low-high features and reconstructs the residual images, while the latter explores more representative features learning from the high-resolution ones to facilitate better restoration. To better improve feature representation in low-resolution space, we propose to build feature transformation from the high-resolution space to the low-resolution one. To that end, we propose two new modules: Dual-path Correlation Matching Transformation module (DualCMT) and Adaptive Channel Modulator (ACM). The DualCMT selects top C/r (r is greater or equal to 1 which controls the squeezing level) correlation channels from the max-pooling/mean-pooling high-resolution features to replace low-resolution ones in Transformers, which can effectively squeeze useless content to improve the feature representation in low-resolution space to facilitate better recovery. The ACM is exploited to adaptively modulate multi-level high-resolution features, enabling to provide more useful features to low-resolution space for better learning. Experimental results show that our UHDformer reduces about ninety-seven percent model sizes compared with most state-of-the-art methods while significantly improving performance under different training sets on 3 UHD image restoration tasks, including low-light image enhancement, image dehazing, and image deblurring. The source codes will be made available at https://github.com/supersupercong/UHDformer.

Abstract 2329: Cancer region definition using spatial gene expression patterns by super resolution reconstruction algorithm for spatial transcriptomics data

Abstract Background: Spatially resolved transcriptomics (ST) has enabled a variety of cancer research on the heterogeneity of tumor microenvironment. However, when identifying cancer boundaries based on ST, the limited resolution of ST such as Visium hinders the accurate identification of cancerous regions. To address these issues, we have developed a novel approach for accurately classifying regions of the tumor by using deep image prior (DIP) algorithm to convert Visium data into high-resolution images. Methods: We proposed SuperST, which utilizes deep image prior to transform low-resolution ST data into high-resolution images, providing more accurate spatial marker expression patterns. This method was used to define cancer regions by combining with a pre-trained deep learning algorithm that recognizes spatial patterns. ST data of hepatocellular carcinoma (HCC) patients were used to apply SuperST. Firstly, 400 spatially variable genes were identified. SuperST leverages the deep image prior (DIP) algorithm and a U-Net neural network to transform ST data into high-resolution ones without needing ground-truth high-resolution data. The final high-resolution images of gene expression data were used to extract 512-dimensional feature vectors for each gene via a pre-trained convolutional neural network model (VGG16). Subsequently, we apply K-means clustering to these features to identify transcriptionally distinct tissue regions considering spatial information and compared with the human-labeled cancer regions. Results: The cancer region detection using spatial gene expression data and our algorithm yielded results that substantially align with the human annotations on the H&amp;E images. Furthermore, compared to the cancer definition obtained using the CopyKat and SPACET algorithms, our results were more spatially consistent for cancerous regions as well as providing high-resolution image-level definition of cancer region margin. This indicates that our algorithm better utilizes spatial information and more effectively overcomes the low resolution of ST data. Conclusion: This research demonstrates the effective application of our SuperST algorithm to the challenging problem of cancer region detection only using spatial gene expression data. It suggests that our algorithm could be utilized for a variety of problems in spatial biology in cancer research. Citation Format: Jeongbin Park, Seungho Cook, Dongjoo Lee, Jinyeong Choi, Seongjin Yoo, Hyung-Jun Im, Daeseung Lee, Hongyoon Choi. Cancer region definition using spatial gene expression patterns by super resolution reconstruction algorithm for spatial transcriptomics data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2329.

Stereo attention-based all-in-one super-resolution for robot-assisted minimally invasive surgery.

Robot-assisted minimally invasive surgery (MIS) faces challenges in obtaining high-quality imaging results due to the limited spatial environment. In this paper, we present an all-in-one image super-resolution (SR) algorithm designed to tackle this challenge. By utilizing the stereo information from binocular images, we effectively convert low-resolution images into high-resolution ones. Our model architecture amalgamates the prowess of Convolutional Neural Networks (CNNs) and Transformers, capitalizing on the advantages of both methodologies. To achieve super-resolution across all scale factors, we employ a trainable upsampling module within our proposed network. We substantiate the effectiveness of our method through extensive quantitative and qualitative experiments. The results of our evaluations provide strong evidence supporting the superior performance of our approach in enhancing the quality of surgical images. Our method improves the resolution and thus the overall image quality, which allows the surgeon to perform precise operations conveniently. Simultaneously, it also facilitates the scaling of the region of interest (ROI) to achieve high-quality visualization during surgical procedures. Furthermore, it has the potential to enhance the image quality during telesurgery.

M2Trans: Multi-Modal Regularized Coarse-to-Fine Transformer for Ultrasound Image Super-Resolution.

Ultrasound image super-resolution (SR) aims to transform low-resolution images into high-resolution ones, thereby restoring intricate details crucial for improved diagnostic accuracy. However, prevailing methods relying solely on image modality guidance and pixel-wise loss functions struggle to capture the distinct characteristics of medical images, such as unique texture patterns and specific colors harboring critical diagnostic information. To overcome these challenges, this paper introduces the Multi-Modal Regularized Coarse-to-fine Transformer (M2Trans) for Ultrasound Image SR. By integrating the text modality, we establish joint image-text guidance during training, leveraging the medical CLIP model to incorporate richer priors from text descriptions into the SR optimization process, enhancing detail, structure, and semantic recovery. Furthermore, we propose a novel coarse-to-fine transformer comprising multiple branches infused with self-attention and frequency transforms to efficiently capture signal dependencies across different scales. Extensive experimental results demonstrate significant improvements over state-of-the-art methods on benchmark datasets, including CCA-US, US-CASE, and our newly created dataset MMUS1K, with a minimum improvement of 0.17dB, 0.30dB, and 0.28dB in terms of PSNR. Our code and dataset will be available at: https://github.com/eezkni/M2Trans.

EVALUATION OF PSNR VALUE FOR IMAGE SUPER-RESOLUTION USING DEEP LEARNING

Deep learning is a branch of study that simulates the human brain and allows learning from large data. In recent years, deep learning techniques have become more prevalent in a range of applications. These techniques have been widely employed in image processing, especially for object detection, image augmentation, and super-resolution. Single-image super-resolution, which converts low-resolution images into high-resolution ones, is a significant application of deep learning. Outstanding perceptual and antagonistic outcomes have been achieved using multiple single-image super-resolution models. In this study, several noteworthy methods, including Super Resolution with Convolutional Neural Network, Super Resolution Residual Network, Super Resolution Generative Adversarial Network, and Enhanced Super Resolution Generative Adversarial Network are used for single image super-resolution. Each layer in these deep learning models carries out certain operations on low-resolution images to transform them into high-resolution equivalents. As a result, many high-resolution images can be produced from a single low-resolution image. We created a dataset of random images for our research. The original low-resolution images are compared to the high-resolution images that were created using a deep learning technique. Peak Signal-to-Noise Ratio and the Structural Similarity Index are used in the comparison. Additionally, the evaluation process includes computing the ground truth value. The highest similarity indexed is achieved by VGG54 with 96% of similarity.

Forest Single-Frame Remote Sensing Image Super-Resolution Using GANs

Generative Adversarial Networks (GANs) possess remarkable fitting capabilities and play a crucial role in the field of computer vision. Super-resolution restoration is the process of converting low-resolution images into high-resolution ones, providing more detail and information. This is of paramount importance for monitoring and managing forest resources, enabling the surveillance of vegetation, wildlife, and potential disruptive factors in forest ecosystems. In this study, we propose an image super-resolution model based on Generative Adversarial Networks. We incorporate Multi-Scale Residual Blocks (MSRB) as the core feature extraction component to obtain image features at different scales, enhancing feature extraction capabilities. We introduce a novel attention mechanism, GAM Attention, which is added to the VGG network to capture more accurate feature dependencies in both spatial and channel domains. We also employ the adaptive activation function Meta ACONC and Ghost convolution to optimize training efficiency and reduce network parameters. Our model is trained on the DIV2K and LOVEDA datasets, and experimental results indicate improvements in evaluation metrics compared to SRGAN, with a PSNR increase of 0.709/2.213 dB, SSIM increase of 0.032/0.142, and LPIPS reduction of 0.03/0.013. The model performs on par with Real-ESRGAN but offers significantly improved speed. Our model efficiently restores single-frame remote sensing images of forests while achieving results comparable to state-of-the-art methods. It overcomes issues related to image distortion and texture details, producing forest remote sensing images that closely resemble high-resolution real images and align more closely with human perception. This research has significant implications on a global scale for ecological conservation, resource management, climate change research, risk management, and decision-making processes.

Coupled discriminative manifold alignment for low-resolution face recognition

In practical applications, due to a long distance between the monitored population and monitoring equipment, the face images or human pose captured by the cameras often incur low-resolution (LR), small size, and poor quality, which leads to extreme difficulty in directly matching an LR face with the high-resolution (HR) ones in the gallery. In this paper, we propose a novel coupled discriminative manifold alignment (CDMA) method for LR face recognition. Specifically, in the training stage the principal component analysis (PCA) is first used to reduce the dimensional gap between LR and HR facial features. Next the LR face images and the corresponding HR face images are converted into a common shared feature subspace by learning two linear mappings in a supervised manner, where the neighborhood samples within the same class and from different classes are jointly exploited to align the manifold structures of LR and HR faces. In the test stage, for a given LR face in the probe set, two learned coupled mappings (CMs) are applied to match the HR images in the gallery set through the correlative metric. Thorough experimental results on three representative face databases verify the effectiveness of the proposed method in comparing with other state-of-the-art competitors. In particular, the proposed method is capable of yielding more competitive recognition performance than other predecessors when lower dimensional feature subspaces are applied to match the expected HR faces.

Fourier Ptychographic Microscopic Reconstruction Method Based on Residual Hybrid Attention Network.

Fourier ptychographic microscopy (FPM) is a novel technique for computing microimaging that allows imaging of samples such as pathology sections. However, due to the influence of systematic errors and noise, the quality of reconstructed images using FPM is often poor, and the reconstruction efficiency is low. In this paper, a hybrid attention network that combines spatial attention mechanisms with channel attention mechanisms into FPM reconstruction is introduced. Spatial attention can extract fine spatial features and reduce redundant features while, combined with residual channel attention, it adaptively readjusts the hierarchical features to achieve the conversion of low-resolution complex amplitude images to high-resolution ones. The high-resolution images generated by this method can be applied to medical cell recognition, segmentation, classification, and other related studies, providing a better foundation for relevant research.

A Machine Learning-Assisted Inversion Method for Solving Biomedical Imaging Based on Semi-Experimental Data

Machine learning approaches have been extensively utilized in the field of inverse scattering problems. Typically, the training dataset is generated synthetically using ideal radiation sources such as plane waves or cylindrical waves. However, the testing data often consist of experimental data that take into account the antenna port couplings and waveform distortions within the system. While noise can be artificially added to synthetic data, it may not accurately represent the real experimental noise. Consequently, the application of machine learning-assisted inversion techniques may encounter challenges when the training dataset differs significantly from the experimental data. In this paper, we propose an experimental system specifically designed for human body imaging. A semi-experimental training dataset is constructed using full-wave simulation software, incorporating the relative permittivities of common human tissues. Furthermore, the system noise is meticulously considered through full-wave simulation, enhancing the authenticity of the dataset. A back-propagation scheme is firstly employed to obtain the rough reconstructed images. Then, the U-net convolutional neural network (CNN) is employed to map these rough images to high-resolution ones. Numerical results demonstrate that, in comparison to networks trained solely on synthetic data, the network trained using semi-experimental data achieves superior reconstruction results with lower errors and improved image quality.

A high resolution compact all-fiber spectrometer based on periodic refractive index modulation

The realization of a miniaturized spectrometer with high resolution is highly desired but is still a big challenge. Although all-fiber spectrometers based on speckle detection show their great potential for high resolution ones, their long fiber lengths set the greatest obstacle for the miniaturized design. Here, we demonstrate a compact all-fiber speckle spectrometer by using cascading coreless fibers and photonic crystal fibers. A unique cascaded structure readily excites more guided modes, in which the speckle patterns are formed by modal interferences. Using only a 10 cm-long fiber with 20-segment spliced elements, a resolution of 0.03 nm over a bandwidth from 1540 to 1560 nm is achieved. The spectral resolution is comparable to that of a 2 m multimode fiber spectrometer and approximately 20 times higher than that of the same length multimode fiber. Narrow linewidth and broadband spectra are individually reconstructed to demonstrate the excellent performance of the spectrometer. The proposed processing technique of the dispersive element is versatile, reproducible, and controllable, promising for different application scenarios.

Downscaling long lead time daily rainfall ensemble forecasts through deep learning

Skilful and localised daily weather forecasts for upcoming seasons are desired by climate-sensitive sectors. Various General circulation models routinely provide such long lead time ensemble forecasts, also known as seasonal climate forecasts (SCF), but require downscaling techniques to enhance their skills from historical observations. Traditional downscaling techniques, like quantile mapping (QM), learn empirical relationships from pre-engineered predictors. Deep-learning-based downscaling techniques automatically generate and select predictors but almost all of them focus on simplified situations where low-resolution images match well with high-resolution ones, which is not the case in ensemble forecasts. To downscale ensemble rainfall forecasts, we take a two-step procedure. We first choose a suitable deep learning model, very deep super-resolution (VDSR), from several outstanding candidates, based on an ensemble forecast skill metric, continuous ranked probability score (CRPS). Secondly, via incorporating other climate variables as extra input, we develop and finalise a very deep statistical downscaling (VDSD) model based on CRPS. Both VDSR and VDSD are tested on downscaling 60 km rainfall forecasts from the Australian Community Climate and Earth-System Simulator Seasonal model version 1 (ACCESS-S1) to 12 km with lead times up to 217 days. Leave-one-year-out testing results illustrate that VDSD has normally higher forecast accuracy and skill, measured by mean absolute error and CRPS respectively, than VDSR and QM. VDSD substantially improves ACCESS-S1 raw forecasts but does not always outperform climatology, a benchmark for SCFs. Many more research efforts are required on downscaling and climate modelling for skilful SCFs.

Super-Resolution Techniques in Photogrammetric 3D Reconstruction from Close-Range UAV Imagery

Current Multi-View Stereo (MVS) algorithms are tools for high-quality 3D model reconstruction, strongly depending on image spatial resolution. In this context, the combination of image Super-Resolution (SR) with image-based 3D reconstruction is turning into an interesting research topic in photogrammetry, around which however only a few works have been reported so far in the literature. Here, a thorough study is carried out on various state-of-the-art image SR techniques to evaluate the suitability of such an approach in terms of its inclusion in the 3D reconstruction process. Deep-learning techniques are tested here on a UAV image dataset, while the MVS task is then performed via the Agisoft Metashape photogrammetric tool. The data under experimentation are oblique cultural heritage imagery. According to results, point clouds from low-resolution images present quality inferior to those from upsampled high-resolution ones. The SR techniques HAT and DRLN outperform bicubic interpolation, yielding high precision/recall scores for the differences of reconstructed 3D point clouds from the reference surface. The current study indicates spatial image resolution increased by SR techniques may indeed be advantageous for state-of-the art photogrammetric 3D reconstruction.