High-resolution Ones Research Articles

SRARNet: A Unified Framework for Joint Superresolution and Aircraft Recognition

Aircraft recognition in high-resolution remote sensing images has rapidly progressed with the advance of convolutional neural networks (CNNs). However, the previous CNN-based methods may not work well for recognizing aircraft in low-resolution remote sensing images because the blurred aircraft in these images offer insufficient details to distinguish them from similar types of targets. An intuitive solution is to introduce superresolution preprocessing. However, conventional superresolution methods mainly focus on reconstructing natural images with detailed texture rather than constructing a high-resolution object with strong discriminative information for the recognition task. To address these problems, we propose a unified framework for joint superresolution and aircraft recognition (Joint-SRARNet) that tries to improve the recognition performance by generating discriminative, high-resolution aircraft from low-resolution remote sensing images. Technically, this network integrates superresolution and recognition tasks into the generative adversarial network (GAN) framework through a joint loss function. The generator is constructed as a joint superresolution and refining subnetwork that can upsample small blurred images into high-resolution ones and restore high-frequency information. In the discriminator, we introduce a new classification loss function that forces the discriminator to distinguish between real and fake images while recognizing the type of aircraft. In addition, the classification loss function is back-propagated to the generator to obtain high-resolution images with discriminative information for easier recognition. Extensive experiments on the challenging multitype aircraft of remote sensing images (MTARSI) dataset demonstrate the effectiveness of the proposed method in restoring a clear super-resolved image from a small blurred image and significant improvement in the recognition performance. To our knowledge, this is the first work on joint superresolution and aircraft recognition tasks.

USE OF SUPER-RESOLUTION IN TEXT-DETECTION

USE OF SUPER-RESOLUTION IN TEXT-DETECTION

Lightweight image super-resolution with feature enhancement residual network

Recently, single image super-resolution (SR) methods based on deep convolutional neural network (CNN) have demonstrated remarkable progress. The essence of most CNN-based models is to learn the non-linear mapping between low-resolution patches and corresponding high-resolution ones. However, numerous convolutions are applied to implement this mapping, which directly contributes to large model sizes and huge graphics memory consumption. In this paper, we propose a lightweight feature enhancement residual network (FERN) to achieve prominent performance by incorporating lightweight non-local operations into the residual block. By taking advantage of utilizing this non-locally enhanced residual block, the proposed model can capture long-range dependencies. For further improving performance, we design the structure-aware channel attention layer that explicitly boosts feature maps with more structural and textural details. Extensive experiments suggest that the proposed approach performs favorably against the state-of-the-art SR algorithms in terms of visual quality and inference time.

PGA-Net: Pyramid Feature Fusion and Global Context Attention Network for Automated Surface Defect Detection

Surface defect detection is a critical task in industrial production process. Nowadays, there are lots of detection methods based on computer vision and have been successfully applied in industry, they also achieved good results. However, achieving full automation of surface defect detection remains a challenge, due to the complexity of surface defect, in intraclass. While the defects between interclass contain similar parts, there are large differences in appearance of the defects. To address these issues, this article proposes a pyramid feature fusion and global context attention network for pixel-wise detection of surface defect, called PGA-Net. In the framework, the multiscale features are extracted at first from backbone network. Then the pyramid feature fusion module is used to fuse these features into five resolutions through some efficient dense skip connections. Finally, the global context attention module is applied to the fusion feature maps of adjacent resolution, which allows effective information propagate from low-resolution fusion feature maps to high-resolution fusion ones. In addition, the boundary refinement block is added to the framework to refine the boundary of defect and improve the result of the prediction. The final prediction is the fusion of the five resolutions fusion feature maps. The results of evaluation on four real-world defect datasets demonstrate that the proposed method outperforms the state-of-the-art methods on mean intersection of union and mean pixel accuracy (NEU-Seg: 82.15%, DAGM 2007: 74.78%, MT_defect: 71.31%, Road_defect: 79.54%).

Look One and More: Distilling Hybrid Order Relational Knowledge for Cross-Resolution Image Recognition

In spite of great success in many image recognition tasks achieved by recent deep models, directly applying them to recognize low-resolution images may suffer from low accuracy due to the missing of informative details during resolution degradation. However, these images are still recognizable for subjects who are familiar with the corresponding high-resolution ones. Inspired by that, we propose a teacher-student learning approach to facilitate low-resolution image recognition via hybrid order relational knowledge distillation. The approach refers to three streams: the teacher stream is pretrained to recognize high-resolution images in high accuracy, the student stream is learned to identify low-resolution images by mimicking the teacher's behaviors, and the extra assistant stream is introduced as bridge to help knowledge transfer across the teacher to the student. To extract sufficient knowledge for reducing the loss in accuracy, the learning of student is supervised with multiple losses, which preserves the similarities in various order relational structures. In this way, the capability of recovering missing details of familiar low-resolution images can be effectively enhanced, leading to a better knowledge transfer. Extensive experiments on metric learning, low-resolution image classification and low-resolution face recognition tasks show the effectiveness of our approach, while taking reduced models.

FISR: Deep Joint Frame Interpolation and Super-Resolution with a Multi-Scale Temporal Loss

Super-resolution (SR) has been widely used to convert low-resolution legacy videos to high-resolution (HR) ones, to suit the increasing resolution of displays (e.g. UHD TVs). However, it becomes easier for humans to notice motion artifacts (e.g. motion judder) in HR videos being rendered on larger-sized display devices. Thus, broadcasting standards support higher frame rates for UHD (Ultra High Definition) videos (4K@60 fps, 8K@120 fps), meaning that applying SR only is insufficient to produce genuine high quality videos. Hence, to up-convert legacy videos for realistic applications, not only SR but also video frame interpolation (VFI) is necessitated. In this paper, we first propose a joint VFI-SR framework for up-scaling the spatio-temporal resolution of videos from 2K 30 fps to 4K 60 fps. For this, we propose a novel training scheme with a multi-scale temporal loss that imposes temporal regularization on the input video sequence, which can be applied to any general video-related task. The proposed structure is analyzed in depth with extensive experiments.

Non-uniformity correction for medium wave infrared focal plane array-based compressive imaging.

As a super-resolution imaging method, high-resolution medium wave infrared (MWIR) images can be obtained from a low-resolution focal plane array-based (FPA) sensor using compressive imaging (CI) technology. As a common problem in MWIR FPA imaging, the non-uniformity reduces image quality, which is turning worse in MWIR FPA CI. This paper investigates the source of the non-uniformity of MWIR FPA CI, both in the captured low-resolution MWIR images and in the reconstructed high-resolution ones. According to the system model and the image super-resolution computation process of FPA CI, we propose a calibration-based non-uniformity correction (NUC) method for MWIR FPA CI. Based on the actual MWIR FPA CI system, the effectiveness and practicability of the proposed NUC method are verified, obtaining better results than the traditional method. According to the theoretical analysis and experimental results, the particularities of the non-uniformity in MWIR FPA CI are discovered and discussed, which have certain great guiding significance and practical value.

3D Face Model Super-Resolution Based on Radial Curve Estimation

Consumer depth cameras bring about cheap and fast acquisition of 3D models. However, the precision and resolution of these consumer depth cameras cannot satisfy the requirements of some 3D face applications. In this paper, we present a super-resolution method for reconstructing a high resolution 3D face model from a low resolution 3D face model acquired from a consumer depth camera. We used a group of radial curves to represent a 3D face. For a given low resolution 3D face model, we first extracted radial curves on it, and then estimated their corresponding high resolution ones by radial curve matching, for which Dynamic Time Warping (DTW) was used. Finally, a reference high resolution 3D face model was deformed to generate a high resolution face model by using the radial curves as the constraining feature. We evaluated our method both qualitatively and quantitatively, and the experimental results validated our method.

Image super‐resolution using conditional generative adversarial network

Recently, extensive studies on a generative adversarial network (GAN) have made great progress in single image super-resolution (SISR). However, there still exists a significant difference between the reconstructed high-frequency and the real high-frequency details. To address this issue, this study presents an SISR approach based on conditional GAN (SRCGAN). SRCGAN includes a generator network that generates super-resolution (SR) images and a discriminator network that is trained to distinguish the SR images from ground-truth high-resolution (HR) ones. Specifically, the discriminator network uses the ground-truth HR image as a conditional variable, which guides the network to distinguish the real images from the SR images, facilitating training a more stable generator model than GAN without this guidance. Furthermore, a residual-learning module is introduced into the generator network to solve the issue of detail information loss in SR images. Finally, the network is trained in an end-to-end manner by optimizing a perceptual loss function. Extensive evaluations on four benchmark datasets including Set5, Set14, BSD100, and Urban100 demonstrate the superiority of the proposed SRCGAN over state-of-the-art methods in terms of PSNR, SSIM, and visual effect.

High external-efficiency nanofocusing for lens-free near-field optical nanoscopy

Efficient, broadband illumination and collection through a nanometre-sized hotspot carried by a scanning probe will endow light–matter interaction research with nanoscale spatial information. However, near-field scanning optical microscopy probes, particularly the high-resolution ones, demand cumbersome optics but can only concentrate less than 10−3 of the incident light, which has limited its applications. Here, we report a two-step sequential broadband nanofocusing technique with an external nanofocusing efficiency of ~50% over nearly all the visible range on a fibre-coupled nanowire scanning probe, which is capable of both light delivery and spectrum collection with nanoscale spatial resolution. By integrating this with a basic portable scanning tunnelling microscope, we have demonstrated lens-free tip-enhanced Raman spectroscopy and achieved 1 nm spatial resolution. The high performance and vast versatility offered by this fibre-based nanofocusing technique allow for the easy incorporation of nano-optical microscopy into various existing measurement platforms. A two-step sequential broadband nanofocusing technique offers an external efficiency of ~50% over nearly all the visible range on a fibre-coupled plasmonic nanowire probe. Its integration with a scanning tunnelling microscope realizes lens-free tip-enhanced Raman spectroscopy with 1 nm spatial resolution.

Speeding Up the Line-Scan Raman Imaging of Living Cells by Deep Convolutional Neural Network.

Raman imaging is a promising technique that allows the spatial distribution of different components in the sample to be obtained using the molecular fingerprint information on individual species. However, the imaging speed is the bottleneck for the current Raman imaging methods to monitor the dynamic process of living cells. In this paper, we developed an artificial intelligence assisted fast Raman imaging method over the already fast line scan Raman imaging method. The reduced imaging time is realized by widening the slit and laser beam, and scanning the sample with a large scan step. The imaging quality is improved by a data-driven approach to train a deep convolutional neural network, which statistically learns to transform low-resolution images acquired at a high speed into high-resolution ones that previously were only possible with a low imaging speed. Accompanied with the improvement of the image resolution, the deteriorated spectral resolution as a consequence of a wide slit is also restored, thereby the fidelity of the spectral information is retained. The imaging time can be reduced to within 1 min, which is about five times faster than the state-of-the-art line scan Raman imaging techniques without sacrificing spectral and spatial resolution. We then demonstrated the reliability of the current method using fixed cells. We finally used the method to monitor the dynamic evolution process of living cells. Such an imaging speed opens a door to the label-free observation of cellular events with conventional Raman microscopy.

Supervised and Unsupervised End-to-End Deep Learning for Gene Ontology Classification of Neural In Situ Hybridization Images.

In recent years, large datasets of high-resolution mammalian neural images have become available, which has prompted active research on the analysis of gene expression data. Traditional image processing methods are typically applied for learning functional representations of genes, based on their expressions in these brain images. In this paper, we describe a novel end-to-end deep learning-based method for generating compact representations of in situ hybridization (ISH) images, which are invariant-to-translation. In contrast to traditional image processing methods, our method relies, instead, on deep convolutional denoising autoencoders (CDAE) for processing raw pixel inputs, and generating the desired compact image representations. We provide an in-depth description of our deep learning-based approach, and present extensive experimental results, demonstrating that representations extracted by CDAE can help learn features of functional gene ontology categories for their classification in a highly accurate manner. Our methods improve the previous state-of-the-art classification rate (Liscovitch, et al.) from an average AUC of 0.92 to 0.997, i.e., it achieves 96% reduction in error rate. Furthermore, the representation vectors generated due to our method are more compact in comparison to previous state-of-the-art methods, allowing for a more efficient high-level representation of images. These results are obtained with significantly downsampled images in comparison to the original high-resolution ones, further underscoring the robustness of our proposed method.

A Few-Minute Synthesis of CsPbBr3 Nanolasers with a High Quality Factor by Spraying at Ambient Conditions.

Inorganic cesium lead halide perovskite nanowires, generating laser emission in the broad spectral range at room temperature and low threshold, have become powerful tools for the cutting-edge applications in the optoelectronics and nanophotonics. However, to achieve high-quality nanowires with the outstanding optical properties, it was necessary to employ long-lasting and costly methods of their synthesis, as well as postsynthetic separation and transfer procedures that are not convenient for large-scale production. Here we report a novel approach to fabricate high-quality CsPbBr3 nanolasers obtained by rapid precipitation from dimethyl sulfoxide solution sprayed onto hydrophobic substrates at ambient conditions. The synthesis technique allows producing the well-separated nanowires with a broad size distribution of 2-50 μm in 5-7 min, being the fastest method to the best of our knowledge. The formation of nanowires occurs via ligand-assisted reprecipitation triggered by intermolecular proton transfer from (CH3)2CHOH to H2O in the presence of a minor amount of water. The XRD patterns confirm an orthorhombic crystal structure of the as-grown CsPbBr3 single nanowires. Scanning electron microscopy images reveal their regular shape and truncated pyramidal end facets, while high-resolution transmission electron microscopy ones demonstrate their single-crystal structure. The lifetime of excitonic emission of the nanowires is found to be 7 ns, when the samples are excited with energy below the lasing threshold, manifesting the low concentration of defect states. The measured nanolasers of different lengths exhibit pronounced stimulated emission above 13 μJ cm-2 excitation threshold with quality factor Q = 1017-6166. Their high performance is assumed to be related to their monocrystalline structure, low concentration of defect states, and improved end facet reflectivity.

Image super-resolution based on two-level residual learning CNN

In recent years, CNN has been used for single image super-resolution (SR) with its success of in the field of computer vision. However, in the recovery process, there are always some high-frequency components that cant be recovered from low-resolution images to high-resolution ones by using existing CNN-based methods. In this paper, we propose an image super-resolution method based on CNN, which uses a two-level residual learning network to learn residual components, i.e., high-frequency components. We use the Super-Resolution Convolutional Neural Network (SRCNN) as the network structure in each level so that our proposed method can achieve the high-resolution images with high-frequency components that cant be obtained by the existing methods. In addition, we analyze the proposed method with considering three kinds of residual learning networks, which are different in the structure and superimposed layers of the residual learning network. In the experiments, we investigate the performance of the proposed method with various residual learning networks and the effect of image super-resolution to image captioning task.

Blind image deblurring based on multi-resolution ringing removal

Abstract In this paper, a blind deblurring algorithm that is well converged towards the latent sharp image and the blur PSF is proposed. The proposed algorithm employs our recently proposed ringing suppression regularizer in a solution to the blind deblurring problem. Latent sharp image and the blur PSF are estimated jointly via a multi-functional optimization problem addressed by a first-order primal-dual solver. This approach generates a multi-resolution pyramid of the input blurred image and propagates the estimation results of the lower resolution levels to the higher resolution ones. By considering the PSF initiation as an important step in the blind deblurring process, we introduce a PSF initiation method that benefits from the salient structure obtained via an l0-gradient smoothing of blurred image. A first-order primal-dual solution to the l0-gradient smoothing problem is introduced in this paper. Experimental results show that the proposed deblurring algorithm significantly enhances the results favorably against other state-of- the-art image deblurring methods.

Infrared Image Super Resolution by Combining Compressive Sensing and Deep Learning.

Super resolution methods alleviate the high cost and high difficulty in applying high resolution infrared image sensors. In this paper we present a novel single image super resolution method for infrared images by combining compressive sensing theory and deep learning. Low resolution images can be regarded as the compressed sampling results of the high resolution ones in compressive sensing. With sparsity in this theory, higher resolution images can be reconstructed. However, because of diverse level of sparsity for different images, the output contains noise and loss of high frequency information. Deep convolutional neural network provides a solution to relieve the noise and supplement some missing high frequency information. By concatenating two methods, we manage to produce better results in super resolution tasks for infrared images than SRCNN and ScSR. PSNR and SSIM values are used to quantify the performance. Applying our method to open datasets and actual infrared imaging experiments, we also find better visual results are preserved.

Resolution Dependence of an Ab Initio Phasing Method in Protein X-ray Crystallography

For direct phasing of protein crystals, a method based on the hybrid-input-output (HIO) algorithm has been proposed and tested on a variety of structures. So far, however, the diffraction data have been limited to high-resolution ones, i.e., higher than 2 Å. In principle, the methodology can be applied to data of lower resolutions, which might be particularly useful for phasing membrane protein crystals. For resolutions higher than 3.5 Å, it seems the atomic structure is solvable. For data of lower resolutions, information of the secondary structures and the protein boundary can still be obtained. Examples are given to support the conclusions. Real experimental data are used. Two aspects of the observed data have been discussed: removal of the measured low-resolution reflections and involvement of the unmeasured high-resolution reflections. The ab initio phasing employs histogram matching for density modification. A question arises whether the reference histogram used should match the resolution of the diffraction data or not. It seems that there is an optimal histogram which is good to use for data at various resolutions.

High-resolution CT Image Retrieval Using Sparse Convolutional Neural Network.

We propose a high-resolution CT image retrieval method based on sparse convolutional neural network. The proposed framework is used to train the end-to-end mapping from low-resolution to high-resolution images. The patch-wise feature of low-resolution CT is extracted and sparsely represented by a convolutional layer and a learned iterative shrinkage threshold framework, respectively. Restricted linear unit is utilized to non-linearly map the low-resolution sparse coefficients to the high-resolution ones. An adaptive high-resolution dictionary is applied to construct the informative signature which is highly connected to a high-resolution patch. Finally, we feed the signature to a convolutional layer to reconstruct the predicted high-resolution patches and average these overlapping patches to generate high-resolution CT. The loss function between reconstructed images and the corresponding ground truth high-resolution images is applied to optimize the parameters of end-to-end neural network. The well-trained map is used to generate the high-resolution CT from a new low-resolution input. This technique was tested with brain and lung CT images and the image quality was assessed using the corresponding CT images. Peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and mean absolute error (MAE) indexes were used to quantify the differences between the generated high-resolution and corresponding ground truth CT images. The experimental results showed the proposed method could enhance images resolution from low-resolution images. The proposed method has great potential in improving radiation dose calculation and delivery accuracy and decreasing CT radiation exposure of patients.

Residual Super-Resolution Single Shot Network for Low-Resolution Object Detection

For object detection in computer vision, detection models trained by high-resolution images often fail to recognize or localize objects on low-resolution images. To tackle this problem, we propose a fully convolutional network named residual super-resolution single shot network (RSRSSN). RSRSSN consists of two sub-networks, super-resolution sub-network and detection sub-network. The super-resolution sub-network in RSRSSN is achieved by stacking of identity residual blocks while the detection sub-network adopts the single shot multibox detector (SSD). Based on multi-task learning, we design a novel objective function called feature maps multibox loss to enforce low-resolution images to produce similar feature maps with their corresponding high-resolution ones. This information sharing mechanism is proved to be critical for solving the resolution mismatch problem in the experiments. A two-step training scheme is also proposed to train the RSRSSN in an end-to-end manner. Without any data augmentation, RSRSSN outperforms the SSD on both down-sampled PASCAL VOC and MS COCO in real-time object detection.

The operational eEMEP model version 10.4 for volcanic SO<sub>2</sub> and ash forecasting

Abstract. This paper presents a new version of the EMEP MSC-W model called eEMEP developed for transportation and dispersion of volcanic emissions, both gases and ash. EMEP MSC-W is usually applied to study problems with air pollution and aerosol transport and requires some adaptation to treat volcanic eruption sources and effluent dispersion. The operational set-up of model simulations in case of a volcanic eruption is described. Important choices have to be made to achieve CPU efficiency so that emergency situations can be tackled in time, answering relevant questions of ash advisory authorities. An efficient model needs to balance the complexity of the model and resolution. We have investigated here a meteorological uncertainty component of the volcanic cloud forecast by using a consistent ensemble meteorological dataset (GLAMEPS forecast) at three resolutions for the case of SO2 emissions from the 2014 Barðarbunga eruption. The low resolution (40 × 40 km) ensemble members show larger agreement in plume position and intensity, suggesting that the ensemble here does not give much added value. To compare the dispersion at different resolutions, we compute the area where the column load of the volcanic tracer, here SO2, is above a certain threshold, varied for testing purposes between 0.25 and 50 Dobson units. The increased numerical diffusion causes a larger area (+34 %) to be covered by the volcanic tracer in the low resolution simulations than in the high resolution ones. The higher resolution (10 × 10 km) ensemble members show higher column loads farther away from the volcanic eruption site in narrower clouds. Cloud positions are more varied between the high resolution members, and the cloud forms resemble the observed clouds more than the low resolution ones. For a volcanic emergency case this means that to obtain quickly results of the transport of volcanic emissions, an individual simulation with our low resolution is sufficient; however, to forecast peak concentrations with more certainty for forecast or scientific analysis purposes, a finer resolution is needed. The model is further developed to simulate ash from highly explosive eruptions. A possibility of increasing the number of vertical layers, achieving finer vertical resolution, as well as a higher model top, is included in the eEMEP version. Ash size distributions may be altered for different volcanic eruptions and assumptions. Since ash particles are larger than typical particles in the standard model, gravitational settling across all vertical layers is included. We attempt finally a specific validation of the simulation of ash and its vertical distribution. Model simulations with and without gravitational settling for the 2010 Eyjafjallajökull eruption are compared to lidar observations over central Europe. The results show that with gravitation the centre of the ash mass can be 1 km lower over central Europe than without gravitation. However, the height variations in the ash layer caused by real weather situations are not captured perfectly well by either of the two simulations, playing down the role of gravitation parameterization imperfections. Both model simulations have on average an ash centre of mass below the observed values. Correlations between the observed and corresponding model centres of mass are higher for the model simulation with gravitational settling for four of the six stations studied here. The inclusion of gravitational settling is suggested to be required for a volcanic ash model.