High Resolution Visible Images Research Articles

Enhancement of guided thermal image super-resolution approaches

Guided image processing techniques are widely used to extract meaningful information from a guiding image and facilitate the enhancement of the guided one. This paper specifically addresses the challenge of guided thermal image super-resolution, where a low-resolution thermal image is enhanced using a high-resolution visible spectrum image. We propose a new strategy that enhances outcomes from current guided super-resolution methods. This is achieved by transforming the initial guiding data into a representation resembling a thermal-like image, which is more closely in sync with the intended output. Experimental results with upscale factors of ×8 and ×16, demonstrate the outstanding performance of our approach in guided thermal image super-resolution obtained by mapping the original guiding information to a thermal-like image representation.

Thermal UAV Image Super-Resolution Guided by Multiple Visible Cues

Unmanned aerial vehicle (UAV) thermal-imaging has received much attention, but the insufficient image resolution caused by thermal imaging systems is still a crucial problem that limits the understanding of thermal UAV images. However, high-resolution visible images are relatively easy to access, and it is thus valuable for exploring useful information from visible image to assist thermal UAV image super-resolution (SR). In this article, we propose a novel multiconditioned guidance network (MGNet) to effectively mine the information of visible images for thermal UAV image SR. High-resolution visible UAV images usually contain salient appearance, semantic, and edge information, which plays a critical role in boosting the performance of thermal UAV image SR. Therefore, we design an effective multicue guidance module (MGM) to leverage the appearance, edge, and semantic cues from visible images to guide thermal UAV image SR. In addition, we build the first benchmark dataset for the task of thermal UAV image SR guided by visible images. It is collected by a multimodal UAV platform and composes of 1025 pairs of manually aligned visible and thermal images. Extensive experiments on the built dataset show that our MGNet can effectively leverage useful information from visible images to improve the performance of thermal UAV image SR and perform well against several state-of-the-art methods. The dataset is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mmic-lcl/Datasets-and-benchmark-code</uri> .

High-resolution visible imaging with piezoelectric deformable secondary mirror: experimental results at the 1.8-m adaptive telescope

High-resolution visible imaging with piezoelectric deformable secondary mirror: experimental results at the 1.8-m adaptive telescope

ShuffleCloudNet: A Lightweight Composite Neural Network-Based Method for Cloud Computation in Remote-Sensing Images

The occlusion of cloud layers affects the accurate acquisition of ground object information and causes a large amount of useless remote-sensing data transmission and processing, wasting storage, as well as computing resources. Therefore, in this paper, we designed a lightweight composite neural network model to calculate the cloud amount in high-resolution visible remote-sensing images by training the model using thumbnail images and browsing images in remote-sensing images. The training samples were established using paired thumbnail images and browsing images, and the cloud-amount calculation model was obtained by training a proposed composite neural network. The strategy used the thumbnail images for preliminary judgment and the browsing images for accurate calculation, and this combination can quickly determine the cloud amount. The multi-scale confidence fusion module and bag-of-words loss function were redesigned to achieve fast and accurate calculation of cloud-amount data from remote-sensing images. This effectively alleviates the problem of low cloud-amount calculation, thin clouds not being counted as clouds, and that of ice and clouds being confused as in existing methods. Furthermore, a complete dataset of cloud-amount calculation for remote-sensing images, CTI_RSCloud, was constructed for training and testing. The experimental results show that, with less than 13 MB of parameters, the proposed lightweight network model greatly improves the timeliness of cloud-amount calculation, with a runtime is in the millisecond range. In addition, the calculation accuracy is better than the classic lightweight networks and backbone networks of the best cloud-detection models.

Quantifying Sub-Meter Surface Heterogeneity on Mars Using Off-Axis Thermal Emission Imaging System (THEMIS) Data.

Surface heterogeneities below the spatial resolution of thermal infrared (TIR) instruments result in anisothermality and can produce emissivity spectra with negative slopes toward longer wavelengths. Sloped spectra arise from an incorrect assumption of either a uniform surface temperature or a maximum emissivity during the temperature-emissivity separation of radiance data. Surface roughness and lateral mixing of different sub-pixel surface units result in distinct spectral slopes with magnitudes proportional to the degree of temperature mixing. Routine Off-nadir Targeted Observations (ROTO) of the Thermal Emission Imaging Spectrometer (THEMIS) are used here for the first time to investigate anisothermality below the spatial resolution of THEMIS. The southern flank of Apollinaris Mons and regions within the Medusae Fossae Formation are studied using THEMIS ROTO data acquired just after local sunset. We observe a range of sloped TIR emission spectra dependent on the magnitude of temperature differences within a THEMIS pixel. Spectral slopes and wavelength-dependent brightness temperature differences are forward-modeled for a series of two-component surfaces of varying thermal inertia values. Our results imply that differing relative proportions of rocky and unconsolidated surface units are observed at each ROTO viewing geometry and suggest a local rock abundance six times greater than published results that rely on nadir data. High-resolution visible images of these regions indicate a mixture of surface units from boulders to dunes, providing credence to the model.

Unsupervised Remote Sensing Image Super-Resolution Guided by Visible Images

Remote sensing images are widely used in many applications. However, due to being limited by the sensors, it is difficult to obtain high-resolution (HR) images from remote sensing images. In this paper, we propose a novel unsupervised cross-domain super-resolution method devoted to reconstructing a low-resolution (LR) remote sensing image guided by an unpaired HR visible natural image. Therefore, an unsupervised visible image-guided remote sensing image super-resolution network (UVRSR) is built. The network is divided into two learnable branches: a visible image-guided branch (VIG) and a remote sensing image-guided branch (RIG). As HR visible images can provide rich textures and sufficient high-frequency information, the purpose of VIG is to treat them as targets and make full use of their advantages in reconstruction. Specially, we first use a CycleGAN to drag the LR visible natural images to the remote sensing domain; then, we apply an SR network to upscale these simulated remote sensing domain LR images. However, the domain gap between SR remote sensing images and HR visible targets is massive. To enforce domain consistency, we propose a novel domain-ruled discriminator in the reconstruction. Furthermore, inspired by the zero-shot super-resolution network (ZSSR) to explore the internal information of remote sensing images, we add a remote sensing domain inner study to train the SR network in RIG. Sufficient experimental works show UVRSR can achieve superior results with state-of-the-art unpaired and remote sensing SR methods on several challenging remote sensing image datasets.

Cotton yield estimation model based on machine learning using time series UAV remote sensing data

Crop yield prediction is of great practical significance for farmers to make reasonable decisions, such as decisions on crop insurance, storage demand, cash flow budget, fertilizer, water and other input factors. The traditional yield measurement method is sampling surveys, which require a large area of destructive sampling of cotton fields and consume considerable time and labor costs. This study established a cotton yield estimation model based on time series Unmanned Aerial Vehicle (UAV) remote sensing data. The U-Net semantic segmentation network is used to recognize and extract the boll opening pixels in high-resolution visible images, and the boll opening pixel percentage (BOP) is calculated according to the network extraction results. By combining the multispectral images and the pixel coverage of cotton bolls, a Bayesian regularization BP (back propagation) neural network was used to predict cotton yields. In order to simplify the input parameters of the model, the stepwise sensitivity analysis method is used to eliminate redundant variables and obtain the optimal input feature set. The experimental results show that the R2 of the proposed model is 0.853 at the scale of 0.81 m2 (average results of ten-fold cross validation). This study provides a method that can simultaneously meet the requirements of large-area and small-scale forecasting of cotton yields and provides a new idea for cotton yield measurement and breeding screening.

An ensemble deep learning method as data fusion system for remote sensing multisensor classification

Because of the great achievements in designing remote sensing sensors, the extraction of useful information from multisource remote sensing data remains a challenging problem. Most of the recent research projects have applied single deep learning systems for data fusion and classification. The idea of using ensemble deep learning algorithms through a multisensor fusion system can improve the performance of data fusion tasks. In this research, however, a multi-sensor classification strategy, which is based on deep learning ensemble procedure and decision fusion framework, is investigated for the fusion of Light Detection and Ranging (LiDAR), Hyperspectral Images (HS), and very high-resolution Visible (Vis) images. This research proposes a basic classifier based on deep Convolutional Neural Network (CNN) in which the softmax layer is replaced by a Support Vector Machine (CNN-SVM). Then, a random feature selection is applied to generate two separate CNN-SVM ensemble systems, one for LiDAR and Vis and the other one for HS data. To overcome the similarity and overfitting between the deep features and the classifiers provided by two ensemble systems and to select the best subsets of the classifiers, two diversity measures select the most diverse combinations of the classifiers. Finally, a decision fusion method combines the obtained diverse classifiers from CNN ensembles. Results demonstrate that the proposed method achieves higher accuracy, and its performance outperforms some of the existing methods. The proposed ensemble CNN method improved single deep CNN, random forest, and Adaboost between 2% to 10% in terms of classification accuracy.

Apple powdery mildew infestation detection and mapping using high-resolution visible and multispectral aerial imaging technique

Powdery mildew (PM) in apple orchards is a critical fungal disease that considerably reduces yield, harvested fruit quality, and orchard health. Rapid detection and mapping of resulting infestation at orchard scale is a challenge with existing laborious manual scouting approaches. Therefore, this study explored the feasibility of detecting and mapping PM infestation in an apple orchard block using high-resolution visible (red-green-blue [RGB]) and multispectral imaging technique. Imaging campaigns were conducted over an experimental orchard using small unmanned aerial systems (UAS) integrated with the above optical sensors. K-means classifier trained on individual snapshots of RGB imagery had mean disease detection accuracy of 77%. Eight multispectral vegetation indices also showed significant differences (p < 0.001) between healthy (Mean: 0.25–0.84) and infected (Mean: 0.01–0.25) leaves. Modified Simple Ratio-Red (MSRR), Modified Simple Ratio-Blue (MSRB), and Optimized Soil Adjusted Vegetation Index (OSAVI) showed the highest contrast (Mean: 0.46–0.79). Orchard block-scale mapping using RGB orthomosaic layer, classified with k-means and spectral angle mapper techniques had accuracies up to 73%. Overall, high resolution imagery in the visible and multispectral domain could sufficiently detect the infection and generate site-specific disease heat maps. These maps could be useful to apple growers in directing prescriptive resources ( pruning labor or fungicide application) for PM management.

Insulation Failure Quantification Based on the Energy of Digital Images Using Low-Cost Imaging Sensors

Insulation faults in high-voltage applications often generate partial discharges (PDs) accompanied by corona activity, optical radiation mainly in the ultraviolet (UV) and visible bands. Recent developments in low-cost, small-size, and high-resolution visible imaging sensors, which are also partially sensitive to the UV spectral region, are gaining attention due to their many industrial applications. This paper proposes a method for early PD detection by using digital imaging sensors, which allows the severity of insulation faults to be assessed. The electrical power dissipated by the PDs is correlated to the energy of the acquired visible images, and thus, the severity of insulation faults is determined from the energy of the corona effect. A criterion to quantify the severity of insulation faults based on the energy of the corona images is proposed. To this end, the point-to-plane gap configuration is analyzed in a low-pressure chamber, where digital image photographs of the PDs are taken and evaluated under different pressure conditions ranging from 10 to 100 kPa, which cover the typical pressure range of aeronautic applications. The use of digital imaging sensors also allows an early detection, location and quantification of the PD activity, and thus assessing the severity of insulation faults to perform predictive maintenance tasks, while enabling the cost and complexity of the instrumentation to be reduced. Although the approach proposed in this paper has been applied to detect PDs in aeronautic applications, it can be applied to many other high-voltage applications susceptible of PD occurrence.

A Multiple Remote Sensing Sensor Fusion System Using Choquet Fuzzy Integral and Modified Particle Swarm Optimization (FI-MPSO)

The simultaneous use of remote sensing sensors has been of great interest due to reasons such as the weaknesses of each sensor individually and the improvement in results by integrating multisensor information. In this context, decision fusion of multiremote sensing sensors is one of the hot research topics for improvement in land cover classification. This paper investigates a multisensor fusion system based on Choquet fuzzy integral and a modified particle swarm optimization for fusion of high-resolution visible RGB image and thermal infrared hyperspectral. Instead of using common optimization algorithms, a new version of PSO is used to reduce the running time and to increase the accuracy. After some processes and fuzzy classifications, a fuzzy integral–fuzzy measure fusion methodology which uses modified PSO to find the best subset of fuzzy measures is utilized to fuse the resulted decision profiles of soft decision-making system. Also, comparison between proposed method and other successful decision fusion techniques such as Dempsters shafer, AdaBoost and Bagging is investigated. Experiments are executed on high-resolution RGB image and thermal infrared hyperspectral data from Quebec of Canada. The results show that the suggested method obtains 92% of overall accuracy and outperforms the other decision fusion and optimization methods in classification performance.

Fast Detection of Olive Trees Affected by Xylella Fastidiosa from UAVs Using Multispectral Imaging.

Xylella fastidiosa (Xf) is a well-known bacterial plant pathogen mainly transmitted by vector insects and is associated with serious diseases affecting a wide variety of plants, both wild and cultivated; it is known that over 350 plant species are prone to Xf attack. In olive trees, it causes olive quick decline syndrome (OQDS), which is currently a serious threat to the survival of hundreds of thousands of olive trees in the south of Italy and in other countries in the European Union. Controls and countermeasures are in place to limit the further spreading of the bacterium, but it is a tough war to fight mainly due to the invasiveness of the actions that can be taken against it. The most effective weapons against the spread of Xf infection in olive trees are the detection of its presence as early as possible and attacks to the development of its vector insects. In this paper, image processing of high-resolution visible and multispectral images acquired by a purposely equipped multirotor unmanned aerial vehicle (UAV) is proposed for fast detection of Xf symptoms in olive trees. Acquired images were processed using a new segmentation algorithm to recognize trees which were subsequently classified using linear discriminant analysis. Preliminary experimental results obtained by flying over olive groves in selected sites in the south of Italy are presented, demonstrating a mean Sørensen–Dice similarity coefficient of about 70% for segmentation, and 98% sensitivity and 93% precision for the classification of affected trees. The high similarity coefficient indicated that the segmentation algorithm was successful at isolating the regions of interest containing trees, while the high sensitivity and precision showed that OQDS can be detected with a low relative number of both false positives and false negatives.

Spinal Nerve Modeling Within The Spinal Canal

The objective of this work is to develop a high‐resolution model of the brachial plexus with detail including individual spinal nerve rootlets. Medical illustrations may include this level of detail with artistic enhancement of structures of interest and suppression of background visual noise. Careful cadaver dissection of an open spinal canal and separated dural sheath will reveal the detail we seek for our digital model, but such dissection usually demonstrates them in positions unlike that of the living and intact nervous system. Our goal is to segment the nerves forming the brachial plexus, retaining their positions in‐vivo. We use a high resolution Visible Human Project type block face image dataset. The image database is from whole‐body imaging of an elderly, female donor with visible pathology. The focus of this work was on modeling the spinal nerves from C3 to T1. The brachial plexus and some peripheral nerves were included for orientation. This image database is made up of voxels spaced orthogonally 63μ apart, compared to the Visible Human Male image database with voxels spaced 1000μ apart. This voxel size yields a spatial resolution on the order of 300μ for this new dataset compared to the 5mm resolution of the VH Male. We can now see individual rootlets exiting the spinal cord to form the anterior and posterior spinal nerve roots but it is still challenging to follow each rootlet from beginning to end. We concentrated on nerves C3 to T1 which are the most visible as they diverge from the spinal cord soon after exiting the spinal columns. We were able to identify an average of five rootlets for each anterior and posterior root for nerves C3 through T1. Splines were constructed with control points to fit the path of the nervous structures. Each control point was assigned a radius to approximate the size of the structure at that point. We identified an anatomic anomaly demonstrating a conjoined posterior rootlet, which emerged from the posterior gray horn as one rootlet but split to contribute to the roots of C6 and C7. Within the brachial plexus we were able to identify the posterior rami for spinal nerves C3 to C5, which we were unable to find in VH Male data. For nerves C5 through C8 we followed anterior rami into the trunks, divisions, and cords of the brachial plexus. Following divisions and cords proved difficult because the cadaver was oriented with arms at her side such that the brachial plexus was compact as it is in the anatomical position. Freezing artifacts occasionally obscured the tissue, making it hard to confidently follow specific nerve components in a cluster of multiple trunks or divisions. We were still able to identify the superior, middle, and inferior trunks of the brachial plexus, the posterior and anterior divisions of each trunk, the lateral, posterior, and medial cords, the terminal nerves, and branching peripheral nerves. This modeling of the cervical spinal nerves and the brachial plexus is not only unique because of the extraordinary detail of the nervous structures, but also advantageous because it allows anatomy students to visualize these nerves in the same spatial configuration as in the living human.Figure shows 3D splines of brachial plexus and spinal nerve rami, roots, and and rootlets of C3 through T1.Figure 1Figure shows anatomical anomaly found where two nerve rootlets coming out from the root of C6 and C7 conjoin into a single rootlet before entering the posterior gray horn.Figure 2

Semantic Segmentation Using Deep Learning with Vegetation Indices for Rice Lodging Identification in Multi-date UAV Visible Images

A rapid and precise large-scale agricultural disaster survey is a basis for agricultural disaster relief and insurance but is labor-intensive and time-consuming. This study applies Unmanned Aerial Vehicles (UAVs) images through deep-learning image processing to estimate the rice lodging in paddies over a large area. This study establishes an image semantic segmentation model employing two neural network architectures, FCN-AlexNet, and SegNet, whose effects are explored in the interpretation of various object sizes and computation efficiency. Commercial UAVs imaging rice paddies in high-resolution visible images are used to calculate three vegetation indicators to improve the applicability of visible images. The proposed model was trained and tested on a set of UAV images in 2017 and was validated on a set of UAV images in 2019. For the identification of rice lodging on the 2017 UAV images, the F1-score reaches 0.80 and 0.79 for FCN-AlexNet and SegNet, respectively. The F1-score of FCN-AlexNet using RGB + ExGR combination also reaches 0.78 in the 2019 images for validation. The proposed model adopting semantic segmentation networks is proven to have better efficiency, approximately 10 to 15 times faster, and a lower misinterpretation rate than that of the maximum likelihood method.

DDcGAN: A Dual-discriminator Conditional Generative Adversarial Network for Multi-resolution Image Fusion.

In this paper, we proposed a new end-to-end model, termed as dual-discriminator conditional generative adversarial network (DDcGAN), for fusing infrared and visible images of different resolutions. Our method establishes an adversarial game between a generator and two discriminators. The generator aims to generate a real-like fused image based on a specifically designed content loss to fool the two discriminators, while the two discriminators aim to distinguish the structure differences between the fused image and two source images, respectively, in addition to the content loss. Consequently, the fused image is forced to simultaneously keep the thermal radiation in the infrared image and the texture details in the visible image. Moreover, to fuse source images of different resolutions, e.g., a low-resolution infrared image and a high-resolution visible image, our DDcGAN constrains the downsampled fused image to have similar property with the infrared image. This can avoid causing thermal radiation information blurring or visible texture detail loss, which typically happens in traditional methods. In addition, we also apply our DDcGAN to fusing multi-modality medical images of different resolutions, e.g., a low-resolution positron emission tomography image and a high-resolution magnetic resonance image. The qualitative and quantitative experiments on publicly available datasets demonstrate the superiority of our DDcGAN over the state-of-the-art, in terms of both visual effect and quantitative metrics.

Colorimetric Method for Discharge Status Diagnostics based on Optical Spectroscopy and Digital Images

Discharge is a process, which involves changes in multiple species, and it is important for researchers to be able to describe changes in discharge status holistically. In this study, we applied AC and DC high voltage power sources to a short-gap needle-plane device in an atmospheric environment. We propose two new corona diagnosis methods based on spectral data and visible images. The experimental results demonstrate that, in the RGB system, the proportion of the blue component in AC corona images has the characteristic of decrease-stable-increase with voltage, and the corresponding result of the spectra is stable-increase. In the XYZ system, the AC corona chromaticity coordinates of both images and spectra demonstrate a linear fitting result with the voltage, and the image results contain a retracement process, showing shows the increase-stable-decrease trend of chromaticity coordinates x and y, which correlates well with oscilloscope results. The difference between the image and spectra results indicates that, compared to the spectra, high-resolution visible images are more sensitive to imperceptible changes in colorimetry information. The change of DC corona colorimetry characteristics can illustrate the change in AC corona colorimetry characteristics. Additionally, the dominant wavelength is proven to reflect the discharge species, and can reflect the vibrational temperatures. To the best of our knowledge, this paper represents the first attempt to systematically introduce colorimetry theory into discharge images and spectra. We successfully obtained colorimetric indicators with the linearity and low-dimension characteristics, which correspond well with known physical parameters.

SmART_scan: A method to produce composition maps using any elemental, molecular and image data

Abstract XRF scanners recently have become the most effective tools for analyzing the distribution of pigments on paintings. They are noninvasive and portable and can be used on flat surfaces of notable dimensions. Powerful miniaturized X-Ray tubes combined with more sensitive detectors make it possible to mount instruments on sturdy motorized X,Y devices, and reduce time needed for a scan. The high-resolution results obtained are of the utmost interest for conservators, art historians and conservation scientists. Drawbacks of XRF scanners are the elevated cost and the bulk of the X,Y devices, which makes them more difficult to transport. It is our opinion that the amount of information collected by a scanner is somewhat redundant. Uniform portions of the painting may be treated as a unity. Following this basic idea, a procedure is proposed to combine a high-resolution visible image with compositional measurement carried out on a grid with a much smaller number of points. Different types of images, spatially registered with respect to the visible one, can be used to increase the amount of information available. A computer program, named SmART_scan, statistically combines all available data, and generates false color maps that show the distribution of elements (XRF), or of compounds [XRD or Raman (Tracey et al., 2006)] over the surface. Even so, the collected information still seemed overabundant. For this reason, the program was adapted to work with a limited number of representative points selected by the user. The measuring time was enormously reduced, and the results were reasonably good. The program can be applied retroactively to previously collected data, provided that a good image, the accurate position of the measurements and their numerical results are available. Examples of maps relative to mockups, real paintings, cross sections and three-dimensional objects are shown.

Dictionary Alignment With Re-Ranking for Low-Resolution NIR-VIS Face Recognition

Recently, near-infrared (NIR) images are increasingly being captured for recognizing faces in low-light/night-time conditions. Matching these images against the controlled high-resolution visible facial images usually present in the database is a challenging task. In surveillance scenarios, the NIR images can have very low resolution and also non-frontal pose which makes the problem even more challenging. In this paper, we propose an orthogonal dictionary alignment approach for addressing this problem. We also propose a re-ranking approach to further improve the recognition performance for each probe by combining the rank list given by the proposed algorithm with that given by another complementary feature/algorithm. Finally, we have also collected our own database Heterogeneous face recognition across Pose and Resolution (HPR) that has facial images captured from two surveillance quality NIR cameras and one high-resolution visible camera, with significant variations in head pose and resolution. Extensive experiments on the modified CASIA NIR-VIS 2.0 database, the Surveillance Camera face database, and our HPR database show the effectiveness of the proposed approaches and the collected database.

FusionGAN: A generative adversarial network for infrared and visible image fusion

Infrared images can distinguish targets from their backgrounds on the basis of difference in thermal radiation, which works well at all day/night time and under all weather conditions. By contrast, visible images can provide texture details with high spatial resolution and definition in a manner consistent with the human visual system. This paper proposes a novel method to fuse these two types of information using a generative adversarial network, termed as FusionGAN. Our method establishes an adversarial game between a generator and a discriminator, where the generator aims to generate a fused image with major infrared intensities together with additional visible gradients, and the discriminator aims to force the fused image to have more details existing in visible images. This enables that the final fused image simultaneously keeps the thermal radiation in an infrared image and the textures in a visible image. In addition, our FusionGAN is an end-to-end model, avoiding manually designing complicated activity level measurements and fusion rules as in traditional methods. Experiments on public datasets demonstrate the superiority of our strategy over state-of-the-arts, where our results look like sharpened infrared images with clear highlighted targets and abundant details. Moreover, we also generalize our FusionGAN to fuse images with different resolutions, say a low-resolution infrared image and a high-resolution visible image. Extensive results demonstrate that our strategy can generate clear and clean fused images which do not suffer from noise caused by upsampling of infrared information.

Infrared image super-resolution using auxiliary convolutional neural network and visible image under low-light conditions

Convolutional neural networks (CNN) have been successfully applied to visible image super-resolution (SR) methods. In this study, we propose a CNN-based SR algorithm for up-scaling near-infrared (NIR) images under low-light conditions, using corresponding visible images. Our algorithm first extracts high-frequency (HF) components from the up-scaled low-resolution (LR) NIR image and its corresponding high-resolution (HR) visible image, and then takes them as multiple inputs of the CNN. Next, the CNN outputs the HR HF component of the input NIR image. Finally, an HR NIR image is synthesized by adding the HR HF component to the up-scaled LR NIR image. The simulation results show that the proposed algorithm outperforms the state-of-the-art methods, in terms of both qualitative and quantitative aspects.