Super-resolution Mapping Research Articles

Locally adaptive super-resolution waterline mapping with MODIS imagery

ABSTRACTMapping inland water bodies is important for relevant research fields and water resource management. Satellite remote sensing is a routine approach, and various remotely sensed images have been applied to map waterlines. MODerate-resolution Imaging Spectroradiometer (MODIS) images have advantages for waterline mapping in large areas, thanks to its wide scan width and high frequent revisiting period; however, the spatial resolution of MODIS images is too coarse to map waterlines accurately. In this article, a super-resolution mapping (SRM) model was proposed for fine spatial resolution waterline mapping with MODIS images. The proposed SRM model was directly applied in the MODIS band 2 images, which have a fine spatial resolution and a high spectral separability between water and land. In order to further take account of the spatial heterogeneity of endmembers, the reflectance values of water and land were locally calculated for each coarse resolution pixel. The proposed SRM model was assessed in two study areas located in the Tibetan Plateau, China and Wisconsin, United States, including water bodies with different areas and boundary shapes. The results showed that the accuracy values of the proposed SRM model using local endmembers were higher than those of hard classification and the SRM model using average endmembers, showing the effectiveness of the proposed model in fine spatial resolution waterline mapping with MODIS imagery.

Improved Urban Flooding Mapping from Remote Sensing Images Using Generalized Regression Neural Network-Based Super-Resolution Algorithm

Urban flooding is a serious natural hazard to many cities all over the world, which has dramatic impacts on the urban environment and human life. Urban flooding mapping has practical significance for the prevention and management of urban flood disasters. Remote sensing images with high temporal resolutions are widely used for urban flooding mapping, but have a limitation of relatively low spatial resolutions. In this study, a new method based on a generalized regression neural network (GRNN) is proposed to achieve improved accuracy in super-resolution mapping of urban flooding (SMUF) from remote sensing images. The GRNN-SMUF algorithm was proposed and then assessed using Landsat 5 and Landsat 8 images of Brisbane city in Australia and Wuhan city in China. Compared to three traditional methods, GRNN-SMUF mapped urban flooding more accurately according to both visual and quantitative assessments. The results of this study will improve the accuracy of urban flooding mapping using easily-available remote sensing images with medium-low spatial resolutions and will be propitious to the prevention and management of urban flood disasters.

Learning-Based Superresolution Land Cover Mapping

Superresolution mapping (SRM) is a technique for generating a fine-spatial-resolution land cover map from coarse-spatial-resolution fraction images estimated by soft classification. The prior model used to describe the fine-spatial-resolution land cover pattern is a key issue in SRM. Here, a novel learning-based SRM algorithm, whose prior model is learned from other available fine-spatial-resolution land cover maps, is proposed. The approach is based on the assumption that the spatial arrangement of the land cover components for mixed pixel patches with similar fractions is often similar. The proposed SRM algorithm produces a learning database that includes a large number of patch pairs for which there is a fine- and coarse-spatial-resolution representation for the same area. From the learning database, patch pairs that have similar coarse-spatial-resolution patches as those in the input fraction images are selected. Fine-spatial-resolution patches in these selected patch pairs are then used to estimate the latent fine-spatial-resolution land cover map by solving an optimization problem. The approach is illustrated by comparison against state-of-the-art SRM methods using land cover map subsets generated from the USA's National Land Cover Database. Results show that the proposed SRM algorithm better maintains the spatial pattern of land covers for a range of different landscapes. The proposed SRM algorithm has the highest overall accuracy and kappa values in all of these SRM algorithms, by using the entire maps in the accuracy assessment.

Improving super-resolution mapping through combining multiple super-resolution land-cover maps

ABSTRACTSuper-resolution mapping (SRM) is an ill-posed problem, and different SRM algorithms may generate non-identical fine-spatial resolution land-cover maps (sub-pixel maps) from the same input coarse-spatial resolution image. The output sub-pixels maps may each have differing strengths and weaknesses. A multiple SRM (M-SRM) method that combines the sub-pixel maps obtained from a set of SRM analyses, obtained from a single or multiple set of algorithms, is proposed in this study. Plurality voting, which selects the class with the most votes, is used to label each sub-pixel. In this study, three popular SRM algorithms, namely, the pixel-swapping algorithm (PSA), the Hopfield neural network (HNN) algorithm, and the Markov random field (MRF)-based algorithm, were used. The proposed M-SRM algorithm was validated using two data sets: a simulated multispectral image and an Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) hyperspectral image. Results show that the highest overall accuracies were obtained by M-SRM in all experiments. For example, in the AVIRIS image experiment, the highest overall accuracies of PSA, HNN, and MRF were 88.89, 93.81, and 82.70%, respectively, and these increased to 95.06, 95.37, and 85.56%, respectively for M-SRM obtained from the multiple PSA, HNN, and MRF analyses.

Super-Resolution Imaging of Molecular Emission Spectra and Single Molecule Spectral Fluctuations.

Localization microscopy can image nanoscale cellular details. To address biological questions, the ability to distinguish multiple molecular species simultaneously is invaluable. Here, we present a new version of fluorescence photoactivation localization microscopy (FPALM) which detects the emission spectrum of each localized molecule, and can quantify changes in emission spectrum of individual molecules over time. This information can allow for a dramatic increase in the number of different species simultaneously imaged in a sample, and can create super-resolution maps showing how single molecule emission spectra vary with position and time in a sample.

Super-Resolution Mapping of Neuronal Circuitry With an Index-Optimized Clearing Agent.

Super-resolution imaging deep inside tissues has been challenging, as it is extremely sensitive to light scattering and spherical aberrations. Here, we report an optimized optical clearing agent for high-resolution fluorescence imaging (SeeDB2). SeeDB2 matches the refractive indices of fixed tissues to that of immersion oil (1.518), thus minimizing both light scattering and spherical aberrations. During the clearing process, fine morphology and fluorescent proteins were highly preserved. SeeDB2 enabled super-resolution microscopy of various tissue samples up to a depth of >100 μm, an order of magnitude deeper than previously possible under standard mounting conditions. Using this approach, we demonstrate accumulation of inhibitory synapses on spine heads in NMDA-receptor-deficient neurons. In the fly medulla, we found unexpected heterogeneity in axon bouton orientations among Mi1 neurons, a part of the motion detection circuitry. Thus, volumetric super-resolution microscopy of cleared tissues is a powerful strategy in connectomic studies at synaptic levels.

Spatial continuity and self-similarity in super-resolution mapping: self-similar pixel swapping

ABSTRACTSelf-similarity of fractal geometry refers to that a part of an object is similar to the whole. This scale-invariant feature has a certain role to play in super-resolution mapping which is a mapping technique across scale aiming at enhancing spatial resolution of remote-sensing imagery. Unlike other super-resolution mapping methods depending solely on spatial continuity, a self-similar pixel swapping (SSPS) method combining spatial continuity and self-similarity of fractal geometry into original pixel swapping (PS) algorithm is presented here. A self-similar weight function defined from the composition information at pixel scale within a predetermined window is added to the calculation of attractiveness in the standard PS method. The self-similar weight function guides the subpixels within a pixel to arrange spatially similar to the appearance of the composition information at pixel scale. Evaluating with synthetic images and satellite image, the performance of the SSPS is particularly obvious in reproducing objects with sharp corners, linear features and adjacent small objects.

Reconstructing contralateral fiber tracts: methodological aspects of cerebello-thalamocortical pathway reconstruction.

The identification of pathways connecting the cerebral cortex with subcortical structures is critical to understanding how large-scale brain networks operate. The cerebellum, for example, is known to project numerous axonal bundles to thecerebral cortex passing through the thalamus. This paper focuses on the technical details of cerebello-thalamo-cortical pathway reconstruction using advanced diffusion MRI techniques in humans in vivo. Pathways reconstructed using seed/target placement on super-resolution maps, created with track density imaging (TDI), were compared with those reconstructed by defining regions of interest (ROIs) on non-diffusion weighted images (b0). We observed that the reconstruction of the pathways was more anatomically accurate when using ROIs placed on TDI rather than on b0 maps, while inter-subject variability and reproducibility were similar between the two methods. Diffusion indices along pathways showed a position-dependent specificity that will need to be taken into consideration in future clinical investigations.

Influence of Protein Surface Coverage on Anomalously Strong Adsorption Sites.

Serum albumin is commonly used as a blocking agent to reduce nonspecific protein adsorption in bioassays and biodevices; however, the details of this process remain poorly understood. Using single molecule techniques, we investigated the dynamics of human serum albumin (HSA) on four model surfaces as a function of protein concentration. By constructing super-resolution maps, identifying anomalously strong adsorption sites, and quantifying surface heterogeneity, we found that the concentration required for site blocking varied dramatically with surface chemistry. When expressed in terms of protein surface coverage, however, a more consistent picture emerged, where a significant fraction of strong sites were passivated at a fractional coverage of 10(-4). On fused silica (FS), "non-fouling" oligo (ethylene glycol) functionalized FS, and hydrophobically modified FS, a modest additional site blocking effect continued at higher coverage. However, on amine-functionalized surfaces, the surface heterogeneity exhibited a minimum at a coverage of ∼10(-4). Using intermolecular Förster resonance energy transfer (FRET), we determined that new anomalous strong sites were created at higher coverage on amine surfaces and that adsorption to these sites was associated with protein-protein interactions, i.e., surface-induced aggregation.

Superresolution Land-Cover Mapping Based on High-Accuracy Surface Modeling

A new superresolution mapping (SRM) method based on high-accuracy surface modeling (HASM) is proposed to generate land-cover maps at the subpixel scale. HASM uses the fundamental theorem of surfaces to uniquely define a land surface, which can produce less errors in interpolation results than classic methods, and thus, the proposed SRM method first uses it to estimate the soft class values of subpixels according to the fraction images of soft classification. Then, it transforms the soft class values into a hard-classified land-cover map using class allocation under the constraints of fraction images. Experiments on a synthetic image and a real remote sensing image show that the proposed method produces more accurate SRM maps than four existing SRM methods. Hence, the proposed method provides a new option for superresolution land-cover mapping.

Super-Resolution Land Cover Mapping Using Multiscale Self-Similarity Redundancy

Super-resolution mapping (SRM) is a method to estimate a fine-resolution land cover map from coarse-resolution fraction images. SRM is an ill-posed problem and regularization terms are always needed to be introduced to well-pose the solution. The regularization term based on the maximal spatial dependence has been widely used in SRM; however, it is often too simple to provide detailed land cover pattern information. In this paper, a novel SRM algorithm, which adopts a new regularization term that is generated using the multiscale self-similarity redundancy feature in fraction images, is proposed. Based on the multiscale self-similarity redundancy, the proposed SRM algorithm magnifies the input coarse-resolution fraction images to the images with the same spatial resolution as the final fine-resolution map to be estimated. A coarse-to-fine strategy is applied to insure the stability of the magnifying process. During the magnifying process, support vector regression is used to learn the relationship between image patches in fraction images and the down-sampled images, which is applied to estimate the magnified fraction images. The new regularization term is then constructed based on the final magnified fraction images and used to provide additional information for SRM. The proposed SRM algorithm was compared with popular SRM algorithms using both synthetic and real images. Experimental results show that the multiscale self-similarity redundancy feature widely exists in fraction images, and provides valuable land cover information for SRM. Resultant fine-resolution land cover maps generated by the proposed SRM algorithm have higher accuracy than other algorithms.

Super-resolution mapping of glutamate receptors in C. elegans by confocal correlated PALM.

Photoactivated localization microscopy (PALM) is a super-resolution imaging technique based on the detection and subsequent localization of single fluorescent molecules. PALM is therefore a powerful tool in resolving structures and putative interactions of biomolecules at the ultimate analytical detection limit. However, its limited imaging depth restricts PALM mostly to in vitro applications. Considering the additional need for anatomical context when imaging a multicellular organism, these limitations render the use of PALM in whole animals difficult. Here we integrated PALM with confocal microscopy for correlated imaging of the C. elegans nervous system, a technique we termed confocal correlated PALM (ccPALM). The neurons, lying below several tissue layers, could be visualized up to 10 μm deep inside the animal. By ccPALM, we visualized ionotropic glutamate receptor distributions in C. elegans with an accuracy of 20 nm, revealing super-resolution structure of receptor clusters that we mapped onto annotated neurons in the animal. Pivotal to our results was the TIRF-independent detection of single molecules, achieved by genetic regulation of labeled receptor expression and localization to effectively reduce the background fluorescence. By correlating PALM with confocal microscopy, this platform enables dissecting biological structures with single molecule resolution in the physiologically relevant context of whole animals.

Improvement of the Example-Regression-Based Super-Resolution Land Cover Mapping Algorithm

Super-resolution mapping (SRM) is a method for generating a fine-resolution land cover map from coarse-resolution fraction images. Example-regression-based SRM algorithms can estimate a fine-resolution land cover map with detailed spatial information by learning land cover spatial patterns from available land cover maps. Existing example-regression-based SRM algorithms are sensitive to fraction errors, and the results often include many linear artifacts and speckles. To overcome these shortcomings, this study proposes an improved example-regression-based SRM algorithm. The objective function of the proposed SRM algorithm comprises three terms. The first term is used to minimize the difference between the fraction values of the estimated fine-resolution land cover map and the input fraction values. The second term is used to maximize the class membership possibility values of the fine pixels in the result. The final term is used to make the result locally smooth. The proposed SRM algorithm is compared with several popular SRM algorithms using both synthetic and real fraction images. Experimental results indicate that the proposed SRM algorithm can produce results with less speckles and linear artifacts, more spatial details, smoother boundaries, and higher accuracies than the SRM results used for comparison.

Superresolution fluorescence mapping of single-nanoparticle catalysts reveals spatiotemporal variations in surface reactivity

For the practical application of nanocatalysts, it is desirable to understand the spatiotemporal fluctuations of nanocatalytic activity at the single-nanoparticle level. Here we use time-lapsed superresolution mapping of single-molecule catalysis events on individual nanoparticles to observe time-varying changes in the spatial distribution of catalysis events on Sb-doped TiO2 nanorods and Au triangle nanoplates. Compared with the active sites on well-defined surface facets, the defects of the nanoparticle catalysts possess higher intrinsic reactivity but lower stability. Corners and ends are more reactive but also less stable than flat surfaces. Averaged over time, the most stable sites dominate the total apparent activity of single nanocatalysts. However, the active sites with higher intrinsic activity but lower stability show activity at earlier time points before deactivating. Unexpectedly, some active sites are found to recover their activity ("self-healing") after deactivation, which is probably due to desorption of the adsorbate. Our superresolution measurement of different types of active catalytic sites, over both space and time, leads to a more comprehensive understanding of reactivity patterns and may enable the design of new and more productive heterogeneous catalysts.

Markov random field-based method for super-resolution mapping of forest encroachment from remotely sensed ASTER image

Forest encroachment (FE) is a problem in Andaman and Nicobar Islands (ANI) in India for environment and planning. Small gaps created in the forest slowly expand its periphery disturbing the biodiversity. Therefore, intrusion of poachers, slash and burn and other factors causing FE must be carefully detected and monitored. Remote sensing offers a great opportunity to accomplish this task because of its synoptic view. Conventional classification methods with remotely sensed images are problematic because of small size of FE and mixed landcover composition. This study presents an application of super-resolution mapping (SRM) based on Markov random field for detection of FE using ASTER (15 m) images. The SRM results were validated using multispectral IRS LISS-IV (5.8 m) image. Non-contiguous FE patches of various sizes and shapes are characterized using the spatial contextual information. The novelty of this approach lies in the identification and separability of small FE pockets which could not be achieved with pixel-based maximum likelihood classifier (MLC). The SRM parameters were optimized and found comparable to previous studies. Classification accuracy obtained with SRM at scale factor 3 is κ = 0.62 that is superior to accuracy of MLC (κ = 0.51). SRM is a promising tool for detection and monitoring of FE at Rutland Island in ANI, India.

Fully spatially adaptive smoothing parameter estimation for Markov random field super-resolution mapping of remotely sensed images

This article presents a fully spatially adaptive Markov random field (MRF)-based super-resolution mapping (SRM) technique to produce land-cover maps at a finer spatial resolution than the original coarse-resolution image. MRF combines the spectral and spatial energies; hence, an MRF-SRM technique requires a smoothing parameter to manage the contributions of these energies. The main aim of this article is to introduce a new method called fully spatially adaptive MRF-SRM to automatically determine the smoothing parameter, overcoming limitations of the previously proposed approaches. This method estimates the number of endmembers in each image and uses them to assess the proportions of classes within each coarse pixel by a linear spectral unmixing method. Then, the real pixel intensity vectors and the local properties of each coarse pixel are used to compute the local spectral energy change matrix and the local spatial energy change matrix for each coarse pixel. Each pair of matrices represents all possible situations in spatial and spectral energy change for each coarse pixel and can be used to examine the balance between spatial and spectral energies, and hence to estimate a smoothing parameter for each coarse pixel. Thus, the estimated smoothing parameter is fully spatially adaptive with respect to real pixel spectral vectors and their local properties. The performance of this method is evaluated using two synthetic images and an EO1-ALI (The Advanced Land Imager instrument on Earth Observing-1 satellite) multispectral remotely sensed image. Our experiments show that the proposed method outperforms the state-of-the-art techniques.

Super-resolution mapping of wetland inundation from remote sensing imagery based on integration of back-propagation neural network and genetic algorithm

Mapping the spatio-temporal characteristics of wetland inundation has an important significance to the study of wetland environment and associated flora and fauna. High temporal remote sensing imagery is widely used for this purpose with the limitations of relatively low spatial resolutions. In this study, a novel method based on integration of back-propagation neural network (BP) and genetic algorithm (GA), so-called IBPGA, is proposed for super-resolution mapping of wetland inundation (SMWI) from multispectral remote sensing imagery. The IBPGA-SMWI algorithm is developed, including the fitness function and integration search strategy. IBPGA-SMWI was evaluated using Landsat TM/ETM+ imagery from the Poyanghu wetland in China and the Macquarie Marshes in Australia. Compared with traditional SMWI methods, IBPGA-SMWI consistently achieved more accurate super-resolution mapping results in terms of visual and quantitative evaluations. In comparison with GA-SMWI, IBPGA-SMWI not only improved the accuracy of SMWI, but also accelerated the convergence speed of the algorithm. The sensitivity analysis of IBPGA-SMWI in relation to standard crossover rate, BP crossover rate and mutation rate was also carried out to discuss the algorithm performance. It is hoped that the results of this study will enhance the application of median-low resolution remote sensing imagery in wetland inundation mapping and monitoring, and ultimately support the studies of wetland environment.

Hybrid Constraints of Pure and Mixed Pixels for Soft-Then-Hard Super-Resolution Mapping With Multiple Shifted Images

Multiple shifted images (MSIs) have been widely applied to many super-resolution mapping (SRM) approaches to improve the accuracy of fine-scale land-cover maps. Most SRM methods with MSIs involve two processes: subpixel sharpening and class allocation. Complementary information from the MSIs has been successfully adopted to produce soft attribute values of subpixels during the subpixel sharpening process. Such information, however, is not used in the second process of class allocation. In this paper, a new class-allocation algorithm, named “hybrid constraints of pure and mixed pixels” (HCPMP), is proposed to allocate land-cover classes to subpixels using MSIs. HCPMP first determines the classes of subpixels that overlap with the pure pixels of auxiliary images in MSIs, after which the remaining subpixels are classified using information derived from the mixed pixels of the base image in MSIs. An artificial image and two remote sensing images were used to evaluate the performance of the proposed HCPMP algorithm. The experimental results demonstrate that HCPMP successfully applied MSIs to produce SRM maps that are visually closer to the reference images and that have greater accuracy than five existing class-allocation algorithms. Especially, it can produce more accurate SRM maps for high-resolution land-cover classes than low-resolution cases. The algorithm takes slightly less runtime than class allocation using linear optimization techniques. Hence, HCPMP provides a valuable new solution for class allocation in SRM using auxiliary data from MSIs.

Super-Resolution Land Cover Mapping Based on Multiscale Spatial Regularization

Super-resolution mapping (SRM) is a method for allocating land cover classes at a fine scale according to coarse fraction images. Based on a spatial regularization framework, this paper proposes a new regularization method for SRM that integrates multiscale spatial information from the fine scale as a smooth term and from the coarse scale as a penalty term. The smooth term is considered a homogeneity constraint, and the penalty term is used to characterize the heterogeneity constraint. Specifically, the smooth term depends on the local fine scale spatial consistency, and is used to smooth edges and eliminate speckle points. The penalty term depends on the coarse scale local spatial differences, and suppresses the over-smoothing effect from the fine scale information while preserving more details (e.g., connectivity and aggregation of linear land cover patterns). We validated our method using simulated and synthetic images, and compared the results to four representative SRM algorithms. Our numerical experiments demonstrated that the proposed method can produce more accurate maps, reduce differences in the number of patches, visually preserve smoother edges and more details, reject speckle points, and suppress over-smoothing.

Landscape heterogeneity and scale considerations for super-resolution mapping

Super-resolution mapping (SRM) is a rapidly emerging field of remote sensing that seeks to map the spatial distribution of land cover proportions resulting from soft classification techniques. Many methods have been proposed, but there has been little impetus to converge these efforts or compare the results. One reason for this lack of comparison is the issue of landscape heterogeneity and the range of scaling issues it embodies. Landscape heterogeneity refers to the complex distribution of land cover types across the landscape and the spatial patterns or spatial frequency of those land cover types. Landscape heterogeneity and the spatial patterns it produces are inherently scale-dependent while SRM is fundamentally an inverse scaling challenge being applied to real landscapes. The result is that SRM techniques developed for a particular landscape or scaling factor may not translate well when applied to other landscapes with different heterogeneity or computed for a different scaling factor. This lack of transferability due to landscape heterogeneity has largely been ignored in the literature. Heterogeneity is rarely reported in SRM studies and its behaviour across resolutions rarely integrated into SRM techniques. This article discusses the importance of heterogeneity in SRM, demonstrates how heterogeneity impacts SRM results, proposes several solutions for reporting landscape heterogeneity, and discusses the difficulties associated with incorporating heterogeneity into SRM. By highlighting the interrelatedness between landscape heterogeneity and SRM, the aim is for studies to begin reporting heterogeneity to facilitate inter-comparisons and ultimately incorporate the scale-dependency of landscape heterogeneity into SRM techniques to improve accuracy and applicability.