High Resolution Multispectral Images Research Articles

Assessing the Severity of Verticillium Wilt in Cotton Fields and Constructing Pesticide Application Prescription Maps Using Unmanned Aerial Vehicle (UAV) Multispectral Images

Cotton Verticillium wilt is a common fungal disease during the growth of cotton, leading to the yellowing of leaves, stem dryness, and root rot, severely affecting the yield and quality of cotton. Current monitoring methods for Verticillium wilt mainly rely on manual inspection and field investigation, which are inefficient and costly, and the methods of applying pesticides in cotton fields are singular, with issues of low pesticide efficiency and uneven application. This study aims to combine UAV remote sensing monitoring of cotton Verticillium wilt with the precision spraying characteristics of agricultural drones, to provide a methodological reference for monitoring and precision application of pesticides for cotton diseases. Taking the cotton fields of Shihezi City, Xinjiang as the research subject, high-resolution multispectral images were collected using drones. Simultaneously, 150 sets of field samples with varying degrees of Verticillium wilt were collected through ground data collection, utilizing data analysis methods such as partial least squares regression (PLSR) and neural network models; additionally, a cotton Verticillium wilt monitoring model based on drone remote sensing images was constructed. The results showed that the estimation accuracy R2 of the PLSR and BP neural network models based on EVI, RENDVI, SAVI, MSAVI, and RDVI vegetation indices were 0.778 and 0.817, respectively, with RMSE of 0.126 and 0.117, respectively. Based on this, an analysis of the condition of the areas to be treated was performed, combining the operational parameters of agricultural drones, resulting in a prescription map for spraying against cotton Verticillium wilt.

Research on the Optimization of Multi-Class Land Cover Classification Using Deep Learning with Multispectral Images

With the advancement of artificial intelligence, deep learning has become instrumental in land cover classification. While there has been a notable emphasis on refining model structures to improve classification accuracy, it is imperative to also emphasize the pivotal role of data-driven optimization techniques. This paper presents an in-depth investigation into optimizing multi-class land cover classification using high-resolution multispectral images from Worldview3. We explore various optimization strategies, including refined sampling strategies, data band combinations, loss functions, and model enhancements. Our optimizations led to a substantial increase in the Mean Intersection over Union (mIoU) classification accuracy, improving from a baseline of 0.520 to a final accuracy of 0.709, which represents a 35.2% enhancement. Specifically, by optimizing the classic semantic segmentation network in four key aspects, we improved the mIoU by 15.5%. Further improvements through changes in data combinations, sampling methods, and loss functions led to an overall 17.2% increase in mIoU. The proposed model optimization methods enabled the OUNet to outperform the baseline model by providing more precise edge detection and feature representation, while reducing the model parameters scale. Experimental evidence shows that in the application of multi-class land surface classification, increasing the quantity and diversity of samples, avoiding data imbalance issues, is equally valuable for improving overall classification accuracy as it is for enhancing model performance.

Bayesian fusion of multispectral and panchromatic images using a multi-mode and multiorder gradient tensor-based l 1 / 2 sparse model

ABSTRACT In this letter, based on the tensor representation modelling, we propose a multi-mode and multi-order gradient tensor-based non-convex model (M2GTNM) for Bayesian fusion of multispectral (MS) and panchromatic (Pan) images, which aims at producing the high-resolution MS (HRMS) images. Specifically, by modelling the MS image as the order-3 tensor, we mainly develop the multi-mode and multi-order gradient tensor sparse priors of MS image for fusion. For the spectral preservation of low-resolution MS (LRMS) image, the spectral fidelity constraint between HRMS and LRMS images is imposed. For the spatial-mode prior modelling, the multi-order spatial gradient tensor-based non-convex l 1 / 2 sparse prior between HRMS and Pan images is particularly imposed. Moreover, for the spectral-mode prior modelling, the spectral gradient tensor-based non-convex l 1 / 2 sparse prior between HRMS and upsampled LRMS images is further imposed. Then, we apply the alternating direction method of multipliers to optimize the proposed model. Finally, the reduced-scale and full-scale fusion experiments both validate the effectiveness of M2GTNM method.

Unsupervised multi-level spatio-spectral fusion transformer for hyperspectral image super-resolution

Fusing a low spatial resolution hyperspectral image (LR-HSI) with a high spatial resolution multispectral image (HR-MSI) is widely used for HSI super-resolution. Recent works still face problems in exploring global spatio-spectral correlation and lack effective utilization of multi-level features of the inputs (i.e., HR-MSI and LR-HSI), which results in a lack of similarity between the reconstruction and the inputs, ultimately causing significant spatio-spectral distortion. To solve these problems, we design an Unsupervised Multi-level Spatio-spectral Fusion Transformer (UMSFT). In UMSFT, a novel multi-level features fusion strategy is constructed, which fuses the hierarchy features of the inputs via proposed Spatio-spectral Feature Fusion Attention Blocks (S2F-FAB) in a level-by-level manner, thereby fully exploring the interaction between the hierarchical features. The S2F-FAB is specially designed for HSI-MSI fusion consisting of two components: (1) a spatial fusion module (Spa-FM) is designed for spatial domain fusion, and its output is set as Values (V) of a transformer; (2) a novel spectral feature cross attention (Spe- FCA) formulates the features of LR-HSI and HR-MSI as Queries (Q) and Keys (K), respectively, and achieves spectral domain fusion by applying attention mechanism along the spectral dimension. Incorporating spatial detail reconstruction and spectral feature integration into the attention mechanism, the S2F-FAB efficiently exploits the spatio-spectral correlation between target HR-HSI and inputs. Experimental results on three public datasets and our real-world images show the superiority of our method as compared with eleven state-of-the-art methods. Codes will be available at https://github.com/Caoxuheng/HIFtool.

Semi-blind hyperspectral and multispectral image fusion via generalized inverse matrix optimization

Combining a low spatial resolution hyperspectral image (HSI) with a high spatial resolution multispectral image (MSI) is an economical way to obtain a high spatial resolution hyperspectral image (HR-HSI). However, existing fusion methods often rely on a critical variable, the point spread function (PSF), which is difficult to obtain in practice. Additionally, these methods often require a long fusion time and have many parameters that need re-debugging on different datasets to achieve satisfactory results. Many existing methods lack robustness in handling complex scenarios, such as complex PSF, changing spatial downsampling factors, and large-scale datasets. These issues can result in degraded performance or render the methods inapplicable to practical scenarios. Therefore, we propose an efficient and parameterless semi-blind fusion method based on generalized inverse matrix optimization (GIMO), which is robust for multiple complex situations. Our method uses the definition of generalized inverse matrix and the relationship among HSI, MSI, and HR-HSI to approximate the function module of PSF. Then, we calculate the generalized inverse matrix of the spectral response function (SRF). Finally, we optimize the generalized inverse matrix of SRF using the Sylvester equation and multiply it with MSI to obtain HR-HSI. This is the first proposed method to achieve hyperspectral image super-resolution by optimizing the inverse matrix of the spectral response function. Experiments on multiple simulated datasets and one real dataset have validated that our GIMO method outperforms existing state-of-the-art methods, especially regarding fusion time and robustness for various complex scenarios.

Contributions to a global understanding of socioenvironmental justice related to urban forest: Trends from Brazilian cities in the southeastern Paraná State

Global studies on urban forests (UFs) have extensively covered both environmental and social aspects, including concerns about public health and green solutions provided by trees and forests in cities. However, this trend has not been widely observed in South America, particularly in cities located outside and distant from capital cities and wealthier regions. So, this study examines disparities in socioenvironmental justice of cities in the poorest region of Paraná State, Brazil, comparing to Curitiba (CWB), the capital and one of the greenest cities of Brazil, and proposes a methodology to surpass the scarcity of adequate dataset. A four-dimensional analysis covers the configuration of UF landscapes, the provision of ecosystem services, accessibility to UF, and their interrelation with socioeconomic indicators. We used high-resolution multispectral images from PlanetScope Dove and employed a machine learning algorithm to accurately map UF. We quantified landscape patterns of UFs using a variety of landscape metrics (LM), including mean patch area (AREA_MN), percentage of the landscape class (PLAND), largest patch index (LPI), mean shape index (SHAPE_MN), and the percentage of similar adjacencies (PLADJ). Additionally, we estimated ecosystem services such as total biomass (TB), carbon stock (TCS), carbon dioxide equivalent (CO2eq), urban heat island (UHI), and available public green areas (PGA). We created accessibility maps to PGA at distances of 300 and 800 m. A Mann-Kendall correlation analysis was employed to explore the relationships between these variables and various socioeconomic indicators, including the percentage of people in households without adequate water supply and sewage (TAEE), the percentage of the population living in urban households without garbage collection service (TPSL), the illiteracy rate of the population aged 15 years or older (TANAL), the proportion of people with per capita household income equal to or less than half minimum wage at year of 2010 (TVULNER), the number of schools (SCHOOLS), and the number of permanent residents (MPERM). The Sen's slope test was performed to determine the magnitude of these correlations. Our four-dimensional analysis revealed significant disparities in the distribution and accessibility of ecosystem services within the cities of southeastern Paraná State when compared to CWB. Correlation analyses revealed significant discrepancies between different socioeconomic conditions and the availability of ecosystem services. Financial vulnerability is positively correlated with ecosystem services, while the lack of sanitation negatively impacts their availability. The results highlight the urgent need for equitable distribution of UF ecosystem services.

DUCD: Deep Unfolding Convolutional-Dictionary network for pansharpening remote sensing image

The goal of pansharpening methods is to complement the spectral and spatial information contained in Multi-spectral (MS) and panchromatic (PAN) images to obtain the desired High-resolution multispectral (HRMS) image. The existing majority of pansharpening methods either extract feature information separately from the MS image and PAN image, or extract feature information after concatenating the MS image and PAN image. However, the entire extraction process lacks the utilization of complementary information and tends to generate redundant information, thereby leading to the loss of certain important information during the extraction process, which in turn affects the overall performance. In order to better utilize the complementary information between the MS image and PAN image and enhance the interpretability of the network, we propose the Deep Unfolding Convolutional-Dictionary Network (DUCD) for pansharpening in this paper. This network fully integrates complementary information between the MS image and PAN image to generate the final fused image. The entire network structure consists of two parts: The encoder and the decoder. In the encoder part of the network, we clarify the common and unique feature information between MS and PAN images by constructing an observation model. Simultaneously, we use the approximate gradient algorithm to continuously optimize the model and iteratively unfold it into a deep network structure. In the decoder part of the network, we concatenate the obtained common and specific information from MS and PAN images and pass them through convolutional and activation layers. Subsequently, they are input into the introduced Frequency Domain-based Transformer (FDT) module and an information-lossless inversible neural network(INN). This provides a more efficient method for establishing long-range dependency relationships between feature extraction and feature fusion. To demonstrate the effectiveness of our proposed method, we conduct extensive experiments on three benchmark datasets QB, GF2 and WV3. Experimental results show that our method outperforms the current SOTA Pansharpening methods in terms of performance.

Measuring Coupled Fire–Atmosphere Dynamics: The California Fire Dynamics Experiment (CalFiDE)

Abstract The social, economic, and ecological impacts of wildfires are increasing over much of the United States and globally, partially due to changing climate and build-up of fuels from past forest management practices. This creates a need to improve coupled fire–atmosphere forecast models. However, model performance is difficult to evaluate due to scarcity of observations for many key fire–atmosphere interactions, including updrafts and plume injection height, plume entrainment processes, fire intensity and rate-of-spread, and plume chemistry. Intensive observations of such fire–atmosphere interactions during active wildfires are rare due to the logistical challenges and scales involved. The California Fire Dynamics Experiment (CalFiDE) was designed to address these observational needs, using Doppler lidar, high-resolution multispectral imaging, and in situ air quality instruments on a NOAA Twin Otter research aircraft, and Doppler lidars, radar, and other instrumentation on multiple ground-based mobile platforms. Five wildfires were studied across northern California and southern Oregon over 16 flight days from 28 August to 25 September 2022, including a breadth of fire stages from large blow-up days to smoldering air quality observations. Missions were designed to optimize the observation of the spatial structure and temporal evolution of each fire from early afternoon until sunset during multiple consecutive days. The coordination of the mobile platforms enabled four-dimensional sampling strategies during CalFiDE that will improve understanding of fire–atmosphere dynamics, aiding in model development and prediction capability. Satellite observations contributed aerosol measurements and regional context. This article summarizes the scientific objectives, platforms and instruments deployed, coordinated sampling strategies, and presents first results.

Crop mapping in smallholder farms using unmanned aerial vehicle imagery and geospatial cloud computing infrastructure

Smallholder farms are major contributors to agricultural production, food security, and socio-economic growth in many developing countries. However, they generally lack the resources to fully maximize their potential. Subsequently they require innovative, evidence-based and lower-cost solutions to optimize their productivity. Recently, precision agricultural practices facilitated by unmanned aerial vehicles (UAVs) have gained traction in the agricultural sector and have great potential for smallholder farm applications. Furthermore, advances in geospatial cloud computing have opened new and exciting possibilities in the remote sensing arena. In light of these recent developments, the focus of this study was to explore and demonstrate the utility of using the advanced image processing capabilities of the Google Earth Engine (GEE) geospatial cloud computing platform to process and analyse a very high spatial resolution multispectral UAV image for mapping land use land cover (LULC) within smallholder farms. The results showed that LULC could be mapped at a 0.50 m spatial resolution with an overall accuracy of 91%. Overall, we found GEE to be an extremely useful platform for conducting advanced image analysis on UAV imagery and rapid communication of results. Notwithstanding the limitations of the study, the findings presented herein are quite promising and clearly demonstrate how modern agricultural practices can be implemented to facilitate improved agricultural management in smallholder farmers.

Hyperspectral and multispectral image fusion via residual selective kernel attention-based U-net

ABSTRACT The fusion of low-resolution hyperspectral image (LR-HSI) and high-resolution multispectral image (HR-MSI) is a crucial technology for producing high-resolution hyperspectral images. Most existing image fusion algorithms based on deep learning do not fully utilize the ability of neural network to extract and process multi-scale features, which leads to the problem of difficulty in fully learning features and ambiguity of features. In order to overcome these issues, a residual selective kernel attention-based U-net named RSKAU-net is designed for LR-HSI and HR-MSI fusion. RSKAU-net is constructed by a residual selective kernel module with an attention mechanism and a channel attention block. The residual selective kernel attention-based (RSKA) module is designed to process images of different resolutions, which adaptively extracts multi-scale features and efficiently emphasizes significant features through the attention mechanism. The channel attention (CA) module retains important spectral information by assigning different weights to each channel of LR-HSI. The proposed network can enhance the spatial information of LR-HSI while preserving its spectral information. Meanwhile, it effectively fuses the features from the source image to obtain the HR-HSI with rich details. The experimental results demonstrate that the proposed network has advantages in terms of both visual effect and objective quantitative indices when compared to existing HSI-MSI fusion approaches.

Transformer-based dual path cross fusion for pansharpening remote sensing images

ABSTRACT The purpose of pan-sharpening is to generate high-resolution multispectral (HRMS) images by combining multispectral images (MS) and panchromatic images (PAN), which have become an important part of remote sensing image processing. Therefore, how to better extract complete feature information from MS images and PAN images has become the focus of our attention. In this paper, we propose a new pansharpening method, called Transformer-based Dual-path cross fusion network (TDF) for Pan-sharpening remote sensing images, which aims to extract the spatial details of PAN images while maintaining the spectral fidelity of MS images. The whole network structure can be divided into two parts: in the encoder part, we adopt the Swin-Transformer module for the downsampling operation, which expands the sensory field of the network to the feature map, and then extracts the global information. However, since the Swin-Transformer module is not good at extracting pixel-level details, we introduce the Base Feature Extraction (BFE) and the Invertible Neural Network Block (INNB) modules for the interaction between the local and the global feature information. we also introduce the Edge-Enhancement Block (EEB) to further enhance the feature extraction at multi-scales during the image fusion process. In the decoder section, we once again employ the Swin Transformer module for downsampling. After the convolution operation and activation function, we utilize the Sub-Pixel Convolutional Neural Network for upsampling to generate the ultimate high-resolution multispectral images. Simulation experiments and real experiments are conducted on QuickBird (QB) and WorldView2 (WV2) datasets, which demonstrated our method are superior to the current methods.

A framework of freshwater and saline lake typology classification through leveraging hydroclimate, spectral, and literature evidence

Separating saline and freshwater lakes is of immense importance for monitoring lentic ecosystems and assessing surface freshwater availability. However, conventional lake typology often relies on in situ salinity measurements, thus limiting the feasibility of a global application. To enable an effective workaround, we here propose and test a framework that classifies saline and freshwater lakes through systematic, integrated leveraging of hydrological, climatological, spectral, and literature evidence. In principle, saline lakes were sequentially identified if a lake is (1) located at the drainage terminal where salinity tends to build up as a result of lacking surface outflow (hydrological evidence), (2) distributed in an arid/semiarid climate with lacustrine evaporites visible from a high-resolution multispectral image (climatological and spectral evidence), or (3) documented to be (sub)saline or have measured salinity exceeding the maximum freshwater threshold (literature evidence). We applied the HydroSHEDS hydrography dataset, the Köppen-Geiger climate classification, and a large volume of spectral satellite images and literature to implement this framework. This framework was tested on the Tibetan Plateau using the Landsat-based, 30-m-resolution UCLA Circa-2015 global lake mask, and the result was benchmarked against the Chinese national lake survey. Our collected literature shows a 95% agreement between the lake types classified using hydrological, climatological, and spectral evidence alone and those documented in the literature. After further corrections based on the literature evidence, our final classification reached an overall accuracy of nearly 70% in terms of lake count and 94% in lake area, implying that the efficacy of our method tends to favor relatively large lakes. Given such promising accuracies, we further applied our framework to Australia, showcasing its effectiveness for lake typology classification without additional in situ surveys across the world’s most saline continent. While our method, in theory, results in a conservative recognition of saline lakes and should not be considered a complete substitute for in situ surveys, the proposed framework demonstrates potential for performing global-scale freshwater and saline lake typology classification with reasonable accuracies, using only remote sensing images, open-source hydrography and climate data, and existing literature.

Pan-sharpening via intrinsic decomposition knowledge distillation

Existing deep-learning-based pan-sharpening strategies mainly involve the fusion of panchromatic and multispectral (MS) information at both the pixel and feature levels. In this paper, we hypothesize that the MS image can be expressed as the multiplication of reflectance and illumination components, and that the reflection components of low-resolution (LR) MS and high-resolution (HR) MS images are invariant. Here, the spectral reflection component can effectively describe the spectral response of an object, while the illumination component can effectively describe its texture. Based on this hypothesis, we propose a novel and concise pan-sharpening framework called intrinsic decomposition knowledge distillation. Specifically, the teacher network decomposes the HR MS image into reflectance and illumination components, which are then combined in the student network with the reflectance component and the enhanced illumination component from LR MS to reconstruct the pan-sharpened image. To approximate the component distributions from the teacher network, we introduce a novel three-stage knowledge distillation strategy that can transfer knowledge about the relationships between components and constrain the student network. Our quantitative and qualitative comparisons demonstrate the reasonableness of our hypothesis and the effectiveness of our proposed method in significantly improving perception quality.

Triple disentangled network with dual attention for remote sensing image fusion

Remote sensing applications, such as detection and recognition of objects, need a large number of high spatial resolution (HR) images. As a feasible technique, image fusion is considered to produce HR images. However, most of remote sensing image fusion methods based on deep neural network (DNN) are mainly limited by the extraction of spatial and spectral features. These methods ignore the redundancy among these features, which causes spatial and spectral distortions in the fused image. In this paper, we propose a novel image fusion method based on a triple disentangled network (TDNet) with dual attention to reduce the redundancy in the extracted features. In the proposed method, it is assumed that the information in panchromatic (PAN) and low spatial resolution multispectral (LRMS) images can be encoded as the spatial, spectral, and common features. Specifically, we construct a triple-stream network to extract these features. To efficiently model the spatial and spectral information in PAN and LRMS images, local–global attention and interdependency attention are designed and integrated into the network. Then, the redundancy among these features is reduced by disentangled learning, in which these features are recombined to reconstruct the PAN and LRMS images. Besides, these features should complement each other. So, we utilize the maximal coding rate reduction to balance the redundancy and complementarity among them. Finally, all features are recombined to synthesize the high spatial resolution multispectral image. The experimental results demonstrate that the proposed TDNet has a superior performance in terms of qualitative and quantitative evaluations. The code link is https://github.com/RSMagneto/TDNet.

Unsupervised Deep Tensor Network for Hyperspectral-Multispectral Image Fusion.

Fusing low-resolution (LR) hyperspectral images (HSIs) with high-resolution (HR) multispectral images (MSIs) is a significant technology to enhance the resolution of HSIs. Despite the encouraging results from deep learning (DL) in HSI-MSI fusion, there are still some issues. First, the HSI is a multidimensional signal, and the representability of current DL networks for multidimensional features has not been thoroughly investigated. Second, most DL HSI-MSI fusion networks need HR HSI ground truth for training, but it is often unavailable in reality. In this study, we integrate tensor theory with DL and propose an unsupervised deep tensor network (UDTN) for HSI-MSI fusion. We first propose a tensor filtering layer prototype and further build a coupled tensor filtering module. It jointly represents the LR HSI and HR MSI as several features revealing the principal components of spectral and spatial modes and a sharing code tensor describing the interaction among different modes. Specifically, the features on different modes are represented by the learnable filters of tensor filtering layers, the sharing code tensor is learned by a projection module, in which a co-attention is proposed to encode the LR HSI and HR MSI and then project them onto the sharing code tensor. The coupled tensor filtering module and projection module are jointly trained from the LR HSI and HR MSI in an unsupervised and end-to-end way. The latent HR HSI is inferred with the sharing code tensor, the features on spatial modes of HR MSIs, and the spectral mode of LR HSIs. Experiments on simulated and real remote-sensing datasets demonstrate the effectiveness of the proposed method.

CS2DIPs: Unsupervised HSI Super-Resolution Using Coupled Spatial and Spectral DIPs.

In recent years, fusing high spatial resolution multispectral images (HR-MSIs) and low spatial resolution hyperspectral images (LR-HSIs) has become a widely used approach for hyperspectral image super-resolution (HSI-SR). Various unsupervised HSI-SR methods based on deep image prior (DIP) have gained wide popularity thanks to no pre-training requirement. However, DIP-based methods often demonstrate mediocre performance in extracting latent information from the data. To resolve this performance deficiency, we propose a coupled spatial and spectral deep image priors (CS2DIPs) method for the fusion of an HR-MSI and an LR-HSI into an HR-HSI. Specifically, we integrate the nonnegative matrix-vector tensor factorization (NMVTF) into the DIP framework to jointly learn the abundance tensor and spectral feature matrix. The two coupled DIPs are designed to capture essential spatial and spectral features in parallel from the observed HR-MSI and LR-HSI, respectively, which are then used to guide the generation of the abundance tensor and spectral signature matrix for the fusion of the HSI-SR by mode-3 tensor product, meanwhile taking some inherent physical constraints into account. Free from any training data, the proposed CS2DIPs can effectively capture rich spatial and spectral information. As a result, it exhibits much superior performance and convergence speed over most existing DIP-based methods. Extensive experiments are provided to demonstrate its state-of-the-art overall performance including comparison with benchmark peer methods.

Progressive Spatial Information-Guided Deep Aggregation Convolutional Network for Hyperspectral Spectral Super-Resolution.

Fusion-based spectral super-resolution aims to yield a high-resolution hyperspectral image (HR-HSI) by integrating the available high-resolution multispectral image (HR-MSI) with the corresponding low-resolution hyperspectral image (LR-HSI). With the prosperity of deep convolutional neural networks, plentiful fusion methods have made breakthroughs in reconstruction performance promotions. Nevertheless, due to inadequate and improper utilization of cross-modality information, the most current state-of-the-art (SOTA) fusion-based methods cannot produce very satisfactory recovery quality and only yield desired results with a small upsampling scale, thus affecting the practical applications. In this article, we propose a novel progressive spatial information-guided deep aggregation convolutional neural network (SIGnet) for enhancing the performance of hyperspectral image (HSI) spectral super-resolution (SSR), which is decorated through several dense residual channel affinity learning (DRCA) blocks cooperating with a spatial-guided propagation (SGP) module as the backbone. Specifically, the DRCA block consists of an encoding part and a decoding part connected by a channel affinity propagation (CAP) module and several cross-layer skip connections. In detail, the CAP module is customized by exploiting the channel affinity matrix to model correlations among channels of the feature maps for aggregating the channel-wise interdependencies of the middle layers, thereby further boosting the reconstruction accuracy. Additionally, to efficiently utilize the two cross-modality information, we developed an innovative SGP module equipped with a simulation of the degradation part and a deformable adaptive fusion part, which is capable of refining the coarse HSI feature maps at pixel-level progressively. Extensive experimental results demonstrate the superiority of our proposed SIGnet over several SOTA fusion-based algorithms.

SSTF-Unet: Spatial-Spectral Transformer-Based U-Net for High-Resolution Hyperspectral Image Acquisition.

To obtain a high-resolution hyperspectral image (HR-HSI), fusing a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI) is a prominent approach. Numerous approaches based on convolutional neural networks (CNNs) have been presented for hyperspectral image (HSI) and multispectral image (MSI) fusion. Nevertheless, these CNN-based methods may ignore the global relevant features from the input image due to the geometric limitations of convolutional kernels. To obtain more accurate fusion results, we provide a spatial-spectral transformer-based U-net (SSTF-Unet). Our SSTF-Unet can capture the association between distant features and explore the intrinsic information of images. More specifically, we use the spatial transformer block (SATB) and spectral transformer block (SETB) to calculate the spatial and spectral self-attention, respectively. Then, SATB and SETB are connected in parallel to form the spatial-spectral fusion block (SSFB). Inspired by the U-net architecture, we build up our SSTF-Unet through stacking several SSFBs for multiscale spatial-spectral feature fusion. Experimental results on public HSI datasets demonstrate that the designed SSTF-Unet achieves better performance than other existing HSI and MSI fusion approaches.

Maize Evapotranspiration Estimates Using Planet Dove Mini-Satellites and Field-Level InfraRed Thermometers

Highlights Multispectral images from mini-satellites can be used in a reflectance-based crop coefficient model to estimate crop water use. Surface reflectance from mini-satellites and temperature from in-situ infrared thermometers can be used in a land surface energy balance model to more accurately estimate maize actual evapotranspiration. It is feasible to monitor daily maize actual evapotranspiration using Planet Dove mini-satellites on cloud-free days. Abstract. Water scarcity, climate change, and the need to sustain irrigated agriculture demand improved methods to manage irrigation water. Irrigation scheduling can be improved when remote sensing (RS) images are used because RS captures the actual crop field conditions. Mini-satellites (mS) are small spaceborne RS platforms that can provide high spatial and temporal resolution multispectral (MS) images. These multispectral images can be used to estimate maize actual evapotranspiration (ETa) on a daily basis. Thus, the main objective of this study was to assess the accuracy of maize ETa estimates when mS MS images were used in two different ETa algorithms. This study was conducted at two research sites in northern Colorado, USA. A reflectance-based crop coefficient and a one-source surface energy balance (OSSEB) models were applied, using MS Planet Dove images, to map maize ETa. An Eddy Covariance energy balance system was used to assess the accuracy of distributed ETa estimates. Results indicate that the reflectance-based crop coefficient method can be used with mS MS images yielding acceptable estimates. Both ETa models performed better in the case of the sub-surface irrigated maize. Further, OSSEB-based estimates of ETa model were more accurate over the surface irrigated maize and homogeneous soils; when surface reflectance images were paired with radiometric surface temperature data from ground-based infrared thermometers. These results imply that potentially maize ETa could be mapped, on a daily basis with good accuracy, using Planet Dove mS-based MS reflectance images and ground-based radiometric surface temperatures. Keywords: Agricultural water management, Crop water use, Irrigation, Land surface energy balance, Reflectance-based crop coefficients, Remote sensing.

Unsupervised Test-Time Adaptation Learning for Effective Hyperspectral Image Super-Resolution With Unknown Degeneration.

Fusing a low-resolution hyperspectral image (HSI) with a high-resolution (HR) multi-spectral image has provided an effective way for HSI super-resolution (SR). The key lies on inferring the posteriori of the latent (i.e., HR) HSI using an appropriate image prior and the likelihood determined by the degeneration between the latent HSI and the observed images. However, in scenarios with complex imaging environments and various imaging scenes, the prior of HSIs can be prohibitively complicated and the degeneration is often unknown, which causes it difficult to accurately infer the posteriori of each latent HSI. To tackle this problem, we present an unsupervised test-time adaptation learning (UTAL) framework for HSI SR under unknown degeneration. Instead of directly modeling the complicated image prior, it first implicitly learns a content-agnostic prior shared across different images through supervisedly pre-training a mutual-guiding fusion module on extensive synthetic data. Then, it adapts the shared prior to those private characteristics in the latent HSI for posteriori inference through unsupervisedly learning a self-guiding adaptation module and a degeneration estimation network on two observed images in the test phase. Such a two-stage learning scheme models the complicated image prior in a divide-and-conquer manner, which eases the modeling difficulty and improves the prior accuracy. Moreover, the unknown degeneration can be estimated properly. Both of these two advantages empower us to accurately infer the posteriori of the latent HSI, thereby increasing the generalization performance in real applications. Additionally, in order to further mitigate the over-fitting in coping with more challenging cases (e.g., degenerations in both spectral and spatial domains are unknown) and speed up, we propose to meta-train UTAL on extensive synthetic SR tasks and solve it using an alternative optimization strategy such that UTAL learns to produce good generalization performance in real challenging cases with a small number of gradient descent steps. To verify the efficacy of UTAL, we evaluate it on HSI SR tasks with different unknown degenerations as well as some other HSI restoration tasks (e.g., compressive sensing), and report strong results superior to that of existing competitors.