Spectral Recovery Research Articles

Spectral peak recovery in parametrically amplified THz-repetition-rate bursts.

Multi-photon resonant spectroscopies require tunable narrowband excitation to deliver spectral selectivity and, simultaneously, high temporal intensity to drive a nonlinear-optical process. These contradictory requirements are achievable with bursts of ultrashort pulses, which provides both high intensity and tunable narrowband peaks in the frequency domain arising from spectral interference. However, femtosecond pulse bursts need special attention during their amplification [Optica7, 1758 (2020)10.1364/OPTICA.403184], which requires spectral peak suppression to increase the energy safely extractable from a chirped-pulse amplifier (CPA). Here, we present a method combining safe laser CPA, relying on spectral scrambling, with a parametric frequency converter that automatically restores the desired spectral peak structure and delivers narrow linewidths for bursts of ultrashort pulses at microjoule energies. The shown results pave the way to new high-energy ultrafast laser sources with controllable spectral selectivity.

Optimized clustering sample selection for spectral reflectance recovery

The accuracy of spectral recovery depends heavily on the selection of an appropriate sample set, so the optimized sample selection by clustering strategy can improve the spectral recovery results. This paper presents a sample optimization method that combines hierarchical clustering and K-mean angle similar clustering to achieve this process. The proposed method employs the hierarchical clustering to divide the training sample dataset into 15 subspaces and obtain 15 subspace centroids. The similarity distance is then calculated between the testing sample and each subspace samples, and the subspace with the sample having the smallest distance is selected. The testing sample is utilized as a priori centroid, which clusters the optimal subspace by competition with the centroid of the subspace selected. This iterative process continues until the centroid of the subspace remains unaltered. Finally, the training samples within the optimal subspace use to recover spectral reflectance through Euclidean distance weighting. Experimental results demonstrate that the proposed method outperforms existing methods in terms of spectral and colorimetric accuracy, as well as stability and robustness. This research provides a solution to the problem of data redundancy in the spectral recovery process and enhances the accuracy and efficiency of spectral recovery.

Pan-sharpening via conditional invertible neural network

In the realm of conventional deep-learning-based pan-sharpening approaches, there has been an ongoing struggle to harmonize the input panchromatic (PAN) and multi-spectral (MS) images across varied channels. Existing methods have often been stymied by spectral distortion and an inadequate texture representation. To address these limitations, we present an innovative constraint-based image generation strategy tailored for the pan-sharpening task. Our method employs a multi-scale conditional invertible neural network, named PSCINN, which is capable of converting the ground truth MS image into a downscaled MS image and a latent variable, all under the guidance of the PAN image. Subsequently, the resampled latent variable, obtained from a prior distribution, and the low-resolution MS image are harnessed to predict the pan-sharpened image in an information-preserving manner, with the PAN image providing essential guidance during the reversion process. Furthermore, we meticulously architect a conditional invertible block to construct a Jacobian Determinant for the spectral information recovery. This structure effectively pre-processes the conditioning PAN image into practical texture information, thereby preventing the spectral information in the pan-sharpened result from potential contamination. The proposed PSCINN outperforms existing state-of-the-art pan-sharpening methodologies, both in terms of objective and subjective results. Post-treatment experiments underscore a substantial enhancement in the perceived quality attributed to our method. The source code for PSCINN will be accessible at https://github.com/jiaming-wang/PSCINN.

DBCT-Net:A dual branch hybrid CNN-transformer network for remote sensing image fusion

Remote sensing image fusion aims at fusing high spatial resolution single-band panchromatic (PAN) image with spectrally informative multispectral (MS) image to generate panchromatic sharpened image with high resolution and color information, it is also called pansharpening. Most of the proposed single convolutional neural network (CNN) or transformer-based pansharpening methods own several problems, such as inability to acquire long-range features or difficult to train, resulting the loss of spatial details and colors. In addition, the computational complexity of transformer cannot be ignored. In this work, we propose a dual-branch hybrid CNN-Transformer network (DBCT-Net) that utilizes the local specificity of CNN and models the global dependencies by transformer. First, a multi-branch dense connected block (MDCB-4) network is designed to obtain spectral and textural information in MS and PAN images, respectively. Next, an encoder–decoder transformer based on the self-attention and co-attention modules is able to inject the missing local and global information, which can further enhance the results. It is worth noting that an inverted multi-head transposed attention (IMTA) is applied here to build attention maps from feature dimensions, which greatly reduces the computation time. Finally, an image reconstruction module is employed to effectively fuse the acquired texture and spectral features. Furthermore, to generate visually better pansharpened images, we propose a combined loss function that includes a focal frequency loss. Extensive experiments on WorldView II (WV2), GF-2,and QuickBird (QB) datasets show that DBCT-Net can perform better in spatial preservation and spectral feature recovery.

Spectral Filter Selection Based on Human Color Vision for Spectral Reflectance Recovery.

Spectral filters are an important part of a multispectral acquisition system, and the selection of suitable filters can improve the spectral recovery accuracy. In this paper, we propose an efficient human color vision-based method to recover spectral reflectance by the optimal filter selection. The original sensitivity curves of the filters are weighted using the LMS cone response function. The area enclosed by the weighted filter spectral sensitivity curves and the coordinate axis is calculated. The area is subtracted before weighting, and the three filters with the smallest reduction in the weighted area are used as the initial filters. The initial filters selected in this way are closest to the sensitivity function of the human visual system. After the three initial filters are combined with the remaining filters one by one, the filter sets are substituted into the spectral recovery model. The best filter sets under L-weighting, M-weighting, and S-weighting are selected according to the custom error score ranking. Finally, the optimal filter set is selected from the three optimal filter sets according to the custom error score ranking. The experimental results demonstrate that the proposed method outperforms existing methods in spectral and colorimetric accuracy, which also has good stability and robustness. This work will be useful for optimizing the spectral sensitivity of a multispectral acquisition system.

Exploring the Influence of Forest Tenure and Protection Status on Post-Fire Recovery in Southeast Australia

Research Highlights: We used Landsat time series data to investigate the role forest tenure and protection status play in the recovery of a forest after a fire. Background and Objectives: Changing fire regimes put forests in southeast Australia under increasing pressure. Our investigation aimed to explore the impact of different forest management structures on a forest’s resilience to fire by looking at the post-fire recovery duration. Materials and Methods: The analysis included a total of 60.6 Mha of land containing 25.4 Mha of forest in southeast Australia. Multispectral time series data from Landsat satellites and a local reference dataset were used to model attributes of disturbance and recovery over a period of 33 years. Results: Protected public forest spectrally recovered 0.4 years faster than protected private forest. No other significant effects in relation to different tenure and protection status were found. Climatic and topographic variables were found to have much greater influence on post-fire spectral recovery. Conclusions: Protected area status in public forests resulted in slightly faster recovery, compared with the private protected forest estate. However, factors outside the control of land managers and policy makers, i.e., climatic and topographic variables, appear to have a much greater impact on post-fire recovery.

Spectral top-down recovery of latent tree models.

Modeling the distribution of high-dimensional data by a latent tree graphical model is a prevalent approach in multiple scientific domains. A common task is to infer the underlying tree structure, given only observations of its terminal nodes. Many algorithms for tree recovery are computationally intensive, which limits their applicability to trees of moderate size. For large trees, a common approach, termed divide-and-conquer, is to recover the tree structure in two steps. First, separately recover the structure of multiple, possibly random subsets of the terminal nodes. Second, merge the resulting subtrees to form a full tree. Here, we develop spectral top-down recovery (STDR), a deterministic divide-and-conquer approach to infer large latent tree models. Unlike previous methods, STDR partitions the terminal nodes in a non random way, based on the Fiedler vector of a suitable Laplacian matrix related to the observed nodes. We prove that under certain conditions, this partitioning is consistent with the tree structure. This, in turn, leads to a significantly simpler merging procedure of the small subtrees. We prove that STDR is statistically consistent and bound the number of samples required to accurately recover the tree with high probability. Using simulated data from several common tree models in phylogenetics, we demonstrate that STDR has a significant advantage in terms of runtime, with improved or similar accuracy.

Spectral super-resolution meets deep learning: Achievements and challenges

Spectral super-resolution (sSR) is a very important technique to obtain hyperspectral images from only RGB images, which can effectively overcome the high acquisition cost and low spatial resolution of hyperspectral imaging. From linear interpolation to sparse recovery, spectral super-resolution have gained rapid development. In the past five years, as deep learning has taken off in various computer vision tasks, spectral super-resolution algorithms based on deep learning have also exploded. From residual learning to physical modeling, deep learning-based models used in spectral super-resolution is multifarious. This paper has collected almost all deep learning-based sSR algorithms and reviewed them according to their main contributions, involving network architecture, feature extraction, and physical modeling. This paper proposed a benchmark about deep learning-based spectral super-resolution algorithms: https://github.com/JiangHe96/DL4sSR, and besides spectral recovery, their potential in colorization and spectral compressive imaging is also systematically discussed. Furthermore, we presented our views about challenges and possible further trends of deep learning-based sSR. Light-weight model architecture with generalization is crucial to in-camera processing. Model robustness should be considered carefully to manage data with various degradation. Finally, multi-task sSR meets the multiple needs of humans and meanwhile achieves inter-task mutual improvement, including low-level with low-level, low-level with high-level, and data reconstruction with parameter inversion.

Snapshot multispectral imaging using a diffractive optical network

Multispectral imaging has been used for numerous applications in e.g., environmental monitoring, aerospace, defense, and biomedicine. Here, we present a diffractive optical network-based multispectral imaging system trained using deep learning to create a virtual spectral filter array at the output image field-of-view. This diffractive multispectral imager performs spatially-coherent imaging over a large spectrum, and at the same time, routes a pre-determined set of spectral channels onto an array of pixels at the output plane, converting a monochrome focal-plane array or image sensor into a multispectral imaging device without any spectral filters or image recovery algorithms. Furthermore, the spectral responsivity of this diffractive multispectral imager is not sensitive to input polarization states. Through numerical simulations, we present different diffractive network designs that achieve snapshot multispectral imaging with 4, 9 and 16 unique spectral bands within the visible spectrum, based on passive spatially-structured diffractive surfaces, with a compact design that axially spans ~72λm, where λm is the mean wavelength of the spectral band of interest. Moreover, we experimentally demonstrate a diffractive multispectral imager based on a 3D-printed diffractive network that creates at its output image plane a spatially repeating virtual spectral filter array with 2 × 2 = 4 unique bands at terahertz spectrum. Due to their compact form factor and computation-free, power-efficient and polarization-insensitive forward operation, diffractive multispectral imagers can be transformative for various imaging and sensing applications and be used at different parts of the electromagnetic spectrum where high-density and wide-area multispectral pixel arrays are not widely available.

Pre-fire measures of boreal forest structure and composition inform interpretation of post-fire spectral recovery rates

Wildfire is a key driver of forest dynamics in boreal forests; however, the annual area burned in boreal forests is highly variable, with increasing wildfire activity documented over the past half century. Post-fire recovery has important implications for carbon balance, and for the wide range of ecosystem goods and services provisioned by the boreal forest. Monitoring post-fire recovery is challenging given the vast and often inaccessible areas impacted, coupled with the marked variability in post-fire conditions and recovery processes. Field assessments of recovery are constrained in space and time, whereas remotely sensed data are spatially explicit, cover large areas, and can retrospectively provide assessments of pre- and post-fire conditions and establish spectral recovery baselines. However, there is a need to link spectral measures of recovery to manifestations of post-fire forest recovery on the ground. Understanding how different forest characteristics influence spectral recovery is key to the successful application of spectral recovery metrics for forest management applications in different forest environments. Herein, using a synthesis of plot data for the North American boreal forest, we assess the influence of pre- and post-fire field-measured characteristics on spectral recovery rates, with a focus on stem density and composition in Canadian boreal forests. Plots that experienced rapid spectral recovery (as measured using Landsat-derived NBR time series data) were associated with a transition from conifer to broadleaf or were broadleaf pre-fire. Plots that experienced a decrease in stem density took significantly longer to spectrally recover than plots that experienced no change in stem density. Plots that had not yet spectrally recovered by the end of the time series were associated with higher elevations, drier sites, greater pre-fire basal area, and had the greatest change magnitude. Our results emphasize that recovery is a process that is highly variable and that knowledge of pre-fire condition is important for characterizing and interpreting measures of post-fire spectral recovery for forest management applications. Remotely sensed time series data are uniquely able to provide information on both pre- and post-fire condition. Our analysis of pre- and post-fire field measures provided novel insights regarding the influence of forest composition and changes in stem density on measures of post-fire spectral recovery in boreal forests.

Optimized Method Based on Subspace Merging for Spectral Reflectance Recovery.

The similarity between samples is an important factor for spectral reflectance recovery. The current way of selecting samples after dividing dataset does not take subspace merging into account. An optimized method based on subspace merging for spectral recovery is proposed from single RGB trichromatic values in this paper. Each training sample is equivalent to a separate subspace, and the subspaces are merged according to the Euclidean distance. The merged center point for each subspace is obtained through many iterations, and subspace tracking is used to determine the subspace where each testing sample is located for spectral recovery. After obtaining the center points, these center points are not the actual points in the training samples. The nearest distance principle is used to replace the center points with the point in the training samples, which is the process of representative sample selection. Finally, these representative samples are used for spectral recovery. The effectiveness of the proposed method is tested by comparing it with the existing methods under different illuminants and cameras. Through the experiments, the results show that the proposed method not only shows good results in terms of spectral and colorimetric accuracy, but also in the selection representative samples.

Speaker-Independent Spectral Enhancement for Bone-Conducted Speech

Because of the acoustic characteristics of bone-conducted (BC) speech, BC speech can be enhanced to better communicate in a complex environment with high noise. Existing BC speech enhancement models have weak spectral recovery capability for the high-frequency part of BC speech and have poor enhancement and robustness for the speaker-independent BC speech datasets. To improve the enhancement effect of BC speech for speaker-independent speech enhancement, we use a GANs method to establish the feature mapping between BC and air-conducted (AC) speech to recover the missing components of BC speech. In addition, the method adds the training of the spectral distance constraint model and, finally, uses the enhanced model completed by the training to reconstruct the BC speech. The experimental results show that this method is superior to the comparison methods such as CycleGAN, BLSTM, GMM, and StarGAN in terms of speaker-independent BC speech enhancement and can obtain higher subjective and objective evaluation results of enhanced BC speech.

Multi-harmonic reconstruction of photoelastic modulator-based Mueller polarimeters using the complex Q-matrix.

Analysis of data generated by Mueller matrix polarimeters using two photoelastic modulators has been evolving with the improvements in data acquisition and digital signal processing (DSP). Historical processing of the temporal data generated by these devices has involved isolating the frequencies via hardware signal processing (e.g.,lock-in amplifiers) or the numerical computation of Fourier integrals of recorded temporal data. Both avenues have their advantages, but the DSP aspects of the latter provide greater flexibility in choice of harmonics for processing. While conventional processing uses one harmonic for each desired Mueller matrix element, recent work has demonstrated that theoretical improvements are possible by coherently combining the information in multiple harmonic channels for each element. We demonstrate some recent progress in DSP that enables these polarimeters' data to be more fully exploited by addressing two key issues in the Fourier domain: spectral leakage and phase recovery. Adequately addressing these issues enables numerical analysis of the temporal data in the complex Fourier domain and delivers Mueller matrix results in which spectral phase information is used to recover the matrix elements and determine their signs automatically. We explore the application of this complex analysis and how the precision and accuracy of the results are affected by common experimental and DSP limitations compared to the usual magnitude-only analysis in the Fourier domain. The multi-harmonic method can provide a theoretical factor of 1.3-1.7 improvement in instrumental precision, and our experimental results approach that theoretical range.

Using dense Sentinel-2 time series to explore combined fire and drought impacts in eucalypt forests

Following one of the driest years on record, millions of hectares of forests in southeast Australia were burned in the 2019–2020 “Black Summer” wildfires. In addition to the areas burned, drought related canopy collapse, dieback and tree mortality was widely observed. In this paper, we present a method to map canopy damage due to drought and fire across a large area. Sentinel-2 satellite imagery was used in a monthly time series to highlight areas of forest where the Normalized Burn Ratio index was significantly below a pre-disturbance “stable” period. The stable period was defined as the 3 years prior to 2019 and the disturbance thresholds are based on bioregion specific standard deviations below pre-disturbance means. The novel methods enabled drought impacted forests to be identified, including those which were subsequently burned by wildfire. Across the 20 Mha of forests studied, 9.9 Mha (49%) fell below the disturbance threshold. Of that, 5.8 Mha was disturbed by fire and a further 4.1 Mha by drought outside of the fire extent. Within the fire extent, almost 0.9 Mha was identified as being significantly drought affected prior to being burned. An analysis of spectral recovery following substantial rainfall from February 2020 onward indicates that most of the areas impacted by both drought and fire have similar rates of recovery to those impacted only by fire. There are some areas, however, where the combined effects of the “double disturbance” appears to be hindering recovery. The methods presented here are easily transferrable and demonstrate an approach for monitoring forest disturbance at higher temporal and spatial scales than those typically used.

Estimation of temperature, local and global radiation of flames, using retrieved hyperspectral imaging

A flame emits energy over a wide spectral region, providing important information about the combustion process. Its associated spectra, classified into continuous and discontinuous features, provide information concerning the temperature of the soot particle and atomic or molecular reactions. However, because of the non-linear and turbulent nature of the flame, added to the high-dimensional nature of the spectral data and the overlap between continuous and discontinuous spectral emissions, complex methods are needed to obtain valuable information for combustion closed-loop control and optimization. In this paper, we propose a method for retrieval of hyperspectral flame images from Liquefied Petroleum Gas (LPG) fuel, using only images captured by a trichromatic camera, but combined with a spectral recovery method. The hyperspectral flame data recovered was used to estimate indexes, such as the flame temperature (in K), the local emitted radiance (at the pixel level) and the global radiance (both in μW/cm2), thus performing a dynamic, spectral and spatial combustion diagnosis in different air/fuel ratios. Our method was validated by performing measurements with a calibrated spectrophotometer, achieving small errors in the retrieval procedure (less than 1%) and in the temperature estimation (less than 4%). The results show that our approach can play an essential role in flame sensing, providing an important tool for monitoring or controlling combustion processes.

Miniaturized infrared spectrometer based on the tunable graphene plasmonic filter.

Miniaturization of a conventional spectrometer is challenging because of the tradeoffs of size, cost, signal-to-noise ratio, and spectral resolution, etc. Here, a new type of miniaturized infrared spectrometer based on the integration of tunable graphene plasmonic filters and infrared detectors is proposed. The transmittance spectrum of a graphene plasmonic filter can be tuned by varying the Fermi energy of the graphene, allowing light incident on the graphene plasmonic filter to be dynamically modulated in a way that encodes its spectral information in the receiving infrared detector. The incident spectrum can then be reconstructed by using decoding algorithms such as ridge regression and neural networks. The factors that influence spectrometer performance are investigated in detail. It is found that the graphene carrier mobility and the signal-to-noise ratio are two key parameters in determining the resolution and precision of the spectrum reconstruction. The mechanism behind our observations can be well understood in the framework of the Wiener deconvolution theory. Moreover, a hybrid decoding (or recovery) algorithm that combines ridge regression and a neural network is proposed that demonstrates a better spectral recovery performance than either the ridge regression or a deep neural network alone, being able to achieve a sub-hundred nanometer spectral resolution across the 8∼14 µm wavelength range. The size of the proposed spectrometer is comparable to a microchip and has the potential to be integrated within portable devices for infrared spectral imaging applications.

Maize disease detection based on spectral recovery from RGB images.

Maize is susceptible to infect pest disease, and early disease detection is key to preventing the reduction of maize yields. The raw data used for plant disease detection are commonly RGB images and hyperspectral images (HSI). RGB images can be acquired rapidly and low-costly, but the detection accuracy is not satisfactory. On the contrary, using HSIs tends to obtain higher detection accuracy, but HSIs are difficult and high-cost to obtain in field. To overcome this contradiction, we have proposed the maize spectral recovery disease detection framework which includes two parts: the maize spectral recovery network based on the advanced hyperspectral recovery convolutional neural network (HSCNN+) and the maize disease detection network based on the convolutional neural network (CNN). Taking raw RGB data as input of the framework, the output reconstructed HSIs are used as input of disease detection network to achieve disease detection task. As a result, the detection accuracy obtained by using the low-cost raw RGB data almost as same as that obtained by using HSIs directly. The HSCNN+ is found to be fit to our spectral recovery model and the reconstruction fidelity was satisfactory. Experimental results demonstrate that the reconstructed HSIs efficiently improve detection accuracy compared with raw RGB image in tested scenarios, especially in complex environment scenario, for which the detection accuracy increases by 6.14%. The proposed framework has the advantages of fast, low cost and high detection precision. Moreover, the framework offers the possibility of real-time and precise field disease detection and can be applied in agricultural robots.

WITHDRAWN: Measurement and calibration of optical instruments based on metrological calibration method and artificial neural network

At present, compared with foreign advanced optical instrument technology, the detection rate and accuracy of Chinese optical instrument system are relatively low. Therefore, it is necessary to update and discuss the measurement and calibration of existing optical instruments. The objective of this paper is to design a new calibration method by combining the traditional measurement calibration method with computer technology. This paper first studied the artificial neural network related knowledge. Then, the measurement parameters of the optical instrument are determined from three aspects: spectral resolution, optical axis magnification and optical axis parallelism measurement error. Then, based on the artificial neural network model and the measurement parameters of optical instruments, the measurement and calibration model of optical instruments is constructed, the measurement and calibration process of optical instruments is analyzed, and the systematic research method is obtained. The system model is combined with the spectral phase recovery method to complete the optical instrument testing and calibration experiments. Finally, the calibration results are tested with the center of mass method. By analyzing the experimental results of this measurement and calibration, it can be seen that updating the measurement and metrological calibration method of optical instruments can improve precision of optical instruments, and improve the application efficiency of optical instruments and promote the development of optical industry. So that, in this paper, the artificial neural network technology is introduced into the field of optical instrument testing as to achieve the calibration of optical instruments. Data AvailabilityData will be made available on request.

Optimized clustering method for spectral reflectance recovery.

An optimized method based on dynamic partitional clustering was proposed for the recovery of spectral reflectance from camera response values. The proposed method produced dynamic clustering subspaces using a combination of dynamic and static clustering, which determined each testing sample as a priori clustering center to obtain the clustering subspace by competition. The Euclidean distance weighted and polynomial expansion models in the clustering subspace were adaptively applied to improve the accuracy of spectral recovery. The experimental results demonstrated that the proposed method outperformed existing methods in spectral and colorimetric accuracy and presented the effectiveness and robustness of spectral recovery accuracy under different color spaces.

Simultaneous spectral recovery and CMOS micro-LED holography with an untrained deep neural network

Lensless holography promises compact, low-cost optical apparatus designs with similar performance to traditional imaging setups. Here, we propose the use of a silicon micro-LED fabricated in a commercial CMOS microelectronics process as the illumination source in a lensless holographic microscope. Its small emission area ( < 4 µ m 2 ) ensures high spatial coherence without the need for a pinhole and results in a large NA setup, circumventing the limits to the source-to-sample distance encountered by conventional lensless holography apparatus. The scene is reconstructed using an untrained deep neural network architecture that simultaneously performs spectral recovery by learning from the given single experimental diffraction intensity. We envision this synergetic combination of CMOS micro-LEDs and the machine learning framework can be used in other computational imaging applications, such as a compact microscope for live-cell tracking or spectroscopic imaging of biological materials.