Pixel Sampling Research Articles

A methodology to assess the phenology patterns of a eutrophic water body, using remote sensing data and principal component analysis

Chlorophyll-a concentrations in coastal and inland waters is acknowledged as an important factor to estimate Water Bodies Quality Status. Remote sensing analysis is a fast, cost-effective, and reliable methodology that can be used to assess the phenological patterns of aquatic ecosystems. The purpose of this study is to assess the phenology patterns of a eutrophic water body and present a novel methodology to fill in the missing values of the daily Surface Reflectance MODIS Aqua dataset. The proposed methodology is to apply Principal Component Analysis to generate the missing values from the chlorophyll-a concentrations and better assess Water Body's Trophic Status. The pilot study is Aitoliko Lagoon, a semi-closed Greek water body with eutrophication. The analysis reveals that sampling stations of swallow waters have higher variance in chlorophyll-a concentrations, favoring the deep waters sampling pixels for evaluating the water body's trophic status. The proposed PCA methodology showed a clear phenological pattern of chlorophyll-a with algae bloom from April and high chlorophyll-a concentrations until August, which is what is expected from a eutrophic lagoon and specifically for Aitoliko lagoon.

Efficient Biomedical Waste Management: Optimizing Segmentation and Classification with EnU-Net-DNN-BMWC

<p style="margin-bottom:8px;text-align:justify;"><span style="font-family:"Times New Roman",serif;font-size:12.0pt;line-height:200%;tab-stops:0in 21.3pt;">In India, biomedical waste (BMW) management is governed by strict regulations to mitigate health and environmental risks. It includes materials such as sharps, infectious wastes, pharmaceuticals, and non-hazardous items contaminated with potentially infectious substances. Proper management and disposal of BMW are critical to prevent environmental contamination and health risks.  This article introduces an Enhanced U-Network (EnU-Net) integrated with Deep Neural Network BMW Classification (EnU-Net-DNN-BMWC) to enhance accuracy in this critical task. Leveraging the U-Net framework with an Encoder-Decoder Network (EDN) and pixel-wise classification layer initially optimizes image segmentation. Bayesian functions mitigate segmentation uncertainties, while Content-Sensitive Sampling (CSS) refines pixel sampling to prioritize data-sparse regions. Data collected from G. Kuppuswamy Naidu Memorial Hospital and Kovai Medical Center and Hospital (KMCH) in Coimbatore over six months, categorizing sharps, infectious materials, pharmaceuticals, and non-hazardous waste, informs waste management strategies. Experimental validation using 100 biomedical waste images demonstrates EnU-Net-DNN-BMWC achieving good accuracy, surpassing standalone DNN-BMWC using Matlab 2022. The comparative analysis across different metrics such as accuracy, precision, F-measure, recall, error rate, and RMSE between DNN-BMWC and the proposed EnU-Net-DNN-BMWC framework were evaluated in this study, highlighting EnU-Net-DNN-BMWC's superior performance. Finally, this study underscores EnU-Net-DNN-BMWC’s efficacy in enhancing biomedical waste classification, crucial for sustainable waste management practices and regulatory compliance in healthcare settings.</span></p>

An Improved Physics-Based Dual-Band Model for Satellite-Derived Bathymetry Using SuperDove Imagery

Shallow water bathymetry is critical for environmental monitoring and maritime security. Current widely used statistical models based on passive optical satellite remote sensing often rely on prior bathymetric data, limiting their application to regions lacking such information. In contrast, the physics-based dual-band log-linear analytical model (P-DLA) can estimate shallow water bathymetry without in situ measurements, offering significant potential. However, the quasi-analytical algorithm (QAA) used in the P-DLA is sensitive to non-ideal pixels, resulting in unstable bathymetry estimation. To address this issue and evaluate the potential of SuperDove imagery for bathymetry estimation in regions without prior bathymetric data, this study proposes an improved physics-based dual-band model (IPDB). The IPDB replaces the QAA with a spectral optimization algorithm that integrates deep and shallow water sample pixels to estimate diffuse attenuation coefficients for the blue and green bands. This allows for more accurate estimation of shallow water bathymetry. The IPDB was tested on SuperDove images of Dongdao Island, Yongxing Island, and Yongle Atoll. The results showed that SuperDove images are capable of estimating shallow water bathymetry in regions without prior bathymetric data. The IPDB achieved Root Mean Square Error (RMSE) values below 1.7 m and R2 values above 0.89 in all three study areas, indicating strong performance in bathymetric estimation. Notably, the IPDB outperformed the standard P-DLA model in accuracy. Furthermore, this study outlines four sampling principles that, when followed, ensure that variations in the spatial distribution of sampling pixels do not significantly impact model performance. This study also showed that the blue–green band combination is optimal for the analytical expression of the physics-based dual-band model.

Category-sensitive semi-supervised semantic segmentation framework for land-use/land-cover mapping with optical remote sensing images

High-quality land-use/land-cover mapping with optical remote sensing images yet presents significant work. Even though fully convolutional semantic segmentation models have recently contributed to popular solutions, the lack of annotation data may lead to severe degradations in their inference performance. Besides, the category confusion in high-resolution representations will further exacerbate the adverse effects. In this paper, we propose a category-sensitive semi-supervised semantic segmentation framework to address these weaknesses by employing massive unlabeled data. With the perturbations from adopted hybrid data augmentation structures, we first focus on the output space and execute regularization constraints to learn category-specific discriminative features. It is formulated with a consistency self-training procedure where a dynamic class-balanced threshold selection scheme is proposed to provide high-confident pseudo supervisions for each category. In addition, we introduce pixel-wise contrastive learning on the common embedding space from both labeled and unlabeled data domains to further facilitate the semantic dependencies among category features, in which the reliable labels are leveraged as guidance for pixel sample selection. We verify the proposed framework on two benchmark land-use/land-cover datasets, and the experimental results demonstrate its competitive performance to other state-of-the-art semi-supervised methods.

Effects of increasing spatial resolution on the spatial information content and accuracy of downward surface shortwave radiation

Downward Surface Shortwave Radiation (DSSR) plays a crucial role in ecological, hydrological, and solar photovoltaic studies. The estimation of DSSR at higher spatial resolutions is increasingly popular. However, the impact of increasing spatial resolution on spatial information content and accuracy of DSSR estimates is not well understood. Addressing this knowledge gap, an all-sky DSSR dataset at varying spatial resolutions (0.8° to 0.01°) in China was estimated using MODIS top-of-atmosphere reflectance data. Subsequently, the spatial heterogeneity and accuracy of the DSSR dataset as well as its driving factors were analyzed. The results indicate that the magnitude of the increase in spatial heterogeneity varies across different regions as the spatial resolution increases. On the edge of the Qinghai-Tibet Plateau, the spatial heterogeneity experiences the most significant increase. To quantify the factors driving the spatial heterogeneity of DSSR, the Correlation Coefficients (CC) were calculated between the Coefficient Of Variation (COV) of DSSR and the COVs of relevant factors. In cloudy-sky conditions, the CC between the COV of DSSR and cloud optical thickness ranged from 0.54 to 0.62, whereas the CC between DSSR COV and Column Water Vapor (CWV) COV was between 0.72 and 0.78. Under clear-sky conditions, the COV of CWV, aerosol optical depth, and PM2.5 showed more significant correlations with DSSR COV compared to altitude COV. Validation using ground-based measurements revealed that the accuracy of DSSR decreased at coarser spatial resolutions due to the presence of mixed pixel samples. Moreover, it was demonstrated that a spatial resolution of 0.05° offered the highest cost-effectiveness when estimating DSSR. These insights significantly advance our understanding of the role of spatial resolutions in DSSR estimation and are instrumental in guiding the selection of optimal resolutions for such studies.

A deep learning-based combination method of spatio-temporal prediction for regional mining surface subsidence

In coal mining areas, surface subsidence poses significant risks to human life and property. Fortunately, surface subsidence caused by coal mining can be monitored and predicted by using various methods, e.g., probability integral method and deep learning (DL) methods. Although DL methods show promise in predicting subsidence, they often lack accuracy due to insufficient consideration of spatial correlation and temporal nonlinearity. Considering this issue, we propose a novel DL-based approach for predicting mining surface subsidence. Our method employs K-means clustering to partition spatial data, allowing the application of a gate recurrent unit (GRU) model to capture nonlinear relationships in subsidence time series within each partition. Optimization using snake optimization (SO) further enhances model accuracy globally. Validation shows our method outperforms traditional Long Short-Term Memory (LSTM) and GRU models, achieving 99.1% of sample pixels with less than 8 mm absolute error.

UnWarpME: Unsupervised warping map estimation in real-world scenarios

This paper introduces a novel approach to warp an RGB image to a new target pose, where it learns to inpaint the invisible parts of the image by sampling pixels from the visible parts. This technique is particularly relevant in applications such as novel pose image generation, where the quality of the generated samples heavily relies on the warped images. Conventional methods typically utilize an affine warping map estimator and learn to estimate the warping map through a downstream image generation task. However, this approach results in an affine function being returned regardless of the complexity of the deformation between the source and target poses. In contrast, our proposed method involves estimating the warping map using a convolutional function, which learns through alternating between two downstream tasks. Initially, the estimation treats the warping as part of an image generation task, similar to existing methods but without the constraint of an affine transformation. This encourages the estimation of complex transformations beyond affine deformations. Subsequently, disregarding the image generation task, we supervise the convolutional warping map to learn any potential affine transformation between the guiding poses of the sample. Our experimental results demonstrate the high effectiveness of our method in preserving image textures while transforming it into highly complex poses.

Decay Timescales of Chromospheric Condensations in Solar Flare Footpoints

Chromospheric condensations (CCs) are a prominent feature of flare footpoint heating in the solar flare standard model, yet their timescales and velocities are not well understood. Fisher derived several important analytical relationships, which have rarely been examined with modern spectral observations. The Interface Region Imaging Spectrograph (IRIS) provides a wealth of flare data with a high enough cadence to sufficiently capture CC evolution. We analyzed Doppler shifts in Mg ii 2791 and Fe ii 2814 from a sample of flare footpoint pixels observed by IRIS to compare with Fisher's analytics and recent flare models. We found a detection lifetime of 1 minute occurs in 50% of the sample, with Mg ii showing several pixels with longer values and Fe ii almost categorically shorter, and both growing with the maximum velocity, v max. The shifts’ half-life is commonly <40 s and is inversely related to v max, indicating that the first half of the CC evolution has more efficient kinetic energy loss. The lifetime’s wide range and growth with v max indicate that the footpoint atmospherics and heating scenarios can vary more widely than first postulated in Fisher. Around 90% of the sample had observable acceleration periods, lasting an average of 38 and 32 s for Mg ii and Fe ii, respectively. These acceleration periods, as well as serving as flare model diagnostics themselves, could potentially be used to calculate other model diagnostics such as the initially accelerated mass.

UVS-CNNs: Constructing general convolutional neural networks on quasi-uniform spherical images

Omnidirectional images, also known as spherical images, offer a significant advantage for the environmental sensing of mobile robots due to their wide field of view. However, previous studies of constructing convolutional neural networks on spherical images have been limited by non-uniform pixel sampling, leading to suboptimal performance in semantic segmentation. To address this issue, a novel pixel segmentation approach is proposed to achieve near-uniform pixel distribution across the entire spherical surface. The corresponding convolution operation for the resulting image is designed as well, which extends the capabilities of spherical CNNs from semantic segmentation to more complex tasks such as instance segmentation. The method is evaluated on the Stanford 2D3DS dataset and shows superior performance compared to conventional spherical CNNs. Furthermore, the method also achieves impressive instance segmentation results on our experimental LiDAR data, demonstrating the general feasibility of our approach for common CNN tasks. The related code and dataset are released in the following link: https://github.com/YoungRainy/UVS-U-Net.

Enhancing Semi-Supervised Few-Shot Hyperspectral Image Classification via Progressive Sample Selection

Hyperspectral images (HSIs) provide valuable spatial–spectral information for ground analysis. However, in few-shot (FS) scenarios, the limited availability of training samples poses significant challenges in capturing the sample distribution under diverse environmental conditions. Semi-supervised learning has shown promise in exploring the distribution of unlabeled samples through pseudo-labels. Nonetheless, FS HSI classification encounters the issue of high intra-class spectral variability and inter-class spectral similarity, which often lead to the diffusion of unreliable pseudo-labels during the iterative process. In this paper, we propose a simple yet effective progressive pseudo-label selection strategy that leverages the spatial–spectral consistency of HSI pixel samples. By leveraging spatially aligned ground materials as connected regions with the same semantic and similar spectrum, pseudo-labeled samples were selected based on round-wise confidence scores. Samples within both spatially and semantically connected regions of FS samples were assigned pseudo-labels and joined subsequent training rounds. Moreover, considering the spatial positions of FS samples that may appear in diverse patterns, to fully utilize unlabeled samples that fall outside the neighborhood of FS samples but still belong to certain connected regions, we designed a matching active learning approach for expert annotation based on the temporal confidence difference. We identified samples with the highest training value in specific regions, utilizing the consistency between predictive labels and expert labels to decide whether to include the region or the sample itself in the subsequent semi-supervised iteration. Experiments on both classic and more recent HSI datasets demonstrated that the proposed base model achieved SOTA performance even with extremely rare labeled samples. Moreover, the extended version with active learning further enhances performance by involving limited additional annotation.

Filtering After Shading With Stochastic Texture Filtering

2D texture maps and 3D voxel arrays are widely used to add rich detail to the surfaces and volumes of rendered scenes, and filtered texture lookups are integral to producing high-quality imagery. We show that applying the texture filter after evaluating shading generally gives more accurate imagery than filtering textures before BSDF evaluation, as is current practice. These benefits are not merely theoretical, but are apparent in common cases. We demonstrate that practical and efficient filtering after shading is possible through the use of stochastic sampling of texture filters. Stochastic texture filtering offers additional benefits, including efficient implementation of high-quality texture filters and efficient filtering of textures stored in compressed and sparse data structures, including neural representations. We demonstrate applications in both real-time and offline rendering and show that the additional error from stochastic filtering is minimal. We find that this error is handled well by either spatiotemporal denoising or moderate pixel sampling rates.

ПРИМЕНЕНИЕ РАДАРНЫХ И ОПТИЧЕСКИХ ДАННЫХ ДЗЗ ДЛЯ ОЦЕНКИ КОЛИЧЕСТВА ПОСТРАДАВШИХ ДОМОВ, ПЛОЩАДИ И ГРАНИЦ ЗОН ЗАТОПЛЕНИЯ

The article presents the result of determining the flooding zone and the affected houses and buildings of the town of Atbasar from the spring floods of 2017. This result was obtained using three available remote sensing data Sentinel-2A, Landsat-8 and Sentinel-1B of medium spatial resolution. At the first stage, remote sensing data of the near, short-wave, thermal infrared (IR) range and polarization by contrast level were visually analyzed. Images were selected based on a high level of contrast between two classes: a water object and a non-water object (land). At the second stage, threshold values were calculated from a sample of pixels related to water bodies, and then binary images were created. In the third stage, binary images were logically summarized to eliminate the cloud effect and find the resulting binary image of the flood zone from spring floods. In the fourth stage, the resulting binary image was superimposed on the GIS data of the town of Atbasar, where the locations and the number of affected houses and buildings were located. According to this study, it was found that more than 200 country houses were flooded in the northern part, and 9 houses in the eastern part of the town of Atbasar. According to media reports and official bodies, the number of affected houses in Atbasar was about 400. The number of affected houses found by the method of logical summation with three remote sensing data was 52%. The accuracy of the location can be significantly improved by using high spatial resolution remote sensing data. The results of this study may be useful for the emergency service, local government agencies, and insurance companies in assessing damage from spring floods.

Accuracy of coordinate measurements at phase modulation by digital raster

The development of optical-electronic instruments for measuring linear and angular quantities based on focal-plane array (photomatrices) using a digital phase analyzer (digital raster) is considered. It is shown that, unlike a phase raster with a mechanical drive, a digital raster provides the possibility of automation and increased measurement accuracy, reducing the weight and size characteristics of the measuring instrument. A method for constructing a digital raster is proposed, based on the use of photomatrices in combination with computer technologies. To ensure the required accuracy of the developed opto-electronic devices, the sources and components of the measurement error were investigated. The influence of the discrete structure of a digital raster on the error in measuring image coordinates has been studied. Mathematical expressions are obtained for calculating the limiting values of coordinate measurement errors caused by parasitic phase modulation that occurs when using a digital raster (sampling error). The dependence of these errors on image parameters and pixel size has been determined. It is shown that, based on the permissible maximum sampling error, it is possible to calculate the required dimensions of a photomatrix pixel, while the size of the sampling pixel can be larger than the size of the photomatrix pixel, which is very signifi cant from the point of view of energy relations. The results obtained will be useful in the development of optical-electronic means of angular and linear measurements with digital rasters.

Reversible data hiding in encrypted image with secure multi-party for telemedicine applications

Privacy protection of electronic patient information (EPI) is a crucial concern in the telemedicine applications, especially when it comes to communication between patients and doctors. In this paper, we propose a high-capacity reversible data hiding (RDH) algorithm combined with secure multi-party computation for telemedicine applications. The pixel values within a 3 × 3-size image block are divided into embedded pixels (EPs) and sampled pixels (SPs), secret data is embedded into the prediction errors of EPs and SPs in two stages. In the first stage, two edge servers predict EPs using the four SPs in the block to obtain the prediction error of EPs. The secret data is embedded into EPs through addition operation and histogram shift method. In the second stage, the prediction error of SPs is calculated as the difference between EPs and SPs, and the embedding of secret data follows the same method as used in the first stage. The experimental results demonstrate that the proposed algorithm achieves a higher embedding rate compared to classical and state-of-the-art algorithms, with an average embedding rate of 0.47bpp on the UCID dataset. Furthermore, the proposed algorithm exhibits robustness against existing attacks.

PARSEG: a computationally efficient approach for statistical validation of botanical seeds’ images

Human recognition and automated image validation are the most widely used approaches to validate the output of binary segmentation methods but, as the number of pixels in an image easily exceeds several million, they become highly demanding from both practical and computational standpoint. We propose a method, called PARSEG, which stands for PArtitioning, Random Selection, Estimation, and Generalization; being the basic steps within this procedure. Suggested method enables us to perform statistical validation of binary images by selecting the minimum number of pixels from the original image to be used for validation without deteriorating the effectiveness of the validation procedure. It utilizes binary classifiers to accomplish image validation and selects the optimal sample of pixels according to a specific objective function. As a result, the computational complexity of the validation experiment is substantially reduced. The procedure’s effectiveness is illustrated by considering images composed of approximately 13 million pixels from the field of seed recognition. PARSEG provides roughly the same precision of the validation process when extended to the entire image, but it utilizes only about 4% of the original number of pixels, thus reducing, by about 90%, the computing time required to validate a binary segmented image.

Superpixel-wise contrast exploration for salient object detection

Salient object detection (SOD) methods typically consider SOD as a pixel-wise binary classification problem and utilize the binary cross-entropy (BCE) loss for optimization. However, the BCE loss ignores the global dependencies between pixels of the entire image, which is important for ensuring the accuracy and integrity of objects. To address this limitation explicitly, contrastive learning is introduced to enhance both the intra-pixel compactness of the foreground and inter-pixel separability between the foreground and background. Unfortunately, the random pixel sampling strategies of existing contrast algorithms cannot ensure that the sampled pixels cover all semantic contents across the entire image because of the lack of fine-grained semantic categories for pixels in SOD. Hence, in this study, a SuperPixel-wise Contrastive (SPCont) algorithm is proposed, which samples all superpixel centers to improve the diversity and representativeness of the sampled pixels. In addition, a SOD-specific hard pixel sampling strategy is designed to refine the edge details of the objects. An intra-image multi-level negative sampling approach is presented to increase the number of negative samples. Extensive experiments are conducted to demonstrate that incorporating the SPCont algorithm into the state-of-the-art network results in significant performance improvements. We expect that our work will open new paths for the application of contrastive learning to SOD.

Enhancing Mechanical Stimulated Brillouin Scattering Imaging with Physics‐Driven Model Selection

AbstractBrillouin microscopy (BM) is an emerging technique for all‐optical mechanical imaging without the need for physical contact with the sample or for an external mechanical stimulus. However, BM often retrieves a single Brillouin frequency shift for multiple mechanically different materials of structures and/or in regions—sufficiently larger than the phonon wavelength—inside the volume and its surroundings, resulting in significantly limited mechanical specificity in the Brillouin shift images produced. Here, a new physics‐driven model selection framework is developed based on information theory and a physical‐statistical overfit Brillouin water peak threshold that enables the robust identification of single‐ and multi‐peak Brillouin signatures in the sample pixels. The model selection framework is applied to Brillouin data of material interfaces and living NIH/3T3 cells measured by stimulated Brillouin scattering (SBS) microscopy, facilitating the quantification of the Brillouin frequency shift of materials in different regions of the sample and significantly improving mechanical specificity compared with the standard single peak fitting analysis.

Superposed Laguerre‐Gaussian Beams‐Based Orbital Angular Momentum Holography

AbstractOrbital angular momentum (OAM) holography implements OAM of light as an information carrier to reconstruct holographic images, due to a physically unbounded set of orthogonal OAM modes. Here, superposed Laguerre‐Gaussian (LG) beams are employed as incident beams to achieve OAM holography. Notably, this scheme utilizes exact complex‐amplitude modulation, as opposed to mere phase‐only modulation, ensuring the maintenance of amplitude information. In the absence of a matching field, the output images retain the properties of incident beams with a varying number of bright petals in each sampling pixel. On the other hand, when specific incident beams are illuminated, the output images can be reconstructed with multiple matching fields that lead to different intensities. It is essential to recognize that the bright spot in each sampling pixel of the output image achieves its peak intensity only when the matching field is the conjugate of the incident light field. This work offers a novel perspective on reconstructing holographic images through various matching fields and modulating the brightness of distinct holographic images in OAM‐multiplexing holography. Additionally, a theoretically infinite number of bright petals in each sampling pixel can also serve as information carriers in holography.

NIS-SLAM: Neural Implicit Semantic RGB-D SLAM for 3D Consistent Scene Understanding.

In recent years, the paradigm of neural implicit representations has gained substantial attention in the field of Simultaneous Localization and Mapping (SLAM). However, a notable gap exists in the existing approaches when it comes to scene understanding. In this paper, we introduce NIS-SLAM, an efficient neural implicit semantic RGB-D SLAM system, that leverages a pre-trained 2D segmentation network to learn consistent semantic representations. Specifically, for high-fidelity surface reconstruction and spatial consistent scene understanding, we combine high-frequency multi-resolution tetrahedron-based features and low-frequency positional encoding as the implicit scene representations. Besides, to address the inconsistency of 2D segmentation results from multiple views, we propose a fusion strategy that integrates the semantic probabilities from previous non-keyframes into keyframes to achieve consistent semantic learning. Furthermore, we implement a confidence-based pixel sampling and progressive optimization weight function for robust camera tracking. Extensive experimental results on various datasets show the better or more competitive performance of our system when compared to other existing neural dense implicit RGB-D SLAM approaches. Finally, we also show that our approach can be used in augmented reality applications. Project page: https://zju3dv.github.io/nis_slam.

Pixel representations, sampling, and label correction for semantic part detection

Pixel representations, sampling, and label correction for semantic part detection