Pixel Mapping Research Articles

From unwanted to wanted: Blending functional weed traits into weed distribution maps

AbstractSite‐specific weed management (SSWM) is increasingly employed to reduce herbicide inputs. Incorporating functional traits of weed species allows for the selection of SSWM methods that effectively reduce the abundance of weeds with a high competitive potential (disservice) while preserving weeds that provide beneficial ecosystem services (service). In this study, we aim to assess relevant weed functional traits and translate this information into a spatial trait distribution map for weed (dis‐)service provision. The distribution of weed abundance in a field was recorded using a spatial grid. Data on functional traits for the recorded weed species were extracted from published datasets and combined into the two variables, service and disservice. Individual traits (service/disservice) were weighted for each pixel of the weed distribution map based on the number of individual plants per species. Principal component analysis was employed to generate independent variables to describe the potential for service and disservice provision. As a result, two (dis‐)service trait‐based distribution maps were generated: one highlights field areas that provide enhanced ecological services, while the other displays areas with a high disservice potential. The results show that around 61% of the area in the field had a high service potential. The area with a high disservice was slightly higher than the half of the area with a high service, while about 32% of the field has both high service and disservice potential in the same area. This study presents a spatially explicit approach to incorporate information on weed functional traits into SSWM approaches targeted at reducing weed competition while at the same time enhancing weed functional diversity.

Dual-key-based adaptive watermark embedding for light field 3D images.

The existing methods fail to effectively utilize the viewpoint information of light field 3D images for watermark embedding which results in a serious decrease in both invisibility and robustness of the watermark. Therefore, we propose a novel, to the best of our knowledge, light field 3D dual-key-based watermarking network (3D-DKWN). Our method employs a pixel mapping algorithm to obtain the disparity sub-image of the light field 3D image and generates an encoding key (EK). Adaptive watermark embedding is then performed on the disparity sub-image and a steganographic key (SK) is generated. Finally, the light field 3D image with the embedded watermark is reconstructed. Compared with previous approaches, our method reasonably utilizes the viewpoint information of light field 3D images, resulting in the significant improvement of invisibility and robustness of the watermark.

CSDFormer: A cloud and shadow detection method for landsat images based on transformer

Cloud and shadow (CS) detection is crucial prerequisite for application of remote sensing images. Current deep learning-based detection algorithms mainly employ Convolutional Neural Networks (CNNs). However, the local receptive field in CNNs cannot effectively capture global contextual information, which hinders accurate characterization of the dependency between clouds and shadows. In vision Transformers, self-attention mechanisms can effectively capture the long-distance dependencies between different regions in an image. Inspired by this, this paper proposed a new CS Detection algorithm based on a Transformer, called CSDFormer. Specifically, we exclusively employed a hierarchical Transformer structure in the encoder stage to extract features of CS. Each Transformer layer contains several multi-head self-attention mechanisms for calculating pixel-wise long-distance connectivity. The designed structure enables the Transformer to better extract global context information, which helps to strengthen the comprehension of the semantic relationships between clouds and shadows. Benefiting from the global feature extraction capability of the encoder stage, we employed several simple multilayer perceptron layers for multi-scale feature map fusion and pixel classification in the decoder stage. The proposed CSDFormer was validated using 898 Landsat 8 Biome images with 512 × 512 pixels, producing an overall accuracy of 95.28 % and a mean intersection over union of 84.08 %, outperforming three state-of-the-art CNN-based algorithms. CSDFormer is consistently more accurate in detection of both clouds and shadows. Owing to the parallel computing capability of the self-attention mechanism, CSDFormer is computationally more efficient than the three CNN-based benchmark methods. For the input spectral bands, the performance of CSDFormer produced can be further enhanced with additional thermal infrared bands.

FFT-Based Dynamic Token Mixer for Vision

Multi-head-self-attention (MHSA)-equipped models have achieved notable performance in computer vision. Their computational complexity is proportional to quadratic numbers of pixels in input feature maps, resulting in slow processing, especially when dealing with high-resolution images. New types of token-mixer are proposed as an alternative to MHSA to circumvent this problem: an FFT-based token-mixer involves global operations similar to MHSA but with lower computational complexity. However, despite its attractive properties, the FFT-based token-mixer has not been carefully examined in terms of its compatibility with the rapidly evolving MetaFormer architecture. Here, we propose a novel token-mixer called Dynamic Filter and novel image recognition models, DFFormer and CDFFormer, to close the gaps above. The results of image classification and downstream tasks, analysis, and visualization show that our models are helpful. Notably, their throughput and memory efficiency when dealing with high-resolution image recognition is remarkable. Our results indicate that Dynamic Filter is one of the token-mixer options that should be seriously considered. The code is available at https://github.com/okojoalg/dfformer

TCI-Former: Thermal Conduction-Inspired Transformer for Infrared Small Target Detection

Infrared small target detection (ISTD) is critical to national security and has been extensively applied in military areas. ISTD aims to segment small target pixels from background. Most ISTD networks focus on designing feature extraction blocks or feature fusion modules, but rarely describe the ISTD process from the feature map evolution perspective. In the ISTD process, the network attention gradually shifts towards target areas. We abstract this process as the directional movement of feature map pixels to target areas through convolution, pooling and interactions with surrounding pixels, which can be analogous to the movement of thermal particles constrained by surrounding variables and particles. In light of this analogy, we propose Thermal Conduction-Inspired Transformer (TCI-Former) based on the theoretical principles of thermal conduction. According to thermal conduction differential equation in heat dynamics, we derive the pixel movement differential equation (PMDE) in the image domain and further develop two modules: Thermal Conduction-Inspired Attention (TCIA) and Thermal Conduction Boundary Module (TCBM). TCIA incorporates finite difference method with PMDE to reach a numerical approximation so that target body features can be extracted. To further remove errors in boundary areas, TCBM is designed and supervised by boundary masks to refine target body features with fine boundary details. Experiments on IRSTD-1k and NUAA-SIRST demonstrate the superiority of our method.

Reversible Data Hiding for Color Images Using Channel Reference Mapping and Adaptive Pixel Prediction

Reversible data hiding (RDH) is a technique that embeds secret data into digital media while preserving the integrity of the original media and the secret data. RDH has a wide range of application scenarios in industrial image processing, such as intellectual property protection and data integrity verification. However, with the increasing prevalence of color images in industrial applications, traditional RDH methods for grayscale images are inadequate to meet the requirements of image fidelity. This paper proposes an RDH method for color images based on channel reference mapping (CRM) and adaptive pixel prediction. Initially, the CRM mode for a color image is established based on the pixel variation correlation between the RGB channels. Then, the pixel local complexity context is adaptively selected using the CRM mode. Next, each pixel value is adaptively predicted based on the features and characteristics of adjacent pixels and reference channels, and then data is embedded by expanding the prediction error. Finally, we compare seven existing RDH algorithms on the standard image dataset and the Kodak dataset to validate the advantages of our method. The experimental results demonstrate that our approach achieves average peak signal-to-noise ratio (PSNR) values of 63.61 and 60.53 dB when embedding 20,000 and 40,000 bits of data, respectively. These PSNR values surpass those of other RDH methods. These findings indicate that our method can effectively preserve the visual quality of images even under high embedding capacities.

A novel image encryption scheme with adaptive Fourier decomposition

This study creatively introduces a class of adaptive Fourier decomposition (AFD) techniques into the field of image encryption, where AFD and its variants are newly developed signal decomposition methods. The novelty of this article is twofold. First, we use three different AFD techniques, including Core AFD, Unwinding AFD, and Cyclic AFD, to generate two key streams through decomposing a source image. The two key streams generated by different AFD algorithms are different and aperiodic, which can effectively improve the security of encryption algorithm. Furthermore, this work also proposes a novel encryption method that implements the confusion and diffusion operations by index-sorted mapping method and pixel swapping operation based on two generated key streams, respectively. To ensure that the image is fully scrambled, both pixel-level and bit-level permutations are performed, which employs a novel bilateral sequence diffusion method to change the image pixel distribution based on pixel swap operations. The cryptographic performance of the scheme is evaluated through various security analyses, including key sensitivity, statistical, entropy, clipping attack, and differential attack analysis. Experimental results confirm that the proposed image encryption scheme ensures a high level of security and exhibits superior performance in resisting various attacks compared with several traditional and state-of-the-art image encryption methods.

A High-Performance Pixel-Level Fully Pipelined Hardware Accelerator for Neural Networks.

The design of convolutional neural network (CNN) hardware accelerators based on a single computing engine (CE) architecture or multi-CE architecture has received widespread attention in recent years. Although this kind of hardware accelerator has advantages in hardware platform deployment flexibility and development cycle, it is still limited in resource utilization and data throughput. When processing large feature maps, the speed can usually only reach 10 frames/s, which does not meet the requirements of application scenarios, such as autonomous driving and radar detection. To solve the above problems, this article proposes a full pipeline hardware accelerator design based on pixel. By pixel-by-pixel strategy, the concept of the layer is downplayed, and the generation method of each pixel of the output feature map (Ofmap) can be optimized. To pipeline the entire computing system, we expand each layer of the neural network into hardware, eliminating the buffers between layers and maximizing the effect of complete connectivity across the entire network. This approach has yielded excellent performance. Besides that, as the pixel data stream is a fundamental paradigm in image processing, our fully pipelined hardware accelerator is universal for various CNNs (MobileNetV1, MobileNetV2 and FashionNet) in computer vision. As an example, the accelerator for MobileNetV1 achieves a speed of 4205.50 frames/s and a throughput of 4787.15 GOP/s at 211 MHz, with an output latency of 0.60 ms per image. This extremely shorts processing time and opens the door for AI's application in high-speed scenarios.

Detection of chronic lymphocytic leukemia using Deep Neural Eagle Perch Fuzzy Segmentation – A novel comparative approach

With a high mortality rate worldwide, Chronic Lymphocytic Leukemia (CLL) poses a serious health risk. Radiologists view the ability to detect blood tumor cells as both important and difficult. It is challenging for many of the current studies to properly examine these blood cancer cells. An intellectual system for separating healthy cells from tumor cells is suggested in this research work. This research primarily depends on the effective implementation of computed tomography (CT) samples of both benign and malignant blood samples. In this research paper, novel methodologies called public active contour pixel mapping and Deep Neural Eagle Perch Fuzzy Segmentation (DNEPFS) are proposed, which help in detecting the blood cell boundary of the unaffected cells. It is quite similar to an eagle using the claw searching algorithm. Here, the eagle uses its sharp claws to tear the prey’s skin during the search for its food. 99.64% and 99.32% accuracy are obtained during the research work for benign and malignant nodules using MATLAB R2023a. The computational complexity was found to be less, which is 7.13 s. Different segmentation techniques are used for comparative analyses. According to the research results, this proposed flow is superior to all segmentation techniques currently in use for the accurate detection of tumor cells. Based on the expert's annotation on online samples and clinical samples, the methodology has been validated to prove their effectiveness and throw light on the life of affected patients to resume normalcy – live long. The research work was tested in real-time clinical samples which deliver promising and encouraging results in leukemia detection procedures.

Augmented Grad-CAM++: Super-Resolution Saliency Maps for Visual Interpretation of Deep Neural Network

In recent years, deep neural networks have shown superior performance in various fields, but interpretability has always been the Achilles’ heel of deep neural networks. The existing visual interpretation methods for deep neural networks still suffer from inaccurate and insufficient target localization and low-resolution saliency maps. To address the above issues, this paper presents a saliency map generation method based on image geometry augmentation and super-resolution called augmented high-order gradient weighting class activation mapping (augmented grad-CAM++). Unlike previous approaches that rely on a single input image to generate saliency maps, this method first introduces the image geometry augmentation technique to create a set of augmented images for the input image and generate activation mappings separately. Secondly, the augmented activation mappings are combined to form the final saliency map. Finally, a super-resolution technique is introduced to add pixel points to reconstruct the saliency map pixels to improve the resolution of the saliency map. The proposed method is applied to analyze standard image data and industrial surface defect images. The results indicate that, in experiments conducted on standard image data, the proposed method achieved a 3.1% improvement in the accuracy of capturing target objects compared to traditional methods. Furthermore, the resolution of saliency maps was three times higher than that of traditional methods. In the application of industrial surface defect detection, the proposed method demonstrated an 11.6% enhancement in the accuracy of capturing target objects, concurrently reducing the false positive rate. The presented approach enables more accurate and comprehensive capture of target objects with higher resolution, thereby enhancing the visual interpretability of deep neural networks. This improvement contributes to the greater interpretability of deep learning models in industrial applications, offering substantial performance gains for the practical deployment of deep learning networks in the industrial domain.

MSMCNet: Differential context drives accurate localization and edge smoothing of lesions for medical image segmentation

Medical image segmentation plays a crucial role in clinical assistance for diagnosis. The UNet-based network architecture has achieved tremendous success in the field of medical image segmentation. However, most methods commonly employ element-wise addition or channel merging to fuse features, resulting in smaller differentiation of feature information and excessive redundancy. Consequently, this leads to issues such as inaccurate lesion localization and blurred boundaries in segmentation. To alleviate these problems, the Multi-scale Subtraction and Multi-key Context Conversion Networks (MSMCNet) are proposed for medical image segmentation. Through the construction of differentiated contextual representations, MSMCNet emphasizes vital information and achieves precise medical image segmentation by accurately localizing lesions and enhancing boundary perception. Specifically, the construction of differentiated contextual representations is accomplished through the proposed Multi-scale Non-crossover Subtraction (MSNS) module and Multi-key Context Conversion Module (MCCM). The MSNS module utilizes the context of MCCM coding and redistribute the value of feature map pixels. Extensive experiments were conducted on widely used public datasets, including the ISIC-2018 dataset, COVID-19-CT-Seg dataset, Kvasir dataset, as well as a privately constructed traumatic brain injury dataset. The experimental results demonstrated that our proposed MSMCNet outperforms state-of-the-art medical image segmentation methods across different evaluation metrics.

Adaptive super-resolution image reconstruction based on fractal theory

Image super-resolution (SR) reconstruction has been a significant research field in image processing, although the current approaches to dealing with complicated image reconstruction still have some issues. In order to overcome these problems, traditional approaches apply fractal geometry to image super-resolution reconstruction, either in textured or non-textured regions, treating the image as a single fractal set. However, it won't be possible to classify the image specifically, such as edge, common texture, and complex texture, unless identifying the texture complexity of all regions in the image. For the texture region, this paper considers the fractal as a perturbation function of rational interpolation and constructs a new interpolation model with scale factor parameter. The parameter is related to the local fractal dimension (LFD). This paper defines a new function between scale factor and local fractal dimension so as to select the optimal value adaptively. Considering the image edge quality, this paper combines the non-texture regions and edge regions, applies an improved edge interpolation algorithm and sets a new pixel mapping method to meet the needs of different zoom factors. Based on the above, this paper proposes an adaptive super-resolution reconstruction algorithm. Particularly, region segmentation with different texture complexity is a foundational step of our algorithm, this paper proposes an image segmentation algorithm based on the research of local fractal dimension which combines the multi-scale analysis capability of wavelets with the multi-scale self-similarity feature of fractals. The experimental results show that our algorithm can obtain accurate texture details, smooth edges and low noise subjectively, and achieve the best evaluation objectively. This paper lays the theoretical foundation for fractal applications in super-resolution image reconstruction.

High-Resolution Rainfall Maps from Commercial Microwave Links for a Data-Scarce Region in West Africa

Abstract We present high-resolution rainfall maps from commercial microwave link (CML) data in the city of Ouagadougou, Burkina Faso. Rainfall was quantified based on data from 100 CMLs along unique paths and interpolated to achieve rainfall maps with a 5-min temporal and 0.55-km spatial resolution for the monsoon season of 2020. Established processing methods were combined with newly developed filtering methods, minimizing the loss of data availability. The rainfall maps were analyzed qualitatively both at a 5-min and aggregated daily scales. We observed high spatiotemporal variability on the 5-min scale that cannot be captured with any existing measurement infrastructure in West Africa. For the quantitative evaluation, only one rain gauge with a daily resolution was available. Comparing the gauge data with the corresponding CML rainfall map pixel showed a high agreement, with a Pearson correlation coefficient > 0.95 and an underestimation of the CML rainfall maps of ∼10%. Because the CMLs closest to the gauge have the largest influence on the map pixel at the gauge location, we thinned out the CML network around the rain gauge synthetically in several steps and repeated the interpolation. The performance of these rainfall maps dropped only when a radius of 5 km was reached and approximately one-half of all CMLs were removed. We further compared ERA5 and GPM IMERG data with the rain gauge and found that they had much lower correlation than data from the CML rainfall maps. This clearly highlights the large benefit that CML data can provide in the data-scarce but densely populated African cities. Significance Statement In this study, we investigate the possibility of deriving accurate high-resolution rainfall maps from commercial microwave link (CML) data in West Africa. The main challenges are the lack of reference data in this area and the adoption of existing processing tools without reference data. We show CML rainfall maps for Ouagadougou, Burkina Faso, with a resolution of 5 min and 0.55 km, which is unprecedented in this region. The comparison with the only available rain gauge, which provides data only at a daily resolution, yields a Pearson correlation of >0.95. An analysis of synthetically thinned-out networks shows that this accuracy is valid for the whole domain. Comparing reanalysis and satellite data with the rain gauge and CML data showed a poor performance of these gridded reference datasets. Also, a high coincidence of temporal dynamics between CML rainfall maps and satellite products was observed. Overall, these findings support the potential of CMLs for future hydrometeorological applications in West Africa.

3D salient object detection based on light field integral imaging.

Potent usage of the multi-scale light field information for salient object detection (SOD) is the essential requirement of three-dimensional (3D) SOD. On this basis, a light field 3D-SOD scheme is proposed that employs the pixel mapping algorithm to achieve a more distinct representation of spatial and angular information in the four-dimensional (4D) light field, collaboratively mining the global saliency cues via the co-salient object detection (CoSOD) network. Compared with the previous method, our scheme filters out most of the noise by thoroughly leveraging the global dependence of the 4D light field, offering significant enhancements in saliency extraction performance and efficiency. Additionally, the 3D reconstruction results demonstrate the integral retention of the spatial and angular information of the original light field.

Full-resolution and full-dynamic-range coded aperture compressive temporal imaging.

Coded aperture compressive temporal imaging (CACTI) aims to capture a sequence of video frames in a single shot, using an off-the-shelf 2D sensor. This approach effectively increases the frame rate of the sensor while reducing data throughput requirements. However, previous CACTI systems have encountered challenges such as limited spatial resolution and a narrow dynamic range, primarily resulting from suboptimal optical modulation and sampling schemes. In this Letter, we present a highly efficient CACTI system that addresses these challenges by employing precise one-to-one pixel mapping between the sensor and modulator, while using structural gray scale masks instead of binary masks. Moreover, we develop a hybrid convolutional-Transformer deep network for accurate reconstruction of the captured frames. Both simulated and real data experiments demonstrate the superiority of our proposed system over previous approaches, exhibiting significant improvements in terms of spatial resolution and dynamic range.

LPDNet: A Lightweight Network for SAR Ship Detection Based on Multi-Level Laplacian Denoising.

Intelligent ship detection based on synthetic aperture radar (SAR) is vital in maritime situational awareness. Deep learning methods have great advantages in SAR ship detection. However, the methods do not strike a balance between lightweight and accuracy. In this article, we propose an end-to-end lightweight SAR target detection algorithm, multi-level Laplacian pyramid denoising network (LPDNet). Firstly, an intelligent denoising method based on the multi-level Laplacian transform is proposed. Through Convolutional Neural Network (CNN)-based threshold suppression, the denoising becomes adaptive to every SAR image via back-propagation and makes the denoising processing supervised. Secondly, channel modeling is proposed to combine the spatial domain and frequency domain information. Multi-dimensional information enhances the detection effect. Thirdly, the Convolutional Block Attention Module (CBAM) is introduced into the feature fusion module of the basic framework (Yolox-tiny) so that different weights are given to each pixel of the feature map to highlight the effective features. Experiments on SSDD and AIR SARShip-1.0 demonstrate that the proposed method achieves 97.14% AP with a speed of 24.68FPS and 92.19% AP with a speed of 23.42FPS, respectively, with only 5.1 M parameters, which verifies the accuracy, efficiency, and lightweight of the proposed method.

LACC2.0: Improving the LACC Algorithm for Reconstructing Satellite-Derived Time Series of Vegetation Biochemical Parameters

The locally adjusted cubic-spline capping (LACC) algorithm is well recognized for its effectiveness in the global time series reconstruction of vegetation biophysical and biochemical parameters. However, in its application, we often encounter issues, such as identifying positively biased outliers for vegetation biochemical parameters and reducing the influence of long consecutive gaps. In this study, we improved the LACC algorithm to address the above two issues by (1) incorporating a procedure to remove outliers and (2) integrating the spatial information of neighboring pixels for large data gap filling. To evaluate the performance of the new version of LACC (namely LACC2.0), leaf chlorophyll content (LCC) was taken as an example. A reference LCC curve was generated for each pixel of the global map as the true value for global evaluation, and a time series of LCC with real gaps in the original data for each pixel was created by adding Gaussian noises into observations for testing the effectiveness of time series reconstruction algorithms. Results showed that the percentage of pixels with an RMSE smaller than 5 μg/cm2 was improved from 81.2% in LACC to 96.4% in LACC2.0, demonstrating that LACC2.0 had the potential to provide a better reconstruction of global daily satellite-derived vegetation biochemical parameters. This finding highlights the significance of outlier removal and spatial-temporal fusion to enhance the accuracy and reliability of time series reconstruction.

Joint Calibration of a Multimodal Sensor System for Autonomous Vehicles

Multimodal sensor systems require precise calibration if they are to be used in the field. Due to the difficulty of obtaining the corresponding features from different modalities, the calibration of such systems is an open problem. We present a systematic approach for calibrating a set of cameras with different modalities (RGB, thermal, polarization, and dual-spectrum near infrared) with regard to a LiDAR sensor using a planar calibration target. Firstly, a method for calibrating a single camera with regard to the LiDAR sensor is proposed. The method is usable with any modality, as long as the calibration pattern is detected. A methodology for establishing a parallax-aware pixel mapping between different camera modalities is then presented. Such a mapping can then be used to transfer annotations, features, and results between highly differing camera modalities to facilitate feature extraction and deep detection and segmentation methods.

Reparametric Sampling for Small Object Image Recognition

It is difficult to balance detection speed and detection accuracy in current methods for sampling on feature maps. We combine the parameterless interpolation algorithm used in the traditional method with the algorithm in deep learning to improve shortcomings of the traditional interpolation algorithm by introducing learnable parameters into the algorithm. The new algorithm guarantees the features of the original image by calculating the correlation between feature map channels and pixels to ensure the correct restoration of the image features during up-sampling. In this paper, we propose a new up-sampling operation model — Reparametric Blend Up-sampling (RBU), which uses the method of rearranging pixels to obtain a feature map that matches the up-sampling size, uses blending to compute inter-pixel and inter-channel correlations and back-propagates the model to learn the appropriate parameters to ensure feature extraction. In the field of object detection, this model replaces the original up-sampling operation and improves the mAP mean value by 1.1% and the detection of small target objects by 3.1% in the VOC dataset with the richest application scenarios.

Image Encryption Scheme Based on Orbital Shift Pixels Shuffling with ILM Chaotic System.

Image encryption techniques protect private images from unauthorized access while they are being transmitted. Previously used confusion and diffusion processes are risky and time-consuming. Therefore, finding a solution to this problem has become necessary. In this paper, we propose a new image encryption scheme that combines the Intertwining Logistic Map (ILM) and Orbital Shift Pixels Shuffling Method (OSPSM). The proposed encryption scheme applies a technique for confusion inspired by the rotation of planets around their orbits. We linked the technique of changing the positions of planets around their orbits with the shuffling technique of pixels and combined it with chaotic sequences to disrupt the pixel positions of the plain image. First, randomly selected pixels from the outermost orbit are rotated to shift the pixels in that orbit, causing all pixels in that orbit to change their original position. This process is repeated for each orbit until all pixels have been shifted. This way, all pixels are randomly scrambled on their orbits. Later on, the scrambled pixels are converted into a 1D long vector. The cyclic shuffling is applied using the key generated by the ILM to a 1D long vector and reshaped into a 2D matrix. Then, the scrambled pixels are converted into a 1D long vector to apply cyclic shuffle using the key generated by the ILM. After that, the 1D long vector is converted into a 2D matrix. For the diffusion process, using ILM generates a mask image, which is then XORed with the transformed 2D matrix. Finally, a highly secure and unrecognizable ciphertext image is obtained. Experimental results, simulation analysis, security evaluation, and comparison with existing image encryption schemes show that it has a strong advantage in defending against common attacks, and the operating speed of this encryption scheme also performs excellently in practical image encryption applications.