Pixel Location Research Articles

Misshaped boundary classifier model for license plate detection in haze weather using entropy CNN

Weather that creates haze can cover up car license plates, creating warped lines that make it difficult to see and identify them. This paper suggests a novel Primitive Boundary Classifier Model (PBCM) that uses the unique properties of bright and dark boundaries to solve this problem. Iteratively extracting characteristics from the input image, the PBCM draws volatile borders and ends linearity at particular pixel positions. To detect irregular boundaries in the hidden layers through changes in entropy and regularity terminations, this procedure is combined with linear entropy learning, which is accomplished by altering a convolutional neural network. Identifying the license plate area and its related embedding is possible by finding these terminating border pixel locations. The model evaluates its performance during validation by considering similarity and false rate metrics. The comparative analysis, this model improves the 7.34% detection precision with 15.98% high similarity and 8.95% less false rate for the maximum epochs performance ratio of 90.1% and error rate of 11.2%.

UTran-DSR: a novel transformer-based model using feature enhancement for dysarthric speech recognition

Over the past decade, the prevalence of neurological diseases has significantly risen due to population growth and aging. Individuals suffering from spastic paralysis, brain attack, and idiopathic Parkinson’s disease (PD), among other neurological illnesses, commonly suffer from dysarthria. Early detection and treatment of dysarthria in these patients are essential for effectively managing the progression of their disease. This paper provides UTrans-DSR, a novel encoder-decoder architecture for analyzing Mel-spectrograms (generated from audios) and classifying speech as healthy or dysarthric. Our model employs transformer encoder features based on a hybrid design, which includes the feature enhancement block (FEB) and the vision transformer (ViT) encoders. This combination effectively extracts global and local pixel information regarding localization while optimizing the mel-spectrograms feature extraction process. We keep up with the original class-token grouping sequence in the vision transformer while generating a new equivalent expanding route. More specifically, two unique growing pathways use a deep-supervision approach to increase spatial data recovery and expedite model convergence. We add consecutive residual connections to the system to reduce feature loss while increasing spatial data retrieval. Our technique is based on identifying gaps in mel-spectrograms distinguishing between normal and dysarthric speech. We conducted several experiments on UTrans-DSR using the UA speech and TORGO datasets, and it outperformed the existing top models. The model performed significantly in pixel’s localized and spatial feature extraction, effectively detecting and classifying spectral gaps. The Tran-DSR model outperforms previous research models, achieving an accuracy of 97.75%.

In-flight pixel degradation of the Sentinel 5 Precursor TROPOMI-SWIR HgCdTe detector

Abstract The TROPOMI-SWIR HgCdTe detector on the Sentinel-5 Precursor mission has been performing in-orbit measurements of molecular absorption in Earth's atmosphere since its launch in October 2017. In its polar orbit the detector is continuously exposed to potentially harmful energetic particles. Calibration measurements taken during the eclipse are used to inspect the performance of this detector. This paper explores the in-orbit degradation of the HgCdTe detector.
After five years, the detector is still performing within specifications, even though pixels are continuously hit by cosmic radiation. The bulk of the impacts have no lasting effects, and most of the damaged pixels (95%) appear to recover on the order of a few days to several months, attributed to a slow spontaneous recovery of defects in the HgCdTe detector material. This is observed at the operational temperature of 140 K. The distribution of the observed recovery times has a mean around nine days with a significant tail towards several months. Pixels that have degraded have a significant probability to degrade again. The location of faulty pixels follows a Poissonian distribution across the detector. No new clusters have appeared, revealing that impacts are dominated by relatively low energetic protons and electrons. Due to the observed spontaneous recovery of pixels, the fraction of pixels meeting all quality requirements in the nominal operations phase has always been over 98.7%. 
The observed performance of the TROPOMI-SWIR detector in-flight impacts selection criteria of HgCdTe detectors for future space instrumentation.

A Nonintrusive Optical Approach to Characterize Heliostats in Utility-Scale Power Tower Plants: Camera Position Sensitivity Analysis

Abstract Optics plays a major role in the effectiveness of concentrating solar power (CSP) technologies. The nonintrusive optical (NIO) approach developed by the National Renewable Energy Laboratory uses uncrewed aircraft system (UAS)-based imaging to survey heliostats in a commercial-scale power tower CSP plant and characterize their optical errors. The image processing algorithm uses photogrammetry to calculate the camera position for each image frame, and the accuracy of the estimated optical errors is highly sensitive to the calculated camera position accuracy. In this study, we simulate a series of case studies in python to examine the impact of different parameters of the sensitivity of the camera calculation, including the number of facet corners used as control points for the photogrammetric calculation, precision error in the detected pixel locations of the facet corners in the image, and precision error of the canting and mounting positions of the facets of the heliostat. The case studies consider heliostat geometry based on three commercial designs to serve as representative examples of different possible sizes of heliostats that the NIO method could be applied to. The results show that increasing the number of control points can improve accuracy for heliostats with many facets, pixel precision has a significantly larger impact on camera calculation accuracy than facet canting and mounting errors, and the camera distance and focal length must be chosen to ensure adequate pixel accuracy on the heliostat surface depending on the size of heliostat. Based on the results, recommendations for the allowable values of each parameter are provided to achieve the required NIO optical error estimation accuracy depending on the size of heliostat.

Hiding image with inception transformer

AbstractImage steganography aims to hide secret data in the cover media for covert communication. Though many deep‐learning‐based image steganography methods have been presented, these approaches suffer from the inefficiency of building long‐distance connections between the cover and secret images, leading to noticeable modification traces and poor steganalysis resistance. To improve the visual imperceptibility of generated stego images, it is essential to establish a global correlation between the cover and secret images. In this way, the secret image can be dispersed throughout the cover image globally. To bridge this gap, a novel image steganography framework called HiiT is proposed, which takes advantage of CNN and Transformer to learn both the local and global pixel correlation in image hiding. Specifically, a new Transformer structure called Inception Transformer is proposed, which incorporates the Inception Net in the attention‐based Transformer architecture. The Inception Net can learn different scaled image features using multiple convolution kernels, while the attention mechanism can learn the global pixel correlation. By this, the proposed Inception Transformer learns the long‐distance pixel dependency between the cover and secret images. Furthermore, we propose a ‘Skip Connection’ mechanism in the proposed Inception Transformer, which merges the low‐level visual features and high‐level semantic features and achieves better model performance. In detail, The HiiT generates higher‐quality stego images with 45.46 PSNR and 0.9915 SSIM. Besides, accurately restored secret images achieve 47.27 PSNR and 0.9952 SSIM. Extensive experimental results show the proposed HiiT significantly improves the image‐hiding performance compared with state‐of‐the‐art methods.

An approach of 2D convolutional neural network–based seismic data fault interpretation with linear annotation and pixel thinking

AbstractThis article introduces an improved method for geological fault interpretation utilizing a 2D convolutional neural network approach with a focus on coalbed horizons, dealing with the problem as an image classification task. At the beginning, linear annotations, reflecting geological fault features, are applied to multiple seismic sections. By considering texture differences between fault and non‐fault areas, we construct samples that represent these distinct zones for training deep neural networks. Initially, fault annotations are transformed into single dots to facilitate pixel‐based processing. To depict a specific dot's geological structure, we employ a matrix clipped around the point, determined by a combination of range and step parameters. Convolutional layers generate filters equivalent to seismic data transformation, streamlining the need for analysis and selection of seismic attributes. The article discusses enhancing the efficiency of 2D convolutional neural network–based fault interpretation by optimizing sample selection, data extraction and model construction procedures. Through the incorporation of data from two mining areas (totalling 27.09 km2) in sample creation, the overall accuracy exceeds 0.99. Recognition extends seamlessly to unlabelled sections, showcasing the innovative technical route and methodology of fault interpretation with linear annotation and pixel‐based thinking. This study presents a method that integrates planar and raster thinking, transitioning from vision‐oriented geological structure annotation to algorithm‐oriented pixel location. The proposed 2D convolutional neural network–based matrix‐oriented fault/non‐fault binary classification demonstrates feasibility and reproducibility, offering a new automated approach for fault detection in coalbeds through convolutional neural network algorithms.

Detection of fireballs in the Lightning Imager data

ABSTRACT The Meteosat Third Generation - Imager 1 satellite, launched on 13 December 2022, features the first Lightning Imager instrument, a high-speed optical camera providing near real-time lightning detection over Europe and Africa. This study demonstrates that signatures of fireballs (i.e. bright meteors) can be detected in Lightning Imager data. We describe a method to analyse this data to determine the timing, light signal, and trajectory of fireballs, highlighting the instrument’s usefulness for bright meteor observation and proposing avenues for further research. By using known fireball locations and timings from external sources, the Lightning Imager data can be filtered to isolate ‘lightning events’ induced by the meteor. From this data set, the fireball light curve is computed by aggregating signal increases measured by instrument pixels. A trajectory is then deduced using a weighted average of pixel locations based on observed intensity. Three examples of fireballs detected by the Lightning Imager are presented, including a man-made meteor from space debris re-entry, with estimated timing, light curves, and trajectories.

A Method for Sorting High-Quality Fresh Sichuan Pepper Based on a Multi-Domain Multi-Scale Feature Fusion Algorithm.

Post-harvest selection of high-quality Sichuan pepper is a critical step in the production process. To achieve this, a visual system needs to analyze Sichuan pepper with varying postures and maturity levels. To quickly and accurately sort high-quality fresh Sichuan pepper, this study proposes a multi-scale frequency domain feature fusion module (MSF3M) and a multi-scale dual-domain feature fusion module (MS-DFFM) to construct a multi-scale, multi-domain fusion algorithm for feature fusion of Sichuan pepper images. The MultiDomain YOLOv8 Model network is then built to segment and classify the target Sichuan pepper, distinguishing the maturity level of individual Sichuan peppercorns. A selection method based on the average local pixel value difference is proposed for sorting high-quality fresh Sichuan pepper. Experimental results show that the MultiDomain YOLOv8-seg achieves an mAP50 of 88.8% for the segmentation of fresh Sichuan pepper, with a model size of only 5.84 MB. The MultiDomain YOLOv8-cls excels in Sichuan pepper maturity classification, with an accuracy of 98.34%. Compared to the YOLOv8 baseline model, the MultiDomain YOLOv8 model offers higher accuracy and a more lightweight structure, making it highly effective in reducing misjudgments and enhancing post-harvest processing efficiency in agricultural applications, ultimately increasing producer profits.

Image harmonization with Simple Hybrid CNN-Transformer Network

Image harmonization seeks to transfer the illumination distribution of the background to that of the foreground within a composite image. Existing methods lack the ability of establishing global–local pixel illumination dependencies between foreground and background of composite images, which is indispensable for sharp and color-consistent harmonized image generation. To overcome this challenge, we design a novel Simple Hybrid CNN-Transformer Network (SHT-Net), which is formulated into an efficient symmetrical hierarchical architecture. It is composed of two newly designed light-weight Transformer blocks. Firstly, the scale-aware gated block is designed to capture multi-scale features through different heads and expand the receptive fields, which facilitates to generate images with fine-grained details. Secondly, we introduce a simple parallel attention block, which integrates the window-based self-attention and gated channel attention in parallel, resulting in simultaneously global–local pixel illumination relationship modeling capability. Besides, we propose an efficient simple feed forward network to filter out less informative features and allow the features to contribute to generating photo-realistic harmonized results passing through. Extensive experiments on image harmonization benchmarks indicate that our method achieve promising quantitative and qualitative results. The code and pre-trained models are available at https://github.com/guanguanboy/SHT-Net.

Audio-guided implicit neural representation for local image stylization

We present a novel framework for audio-guided localized image stylization. Sound often provides information about the specific context of a scene and is closely related to a certain part of the scene or object. However, existing image stylization works have focused on stylizing the entire image using an image or text input. Stylizing a particular part of the image based on audio input is natural but challenging. This work proposes a framework in which a user provides an audio input to localize the target in the input image and another to locally stylize the target object or scene. We first produce a fine localization map using an audio-visual localization network leveraging CLIP embedding space. We then utilize an implicit neural representation (INR) along with the predicted localization map to stylize the target based on sound information. The INR manipulates local pixel values to be semantically consistent with the provided audio input. Our experiments show that the proposed framework outperforms other audio-guided stylization methods. Moreover, we observe that our method constructs concise localization maps and naturally manipulates the target object or scene in accordance with the given audio input.

PMLPNet: Classifying Multi-Class Pests in Wild Environment via a Novel Convolutional Neural Network

Pest damage is a major factor in reducing crop yield and has negative impacts on the economy. However, the complex background, diversity of pests, and individual differences pose challenges for classification algorithms. In this study, we propose a patch-based neural network (PMLPNet) for multi-class pest classification. PMLPNet leverages spatial and channel contextual semantic features through meticulously designed token- and channel-mixing MLPs, respectively. This innovative structure enhances the model’s ability to accurately classify complex multi-class pests by providing high-quality local and global pixel semantic features for the fully connected layer and activation function. We constructed a database of 4510 images spanning 40 types of plant pests across 4 crops. Experimental results demonstrate that PMLPNet outperforms existing CNN models, achieving an accuracy of 92.73%. Additionally, heat maps reveal distinctions among different pest images, while patch probability-based visualizations highlight heterogeneity within pest images. Validation on external datasets (IP102 and PlantDoc) confirms the robust generalization performance of PMLPNet. In summary, our research advances intelligent pest classification techniques, effectively identifying various pest types in diverse crop images.

A Dual-Branch Fusion of a Graph Convolutional Network and a Convolutional Neural Network for Hyperspectral Image Classification.

Semi-supervised graph convolutional networks (SSGCNs) have been proven to be effective in hyperspectral image classification (HSIC). However, limited training data and spectral uncertainty restrict the classification performance, and the computational demands of a graph convolution network (GCN) present challenges for real-time applications. To overcome these issues, a dual-branch fusion of a GCN and convolutional neural network (DFGCN) is proposed for HSIC tasks. The GCN branch uses an adaptive multi-scale superpixel segmentation method to build fusion adjacency matrices at various scales, which improves the graph convolution efficiency and node representations. Additionally, a spectral feature enhancement module (SFEM) enhances the transmission of crucial channel information between the two graph convolutions. Meanwhile, the CNN branch uses a convolutional network with an attention mechanism to focus on detailed features of local areas. By combining the multi-scale superpixel features from the GCN branch and the local pixel features from the CNN branch, this method leverages complementary features to fully learn rich spatial-spectral information. Our experimental results demonstrate that the proposed method outperforms existing advanced approaches in terms of classification efficiency and accuracy across three benchmark data sets.

A two-stream-network based steganalysis network: TSNet

The three-stage framework consisting of preprocessing, feature extraction and classification is the most commonly used structure in traditional deep learning based steganalysis networks (SNs). Preprocessing, as the core part of the three-stage framework, typically adopts fixed high-pass filters to generate filter residuals, and exploits a feature enhancement module following high-pass filters to improve signal-to-noise ratio of the stego signal, thus strengthening the expression of steganographic features. However, fixed high-pass filters can only provide filter residuals through the linear relationships, rather than complex dependencies automatically learned, among local pixels, which suggests that fixed high-pass filters lack powerful ability of extracting comprehensive steganographic features from the beginning of SNs. Inspired by the motivation, a two-stream-network based SN called TSNet is proposed to extract comprehensive steganographic features from two different perspectives, namely automatic learning and fixed filtering. The discrete cosine transform attention mechanism (DCTAM)-based fusion module (DCTAM-FM) with considering the steganalysis properties is proposed to fuse two streams and enhance channel representation using DCTAM, which is conductive to improving the detection accuracy. Two streams aim to explore different types of residuals from two perspectives, but inevitably lead to a considerable number of parameters and high computational complexity. To this end, the feature extraction stage contains lightweight depthwise separable convolution blocks and the linearly increasing style of the number of channels. Combining the two-stream network, DCTAM-FM as well as the lightweight feature extraction stage, TSNet obtains deep composite features, which can fully express the comprehensive steganographic features, and thus brings a high detection accuracy. Experimental results from three benchmark datasets demonstrate that TSNet outperforms current state-of-the-art methods in detection accuracy and has an acceptable computational complexity.

Deep learning aided computer vision system for automated linear type trait evaluation in dairy cows

The assessment of traits is important in determining production potential, reproductive performance, and overall health of dairy cows. The assessment of these traits typically involves visual inspection and manual measurement, which can be time-consuming, subject to bias, and potentially distressing for the animals. To address these challenges, convolutional neural networks (CNNs)-aided non-invasive computer vision system was developed in the present study. This system consists of a depth camera to acquire the RGB images and depth information of cows. The DeepLabV3+ model, having the ResNet50 model as a backbone, was utilized to segment the body parts of cows from RGB images. Image processing-based algorithms were developed to extract key pixel locations for each trait from these segmented images. The system estimated trait dimensions utilizing 3D data of respective key points. The mean-IoU (intersection-over-union) values for the developed segmentation models were 93.46%, 91.25%, and 99.27% for side-view, back-view traits, and stature, respectively. Additionally, the vision system was able to estimate the trait dimensions with mean absolute percentage error (MAPE) below 6.0%. For a few traits, MAPE, however, exceeded 10.0%, indicating higher error. Inaccurate segmentation, imprecise key point extraction, visual overlaps of specific body parts, and variations in cow postures contribute to such errors. The developed system attained a Ratio of Performance to Deviation (RPD) above 1.2 for all traits, indicating its ability to estimate the dimensions of traits efficaciously. Thus, the present study demonstrated the potential of a CNN-based computer vision-based system for automating the trait measurement process in cows.

X-ray pneumonia detection using angular and radial local binary patterns fusion

Detecting pneumonia via X-ray images is a crucial phase in determining any probable bacterial or viral disease. The automation of this stage is critical since it speeds up the diagnosing process. LBPs have excelled in investigating local conditions within a neighborhood, where they aggregate and analyze local statistical results. In this paper, we present a novel family of LBP descriptors based on local accumulative pixel disparities. We construct the new descriptors using a circular neighborhood acquired from the initial square filter. These descriptors consider angular and radial transitions that occur in both the microstructures and macrostructures of image textures. Eventually, a more accurate and comprehensive visual representation is obtained. Furthermore, we offer a data fusion step based on a voting mechanism to integrate the retrieved data efficiently. We show many types of analyses that demonstrate LBP's and the suggested extensions' capability to extract discriminant features from X-ray images. The suggested method is tested on two different datasets with large diversities that include images from various demographics and regions. Several measures, including accuracy rate, precision, sensitivity, F-measure, and specificity, are used to evaluate the system's efficacy. According to the testing results, the proposed system provides a best successful recognition rate of 82.7 %.

Visual State Space Model for Image Deraining with Symmetrical Scanning

Image deraining aims to mitigate the adverse effects of rain streaks on image quality. Recently, the advent of convolutional neural networks (CNNs) and Vision Transformers (ViTs) has catalyzed substantial advancements in this field. However, these methods fail to effectively balance model efficiency and image deraining performance. In this paper, we propose an effective, locally enhanced visual state space model for image deraining, called DerainMamba. Specifically, we introduce a global-aware state space model to better capture long-range dependencies with linear complexity. In contrast to existing methods that utilize fixed unidirectional scan mechanisms, we propose a direction-aware symmetrical scanning module to enhance the feature capture of rain streak direction. Furthermore, we integrate a local-aware mixture of experts into our framework to mitigate local pixel forgetting, thereby enhancing the overall quality of high-resolution image reconstruction. Experimental results validate that the proposed method surpasses state-of-the-art approaches on six benchmark datasets.

Leveraging foreground–background cues for semantically-driven, training-free moving object detection

Deep learning algorithms and semantic cue-based methods are increasingly used to detect moving objects. The semantic label changes in pixel location over time are used for moving object detection in semantic cue-based studies. In this approach, traditional background subtraction is employed only when there is a lack of semantic background. However, the change in semantic information in each pixel is insufficient for accurate detection of moving objects due to similar labels assignments for different objects and limited use of pixel intensity data. Moreover, challenges arise in detecting moving objects, such as a lack of diversity in labeled objects, an uneven distribution of labels or classes, and an insufficient amount of unseen datasets. In this study, we propose the Moving Semantic Object Detector (MSemOD), a train-less moving object detection method based on a traditional method enhanced with semantic information provided by pre-trained semantic segmentation models. The proposed method includes a region-based decision step, utilizing the traditional methods and semantic information as spatial–temporal cues. Threshold values are applied in the method to determine the temporal and spatial movements of each segment in the image. To improve the consistency of the semantic information, the proposed method converts the semantic segmentation output into a semantic probability map by enhancing the temporal and spatial state. The performance of the proposed method is evaluated on the LASIESTA, CDNet2014, and the BMC datasets and the results are compared with the state-of-the-art algorithms. The results show that using region-based semantic information can significantly reduce false positives and enhance foreground–background segmentation in moving object detection.

High count rate alpha particle spectroscopy by a digital optical camera: A comparison study on edge detection algorithms

Alpha particle spectroscopy is highly significant in technologies related to radioactivity, particularly in radiation safety and the study of natural radioactivity. On the other hand, tools capable of identifying and determining the energies of alpha particles are of great interest. Among these tools, Commercial Off-The-Shelf (COTS) instruments, which can be used for alpha particle detection, have gained significant importance due to their ease of use and low cost. In this study, alpha particle spectroscopy of a Ra-226 source has been performed using a commercial digital camera based-on a CMOS sensor. In order to determine energy spectrum of alpha particles emitted from the source, the performance of various edge detection algorithms was examined. An algorithm based-on local pixel changes was introduced and implemented. By comparing the run times of the algorithms, it was shown that the algorithm proposed in this article exhibited the shortest run-time. Additionally, a method for rejecting pile-up events to correct their destructive effects on the alpha particle energy spectrum was introduced and implemented.

Determination of scintillation pixel location through deep learning using a two-layer DOI detector

Determination of scintillation pixel location through deep learning using a two-layer DOI detector

Quantitative insights into osteocyte density developmental patterns in fossilized bone — A new method

This study presents a new methodology that combines quantitative image analysis, clustering, and statistical techniques to examine the 2D distribution of osteohistological features in an extinct stem-group bird. The driving force of our research was the need to map ontogenetic changes in the spatial density of osteocyte lacunae in the type specimen of Musivavis amabilis from the Lower Cretaceous of China. We particularly focused on developing tools to reveal quantitative aspects of the dynamics in the formation of avascular bone in this active flier. We achieve this goal by proposing an algorithm with the following methodological steps: 1) We obtain relevant coordinate details for pixel locations selected by thresholding the original image with shading as a criterion. 2) We estimate density using the Gaussian kernel estimator and refined it through observations and regression analysis. 3) After slicing the image, we apply k-means clustering to obtain one-dimensional representations of lacunar density. 4) We proceed by implementing weighted averaging employing the k-nearest neighbor approach. Having applied these steps, we are able to quantitatively disclose growth processes previously unnoticed and reveal dynamics in the formation of lacunar bone tissue in the enantiornithine birds capable of power flight.