Pixel Representation Research Articles

Improved Small Object Detection Algorithm CRL-YOLOv5.

Detecting small objects in images poses significant challenges due to their limited pixel representation and the difficulty in extracting sufficient features, often leading to missed or false detections. To address these challenges and enhance detection accuracy, this paper presents an improved small object detection algorithm, CRL-YOLOv5. The proposed approach integrates the Convolutional Block Attention Module (CBAM) attention mechanism into the C3 module of the backbone network, which enhances the localization accuracy of small objects. Additionally, the Receptive Field Block (RFB) module is introduced to expand the model's receptive field, thereby fully leveraging contextual information. Furthermore, the network architecture is restructured to include an additional detection layer specifically for small objects, allowing for deeper feature extraction from shallow layers. When tested on the VisDrone2019 small object dataset, CRL-YOLOv5 achieved an mAP50 of 39.2%, representing a 5.4% improvement over the original YOLOv5, effectively boosting the detection precision for small objects in images.

Sequence Segmentation of Nematodes in Atlantic Cod with Multispectral Imaging Data.

Nematodes pose significant challenges for the fish processing industry, particularly in white fish. Despite technological advances, the industry still depends on manual labor for the detection and extraction of nematodes. This study addresses the initial steps of automatic nematode detection and differentiation from other common defects in fish fillets, such as skin remnants and blood spots. VideometerLab 4, an advanced Multispectral Imaging (MSI) System, was used to acquire 270 images of 50 Atlantic cod fillets under controlled conditions. In total, 173 nematodes were labeled using the Segment Anything Model (SAM), which is trained to automatically segment objects of interest from only few representative pixels. With the acquired dataset, we study the potential of identifying nematodes through their spectral signature. We incorporated normalized Canonical Discriminant Analysis (nCDA) to develop segmentation models trained to distinguish between different components within the fish fillets. By incorporating multiple segmentation models, we aimed to achieve a satisfactory balance between false negatives and false positives. This resulted in 88% precision and 79% recall for our annotated test data. This approach could improve process control by accurately identifying fillets with nematodes. Using MSI minimizes unnecessary inspection of fillets in good condition and concurrently boosts product safety and quality.

Cubixel: a novel paradigm in image processing using three-dimensional pixel representation

Cubixel: a novel paradigm in image processing using three-dimensional pixel representation

Inverse design of functionally graded porous structures with target dynamic responses

Although functionally graded porous structures (FGPS) have been studied comprehensively due to their excellent mechanical and functional properties, the design process of FGPS with target stress response still dependent on continuous trial and error, which is not only time-consuming but also heavily relies on experience and intuition. This study proposes a novel inverse design framework of 2D FGPS with targeted dynamic stress-strain responses via a combination of conditional diffusion model and a residual neural network. Unlike traditional binary pixel representations, this approach utilizes nuclei position maps and color mapping technique to represent the structure configurations and cell wall thicknesses to improve the efficiency and performance of the model. Firstly, various functionally graded porous structures were constructed by employing Voronoi diagram techniques, and finite element simulations were conducted to calculate the nonlinear responses subjected to dynamic loadings. A dataset comprising 2100 FGPS and corresponding nonlinear stress-strain responses was established. Then, an inverse design framework is formulated by integrating a generator that uses a diffusion model to synthesize structures conditioned on specific target responses, with a predictor that employs a residual neural network to assess the responses of these structures. Finally, to demonstrate the effectiveness of the approach, the performance of the predictor and generator was thoroughly investigated. Structures with various target stress-strain responses were generated through the proposed method, and validated by finite element analysis. A detailed impact experiment was also carried out to verify the effectiveness of the proposed methods. The result shows that the framework can effectively generate new structures with objective nonlinear responses. This work offers a fast and efficient way to design FGPS that meet specific performance objectives.

CSAN-UNet: Channel Spatial Attention Nested UNet for Infrared Small Target Detection

Segmenting small infrared targets presents a significant challenge for traditional image processing architectures due to the inherent lack of texture, minimal shape information, and their sparse pixel representation within images. The conventional UNet architecture, while proficient in general segmentation tasks, inadequately addresses the nuances of small infrared target segmentation due to its reliance on downsampling operations, such as pooling, which often results in the loss of critical target information. This paper introduces the Channel Spatial Attention Nested UNet (CSAN-UNet), an innovative architecture designed specifically to enhance the detection and segmentation of small infrared targets. Central to CSAN-UNet’s design is the Cascaded Channel and Spatial Convolutional Attention Module (CSCAM), a novel component that adaptively enhances multi-level features and mitigates the loss of target information attributable to downsampling processes. Additionally, the Channel-priority and Spatial Attention Cascade Module (CPSAM) represents another pivotal advancement within CSAN-UNet, prioritizing channel-level adjustments alongside spatial attention mechanisms to efficiently extract deep semantic information pertinent to small infrared targets. Empirical validation conducted on two public datasets confirms that CSAN-UNet surpasses existing state-of-the-art algorithms in segmentation performance, while simultaneously reducing computational overhead.

Intelligent Detection of Surface Defects in High-Speed Railway Ballastless Track Based on Self-Attention and Transfer Learning

The detection of ballastless track surface (BTS) defects is a prerequisite for ensuring the safe operation of high-speed railways. Traditional convolutional neural networks fail to fully exploit contextual information and lack global pixel representations. The extensive stacking of convolutions leads deep learning models to play a black-box detection role, lacking interpretability. Due to the current lack of sufficient high-quality surface data for ballastless tracks, it is a severe constraint on the accurate identification of the substructure state in high-speed railways. This paper proposes an intelligent detection method for BTS defects named TrackNet based on self-attention and transfer learning. The method enhances the fusion ability of global features of BTS defects using multihead self-attention. The model’s dependence on extensive defect data is reduced by transferring knowledge from large-scale publicly available datasets. Experimental results demonstrate that compared to advanced Swin Transformer model results, the TrackNet model achieves improvements in average accuracy and F1-score by 5.15% and 5.16%, respectively, on limited test data. The TrackNet model visualizes the decision regions of the model in identifying BTS defects, revealing the black-box recognition mechanism of deep learning models. This research performs engineering applications and provides valuable insights for the multiclass recognition of BTS defects in high-speed railways.

Semi-Supervised Multiscale Dynamic Graph Convolution Network for Hyperspectral Image Classification.

In recent years, convolutional neural networks (CNNs)-based methods achieve cracking performance on hyperspectral image (HSI) classification tasks, due to its hierarchical structure and strong nonlinear fitting capacity. Most of them, however, are supervised approaches that need a large number of labeled data to train them. Conventional convolution kernels are fixed shape of rectangular with fixed sizes, which are good at capturing short-range relations between pixels within HSIs but ignore the long-range context within HSIs, limiting their performance. To overcome the limitations mentioned above, we present a dynamic multiscale graph convolutional network (GCN) classifier (DMSGer). DMSGer first constructs a relatively small graph at region-level based on a superpixel segmentation algorithm and metric-learning. A dynamic pixel-level feature update strategy is then applied to the region-level adjacency matrix, which can help DMSGer learn the pixel representation dynamically. Finally, to deeply understand the complex contents within HSIs, our model is expanded into a multiscale version. On the one hand, by introducing graph learning theory, DMSGer accomplishes HSI classification tasks in a semi-supervised manner, relieving the pressure of collecting abundant labeled samples. Superpixels are generally in irregular shapes and sizes which can group only similar pixels in a neighborhood. On the other hand, based on the proposed dynamic-GCN, the pixel-level and region-level information can be captured simultaneously in one graph convolution layer such that the classification results can be improved. Also, due to the proper multiscale expansion, more helpful information can be captured from HSIs. Extensive experiments were conducted on four public HSIs, and the promising results illustrate that our DMSGer is robust in classifying HSIs. Our source codes are available at https://github.com/TangXu-Group/DMSGer.

TENSOR AND VECTOR APPROACHES TO OBJECTS RECOGNITION BY INVERSE FEATURE FILTERS

The investigation of the extraction of image objects features by filters based on tensor and vector data presentation is considered. The tensor data is obtained as a sum of rank-one tensors, given by the tensor product of the vector of lexicographic representation of image fragments pixels with itself. The accumulated tensor is approximated by one rank tensor obtained using singular values decomposition. It has been shown that the main vector of the decomposition can be considered as the object feature vector. The vector data is obtained by accumulating analogous vectors of image fragments pixels. The accumulated vector is also considered as an object feature. The filter banks of a set of objects are obtained by regularized inversion of the matrices compiled by object features vectors. Optimized regularization of the inversion is used to expand the regions of object features capture with minimal error. The object fragments and corresponding feature vectors are selected through a training iterative process. The tensor and vector approaches create two channels for recognition. High efficiency of object recognition can be achieved by choosing the filter capture band and creating filter branches according to the given bands. The filters create a convolutional network to recognize a set of objects. It has been shown that the obtained filters have an advantage over known correlation filters when recognizing objects with small fragments.

SRFormer: Text Detection Transformer with Incorporated Segmentation and Regression

Existing techniques for text detection can be broadly classified into two primary groups: segmentation-based and regression-based methods. Segmentation models offer enhanced robustness to font variations but require intricate post-processing, leading to high computational overhead. Regression-based methods undertake instance-aware prediction but face limitations in robustness and data efficiency due to their reliance on high-level representations. In our academic pursuit, we propose SRFormer, a unified DETR-based model with amalgamated Segmentation and Regression, aiming at the synergistic harnessing of the inherent robustness in segmentation representations, along with the straightforward post-processing of instance-level regression. Our empirical analysis indicates that favorable segmentation predictions can be obtained at the initial decoder layers. In light of this, we constrain the incorporation of segmentation branches to the first few decoder layers and employ progressive regression refinement in subsequent layers, achieving performance gains while minimizing computational load from the mask. Furthermore, we propose a Mask-informed Query Enhancement module. We take the segmentation result as a natural soft-ROI to pool and extract robust pixel representations, which are then employed to enhance and diversify instance queries. Extensive experimentation across multiple benchmarks has yielded compelling findings, highlighting our method's exceptional robustness, superior training and data efficiency, as well as its state-of-the-art performance. Our code is available at https://github.com/retsuh-bqw/SRFormer-Text-Det.

Quantifying of surface urban heat island intensity in Isfahan metropolis using MODIS\Terra\LST data

Heat island characteristics depend on the background climate of the site where the city is located. Therefore, an index was defined for the Isfahan metropolitan area to quantify the surface urban heat island intensity. This new index is based on the representative pixels of urban and non-urban areas. For this purpose, MODIS land cover type product (MCD12Q1) data were used to distinguish between urban and non-urban areas. Also, data from the MODIS/Terra land surface temperature product (MOD11A1) from 2000 to 2018 were utilized for daytime and nighttime to study the surface heat island intensity. Then, the representative pixels of urban and non-urban areas were identified using the spatial correlation method, and the heat island index was calculated for the metropolitan area of Isfahan. The study showed that the frequency distribution of the nighttime heat island index follows a normal distribution and is often 3.5 to 4º K above the temperature of the surrounding areas of the city. The 365-day floating mean of the surface urban heat island reveals that this index has increased in recent years. The research of temporal behavior showed that the intensity of the surface urban heat island reaches its maximum in January and becomes weaker in summer, while the survey of spatial behavior showed that the core of the surface urban heat island extends towards downtown areas, where the oldest part of the city is located.

Quantum annealing-based computed tomography using variational approach for a real-number image reconstruction

Objective. Despite recent advancements in quantum computing, the limited number of available qubits has hindered progress in CT reconstruction. This study investigates the feasibility of utilizing quantum annealing-based computed tomography (QACT) with current quantum bit levels. Approach. The QACT algorithm aims to precisely solve quadratic unconstrained binary optimization problems. Furthermore, a novel approach is proposed to reconstruct images by approximating real numbers using the variational method. This approach allows for accurate CT image reconstruction using a small number of qubits. The study examines the impact of projection data quantity and noise on various image sizes ranging from 4 × 4 to 24 × 24 pixels. The reconstructed results are compared against conventional reconstruction algorithms, namely maximum likelihood expectation maximization (MLEM) and filtered back projection (FBP). Main result. By employing the variational approach and utilizing two qubits for each pixel of the image, accurate reconstruction was achieved with an adequate number of projections. Under conditions of abundant projections and lower noise levels, the image quality in QACT algorithm outperformed that of MLEM and FBP algorithms. However, in situations with limited projection data and in the presence of noise, the image quality in QACT was inferior to that in MLEM. Significance. This study developed the QACT reconstruction algorithm using the variational approach for real-number reconstruction. Remarkably, only 2 qubits were required for each pixel representation, demonstrating their sufficiency for accurate reconstruction.

Semantic segmentation of urban land classes using a multi-scale dataset

ABSTRACT The use of remote sensing imagery for land cover and land use classification has made significant advancements in recent years. However, it becomes particularly challenging to enhance the semantic representation of high-resolution networks while dealing with uneven land categories and merging multi-scale data without compromising the accuracy of semantic segmentation. To tackle this challenge, this paper presents a novel method for classifying high-resolution remote sensing images based on a deep neural network that performs semantic segmentation of urban construction lands into five categories: vegetation, water, buildings, roads, and bare soil. The network incorporates a U-shaped high-resolution neural network and the advanced high-resolution network (HRNet) framework. The parallel storage of feature maps with different resolutions enables the exchange of information between them. The data pre-processing module addresses the issue of data imbalance in the semantic segmentation of urban construction lands, resulting in an increase in Intersection over Union (IoU) values for different land types by 3.75%-12.01%. Additionally, a target context representation module is introduced to enhance the feature representation of pixels by calculating the relationship between pixels and multiple target regions. Moreover, a polarization attention mechanism is proposed to extract the characteristics of geographical objects in all directions and achieve a stronger semantic representation. This method provides a novel approach to accurately and effectively extract information on construction lands and advance the development of monitoring algorithms for urban construction lands. To validate the proposed U-HRNet-OCR+PSA network, a comparative analysis was conducted with six classical networks, including DeepLabv3+, PSPNet, U-Net, U-Net++, HRNet, and HRNet-OCR, as well as the relatively new ViT-adapter-L, Oneformer and InternImage-H. The experiments demonstrate that the U-HRNet-OCR+PSA network achieves higher accuracy compared to the aforementioned networks. Specifically, the corresponding IoU values for the buildings, roads, vegetation, bare soil, and water in the multi-scale dataset are 89.79%, 90.05%, 94.89%, 85.91%, and 88.36%, respectively.

CRSANet: Class Representations Self-Attention network for the segmentation of thyroid nodules

Segmenting accurate thyroid nodules in medical ultrasound images is always non-trivial due to the large variation in size, shape and texture of nodules, and also the presence of surrounding tissues and organs with similar intensity values. Existing algorithms solve this by designing multi-scale spatial context modeling modules and introducing the emerging self-attention based paradigm, while failing to learn the class-wise relational context around nodules. To this end, we propose a novel coarse-to-fine framework, namely, CRSANet, which leverages both class-wise relational context and spatial multi-scale context for comprehensive context perception. Specifically, we first introduce the concept of Class Representations, which describe the overall representation of thyroid nodules in ultrasound images from a categorical perspective. Secondly, we propose a backbone-free dual-branch Class Representation Self-Attention Module (CRSA) to refine the coarse segmentation results from segmentation backbones. Concretely, Fine Class Region (FCR) branch is proposed to impose consistency between pixels and their corresponding Class Representations to correct the wrongly classified pixels. Meanwhile, Class Contextual Representation (CCR) branch is introduced to enhance the ability of pixel representations with the corresponding Class Representations. We evaluate the effectiveness and generality of CRSANet using three types of backbone networks. In the experiments, our proposed U-CRSANet achieves a mIoU of 78.86% and 81.35%, an accuracy of 97.11% and 98.63%, a Dice value of 74.26% and 87.57%, respectively, which outperforms most state-of-the-art methods on the public DDTI and TN-SCUI datasets.

Unsupervised Part Discovery via Dual Representation Alignment.

Object parts serve as crucial intermediate representations in various downstream tasks, but part-level representation learning still has not received as much attention as other vision tasks. Previous research has established that Vision Transformer can learn instance-level attention without labels, extracting high-quality instance-level representations for boosting downstream tasks. In this paper, we achieve unsupervised part-specific attention learning using a novel paradigm and further employ the part representations to improve part discovery performance. Specifically, paired images are generated from the same image with different geometric transformations, and multiple part representations are extracted from these paired images using a novel module, named PartFormer. These part representations from the paired images are then exchanged to improve geometric transformation invariance. Subsequently, the part representations are aligned with the feature map extracted by a feature map encoder, achieving high similarity with the pixel representations of the corresponding part regions and low similarity in irrelevant regions. Finally, the geometric and semantic constraints are applied to the part representations through the intermediate results in alignment for part-specific attention learning, encouraging the PartFormer to focus locally and the part representations to explicitly include the information of the corresponding parts. Moreover, the aligned part representations can further serve as a series of reliable detectors in the testing phase, predicting pixel masks for part discovery. Extensive experiments are carried out on four widely used datasets, and our results demonstrate that the proposed method achieves competitive performance and robustness due to its part-specific attention. The code will be released upon paper acceptance.

Pixel representations, sampling, and label correction for semantic part detection

Pixel representations, sampling, and label correction for semantic part detection

A Radiotherapy Dose Map-Guided Deep Learning Method for Predicting Pathological Complete Response in Esophageal Cancer Patients after Neoadjuvant Chemoradiotherapy Followed by Surgery.

Esophageal cancer is a deadly disease, and neoadjuvant chemoradiotherapy can improve patient survival, particularly for patients achieving a pathological complete response (ypCR). However, existing imaging methods struggle to accurately predict ypCR. This study explores computer-aided detection methods, considering both imaging data and radiotherapy dose variations to enhance prediction accuracy. It involved patients with node-positive esophageal squamous cell carcinoma undergoing neoadjuvant chemoradiotherapy and surgery, with data collected from 2014 to 2017, randomly split into five subsets for 5-fold cross-validation. The algorithm DCRNet, an advanced version of OCRNet, integrates RT dose distribution into dose contextual representations (DCR), combining dose and pixel representation with ten soft regions. Among the 80 enrolled patients (mean age 55.68 years, primarily male, with stage III disease and middle-part lesions), the ypCR rate was 28.75%, showing no significant demographic or disease differences between the ypCR and non-ypCR groups. Among the three summarization methods, the maximum value across the CTV method produced the best results with an AUC of 0.928. The HRNetV2p model with DCR performed the best among the four backbone models tested, with an AUC of 0.928 (95% CI, 0.884-0.972) based on 5-fold cross-validation, showing significant improvement compared to other models. This underscores DCR-equipped models' superior AUC outcomes. The study highlights the potential of dose-guided deep learning in ypCR prediction, necessitating larger, multicenter studies to validate the results.

Hyperspectral Anomaly Detection With Tensor Average Rank and Piecewise Smoothness Constraints.

Anomaly detection in hyperspectral images (HSIs) has attracted considerable interest in the remote-sensing domain, which aims to identify pixels with different spectral and spatial features from their surroundings. Most of the existing anomaly detection methods convert the 3-D data cube to a 2-D matrix composed of independent spectral vectors, which destroys the intrinsic spatial correlation between the pixels and their surrounding pixels, thus leading to considerable degradation in detection performance. In this article, we develop a tensor-based anomaly detection algorithm that can effectively preserve the spatial-spectral information of the original data. We first separate the 3-D HSI data into a background tensor and an anomaly tensor. Then the tensor nuclear norm based on the tensor singular value decomposition (SVD) is exploited to characterize the global low rank existing in both the spectral and spatial directions of the background tensor. In addition, the total variation (TV) regularization is incorporated due to the piecewise smoothness. For the anomaly component, the l2.1 norm is exploited to promote the group sparsity of anomalous pixels. In order to improve the ability of the algorithm to distinguish the anomaly from the background, we design a robust background dictionary. We first split the HSI data into local clusters by leveraging their spectral similarity and spatial distance. Then we develop a simple but effective way based on the SVD to select representative pixels as atoms. The constructed background dictionary can effectively represent the background materials and eliminate anomalies. Experimental results obtained using several real hyperspectral datasets demonstrate the superiority of the proposed method compared with some state-of-the-art anomaly detection algorithms.

StructToken: Rethinking Semantic Segmentation With Structural Prior

In previous deep-learning-based methods, semantic segmentation has been regarded as a static or dynamic per-pixel classification task, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., classify each pixel representation to a specific category. However, these methods only focus on learning better pixel representations or classification kernels while ignoring the structural information of objects, which is critical to human decision-making mechanism. In this paper, we present a new paradigm for semantic segmentation, named structure-aware extraction. Specifically, it generates the segmentation results via the interactions between a set of learned structure tokens and the image feature, which aims to progressively extract the structural information of each category from the feature. Extensive experiments show that our StructToken outperforms the state-of-the-art on three widely-used benchmarks, including ADE20K, Cityscapes, and COCO-Stuff-10K.

A biological sequence comparison algorithm using quantum computers

Genetic information is encoded as linear sequences of nucleotides, represented by letters ranging from thousands to billions. Differences between sequences are identified through comparative approaches like sequence analysis, where variations can occur at the individual nucleotide level or collectively due to various phenomena such as recombination or deletion. Detecting these sequence differences is vital for understanding biology and medicine, but the complexity and size of genomic data require substantial classical computing power. Inspired by human visual perception and pixel representation on quantum computers, we leverage these techniques to implement pairwise sequence analysis. Our method utilizes the Flexible Representation of Quantum Images (FRQI) framework, enabling comparisons at a fine granularity to single letters or amino acids within gene sequences. This novel approach enhances accuracy and resolution, surpassing traditional methods by capturing subtle genetic variations with precision. In summary, our approach offers algorithmic advantages, including reduced time complexity, improved space efficiency, and accurate sequence comparisons. The novelty lies in applying the FRQI algorithm to compare quantum images in genome sequencing, allowing for examination at the individual letter or amino acid level. This breakthrough holds promise for advancing biological data analysis and enables a more comprehensive understanding of genetic information.

Drivable Agricultural Road Region Detection Based on Pixel-Level Segmentation with Contextual Representation Augmentation

Drivable area detection is crucial for the autonomous navigation of agricultural robots. However, semi-structured agricultural roads are generally not marked with lanes and their boundaries are ambiguous, which impedes the accurate segmentation of drivable areas and consequently paralyzes the robots. This paper proposes a deep learning network model for realizing high-resolution segmentation of agricultural roads by leveraging contextual representations to augment road objectness. The backbone adopts HRNet to extract high-resolution road features in parallel at multiple scales. To strengthen the relationship between pixels and corresponding object regions, we use object-contextual representations (OCR) to augment the feature representations of pixels. Finally, a differentiable binarization (DB) decision head is used to perform threshold-adaptive segmentation for road boundaries. To quantify the performance of our method, we used an agricultural semi-structured road dataset and conducted experiments. The experimental results show that the mIoU reaches 97.85%, and the Boundary IoU achieves 90.88%. Both the segmentation accuracy and the boundary quality outperform the existing methods, which shows the tailored segmentation networks with contextual representations are beneficial to improving the detection accuracy of the semi-structured drivable areas in agricultural scene.