Pixel Features Research Articles

An innovative traffic flow detection model based on temporal video frame analysis and grayscale aggregation quantification

AbstractCurrent traffic status detection methods heavily rely on historical traffic flow data and vehicle counts. However, these methods often fail to meet the stringent real‐time requirements of state detection, especially on edge devices with limited computing resources.To address these challenges, this study develops a traffic alert model using temporal video frame analysis and grayscale aggregation quantization techniques. Initially, the model uses distance mapping between pixel features and frames of road traffic videos to construct a comprehensive road environment and vehicle segmentation model. The model also establishes a mapping between pixel equidistant lines and actual distances, enabling precise congestion detection. This approach significantly reduces costs associated with traditional traffic detection methods as it does not rely on historical data. Performance evaluation using fixed‐point road monitoring data indicates that the proposed model outperforms traditional traffic state detection models, with a performance improvement of approximately 4.7% to 9.5%. Additionally, the model improves computing resource efficiency by approximately 72.5% and demonstrates substantial real‐time detection capabilities.

An algorithm for lane detection based on RIME optimization and optimal threshold

In order to address the challenges of low lane line detection rates caused by complex road conditions,we propose a novel algorithm that integrates frost and ice optimisation with optimal thresholding. A pre-processing model based on Retinex theory is used to reduce noise and preserve grey scale detail. The optimal OTSU threshold is determined for segmentation, which is enhanced by tent mapping. To further enhance the precision of the detection process, the binarized image is transformed into a bird’s-eye view, and the lane line pixel features are identified through the use of an adaptive sliding window. Ultimately, the RANSAC algorithm is utilized in conjunction with a parabolic model for lane line fitting. The experimental results demonstrate that, in comparison to similar image segmentation algorithms, the proposed method exhibits a notable advantage in terms of threshold calculation error and computational efficiency. Moreover, in comparison to analogous line detection algorithms, the detection accuracy rate reaches 93.87%, effectively reducing the impact of interference factors and demonstrating remarkable robustness that surpasses the traditional Hough Transform, which has an accuracy of 43.2%, and sliding window and Hough transform, with an accuracy of 89.16%. The code of our research work is publicly available at: https://github.com/zx2000430/rime.

Unsupervised cervical cell instance segmentation method integrating cellular characteristics.

Cell instance segmentation is a key technology for cervical cancer auxiliary diagnosis systems. However, pixel-level annotation is time-consuming and labor-intensive, making it difficult to obtain a large amount of annotated data. This results in the model not being fully trained. In response to these problems, this paper proposes an unsupervised cervical cell instance segmentation method that integrates cell characteristics. Cervical cells have a clear corresponding structure between the nucleus and cytoplasm. This method fully takes this feature into account by building a dual-flow framework to locate the nucleus and cytoplasm and generate high-quality pseudo-labels. In the nucleus segmentation stage, the position and range of the nucleus are determined using the standard cell-restricted nucleus segmentation method. In the cytoplasm segmentation stage, a multi-angle collaborative segmentation method is used to achieve the positioning of the cytoplasm. First, taking advantage of the self-similarity characteristics of pixel blocks in cells, a cytoplasmic segmentation method based on self-similarity map iteration is proposed. The pixel blocks are mapped from the perspective of local details, and the iterative segmentation is repeated. Secondly, using low-level features such as cell color and shape, a self-supervised heatmap-aware cytoplasm segmentation method is proposed to obtain the activation map of the cytoplasm from the perspective of global attention. The two methods are fused to determine cytoplasmic regions, and combined with nuclear locations, high-quality pseudo-labels are generated. These pseudo-labels are used to train the model cyclically, and the loss strategy is used to encourage the model to discover new object masks, thereby obtaining a segmentation model with better performance. Experimental results show that this method achieves good results in cytoplasm segmentation. On the three datasets of ISBI, MS_CellSeg, and Cx22, 54.32%, 44.64%, and 66.52% AJI were obtained, respectively, which is better than other typical unsupervised methods selected in this article.

A calibration-informed deep learning model for three-dimensional particle reconstruction of volumetric particle image velocimetry

Accurately reconstructing three-dimensional particle fields is essential in fluid velocity measurement research. This study addresses the limitations of current three-dimensional (3D) particle reconstruction methods, such as computational efficiency, precision at high particle density, and particle morphology issues, by introducing a calibration-informed deep learning model named the calibrated pixel to voxel convolutional neural network (CPV-CNN) for 3D Particle Reconstruction. This innovative neural network framework employs a unique Gaussian attention mechanism that bridges pixels and voxels, enabling the mapping of pixel features from two-dimensional (2D) particle images to 3D voxel features. This approach eliminates the need for an initial particle field for particle reconstruction, while significantly enhancing reconstruction efficiency. Additionally, the neural network incorporates camera calibration parameters and the physical coordinates of the reconstructed domain, thereby improving the model's generalization capability and flexibility. Numerical experiments demonstrate that CPV-CNN delivers superior results in terms of accuracy, generalization, and robustness in 3D particle reconstruction. The reconstructed particles exhibit favorable morphology, without the elongation issues commonly observed with conventional methods. This achievement illustrates a practical particle reconstruction algorithm based on artificial intelligence (AI) techniques and represents an important step toward developing an end-to-end AI-based particle reconstruction method in the future.

A Low-Measurement-Cost-Based Multi-Strategy Hyperspectral Image Classification Scheme.

The cost of hyperspectral image (HSI) classification primarily stems from the annotation of image pixels. In real-world classification scenarios, the measurement and annotation process is both time-consuming and labor-intensive. Therefore, reducing the number of labeled pixels while maintaining classification accuracy is a key research focus in HSI classification. This paper introduces a multi-strategy triple network classifier (MSTNC) to address the issue of limited labeled data in HSI classification by improving learning strategies. First, we use the contrast learning strategy to design a lightweight triple network classifier (TNC) with low sample dependence. Due to the construction of triple sample pairs, the number of labeled samples can be increased, which is beneficial for extracting intra-class and inter-class features of pixels. Second, an active learning strategy is used to label the most valuable pixels, improving the quality of the labeled data. To address the difficulty of sampling effectively under extremely limited labeling budgets, we propose a new feature-mixed active learning (FMAL) method to query valuable samples. Fine-tuning is then used to help the MSTNC learn a more comprehensive feature distribution, reducing the model's dependence on accuracy when querying samples. Therefore, the sample quality is improved. Finally, we propose an innovative dual-threshold pseudo-active learning (DSPAL) strategy, filtering out pseudo-label samples with both high confidence and uncertainty. Extending the training set without increasing the labeling cost further improves the classification accuracy of the model. Extensive experiments are conducted on three benchmark HSI datasets. Across various labeling ratios, the MSTNC outperforms several state-of-the-art methods. In particular, under extreme small-sample conditions (five samples per class), the overall accuracy reaches 82.97% (IP), 87.94% (PU), and 86.57% (WHU).

Prototype-Based Semantic Segmentation.

Deep learning based semantic segmentation solutions have yielded compelling results over the preceding decade. They encompass diverse network architectures (FCN based or attention based), along with various mask decoding schemes (parametric softmax based or pixel-query based). Despite the divergence, they can be grouped within a unified framework by interpreting the softmax weights or query vectors as learnable class prototypes. In light of this prototype view, we reveal inherent limitations within the parametric segmentation regime, and accordingly develop a nonparametric alternative based on non-learnable prototypes. In contrast to previous approaches that entail the learning of a single weight/query vector per class in a fully parametric manner, our approach represents each class as a set of non-learnable prototypes, relying solely upon the mean features of training pixels within that class. The pixel-wise prediction is thus achieved by nonparametric nearest prototype retrieving. This allows our model to directly shape the pixel embedding space by optimizing the arrangement between embedded pixels and anchored prototypes. It is able to accommodate an arbitrary number of classes with a constant number of learnable parameters. Through empirical evaluation with FCN based and Transformer based segmentation models (i.e., HRNet, Swin, SegFormer, Mask2Former) and backbones (i.e., ResNet, HRNet, Swin, MiT), our nonparametric framework shows superior performance on standard segmentation datasets (i.e., ADE20 K, Cityscapes, COCO-Stuff), as well as in large-vocabulary semantic segmentation scenarios. We expect that this study will provoke a rethink of the current de facto semantic segmentation model design.

A novel data-driven state evaluation approach for photovoltaic arrays in uncertain shading scenarios

Accurately determining the operating state of photovoltaic (PV) power-generation equipment and providing accurate support for maintenance management requires evaluation of the states of PV arrays under partial shading conditions. Results of such evaluation enable the optimization of maintenance measures to reduce maintenance costs and extend the lifespan of PV modules, thereby increasing the overall revenue of the PV power station. Therefore, this study proposes a novel state-evaluation approach based on current-voltage (I-V) curves under partial shading conditions. Based on the size of the pixel values in this curve, the proposed novel state-evaluation was performed using the proportion of the values of the normal and shaded curves under the same environmental conditions. Specifically, the feature variables (open-circuit voltage, short-circuit current) and pixel values within the I-V curve were analyzed and calculated using a sensitivity algorithm and computer vision algorithms (Canny edge-detection algorithm and Green's theorem). Based on the mapping relationship between feature variables and pixels, an improved Gaussian-process regression method with a combined kernel function (dot product and Matérn function) was proposed to predict the pixel values under the corresponding normal curve. This combined kernel function effectively captured causal relationships and handled outliers in the test dataset to improve the robustness of the model. Finally, the ratios of the pixel values were calculated using the predicted pixels within the normal curve and the calculated pixels within the shading curve. Experiments demonstrated that the root mean square error (RMSE), mean absolute error (MAE), R squared (R2) and log likelihood of the improved Gaussian-process regression method reached 0.011, 0.0086, 0.9996, and 24.7096, respectively.

Early detection of thermal image based T1 breast cancer using enhanced multiwavelet denoised convolution neural network with region based analysis

Medical Thermography image is used for the detection of breast cancer at an earlier stage. Thermography image shows the temperature change in the body due to cells. The metabolic rate of cancer cells is high compared to normal cells. A high metabolic rate increases the blood flow in cancer cells. High blood leads to changes in body temperature. The change in body temperature is used for cancer cell detection at an earlier stage. However, T1-stage cancer cells are smaller and have small temperature differences undetected with thermography. In this paper, a T1-stage cancer cell is heated by an external source; then, thermal images are acquired for earlier detection of small-size cancer cells. External heat source amplifies T1 stage cancer cell temperature. Amplified cancer cell images are analyzed using the proposed Multiwavelet-Deep Denoised Convolutional Neural Network (MWTDnCNN) algorithm for T1 cancer cell detection. Amplified T1 stage cancer cell has higher thermal conductivity (k) and heat capacity (Cp), which helps to detect T1 cancer cell tissue due to the enhanced pixel feature. The proposed MWTDnCNN algorithm has a T1-stage cancer cell detection accuracy of about 98% compared with traditional algorithms. The proposed MWTDnCNN algorithm detects T1-stage cancer of size 1.29 mm.

Enhancing Myocardial Infarction Diagnosis: LSTM-based Deep Learning Approach Integrating Echocardiographic Wall Motion Analysis

PurposeOwing to the increased mortality of heart diseases worldwide, especially myocardial infarction (MI), early detection is essential for improved diagnosis and treatment. The main purpose of this study is to develop a myocardial infarction detection method that combines deep learning and image processing, focusing on abnormalities in left ventricular (LV) wall motion.MethodsThe proposed method primarily uses the LV wall motion movement as a feature to train an LSTM network for MI detection. LV wall motion annotated by expert cardiologists was used as the ground truth. Accuracy, sensitivity, specificity, and area under the curve (AUC) were used to evaluate model performance. The proposed method primarily uses LV wall motion as a feature, combined with LV size and image pixels, to improve diagnostic accuracy over existing computer-aided design (CAD) systems.ResultsThe LSTM model achieved the highest diagnostic performance when trained on a combination of LV wall motion, LV size, and image pixel features with an accuracy of 95%, sensitivity of 96%, specificity of 94%, and an AUC value of 0.98. The LSTM model significantly outperformed models trained on individual feature sets or conventional machine learning algorithms. The inclusion of LV wall motion analysis improved accuracy by 10% compared to using only LV size and pixel data.ConclusionOur MI diagnosis system uses echocardiographic image analysis and LSTM-based deep learning to accurately detect LV wall motion issues related to MI. Compared with current CAD systems, the inclusion of LV wall motion analysis significantly improves diagnosis accuracy. The proposed system can help physicians detect MI early, thereby accelerating treatment and improving patient outcomes.

Vehicle Ego-Trajectory Segmentation Using Guidance Cues

Computer vision has significantly influenced recent advancements in autonomous driving by providing cutting-edge solutions for various challenges, including object detection, semantic segmentation, and comprehensive scene understanding. One specific challenge is ego-vehicle trajectory segmentation, which involves learning the vehicle’s path and describing it with a segmentation map. This can play an important role in both autonomous driving and advanced driver assistance systems, as it enhances the accuracy of perceiving and forecasting the vehicle’s movements across different driving scenarios. In this work, we propose a deep learning approach for ego-trajectory segmentation that leverages a state-of-the-art segmentation network augmented with guidance cues provided through various merging mechanisms. These mechanisms are designed to direct the vehicle’s path as intended, utilizing training data obtained with a self-supervised approach. Our results demonstrate the feasibility of using self-supervised labels for ego-trajectory segmentation and embedding directional intentions within the network’s decisions through image and guidance input concatenation, feature concatenation, or cross-attention between pixel features and various types of guidance cues. We also analyze the effectiveness of our approach in constraining the segmentation outputs and prove that our proposed improvements bring major boosts in the segmentation metrics, increasing IoU by more than 12% and 5% compared with our two baseline models. This work paves the way for further exploration into ego-trajectory segmentation methods aimed at better predicting the behavior of autonomous vehicles.

High-Precision Calibration Method and Error Analysis of Infrared Binocular Target Ranging Systems

Infrared binocular cameras, leveraging their distinct thermal imaging capabilities, are well-suited for visual measurement and 3D reconstruction in challenging environments. The precision of camera calibration is essential for leveraging the full potential of these infrared cameras. To overcome the limitations of traditional calibration techniques, a novel method for calibrating infrared binocular cameras is introduced. By creating a virtual target plane that closely mimics the geometry of the real target plane, the method refines the feature point coordinates, leading to enhanced precision in infrared camera calibration. The virtual target plane is obtained by inverse projecting the centers of the imaging ellipses, which are estimated at sub-pixel edge, into three-dimensional space, and then optimized using the RANSAC least squares method. Subsequently, the imaging ellipses are inversely projected onto the virtual target plane, where its centers are identified. The corresponding world coordinates of the feature points are then refined through a linear optimization process. These coordinates are reprojected onto the imaging plane, yielding optimized pixel feature points. The calibration procedure is iteratively performed to determine the ultimate set of calibration parameters. The method has been validated through experiments, demonstrating an average reprojection error of less than 0.02 pixels and a significant 24.5% improvement in calibration accuracy over traditional methods. Furthermore, a comprehensive analysis has been conducted to identify the primary sources of calibration error. Ultimately, this achieves an error rate of less than 5% in infrared stereo ranging within a 55-m range.

Unsupervised domain adaptive building semantic segmentation network by edge-enhanced contrastive learning

Unsupervised domain adaptation (UDA) is a weakly supervised learning technique that classifies images in the target domain when the source domain has labeled samples, and the target domain has unlabeled samples. Due to the complexity of imaging conditions and the content of remote sensing images, the use of UDA to accurately extract artificial features such as buildings from high-spatial-resolution (HSR) imagery is still challenging. In this study, we propose a new UDA method for building extraction, the contrastive domain adaptation network (CDANet), by utilizing adversarial learning and contrastive learning techniques. CDANet consists of a single multitask generator and dual discriminators. The generator employs a region and edge dual-branch structure that strengthens its edge extraction ability and is beneficial for the extraction of small and densely distributed buildings. The dual discriminators receive the region and edge prediction outputs and achieve multilevel adversarial learning. During adversarial training processing, CDANet aligns the cross-domain of similar pixel features in the embedding space by constructing the regional pixelwise contrastive loss. A self-training (ST) strategy based on pseudolabel generation is further utilized to address the target intradomain discrepancy. Comprehensive experiments are conducted to validate CDANet on three publicly accessible datasets, namely the WHU, Austin, and Massachusetts. Ablation experiments show that the generator network structure, contrastive loss and ST strategy all improve the building extraction accuracy. Method comparisons validate that CDANet achieves superior performance to several state-of-the-art methods, including AdaptSegNet, AdvEnt, IntraDA, FDANet and ADRS, in terms of F1 score and mIoU.

Identification of microalgal particles using pixel feature analysis of Mueller matrix images

Fine sorting of aquatic particles is of great significance for water environment monitoring. Natural water contains aquatic particles that exhibit a high degree of diversity and complexity, and the identification of aquatic particles remains a persistent challenge in the field. In this article, we propose a new technique for identifying the target species of microalgal particles by using the pixel feature analysis of Mueller matrix images. This technique is independent of any prior knowledge or data about the existing particles in the environment, which is advantageous when applied to real-world situations. The pixel-level polarimetric features are fully leveraged to construct polarization feature templates, which can be used to characterize and filter specific microalgal particles in complex environments. This method could enable the accurate detection of harmful algal blooms species in natural water, which can facilitate early warning of algal blooms. The preliminary results show that the recall rate reached 97.2%, and the average accuracy is 98.9%, which demonstrate the effectiveness of this approach for identifying the target species of aquatic particles in natural water.

A Dual-Branch Fusion of a Graph Convolutional Network and a Convolutional Neural Network for Hyperspectral Image Classification.

Semi-supervised graph convolutional networks (SSGCNs) have been proven to be effective in hyperspectral image classification (HSIC). However, limited training data and spectral uncertainty restrict the classification performance, and the computational demands of a graph convolution network (GCN) present challenges for real-time applications. To overcome these issues, a dual-branch fusion of a GCN and convolutional neural network (DFGCN) is proposed for HSIC tasks. The GCN branch uses an adaptive multi-scale superpixel segmentation method to build fusion adjacency matrices at various scales, which improves the graph convolution efficiency and node representations. Additionally, a spectral feature enhancement module (SFEM) enhances the transmission of crucial channel information between the two graph convolutions. Meanwhile, the CNN branch uses a convolutional network with an attention mechanism to focus on detailed features of local areas. By combining the multi-scale superpixel features from the GCN branch and the local pixel features from the CNN branch, this method leverages complementary features to fully learn rich spatial-spectral information. Our experimental results demonstrate that the proposed method outperforms existing advanced approaches in terms of classification efficiency and accuracy across three benchmark data sets.

Prototype as query for few shot semantic segmentation

Few-shot Semantic Segmentation (FSS) was proposed to segment unseen classes in a query image, referring to only a few annotated examples named support images. One of the characteristics of FSS is spatial inconsistency between query and support targets, e.g., texture or appearance. This greatly challenges the generalization ability of methods for FSS, which requires to effectively exploit the dependency of the query image and the support examples. Most existing methods abstracted support features into prototype vectors and implemented the interaction with query features using cosine similarity or feature concatenation. However, this simple interaction may not capture spatial details in query features. To address this limitation, some methods utilized pixel-level support information by computing pixel-level correlations between paired query and support features implemented with the attention mechanism of Transformer. Nevertheless, these approaches suffer from heavy computation due to dot-product attention between all pixels of support and query features. In this paper, we propose a novel framework, termed ProtoFormer, built upon the Transformer architecture, to fully capture spatial details in query features. ProtoFormer treats the abstracted prototype of the target class in support features as the Query and the query features as Key and Value embeddings, which are input to the Transformer decoder. This approach enables better capture of spatial details and focuses on the semantic features of the target class in the query image. The output of the Transformer-based module can be interpreted as semantic-aware dynamic kernels that filter the segmentation mask from the enriched query features. Extensive experiments conducted on PASCAL-5i\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$5^{i}$$\\end{document} and COCO-20i\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$20^{i}$$\\end{document} datasets demonstrate that ProtoFormer significantly outperforms the state-of-the-art methods in FSS.

SA-ConvNeXt: A Hybrid Approach for Flower Image Classification Using Selective Attention Mechanism

In response to the current lack of annotations for flower images and insufficient focus on key image features in traditional fine-grained flower image classification based on deep learning, this study proposes the SA-ConvNeXt flower image classification model. Initially, in the image preprocessing stage, a padding algorithm was used to prevent image deformation and loss of detail caused by scaling. Subsequently, the model was integrated using multi-level feature extraction within the Efficient Channel Attention (ECA) mechanism, forming an M-ECA structure to capture channel features at different levels; a pixel attention mechanism was also introduced to filter out irrelevant or noisy information in the images. Following this, a parameter-free attention module (SimAM) was introduced after deep convolution in the ConvNeXt Block to reweight the input features. SANet, which combines M-ECA and pixel attention mechanisms, was employed at the end of the module to further enhance the model’s dynamic extraction capability of channel and pixel features. Considering the model’s generalization capability, transfer learning was utilized to migrate the pretrained weights of ConvNeXt on the ImageNet dataset to the SA-ConvNeXt model. During training, the Focal Loss function and the Adam optimizer were used to address sample imbalance and reduce gradient fluctuations, thereby enhancing training stability. Finally, the Grad-CAM++ technique was used to generate heatmaps of classification predictions, facilitating the visualization of effective features and deepening the understanding of the model’s focus areas. Comparative experiments were conducted on the Oxford Flowers102 flower image dataset. Compared to existing flower image classification technologies, SA-ConvNeXt performed excellently, achieving a high accuracy of 96.7% and a recall rate of 98.2%, with improvements of 4.0% and 3.7%, respectively, compared to the original ConvNeXt. The results demonstrate that SA-ConvNeXt can effectively capture more accurate key features of flower images, providing an effective technical means for flower recognition and classification.

Analysis of Land-Use/Cover-Type Extraction Results of Tamarix Shrub Forest of China Based on Remote Sensing Technology

The endmember spectrum method can improve image classification quality based on the spectral features of pure pixels in remote sensing images. The CART (Classification and Regression Tree) is a powerful machine learning algorithm that can also be used for remote sensing image classification. In this study, the Tamarix chinensis forest in the Changyi National Marine Ecological Special Reserve in Shandong Province was taken as the research object, and the endmember spectrum method and the CART decision tree method were used to compare and analyze the results of land-use/cover-type classification extraction. In the extraction process, the land use/cover types of the Tamarix forest in the study area were first divided into forested land types such as high-density forest land, medium-density forest land, and low-density forest land, as well as non-forested land types such as water bodies, roads, dams, buildings, and bare soil. Through analysis, the following conclusions could be drawn: while the overall forest cover of the Tamarix forest is high, there is still some room for further afforestation and ecological restoration in the protection area; from the results of land-use/cover extraction results based on the endmember spectrum method in the study area, it can be seen that this method has better results when extracting well-grown forested land, such as high-density Tamarix chinensis forests and medium-density Tamarix chinensis forests, and poorer results when extracting non-forested land, such as low-density tamarisk forests, roads, buildings, dams, and water bodies; from the results of land use/cover extraction based on a CART decision tree in the study area, it can be seen that this method is more effective when extracting non-forested land, such as roads, buildings, dams, and water bodies, but less effective when extracting forested land, such as high-density Tamarix chinensis forests, medium-density Tamarix chinensis forests, and low-density Tamarix chinensis forests. The relevant research results and conclusions of this study can provide some reference for the classification and extraction of large-scale shrub forest cover types based on remote sensing images.

Advancements and Challenges in Hand Gesture Recognition: A Comprehensive Review

Hand gesture recognition is a quickly developing field with many uses in human-computer interaction, sign language recognition, virtual reality, gaming, and robotics. This paper reviews different ways to model hands, such as vision-based, sensor-based, and data glove-based techniques. It emphasizes the importance of accurate hand modeling and feature extraction for capturing and analyzing gestures. Key features like motion, depth, color, shape, and pixel values and their relevance in gesture recognition are discussed. Challenges faced in hand gesture recognition include lighting variations, complex backgrounds, noise, and real-time performance. Machine learning algorithms are used to classify and recognize gestures based on extracted features. The paper emphasizes the need for further research and advancements to improve hand gesture recognition systems' robustness, accuracy, and usability. This review offers valuable insights into the current state of hand gesture recognition, its applications, and its potential to revolutionize human-computer interaction and enable natural and intuitive interactions between humans and machines. In simpler terms, hand gesture recognition is a way for computers to understand what people are saying with their hands. It has many potential applications, such as allowing people to control computers without touching them or helping people with disabilities communicate. The paper reviews different ways to develop hand gesture recognition systems and discusses the challenges and opportunities in this area.

Pulmonary PET /CT image instance segmentation based on dense interactive feature fusion Mask RCNN

There are some problems in positron emission tomography/ computed tomography (PET/CT) lung images, such as little information of feature pixels in lesion regions, complex and diverse shapes, and blurred boundaries between lesions and surrounding tissues, which lead to inadequate extraction of tumor lesion features by the model. To solve the above problems, this paper proposes a dense interactive feature fusion Mask RCNN (DIF-Mask RCNN) model. Firstly, a feature extraction network with cross-scale backbone and auxiliary structures was designed to extract the features of lesions at different scales. Then, a dense interactive feature enhancement network was designed to enhance the lesion detail information in the deep feature map by interactively fusing the shallowest lesion features with neighboring features and current features in the form of dense connections. Finally, a dense interactive feature fusion feature pyramid network (FPN) network was constructed, and the shallow information was added to the deep features one by one in the bottom-up path with dense connections to further enhance the model's perception of weak features in the lesion region. The ablation and comparison experiments were conducted on the clinical PET/CT lung image dataset. The results showed that the APdet, APseg, APdet_s and APseg_s indexes of the proposed model were 67.16%, 68.12%, 34.97% and 37.68%, respectively. Compared with Mask RCNN (ResNet50), APdet and APseg indexes increased by 7.11% and 5.14%, respectively. DIF-Mask RCNN model can effectively detect and segment tumor lesions. It provides important reference value and evaluation basis for computer-aided diagnosis of lung cancer.

Bidirectional Feature Fusion and Enhanced Alignment based Multimodal Semantic Segmentation for Remote Sensing Images

Image–text multimodal deep semantic segmentation leverages the fusion and alignment of image and text information and provides more prior knowledge for segmentation tasks. It is worth exploring image–text multimodal semantic segmentation for remote sensing images. In this paper, we propose a bidirectional feature fusion and enhanced alignment-based multimodal semantic segmentation model (BEMSeg) for remote sensing images. Specifically, BEMSeg first extracts image and text features by image and text encoders, respectively, and then the features are provided for fusion and alignment to obtain complementary multimodal feature representation. Secondly, a bidirectional feature fusion module is proposed, which employs self-attention and cross-attention to adaptively fuse image and text features of different modalities, thus reducing the differences between multimodal features. For multimodal feature alignment, the similarity between the image pixel features and text features is computed to obtain a pixel–text score map. Thirdly, we propose a category-based pixel-level contrastive learning on the score map to reduce the differences among the same category’s pixels and increase the differences among the different categories’ pixels, thereby enhancing the alignment effect. Additionally, a positive and negative sample selection strategy based on different images is explored during contrastive learning. Averaging pixel values across different training images for each category to set positive and negative samples compares global pixel information while also limiting sample quantity and reducing computational costs. Finally, the fused image features and aligned pixel–text score map are concatenated and fed into the decoder to predict the segmentation results. Experimental results on the ISPRS Potsdam, Vaihingen, and LoveDA datasets demonstrate that BEMSeg is superior to comparison methods on the Potsdam and Vaihingen datasets, with improvements in mIoU ranging from 0.57% to 5.59% and 0.48% to 6.15%, and compared with Transformer-based methods, BEMSeg also performs competitively on LoveDA dataset with improvements in mIoU ranging from 0.37% to 7.14%.