Global Feature Representation Research Articles

Agent-SwinPyramidNet: an enhanced deep learning model with AMTCF-VMD for anomaly detection in oil and gas pipelines

PurposeThe anomaly detection task for oil and gas pipelines based on acoustic signals faces issues such as background noise coverage, lack of effective features, and small sample sizes, resulting in low fault identification accuracy and slow efficiency. The purpose of this paper is to study an accurate and efficient method of pipeline anomaly detection.Design/methodology/approachFirst, to address the impact of background noise on the accuracy of anomaly signals, the adaptive multi-threshold center frequency variational mode decomposition method(AMTCF-VMD) method is used to eliminate strong noise in pipeline signals. Secondly, to address the strong data dependency and loss of local features in the Swin Transformer network, a Hybrid Pyramid ConvNet network with an Agent Attention mechanism is proposed. This compensates for the limitations of CNN’s receptive field and enhances the Swin Transformer’s global contextual feature representation capabilities. Thirdly, to address the sparsity and imbalance of anomaly samples, the SpecAugment and Scaper methods are integrated to enhance the model’s generalization ability.FindingsIn the pipeline anomaly audio and environmental datasets such as ESC-50, the AMTCF-VMD method shows more significant denoising effects compared to wavelet packet decomposition and EMD methods. Additionally, the model achieved 98.7% accuracy on the preprocessed anomaly audio dataset and 99.0% on the ESC-50 dataset.Originality/valueThis paper innovatively proposes and combines the AMTCF-VMD preprocessing method with the Agent-SwinPyramidNet model, addressing noise interference and low accuracy issues in pipeline anomaly detection, and providing strong support for oil and gas pipeline anomaly recognition tasks in high-noise environments.

Wavelet-fusion image super-resolution model with deep learning for downscaling remotely-sensed, multi-band spectral albedo imagery

Generating granular-scale surface albedo data is extremely important for solar photovoltaic site planning and to optimise renewable energy yield of bifacial panel installations. The albedo effect brings about a significant increase in power in bifacial photovoltaic systems, compared to their mono-facial counterparts, since the spectral response of bifacial solar panels correlates with the incident solar radiation wavelength on the back of the panel, to provide additional power generation capacity. Thus, harnessing the albedo data at relatively local scales is critical towards boosting solar power generation and providing greater power density in local electricity grids. This paper develops novel modelling approaches to produce high-resolution spectral albedo imagery across the Visible and Near Infrared (VNIR) bands, using the Wavelet-Fusion super-resolution model (i.e., Wavelet-FusionSR) trained with the Learned Gamma Correction approach by applying satellite image enhancement methodology. The proposed Wavelet-FusionSR model utilises the low-resolution moderate-resolution Imaging Spectroradiometer (MODIS) as well as high-resolution multi-spectral Advanced Space-borne Thermal Emission Reflection Radiometer (ASTER) satellite images, as critical inputs and ground-truth imagery, respectively, in order to perform sensor-to-sensor deep downscaling, without employing any synthetic or low-resolution satellite imagery data pairs. To augment the proposed deep learning algorithm across the decomposed sub-images of low-resolution inputs, we integrate local and global feature representation learning to train the proposed Wavelet-FusionSR model with Cauchy loss functions. In comparison with five competing benchmark models, the proposed Wavelet-FusionSR model demonstrates performance superiority using quantitative image downscaling metrics and visual assessments of the downscaled images for the visible band of solar radiation. The proposed Wavelet-FusionSR model yielded a Mean Square Error (MSE) of 0.00017, Signal-to-noise-ratio (PSNR) of 37.80, Structural Similarity Index (SSIM) of 0.999 and combined loss, MS-SSIMLoss, based on Multi Structural Similarity and Mean Absolute Error of 2.354 for the Visible Band images, and an MSE of 0.0014, PSNR of 28.43, SSIM of 0.999 and MS-SSIMLoss of 7.426 for the NIR spectral bands, demonstrating high efficacy of the proposed Wavelet-FusionSR method. The Wavelet-FusionSR method therefore attains high-resolution spectral albedo imagery outputs.

EEG-Based Seizure Prediction Using Hybrid DenseNet-ViT Network with Attention Fusion.

Epilepsy seizure prediction is vital for enhancing the quality of life for individuals with epilepsy. In this study, we introduce a novel hybrid deep learning architecture, merging DenseNet and Vision Transformer (ViT) with an attention fusion layer for seizure prediction. DenseNet captures hierarchical features and ensures efficient parameter usage, while ViT offers self-attention mechanisms and global feature representation. The attention fusion layer effectively amalgamates features from both networks, guaranteeing the most relevant information is harnessed for seizure prediction. The raw EEG signals were preprocessed using the short-time Fourier transform (STFT) to implement time-frequency analysis and convert EEG signals into time-frequency matrices. Then, they were fed into the proposed hybrid DenseNet-ViT network model to achieve end-to-end seizure prediction. The CHB-MIT dataset, including data from 24 patients, was used for evaluation and the leave-one-out cross-validation method was utilized to evaluate the performance of the proposed model. Our results demonstrate superior performance in seizure prediction, exhibiting high accuracy and low redundancy, which suggests that combining DenseNet, ViT, and the attention mechanism can significantly enhance prediction capabilities and facilitate more precise therapeutic interventions.

Granformer: A granular transformer net with linear complexity

Recently, transformer models have demonstrated excellent performance across various intelligent applications owing to their ability to understand global context through self-attention mechanism. However, the extensively investigated multiplicative-based attention mechanism is inadequate for capturing relationship-based feature representations, as the dot product cannot depict the intricate semantic information between objects. Moreover, it has a great computational burden with a complexity of O(n2), due to the global feature representation capability achieved by calculating the relationship between each token within the entire feature sequence. To solve the current problem, this paper proposes a granular transformer framework with linear complexity, wherein diverse granulation functions can be employed to supersede the prevailing multiplicative relationships, and an innovative linearization methodology in the form of matrix factorization is designed to reduce the computational burden. Relying on the intricate semantics information embedded within granular structures, the capacity for feature extraction is significantly more comprehensive. Then, a novel matrix factorization methodology is developed for the linearity of granulation-based attention, accomplished by implementing separate deformable convolution sampling and using an approximate iterative algorithm based on cubic equations to calculate the Moore–Penrose inverse. The mathematical proof that our method is approximate with the complete granulation-based attention matrix is investigated in detail. Finally, the performance of Granformer, an innovative reconfiguration of plug-and-play transformer block, is evaluated on representative intelligent applications, including 3D point cloud classification, emotion recognition and sentiment analysis, and object detection. The experimental results suggest that our methodologies outperform the state-of-the-art models.

Multiscale Dynamically Parallel Shrinkage Network for Fault Diagnosis of Aviation Hydraulic Pump and Its Generalizable Applications

Aiming to address the multiscale characteristics and noise corruption problems in the vibration signals of aviation hydraulic pumps, this article develops a novel Multiscale Dynamically Parallel Shrinkage Network (MDPSN) to learn complementary and rich fault-related multiscale features, with the ultimate goal of yielding higher diagnostic accuracy. One significant property is the development of a novel dynamically parallel shrinking module (DPSM) that adaptively generates independent soft thresholds for different scales, effectively shrinking noise-related features to zeros. On one hand, DPSM aggregates and interacts with features at all scales to construct a global feature representation containing richer fault-related information, which is served as the foundation for soft thresholding generation, significantly improving the accuracy and rationality of the generated thresholds. On the other hand, DPSM can adaptively generate individual soft threshold for each scale, allowing each scale to use an independent threshold tailored to its own characteristic to eliminate noise-related information. This avoids the issues of over-denoising or under-denoising caused by the uniform application of thresholds across all scales. Finally, the effectiveness of MDPSN is validated by a series of experiment comparisons on an aviation hydraulic pump dataset and two bearing datasets with various types of noise. The experimental results demonstrate that MDPSN achieves superior diagnostic accuracy compared to five other comparison methods.

MobileAmcT: A Lightweight Mobile Automatic Modulation Classification Transformer in Drone Communication Systems

With the rapid advancement of wireless communication technology, automatic modulation classification (AMC) plays a crucial role in drone communication systems, ensuring reliable and efficient communication in various non-cooperative environments. Deep learning technology has demonstrated significant advantages in the field of AMC, effectively and accurately extracting and classifying modulation signal features. However, existing deep learning models often have high computational costs, making them difficult to deploy on resource-constrained drone communication devices. To address this issue, this study proposes a lightweight Mobile Automatic Modulation Classification Transformer (MobileAmcT). This model combines the advantages of lightweight convolutional neural networks and efficient Transformer modules, incorporating the Token and Channel Conv (TCC) module and the EfficientShuffleFormer module to enhance the accuracy and efficiency of the automatic modulation classification task. The TCC module, based on the MetaFormer architecture, integrates lightweight convolution and channel attention mechanisms, significantly improving local feature extraction efficiency. Additionally, the proposed EfficientShuffleFormer innovatively improves the traditional Transformer architecture by adopting Efficient Additive Attention and a novel ShuffleConvMLP feedforward network, effectively enhancing the global feature representation and fusion capabilities of the model. Experimental results on the RadioML2016.10a dataset show that compared to MobileNet-V2 (CNN-based) and MobileViT-XS (ViT-based), MobileAmcT reduces the parameter count by 74% and 65%, respectively, and improves classification accuracy by 1.7% and 1.09% under different SNR conditions, achieving an accuracy of 62.93%. This indicates that MobileAmcT can maintain high classification accuracy while significantly reducing the parameter count and computational complexity, clearly outperforming existing state-of-the-art AMC methods and other lightweight deep learning models.

A duplex transform heterogeneous feature fusion network for road segmentation

Detecting roads in automatic driving environments poses a challenge due to issues such as boundary fuzziness, occlusion, and glare from light. We believe that two factors are instrumental in addressing these challenges and enhancing detection performance: global context dependency and effective feature representation that prioritizes important feature channels. To tackle these issues, we introduce DTRoadseg, a novel duplex Transformer-based heterogeneous feature fusion network designed for road segmentation. DTRoadseg leverages a duplex encoder architecture to extract heterogeneous features from both RGB images and point-cloud depth images. Subsequently, we introduce a multi-source Heterogeneous Feature Reinforcement Block (HFRB) for fusion of the encoded features, comprising a Heterogeneous Feature Fusion Module (HFFM) and a Reinforcement Fusion Module (RFM). The HFFM leverages the self-attention mechanisms of Transformers to achieve effective fusion through token interactions, while the RFM focuses on emphasizing informative features while downplaying less important ones, thereby reinforcing feature fusion. Finally, a Transformer decoder is utilized to produce the final semantic prediction. Furthermore, we employ a boundary loss function to optimize the segmentation structure area, reduce false detection areas, and improve model accuracy. Extensive experiments are carried out on the KITTI road dataset. The results demonstrate that, compared with state-of-the-art methods, DTRoadseg exhibits superior performance, achieving an average accuracy of 97.01%, a Recall of 96.35%, and runs at a speed of 0.09 s per picture.

SOA: Seed point offset attention for indoor 3D object detection in point clouds

Three-dimensional object detection plays a pivotal role in scene understanding and holds significant importance in various indoor perception applications. Traditional methods based on Hough voting are susceptible to interference from background points or neighboring objects when casting votes for the target’s center from each seed point. Moreover, fixed-size set abstraction modules may result in the loss of structural information for large objects. To address these challenges, this paper proposes a three-dimensional object detection model based on seed point offset attention. The objective of this model is to enhance the model’s resilience to voting noise interference and alleviate feature loss for large-scale objects. Specifically, a seed point offset tensor is first defined, and then the offset tensor self-attention network is employed to learn the weights between votes, thereby establishing a correlation between the voting semantic features and the object structural information. Furthermore, an object surface perception module is introduced, which incorporates detailed features of local object surfaces into global feature representations through vote backtracking and surface mapping. Experimental results indicate that the model achieved excellent performance on the ScanNet-V2 (mAP@0.5, 60.3%) and SUN RGB-D (mAP@0.5, 64.0%) datasets, respectively improving by 2.6% (mAP@0.5) and 5.4% (mAP@0.5) compared to VoteNet.

Local-aware global attention network for person re-identification based on body and hand images

Learning representative, robust and discriminative information from images is essential for effective person re-identification (Re-Id). In this paper, we propose a compound approach for end-to-end discriminative deep feature learning for person Re-Id based on both body and hand images. We carefully design the Local-Aware Global Attention Network (LAGA-Net), a multi-branch deep network architecture consisting of one branch for spatial attention, one branch for channel attention, one branch for global feature representations and another branch for local feature representations. The attention branches focus on the relevant features of the image while suppressing the irrelevant backgrounds. The global and local branches intends to capture global context and fine-grained information, respectively. A set of ablation study shows that each component contributes to the increased performance of the LAGA-Net. Extensive evaluations on four popular body-based person Re-Id benchmarks and two publicly available hand datasets demonstrate that our proposed method consistently outperforms existing state-of-the-art methods.

Cross-modal collaborative feature representation via Transformer-based multimodal mixers for RGB-T crowd counting

RGB-T crowd counting aims to utilize the thermal source data to compensate for the insufficient RGB feature representation under low-illumination conditions. However, how to efficiently use complementary information from different modalities is still a challenging issue for the target task. Existing CNN-based RGB-T crowd counting methods focus on the cross-modal feature representation via the local fixed-receptive-field convolutional operations and the transformer-based works mainly utilize the mutual attention, failing to realize the global cross-modal feature representation. To this end, this paper designs a multimodal counting network based on the multimodal transformer mixer to realize cross-modal collaborative feature representation. Firstly, we design a Transformer-based multimodal mixer to fully fuse the features from both modalities. And then we utilize the Transformer-based network with the multi-head self-attention layer replaced by the average pooling layer to build the two-stream backbone to extract rich multi-scale feature information of the two modalities. Meanwhile, the features extracted from the mixers are used to enhance the intermediate features to achieve the preservation of crowd detail information and suppression of background information. Finally, we design a pyramid regression head to aggregate the multi-scale feature maps to regress the density maps. Extensive experiments, as well as ablation studies, are conducted on challenging RGB-T crowd counting benchmark datasets (RGBT-CC and DroneRGBT), and the achieved competitive experimental results demonstrate the effectiveness of the proposed method on cross-modal collaborative feature representation for target task.

Interpretable domain adaptation transformer: a transfer learning method for fault diagnosis of rotating machinery

Domain adaptation-based transfer learning methods have been widely investigated in fault diagnosis of rotating machinery, but their basic convolution or recurrent structure is subject to poor global feature representation ability, which hinders the learning of domain-irrelevant modulation information. In addition, the “black box” nature of deep learning models limits their applications in high risk-sensitive scenarios. In this paper, an interpretable domain adaptation transformer (IDAT) is proposed for the transferable fault diagnosis of rotating machinery. First, a multi-layer domain adaptation transformer framework is proposed, which can capture the global information that is crucial for learning the modulation information of different domains, and meanwhile reduce the feature distribution discrepancy. Second, an ensemble attention weight is applied to enable the transfer learning framework to be interpretable, which is implemented by averaging the integral values of the multi-head attention maps along the key direction. In addition, the raw vibration signals are embedded as the input of the model, which provides an end-to-end fault diagnosis. The proposed IDAT is tested by various cross-condition and cross-machine bearing fault diagnosis tasks, and results confirm the advantages of the method.

Fine grained food image recognition based on swin transformer

Fine-grained food image recognition is an important research direction in the field of computer vision and machine learning. However, fine-grained food image recognition faces huge challenges when dealing with foods that vary greatly in shape but belong to the same category or subcategories of that food. To improve this problem, this paper proposes a deep convolution module for obtaining local enhanced feature representation and combines it with the global feature representation obtained from Swin Transformer for deep residual, to obtain a deeper enhanced feature representation. An end-to-end fine-grained food universal classifier was also proposed, which can more accurately extract effective feature information from enhanced feature representations and achieve accurate recognition. Our approach can accurately handle foods with widely different shapes but belonging to the same category and is expected to help people better manage their diet and improve their health. Our models were trained and verified on the public fine-grained food datasets Foodx-251 and UEC Food-256 respectively, where the accuracy of the method on the validation set is 81.07% and 82.77% respectively. Compared with other state-of-the-art self-supervised methods, the method proposed in this paper exhibits higher accuracy in fine-grained food image recognition tasks.

Multi-scale 3D-CRU for EEG emotion recognition* *This research has been supported by the National Natural Science Foundation of China Joint Fund Key Project (U2003207), National Natural Science Foundation of China (6220100261902064), Natural Science Foundation of Anhui Province (No. 1908085MF209), Key Projects of Natural Science Foundation of Anhui Province Universities (No. KJ2018A0018).

In this paper, we propose a novel multi-scale 3D-CRU model, with the goal of extracting more discriminative emotion feature from EEG signals. By concurrently exploiting the relative electrode locations and different frequency subbands of EEG signals, a three-dimensional feature representation is reconstructed wherein the Delta (δ) frequency pattern is included. We employ a multi-scale approach, termed 3D-CRU, to concurrently extract frequency and spatial features at varying levels of granularity within each time segment. In the proposed 3D-CRU, we introduce a multi-scale 3D Convolutional Neural Network (3D-CNN) to effectively capture discriminative information embedded within the 3D feature representation. To model the temporal dynamics across consecutive time segments, we incorporate a Gated Recurrent Unit (GRU) module to extract temporal representations from the time series of combined frequency-spatial features. Ultimately, the 3D-CRU model yields a global feature representation, encompassing comprehensive information across time, frequency, and spatial domains. Numerous experimental assessments conducted on publicly available DEAP and SEED databases provide empirical evidence supporting the enhanced performance of our proposed model in the domain of emotion recognition. These findings underscore the efficacy of the features extracted by the proposed multi-scale 3D-GRU model, particularly with the incorporation of the Delta (δ) frequency pattern. Specifically, on the DEAP dataset, the accuracy of Valence and Arousal are 93.12% and 94.31%, respectively, while on the SEED dataset, the accuracy is 92.25%.

Innovative integration of multi-scale residual networks and MK-MMD for enhanced feature representation in fault diagnosis

This article proposes a method based on multi-scale expansion of residual neural networks (ResNets) to address challenges in the operation of rotating components, such as bearings and gears, under complex conditions where they are often affected by environmental noise. This interference leads to weaker fault characteristics, making feature selection difficult and increasing the presence of extraneous information features. To tackle this issue, the proposed method first employs a multi-scale feature ResNet to extract features from vibration signals of rotating machinery. The method decomposes the signal into multiple sub-signals of different scales, extracting local features at each scale. It then uses residual connections to combine these local features to obtain a global feature representation. Furthermore, the article introduces a construction of the maximum mean discrepancy (MMD) and minimization of entropy boundaries to adapt to the differences between two domains. The method utilizes multiple kernel functions to calculate distances between data at different scales and combines these distances to obtain a comprehensive measure. By employing the MMD and minimization of entropy boundary approach, the method can more accurately determine whether signals at different scales belong to the same category, thereby improving diagnostic accuracy and robustness. Experimental results demonstrate the effectiveness of the proposed method in unsupervised cross-domain fault diagnosis tasks. Future work will focus on further optimizing the architecture of ResNets, enhancing feature extraction capabilities, and exploring advanced data augmentation methods to further improve the model’s generalization performance.

Transformer-CNN hybrid network for crowd counting

Efficient feature representation is the key to improving crowd counting performance. CNN and Transformer are the two commonly used feature extraction frameworks in the field of crowd counting. CNN excels at hierarchically extracting local features to obtain a multi-scale feature representation of the image, but it struggles with capturing global features. Transformer, on the other hand, could capture global feature representation by utilizing cascaded self-attention to capture remote dependency relationships, but it often overlooks local detail information. Therefore, relying solely on CNN or Transformer for crowd counting has certain limitations. In this paper, we propose the TCHNet crowd counting model by combining the CNN and Transformer frameworks. The model employs the CMT (CNNs Meet Vision Transformers) backbone network as the Feature Extraction Module (FEM) to hierarchically extract local and global features of the crowd using a combination of convolution and self-attention mechanisms. To obtain more comprehensive spatial local information, an improved Progressive Multi-scale Learning Process (PMLP) is introduced into the FEM, guiding the network to learn at different granularity levels. The features from these three different granularity levels are then fed into the Multi-scale Feature Aggregation Module (MFAM) for fusion. Finally, a Multi-Scale Regression Module (MSRM) is designed to handle the multi-scale fused features, resulting in crowd features rich in high-level semantics and low-level detail. Experimental results on five benchmark datasets demonstrate that TCHNet achieves highly competitive performance compared to some popular crowd counting methods.

Efficient Lightweight Image Denoising with Triple Attention Transformer

Transformer has shown outstanding performance on image denoising, but the existing Transformer methods for image denoising are with large model sizes and high computational complexity, which is unfriendly to resource-constrained devices. In this paper, we propose a Lightweight Image Denoising Transformer method (LIDFormer) based on Triple Multi-Dconv Head Transposed Attention (TMDTA) to boost computational efficiency. LIDFormer first implements Discrete Wavelet Transform (DWT), which transforms the input image into a low-frequency space, greatly reducing the computational complexity of image denoising. However, the low-frequency image lacks fine-feature information, which degrades the denoising performance. To handle this problem, we introduce the Complementary Periodic Feature Reusing (CPFR) scheme for aggregating the shallow-layer features and the deep-layer features. Furthermore, TMDTA is proposed to integrate global context along three dimensions, thereby enhancing the ability of global feature representation. Note that our method can be applied as a pipeline for both convolutional neural networks and Transformers. Extensive experiments on several benchmarks demonstrate that the proposed LIDFormer achieves a better trade-off between high performance and low computational complexity on real-world image denoising tasks.

Electronic eye and electronic tongue data fusion combined with a GETNet model for the traceability and detection of Astragalus.

Astragalus is a widely used traditional Chinese medicine material that is easily confused due to its quality, price and other factors derived from different origins. This article describes a novel method for the rapid tracing and detection of Astragalus via the joint application of an electronic tongue (ET) and an electronic eye (EE) combined with a lightweight convoluted neural network (CNN)-transformer model. First, ET and EE systems were employed to measure the taste fingerprints and appearance images, respectively, of different Astragalus samples. Three spectral transform methods - the Markov transition field, short-time Fourier transform and recurrence plot - were utilized to convert the ET signals into 2D spectrograms. Then, the obtained ET spectrograms were fused with the EE image to obtain multimodal information. A lightweight hybrid model, termed GETNet, was designed to achieve pattern recognition for the Astragalus fusion information. The proposed model employed an improved transformer module and an improved Ghost bottleneck as its backbone network, complementarily utilizing the benefits of CNN and transformer architectures for local and global feature representation. Furthermore, the Ghost bottleneck was further optimized using a channel attention technique, which boosted the model's feature extraction effectiveness. The experiments indicate that the proposed data fusion strategy based on ET and EE devices has better recognition accuracy than that attained with independent sensing devices. The proposed method achieved high precision (99.1%) and recall (99.1%) values, providing a novel approach for rapidly identifying the origin of Astragalus, and it holds great promise for applications involving other types of Chinese herbal medicines. © 2024 Society of Chemical Industry.

Abstract 1025: An AI based CT scan classifier for predicting extranodal extension and nodal metastasis of HPV-associated oropharyngeal carcinoma

Abstract Background: Pathologic extranodal extension (ENE) and nodal metastasis (NM) are of prognostic significance in human papillomavirus (HPV)-associated oropharyngeal squamous cell carcinoma (OPSCC). The pretreatment identifications of these two factors through computed tomography (CT) imaging are of clinical importance since they could guide treatment de-escalation protocols and select candidates suitable for surgery. In an effort to enhance pretreatment decision making, we developed and validated an AI model (specifically a multi-layer perceptron (MLP)-based deep learning model) to predict radiologic NM and pathologic ENE in HPV-associated OPSCC patients. Methods: Radiographic CT scans for 290 OPSCC patients treated with definitive chemoradiation were acquired from The Cancer Imaging Archive (TCIA) and divided into training (D1) and validation (D2) cohorts for NM prediction. Radiologic NM for D1 and D2 were reviewed by two radiologists. Only nodes with concordant NM judgments from both radiologists were included for analysis. An additional 46 OPSCC patients treated with surgery at Emory Winship Cancer Institute were collected to form the test cohorts (D3) for ENE prediction. The largest metastatic or normal nodes were annotated for D1 and D2 while all visible nodes with pathology confirmation of ENE from D3 were annotated. We present a novel deep-learning model utilizing U-shaped multi-layer perceptron mixer (MLP-Mixer) to capture both global and local feature representations by spatially aggregating image patch information through a procedure called token-mixing. The model was trained for 20 epochs with learning rate of0.0001 and batch size of 64. The diagnostic performance of the AI model was evaluated on D2 and D3 using accuracy and area under the ROC curve (AUROC). Results: There were 121 NM negative and 130 NM positive nodes in D1, 14 NM negative and 25 NM positive nodes in D2 and 46 ENE negative and 20 ENE positive nodes in D3. On D2 for NM prediction, the AI model achieved overall accuracy of 0.77 and AUROC of 0.78. On D3 for ENE prediction, the AI model achieved overall accuracy of 0.71, AUROC of 0.69, positive predictive value of 0.52 and negative predictive value of 0.86. Conclusions: The new AI–based deep learning model was able to identify HPV-associated OPSCC NM and ENE with satisfactory diagnostic performance and shows promise to assist clinicians in treatment decision-making. Additional, multi-site prospective validation of the approach is warranted. Citation Format: Bolin Song, Kailin Yang, Shaoyan Pan, Liping Li, Qiyun Liu, Jonathan Lee, Sarah Stock, Nabil F. Saba, Mehmet Asim Bilen, Mihir R. Patel, Anant Madabhushi. An AI based CT scan classifier for predicting extranodal extension and nodal metastasis of HPV-associated oropharyngeal carcinoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 1025.

PFD-Net: Pyramid Fourier Deformable Network for medical image segmentation

Medical image segmentation is crucial for accurately locating lesion regions and assisting doctors in diagnosis. However, most existing methods fail to effectively utilize both local details and global semantic information in medical image segmentation, resulting in the inability to effectively capture fine-grained content such as small targets and irregular boundaries. To address this issue, we propose a novel Pyramid Fourier Deformable Network (PFD-Net) for medical image segmentation, which leverages the strengths of CNN and Transformer. The PFD-Net first utilizes PVTv2-based Transformer as the primary encoder to capture global information and further enhances both local and global feature representations with the Fast Fourier Convolution Residual (FFCR) module. Moreover, PFD-Net further proposes the Dilated Deformable Refinement (DDR) module to enhance the model’s capacity to comprehend global semantic structures of shape-diverse targets and their irregular boundaries. Lastly, Cross-Level Fusion Block with deformable convolution (CLFB) is proposed to combine the decoded feature maps from the final Residual Decoder Block (DDR) with local features from the CNN auxiliary encoder branch, improving the network’s ability to perceive targets resembling the surrounding structures. Extensive experiments were conducted on nine publicly medical image datasets for five types of segmentation tasks including polyp, abdominal, cardiac, gland cells and nuclei. The qualitative and quantitative results demonstrate that PFD-Net outperforms existing state-of-the-art methods in various evaluation metrics, and achieves the highest performance of mDice with the value of 0.826 on the most challenging dataset (ETIS), which is 1.8% improvement compared to the previous best-performing HSNet and 3.6% improvement compared to the next-best PVT-CASCADE. Codes are available at https://github.com/ChaorongYang/PFD-Net.

Few-Shot Fine-Grained Image Classification: A Comprehensive Review

Few-shot fine-grained image classification (FSFGIC) methods refer to the classification of images (e.g., birds, flowers, and airplanes) belonging to different subclasses of the same species by a small number of labeled samples. Through feature representation learning, FSFGIC methods can make better use of limited sample information, learn more discriminative feature representations, greatly improve the classification accuracy and generalization ability, and thus achieve better results in FSFGIC tasks. In this paper, starting from the definition of FSFGIC, a taxonomy of feature representation learning for FSFGIC is proposed. According to this taxonomy, we discuss key issues on FSFGIC (including data augmentation, local and/or global deep feature representation learning, class representation learning, and task-specific feature representation learning). In addition, the existing popular datasets, current challenges and future development trends of feature representation learning on FSFGIC are also described.