Image Feature Information Research Articles

AEFF-SSC: an attention-enhanced feature fusion for 3D semantic scene completion

Abstract Three-dimensional (3D) occupancy perception technology aims to enable autonomous vehicles to observe and understand dense 3D environments. Estimating the complete geometry and semantics of a scene solely from visual images is challenging. However, humans can easily conceive the complete form of objects based on partial key information and their own experience. This ability is crucial for recognizing and interpreting the surrounding environment. To equip 3D occupancy perception systems with a similar capability, a 3D semantic scene completion method called AEFF-SSC is proposed. This method deeply explores boundary and multi-scale information in voxels, aiming to reconstruct 3D geometry more accurately. We have specifically designed an attention-enhanced feature fusion module that effectively fuses image feature information from different scales and focuses on feature boundary information, thereby more efficiently extracting voxel features. Additionally, we introduce a semantic segmentation module driven by a 3D attention-UNet network. This module combines a 3D U-Net network with a 3D attention mechanism. Through feature fusion and feature weighting, it aids in restoring 3D spatial information and significantly improves the accuracy of segmentation results. Experimental verification on the SemanticKITTI dataset demonstrates that AEFF-SSC significantly outperforms other existing methods in terms of both geometry and semantics. Specifically, within a 12.8 m × 12.8 m area ahead, our geometric occupancy accuracy has achieved a significant improvement of 71.58%, and at the same time, the semantic segmentation accuracy has also increased remarkably by 54.20%.

Optimizing flotation froth image segmentation via parallel branch network and hybrid loss supervision

Flotation is a crucial technology for fine coal separation, and accurately acquiring bubble size information during the flotation process is essential for monitoring flotation conditions and achieving intelligent control. However, existing semantic segmentation models encountered issues with boundary disconnection when segmenting flotation bubbles, resulting in deviations between the extracted bubble sizes and their true values. To address the aforementioned challenges, a semantic segmentation model was proposed to maintain high-resolution feature maps throughout the network by designing a parallel branch network structure. Additionally, a ConvTranspose module was proposed to preserve the detailed feature information of images while gradually enhancing the resolution of feature maps. In the model training phase, a hybrid loss function combining pixel classification loss with shape similarity loss was proposed to alleviate the sample imbalance problem caused by the substantial difference in the number of pixels between bubble boundaries and the interior of bubbles. Moreover, since traditional semantic segmentation evaluation metrics, such as MIoU, lack a mechanism for measuring bubble boundary continuity and cannot effectively penalize the problem of boundary disconnection, this paper proposed a new evaluation method for assessing the segmentation performance of flotation froth images. To comprehensively evaluate the effectiveness of the proposed method, this paper conducted tests using flotation froth images collected from actual production processes. Compared with existing methods, the segmentation model proposed in this paper exhibited clear superiority in mitigating the problem of bubble boundary disconnection. The prediction error for the number of bubbles was 6.38 %, which is significantly better than other methods.

A chest imaging diagnosis report generation method based on dual-channel transmodal memory network

Medical imaging is critical for clinical diagnosis. In the past few years, medical imaging data has been steadily growing at an annual rate of 30%, while the annual growth rate in the number of radiologists is only 4. 1%. The traditional manual methods of chest imaging reports result in high information load and workload with low relevance of images and texts. Here we propose a chest CT imaging diagnosis report generation network to effectively integrate image and text feature information, improve the relevance of the cross-modal networks and automatically generate imaging diagnosis reports. The network includes three major modules: a novel ResNetII for information extraction, an image–text dual-channel transmodal memory network (DCTMN), and a dual-channel decoder module combined with LSTM gate. We validate the effectiveness of the proposed network on the IU X-ray and the MIMIC-CXR datasets. Our analysis indicates that the proposed method can strengthen the relationship between chest imaging and text information, ultimately achieving the automatic generation of chest imaging diagnosis reports.

A Zero-Watermarking Algorithm Based on Vortex-like Texture Feature Descriptors

For effective copyright protection of digital images, this paper proposes a zero-watermarking algorithm based on local image feature information. The feature matrix of the algorithm is derived from the keypoint set determined by the Speeded-Up Robust Features (SURF) algorithm, and it calculates both the gradient feature descriptors and the vortex-like texture feature (VTF) descriptors of the keypoint set. Unlike traditional texture feature descriptors, the vortex-like texture feature descriptors proposed in this paper contain richer information and exhibit better stability. The advantage of this algorithm lies in its ability to calculate the keypoints of the digital image and provide a stable vector description of the local features of these keypoints, thereby reducing the amount of erroneous information introduced during attacks. Analysis of experimental data shows that the algorithm has good effectiveness, distinguishability, security, and robustness.

Assessment of fish health status for waterless transportation based on image features and deep learning models

In the current aquaculture industry, the detection of physiological stress indicators in fish during waterless transportation is considered to be an effective method to monitor their health status. However, commonly used detection methods are invasive and interfere with the accuracy of detection results. In order to solve this problem, this study takes grouper as an example, combines multi-sensor physiological stress indicators monitoring, and designs a health status assessment and classification model based on image features and deep learning models, which inputs real-time fish image feature information data acquired during low-temperature waterless transportation into the optimised CNN-BiGRU classification model based on whale optimisation algorithm (WOA) proposed in this study, so as to achieve an accurate assessment of the health status of the fish. By inputting the image feature datasets based on time series and classified into the WOA-CNN-BiGRU, BiGRU, and GRU classification models for training, validation, and testing, respectively, the effectiveness of the classification performance of the models was verified by using accuracy, precision, recall and F1-value composite scores as the metrics. It was found that the WOA-CNN-BiGRU model had excellent classification performance, followed by the BiGRU and GRU models, especially in classifying the health status assessment of small size grouper. This study proposes an efficient and inexpensive method for real-time monitoring and accurate assessment of the health status of fish during waterless transportation, and also provides a reference for vigour monitoring and health assessment of other similar aquatic products.

A Complex Background SAR Ship Target Detection Method Based on Fusion Tensor and Cross-Domain Adversarial Learning

Synthetic Aperture Radar (SAR) ship target detection has been extensively researched. However, most methods use the same dataset division for both training and validation. In practical applications, it is often necessary to quickly adapt to new loads, new modes, and new data to detect targets effectively. This presents a cross-domain detection problem that requires further study. This paper proposes a method for detecting SAR ships in complex backgrounds using fusion tensor and cross-domain adversarial learning. The method is designed to address the cross-domain detection problem of SAR ships with large differences between the training and test sets. Specifically, it can be used for the cross-domain detection task from the fully polarised medium-resolution ship dataset (source domain) to the high-resolution single-polarised dataset (target domain). This method proposes a channel fusion module (CFM) based on the YOLOV5s model. The CFM utilises the correlation between polarised channel images during training to enrich the feature information of single-polarised images extracted by the model during inference. This article proposes a module called the cross-domain adversarial learning module (CALM) to reduce overfitting and achieve adaptation between domains. Additionally, this paper introduces the anti-interference head (AIH) which decouples the detection head to reduce the conflict of classification and localisation problems. This improves the anti-interference and generalisation ability in complex backgrounds. This paper conducts cross-domain experiments using the constructed medium-resolution SAR full polarisation dataset (SFPD) as the source domain and the high-resolution single-polarised ship detection dataset (HRSID) as the target domain. Compared to the best-performing YOLOV8s model among typical mainstream models, this model improves precision by 4.9%, recall by 3.3%, AP by 2.4%, and F1 by 3.9%. This verifies the effectiveness of the method and provides a useful reference for improving cross-domain learning and model generalisation capability in the field of target detection.

Multi-Source Image Fusion Based on BEMD and Region Sharpness Guidance Region Overlapping Algorithm

Multi-focal image and multi-modal image fusion technology can fully take advantage of different sensors or different times, retaining the image feature information and improving the image quality. A multi-source image fusion algorithm based on bidimensional empirical mode decomposition (BEMD) and a region sharpness-guided region overlapping algorithm are studied in this article. Firstly, source images are decomposed into multi-layer bidimensional intrinsic mode functions (BIMFs) and residuals from high-frequency layer to low-frequency layer by BEMD. Gaussian bidimensional intrinsic mode functions (GBIMFs) are obtained by applying Gaussian filtering operated on BIMF and calculating the sharpness value of segmented regions using an improved weighted operator based on the Tenengrad function, which is the key to comparison selection and fusion. Then, the GBIMFs and residuals selected by sharpness comparison strategy are fused by the region overlapping method, and the stacked layers are weighted to construct the final fusion image. Finally, based on qualitative evaluation and quantitative evaluation indicators, the proposed algorithm is compared with six typical image fusion algorithms. The comparison results show that the proposed algorithm can effectively capture the feature information of images in different states and reduce the redundant information.

STU3Net: An Improved U‐Net With Swin Transformer Fusion for Thyroid Nodule Segmentation

ABSTRACTThyroid nodules are a common endocrine system disorder for which accurate ultrasound image segmentation is important for evaluation and diagnosis, as well as a critical step in computer‐aided diagnostic systems. However, the accuracy and consistency of segmentation remains a challenging task due to the presence of scattering noise, low contrast and resolution in ultrasound images. Therefore, we propose a deep learning‐based CAD (computer‐aided diagnosis) method, STU3Net in this paper, aiming at automatic segmentation of thyroid nodules. The method employs a modified Swin Transformer combined with a CNN encoder, which is capable of extracting morphological features and edge details of thyroid nodules in ultrasound images. In decoding through the features for image reconstruction, we introduce a modified three‐layer U‐Net network with cross‐layer connectivity to further enhance image reduction. This cross‐layer connectivity enhances the network's capture and representation of the contained image feature information by creating skip connections between different layers and merging the detailed information of the shallow network with the abstract information of the deeper network. Through comparison experiments with current mainstream deep learning methods on the TN3K and BUSI datasets, we validate the superiority of the STU3Net method in thyroid nodule segmentation performance. The experimental results show that STU3Net outperforms most of the mainstream models on the TN3K dataset, with Dice and IoU reaching 0.8368 and 0.7416, respectively, which are significantly better than other methods. The method demonstrates excellent performance on these datasets and provides radiologists with an effective auxiliary tool to accurately detect thyroid nodules in ultrasound images.

Evolving multispectral sensor configurations using genetic programming for estuary health monitoring

ABSTRACT Assessing ecosystem health on a large scale is crucial for a wide range of management and regulatory decisions. Technologies such as hyperspectral imaging allow noninvasive and rapid estimation of key attributes based on observed reflectance. However, these images are high-dimensional and real-world applications require models based on fewer wavelengths. This paper proposes a new wavelength selection and feature extraction method for hyperspectral image analysis based on genetic programming to automatically select key wavelength regions and informative image features. A dataset of hyperspectral images of sediment in the field was collected and paired with ground-truth measurements of the sediment porosity and organic matter content. Two new program structures were proposed to construct feature extraction trees from either the mean reflectance spectra (spectra-based) or full hyperspectral images (image-based). SVR models were constructed to predict attributes based on the extracted features. Various regression models were used to predict the porosity and organic matter content. Full-wavelength models were constructed to reliably predict the organic matter content. The proposed spectra-based genetic programming solutions show competitive results compared to common wavelength selection methods, such as SPA, CARS, and RC. Finally, the best-evolved solution was applied to predict sediment organic matter content across all collected images.

An intelligent analysis method for power flow of bulk power grid based on fusion of indicator system feature and image feature

Abstract At present, the adjustment of power flow non-convergence in large power grid is still mostly based on manual trial and error, which relies heavily on expert experience and is inefficient. Therefore, an intelligent power flow analysis method based on index system and image feature fusion is proposed. Firstly, the power flow index system is constructed according to the actual demand, which can provide data support for the following power flow convergence judgment problems. Secondly, taking into account the constraints of similarity and ternary loss, the image feature information is used to guide the output of the information feature representation and to realize the organic interaction of the two modal features. On this basis, the calculation result of the index is correlated with the actual state of power flow to judge the convergence of power flow. Finally, the proposed method is verified on the CEPRI36 nodes system and a large power grid in China. The simulation results show that the accuracy of power flow convergence judgment is 100% for CEPRI36 nodes system and 99.83% for some domestic large power grid. It can effectively predict the convergence of the power flow before the calculation of the power flow required by the project and can provide the basis and theoretical support for the power flow optimization adjustment based on the indicator data.

Cnn-trans model: A parallel dual-branch network for fundus image classification

The existence of fundus diseases not only endangers people’s vision, but also brings serious economic burden to the society. Fundus images are an objective and standard basis for the diagnosis of fundus diseases. With the continuous advancement of computer science, deep learning methods dominated by convolutional neural networks (CNN) have been widely used in fundus image classification. However, the current CNN-based fundus image classification research still has a lot of room for improvement: CNN cannot effectively avoid the interference of repeated background information and has limited ability to model the whole world. In response to the above findings, this paper proposes the CNN-Trans model. The CNN-Trans model is a parallel dual-branch network, which is the two branches of CNN-LSTM and Vision Transform (ViT). The CNN-LSTM branch uses Xception after transfer learning. As the original feature extractor, LSTM is responsible for dealing with the gradient disappearance problem in neural network iterations before the classification head, and then introduces a new type of lightweight attention mechanism between Xception and LSTM: Coordinate Attention, so as to emphasize the key information related to classification and suppress the less useful repeated background information; while the self-attention mechanism in the ViT branch is not limited by local interactions, it can establish long-distance dependence on the target and extract global features. Finally, the concatenation (Concat) operation is used to fuse the features of the two branches. The local features extracted by the CNN-LSTM branch and the global features extracted by the ViT branch form complementary advantages. After feature fusion, more comprehensive image feature information is sent to the to the classification layer. Finally, after a large number of experimental tests and comparisons, the results show that: the CNN-Trans model achieved an accuracy of 80.68% on the fundus image classification task, and the CNN-Trans model has a classification that is comparable to the state-of-the-art methods. performance..

Intelligent tongue diagnosis model for gastrointestinal diseases based on tongue images

BackgroundCurrently, the diagnosis of gastrointestinal diseases relies heavily on electronic endoscopy, which is not suitable for all individuals and insufficient for dynamic patient monitoring. Therefore, there is an urgent need for a convenient and noninvasive examination method to enhance the diagnosis and monitoring of gastrointestinal diseases. MethodsSince the tongue is a significant reflection of the digestive system and closely associated with gastrointestinal diseases, our study aims to develop a diagnostic framework based on tongue features. We have collected a dataset of 2,167 tongue coating images from 949 consecutive patients, including 905 images from healthy individuals and 1,262 images from patients with various gastrointestinal conditions. Notably, each patient sample may represent distinct gastrointestinal diseases. This study introduces a novel approach to information fusion detection, integrating hand-crafted and auto-encoded features, alongside disease detection employing Squeeze and Excitation (SE) and Slot attention mechanisms. These two branches respectively analyze image feature and spatial position information when utilizing tongue images for diagnosing gastrointestinal diseases. The amalgamation of predictions from both aspects yields the final detection outcome. ResultsFor both healthy and diseased cohorts, the optimal classification metrics were achieved: AUC = 0.886, ACC = 0.849, and TPR = 0.965, indicating satisfactory performance. Notably, our framework demonstrated promising results in detecting five common gastrointestinal diseases: Helicobacter pylori infection, bile reflux, reflux esophagitis, gastric erosion, and duodenal erosion. Ablation experiments underscored the efficacy of mixed features in multi-color space, showcasing an increase in AUC and ACC by 13.48 % and 12.05 % respectively. Furthermore, the attention mechanism, by focusing on the middle region of tongue images, contributed to a 6.12 % increase in AUC and a 6.35 % increase in ACC. ConclusionsIn this study, we successfully develop and validate a diagnosis framework for gastrointestinal diseases based on tongue image features. By utilizing tongue images, we achieve intelligent diagnosis of gastrointestinal diseases, establishing tongue diagnosis as a convenient, cost-effective, and noninvasive method for diagnosing and screening gastrointestinal diseases.

DdeNet: A dual-domain end-to-end network combining Pale-Transformer and Laplacian convolution for sparse view CT reconstruction

Sparse view (SV) computed tomography (CT) is a clinical diagnostic technique aimed at reducing radiation dose to the human body from X-rays. However, SVCT reconstructed using conventional filtered back-projection (FBP) algorithms will produce severe streak artifacts. In recent years, convolutional neural networks (CNNs) have demonstrated considerable success in addressing the inverse problem of SVCT reconstruction. However, the local convolutional operations of CNNs ignores long-range dependencies of contextual feature information, thereby limiting their capability in feature extraction. To address this limitation, we propose a dual-domain end-to-end network (DdeNet) combining Pale-Transformer and Laplacian convolution for SVCT reconstruction. Initially, in the projection domain, a UNet is employed to restore the interpolated sinogram. Subsequently, in the image domain, we introduce Pale-Transformer and Laplacian convolution to design a dual-stream feature fusion network for finely restoring the CT images reconstructed by FBP. The dual-stream feature fusion network comprises an image restoration network (IRN) module and an edge enhancement network (EEN) module. The depthwise separable convolution based modified pale-shaped self-attention mechanism in Pale-Transformer captures global feature information of the CT image in the IRN, while the Laplacian convolution and multi-scale convolution block with a squeeze and excitation mechanism in the EEN further extract and restore edge feature information of the CT image. Finally, the feature information output from IRN and EEN is fused along the channel dimension and then convolved to obtain the final reconstructed CT image. Experimental results demonstrate that DdeNet exhibits superior generalization and reconstruction performance compared to the previous CNN or Transformer-based reconstruction methods.

A novel infrared and visible image fusion algorithm based on global information-enhanced attention network

The fusion of infrared and visible images aims to extract and fuse thermal target information and texture details to the fullest extent possible, enhancing the visual understanding capabilities of images for both humans and computers in complex scenes. However, existing methods have difficulties in preserving the comprehensiveness of source image feature information and enhancing the saliency of image texture information. Therefore, we put forward a novel infrared and visible image fusion algorithm based on global information-enhanced attention network (GIEA). Specifically, we develop an attention-guided Transformer module (AGTM) to make sure the fused images have enough global information. This module combines the convolutional neural network and Transformer to perform adequate feature extraction from shallow to deep layers, and utilize the attention network for multi-level feature-guided learning. Then, we build the contrast enhancement module (CENM), which enhances the feature representation and contrast of the image so that the fused image contains significant texture information. Furthermore, our network is driven to fully preserve the texture and structure details of the source images with a loss function that consists of content loss and total variance loss. Numerous experiments demonstrate that our fusion approach outperforms other fusion approaches in both subjective and objective assessments.

MISP-Fuse: A progressive fusion network guided by Multi-Information supervision

The fusion of infrared and visible images aims to synthesize an image that combines the advantageous characteristics of both image sources. Nevertheless, existing image fusion algorithms often struggle to extract relevant feature information from the source images and find it challenging to balance the significance of different modality information within the source images. This leads to suboptimal fusion results and inadequately serve advanced downstream visual tasks. In order to confront this challenge, a novel image fusion algorithm, denoted as Multiple Information Supervised Progressive Fusion Network (MISP-Fuse), has been proposed. Specifically, MISP-Fuse employs an innovative and robust multi-stage encoder-decoder network called Full Scale Feature Residual network (FSFR) to extract spatial context and detailed feature information corresponding to infrared and visible images. Within this encoder-decoder network, source image feature information is progressively extracted at different stages, which are recombined at multiple scales and distilled for crucial information. Finally, the fusion image is generated through a spatial localization module named as Spatial Localization Network (SLNet). MISP-Fuse incorporates a multi-information supervision mechanism to establish a linkage between different modality information in the infrared and visible source images. It ensures that the resulting fused image not only aligns with human visual perception but also effectively serves advanced downstream visual tasks. The comparative experiments utilizing diverse image fusion benchmark datasets has been conducted. In comparison to other algorithms, MISP-Fuse demonstrated significant enhancements in comprehensive image fusion metrics, including Average Gradient (AG), Sum of Correlated Differences (SCD), and Correlation Coefficient (CC).

Direction and Residual Awareness Curriculum Learning Network for Rain Streaks Removal.

Single-image rain streaks' removal has attracted great attention in recent years. However, due to the highly visual similarity between the rain streaks and the line pattern image edges, the over-smoothing of image edges or residual rain streaks' phenomenon may unexpectedly occur in the deraining results. To overcome this problem, we propose a direction and residual awareness network within the curriculum learning paradigm for the rain streaks' removal. Specifically, we present a statistical analysis of the rain streaks on large-scale real rainy images and figure out that rain streaks in local patches possess principal directionality. This motivates us to design a direction-aware network for rain streaks' modeling, in which the principal directionality property endows us with the discriminative representation ability of better differing rain streaks from image edges. On the other hand, for image modeling, we are motivated by the iterative regularization in classical image processing and unfold it into a novel residual-aware block (RAB) to explicitly model the relationship between the image and the residual. The RAB adaptively learns balance parameters to selectively emphasize informative image features and better suppress the rain streaks. Finally, we formulate the rain streaks' removal problem into the curriculum learning paradigm which progressively learns the directionality of the rain streaks, rain streaks' appearance, and the image layer in a coarse-to-fine, easy-to-hard guidance manner. Solid experiments on extensive simulated and real benchmarks demonstrate the visual and quantitative improvement of the proposed method over the state-of-the-art methods.

Texture Classification Method Based on Local Enhancement and Non-local Median Patterns

Local binary pattern (LBP) serves as a highly powerful method for texture classification. LBP and its variant methods are extensively applied across various domains of image processing.In increasingly complex imaging environments, LBP faces two issues: (1) Loss of detail information during feature extraction. (2) Sensitivity to noise. To mitigate these issues, this paper proposes a method called Local Enhancement and Non-local Median Pattern (LENMP). It consists of two operators: Local Adaptive Contrast Enhancement Pattern (LEP) and Non-local Median Binary Pattern (NMBP). The LEP operator captures the sign and magnitude details of local image characteristics, while the NMBP operator captures the global information of image features. First, the image is processed using the threshold ACE (thACE) algorithm to enhance the contrast of high-frequency information in the image, extracting image contour edge information. Then, median extraction is performed separately on the two operators to capture larger spatial texture detail information. Finally, comparative experiments are conducted on multiple datasets (Outex, CUReT, Brodatz, KTH-TIPS, and Kylberg) with representative texture classification methods. The results indicate that our proposed LENMP texture classification method exhibits good classification performance and remains competitive compared to the latest descriptors.

A novel RCACycleGAN model is proposed for the high-precision reconstruction of sparse TFM images

The sparse total focusing method (TFM) has been shown to enhance the computational efficacy of ultrasound imaging but the image quality of ultrasound regrettably deteriorates with an increase in the sparsity rate of array elements. Deep learning has made remarkable advancements in image processing and cycle-consistent generative adversarial networks (CycleGANs) have been extensively employed to reconstruct diverse image categories. However, due to the incomplete extraction of image feature information by the generator and discriminator in a CycleGAN, high-quality sparse TFM images cannot be directly reconstructed using CycleGANs. There is also a risk of losing crucial feature information related to minor defects. As a result, this paper modifies the generator and discriminator in the CycleGAN to construct a new relativistic discriminator and coordinate attention CycleGAN (RCACycleGAN) model, which enables high-precision reconstruction of sparse TFM images. The addition of the coordinate attention module to the CycleGAN enhances the defective feature representation by fully considering the channel and spatial correlation between regions and using the fusion of spatially perceived feature maps in different directions. It solves the problem of easy loss of defective key feature information. The relativistic discriminator replaces the PatchGAN discriminator in the CycleGAN and evaluates the quality of both real and sparse TFM reconstructed images to ensure a relative image quality evaluation. This process solves the problem of unstable image quality of the sparse TFM reconstructed image. Experimental results demonstrate that RCACycleGAN can stably reconstruct sparse TFM images even in small sample dataset scenarios. The proposed network model reconstructs images with better accuracy, including in terms of structural similarity, defect roundness and area, and has a shorter training time than several existing network models.

Masked autoencoders with generalizable self-distillation for skin lesion segmentation.

In the field of skin lesion image segmentation, accurate identification and partitioning of diseased regions is of vital importance for in-depth analysis of skin cancer. Self-supervised learning, i.e., MAE, has emerged as a potent force in the medical imaging domain, which autonomously learns and extracts latent features from unlabeled data, thereby yielding pre-trained models that greatly assist downstream tasks. To encourage pre-trained models to more comprehensively learn the global structural and local detail information inherent in dermoscopy images, we introduce a Teacher-Student architecture, named TEDMAE, by incorporating a self-distillation mechanism, it learns holistic image feature information to improve the generalizable global knowledge learning of the student MAE model. To make the image features learned by the model suitable for unknown test images, two optimization strategies are, Exterior Conversion Augmentation (EC) utilizes random convolutional kernels and linear interpolation to effectively transform the input image into one with the same shape but altered intensities and textures, while Dynamic Feature Generation (DF) employs a nonlinear attention mechanism for feature merging, enhancing the expressive power of the features, are proposed to enhance the generalizability of global features learned by the teacher model, thereby improving the overall generalization capability of the pre-trained models. Experimental results from the three public skin disease datasets, ISIC2019, ISIC2017, and PH indicate that our proposed TEDMAE method outperforms several similar approaches. Specifically, TEDMAE demonstrated optimal segmentation and generalization performance on the ISIC2017 and PH datasets, with Dice scores reaching 82.1% and 91.2%, respectively. The best Jaccard values were 72.6% and 84.5%, while the optimal HD95% values were 13.0% and 8.9%, respectively.

Swin-cryoEM: Multi-class cryo-electron micrographs single particle mixed detection method.

Cryo-electron micrograph images have various characteristics such as varying sizes, shapes, and distribution densities of individual particles, severe background noise, high levels of impurities, irregular shapes, blurred edges, and similar color to the background. How to demonstrate good adaptability in the field of image vision by picking up single particles from multiple types of cryo-electron micrographs is currently a challenge in the field of cryo-electron micrographs. This paper combines the characteristics of the MixUp hybrid enhancement algorithm, enhances the image feature information in the pre-processing stage, builds a feature perception network based on the channel self-attention mechanism in the forward network of the Swin Transformer model network, achieving adaptive adjustment of self-attention mechanism between different single particles, increasing the network's tolerance to noise, Incorporating PReLU activation function to enhance information exchange between pixel blocks of different single particles, and combining the Cross-Entropy function with the softmax function to construct a classification network based on Swin Transformer suitable for cryo-electron micrograph single particle detection model (Swin-cryoEM), achieving mixed detection of multiple types of single particles. Swin-cryoEM algorithm can better solve the problem of good adaptability in picking single particles of many types of cryo-electron micrographs, improve the accuracy and generalization ability of the single particle picking method, and provide high-quality data support for the three-dimensional reconstruction of a single particle. In this paper, ablation experiments and comparison experiments were designed to evaluate and compare Swin-cryoEM algorithms in detail and comprehensively on multiple datasets. The Average Precision is an important evaluation index of the evaluation model, and the optimal Average Precision reached 95.5% in the training stage Swin-cryoEM, and the single particle picking performance was also superior in the prediction stage. This model inherits the advantages of the Swin Transformer detection model and is superior to mainstream models such as Faster R-CNN and YOLOv5 in terms of the single particle detection capability of cryo-electron micrographs.