Generalization Performance Of Model Research Articles

Coupling physical constraints with machine learning for satellite-derived evapotranspiration of the Tibetan Plateau

More accurate and process-based satellite evapotranspiration (ET) estimation for the Tibetan Plateau (TP)—the Third Pole of the world—have long been of major interest in hydrometeorology. Combining recent advances in satellite-based ET mechanistic algorithms and data-oriented methods allows ET hybrid modeling by coupling physical constraints with machine learning (ML). Specifically, we developed two hybrid models, a surface conductance-based ML model (ML-Gs) and a soil evaporation-based ML model (ML-Es), to estimate regional ET on the TP. These hybrid models have biophysical framework, under which one of the parameters or components is modeled using ML. Hybrid models make ML complementary to the process-based ET framework, which to find an optimal junction between well physical mechanism and high model performance. The daily ET estimates were evaluated at 28 eddy covariance flux tower sites, as well as by comparison with two process-based ET algorithms (a Penman–Monteith-based ET-PM algorithm and a Priestley-Taylor-based ET-PT algorithm) and a data-oriented pure ML method. The hybrid models decreased the root-mean-square-error (RMSE) of two physical algorithms (1.11 mm/day for ET-PM, 1.09 mm/day for ET-PT) to 0.50 mm/day, and increased the Kling-Gupta efficiency (KGE) (0.35 for ET-PM, 0.36 for ET-PT) to 0.92. Our hybrid models also showed improved performance (KGE of 0.65) than pure ML (KGE of 0.62) at data-sparse regions as well as for the responses to extreme weather events. It indicates that our approach does not only boost the ET simulation accuracy, but also improve the physical understanding of ML-based ET estimation. More importantly, ML-Es focuses on the ET components on the TP and is more well-defined than ML-Gs. An innovation of our approach is that for data-sparse regions and extreme cases, the more robust physical mechanism was coupled, the better generalization performance of hybrid model could achieve. The spatiotemporal ET patterns based on our hybrid models were consistent with the variations in local climatic regions and could provide critical information on the understanding of hydrological processes under the global and regional climate changes.

A Robust Strategy to Account for Data Sampling Variability in the Development of Hydrological Models

AbstractIt is typical to use a single portion of the available data to calibrate hydrological models, and the remainder for model evaluation. To minimize model‐bias, this partitioning must be performed so as to ensure distributional representativeness and mutual consistency. However, failure to account for data sampling variability (DSV) in the underlying Data Generating Process can weaken the model's generalization performance. While “K‐fold cross‐validation” can mitigate this problem, it is computationally inefficient since the calibration/evaluation operations must be repeated numerous times. This paper develops a general strategy for stochastic evolutionary parameter optimization (SEPO) that explicitly accounts for DSV when calibrating a model using any population‐based evolutionary optimization algorithm (EOA), such as Shuffled Complex Evolution (SCE). Inspired in part by the machine‐learning strategy of stochastic gradient descent (SGD), we use various representative random sub‐samples to drive the EOA toward the distribution of the model parameters. Unlike in SGD, derivative information is not required and hence SEPO can be applied to any hydrological model where such information is not readily available. To demonstrate the effectiveness of the proposed strategy, we implement it within the well‐known SCE, to calibrate the GR4J conceptual rainfall‐runoff model to 163 hydro‐climatically diverse catchments. Using only a single optimization run, our Stochastic SCE method converges to population‐based estimates of model parameter distributions (and corresponding simulation uncertainties), without compromising model performance during either calibration or evaluation. Further, it effectively reduces the need to perform independent evaluation tests of model performance under conditions that are represented by the available data.

Parametric unsteady flow modeling by using meta learning

Parametric unsteady flow modeling plays a significant role in the fluid dynamics, since the unsteady flow problems are usually involved with complex physical phenomena. Currently, in the study of data-driven unsteady flow modeling, the convolutional neural network based autoencoder (CNN-AE) model has been widely used. While, its application in engineering field has been limited due to challenges that the CNNs are expensive to be trained and the generalization performance of CNN-AE model needs to be improved. Therefore, based on the CNN-AE model, a meta-learning framework is introduced in the present work for parametric unsteady flow modeling. Specifically, both the model-agnostic meta-learning (MAML) and the reptile methods are used to learn the correlations between flow fields under various physical parameters. First, a meta-initialization CNN-AE model is established, with the training data-set under various physical parameters. Then, for a new physical parameter, the meta-initialization CNN-AE model will be finetuned by using few snapshots in a new task with limited epochs. Finally, the proposed CNN-AE based meta-learning framework is validated in two canonical unsteady flow problems with moving boundaries, including oscillating cylinder and flapping airfoil. The results have shown that the CNN-AE based meta-leaning framework can greatly accelerate the adaptation of CNN-AE model in new physical parameters, which can greatly enhance the generalization performance of CNN-AE model.

Analyzing Out-of-Domain Generalization Performance of Pre-Trained Segmentation Models

Artists illustrate objects to various degrees of complexity. As the amount of detail or the similarity to reality of a depiction decreases, the object tends to be reduced to its simplest, most relevant higher-level features (Harrison, 1981). One of the reasons Deep Neural Networks (DNN) may fail to identify objects in an image is that models are unable to recognize the order of importance of features such as shape, depth, or color within an image, which means even the most minute distortions of pixels within an image that would be imperceptible to humans would greatly impact the performance of the object detection models (Eykholt et al., 2018). However, by training DNN on artworks where the most prominent features defining specific objects are emphasized, perhaps a model can be made to be more resilient against small-scale changes in an image. In this paper, the correlation between the level of similarity to reality of images and artworks of an object and the accuracy of object detection models is investigated to test the ability of object detection models in identifying the most salient features of a particular object. The results of this report can help outline the efficacy of models only trained on real images in identifying increasingly abstract artworks that have simplified an object to its most prominent features. The experiment shows that the accuracies of models decrease as the images or illustrations provided become more abstract or simplified, which suggests the higher level features that identify a particular object are different in object detection models and humans.

E-Nose System Based on Fourier Series for Gases Identification and Concentration Estimation From Food Spoilage

This work presents an electronic nose (EN)-based gas identification and concentration estimation method for the detection of food spoilage. The response data of sensors were acquired through a commercial gas sensor array and data acquisition circuit board and transformed into pictures with the form of the Fourier series. A convolutional neural network (CNN) model was used to identify the pictures from the conversion of sensor data, thus achieving the purpose of identifying the gases (C2H5OH, NH3, and H2S). In order to solve the problem of sample imbalance and to improve the generalization performance of classification models, the synthetic minority oversampling technique (SMOTE) and dropout technique were employed. Fivefold cross-validation was used to evaluate the performance of the model, of which the gas identification accuracy rate reached 96.67%. Moreover, a gas concentration regression model with the advantages of simplicity and strong interpretability was further proposed to estimate the concentrations of C2H5OH, NH3, and H2S. The mean absolute errors and coefficients of determination for the concentration estimation of C2H5OH, NH3, and H2S are (3.71 ppm, 0.968), (0.50 ppm, 0.968), and (0.13 ppm, 0.99), respectively. Furthermore, we used our model to evaluate the freshness of kiwifruit, pork, and beef, and it showed satisfactory predictive performance. The method proposed in this work realizes high-precision detection of gases from food spoilage and has a good application prospect in the rapid judgment of food freshness on the EN system.

Multi-source domain generalization and adaptation toward cross-subject myoelectric pattern recognition

Objective. Myoelectric pattern recognition (MPR) has shown satisfactory performance under ideal laboratory conditions. Nevertheless, the individual variances lead to dramatic performance degradation in cross-user MPR applications. It is crucial to enable the myoelectric interface to adapt to multiple users’ surface electromyography (sEMG) distributions in practical. Approach. Domain adaptation (DA) is a promising approach to tackle cross-user challenges due to its ability to diminish the divergence between individual users’ EMG distributions and escalate model generalization performance. However, existing DA methods in sEMG control are based on single-source domain adaptation (SDA). SDA solely mixes multiple training users’ data as a combined source domain and attempts to align with a novel user. This simple data mixing manner ignores the sEMG distribution variations between disparate training users, leading to an insufficient variance elimination and lower performance. To this end, this paper proposes a multi-source synchronize domain adaptation framework with both DA and domain generalization (DG) capability. This multi-source framework aligns each source user and the new user in individual feature spaces, which better transfers the knowledge of existing users to the new user. Moreover, we retain the source-combined data to preserve the effectiveness of SDA. The property was further confirmed by evaluating the performance of the proposed method on data from nine subjects performing six tasks. Main results. Experiment results prove that the proposed multi-source framework achieved both positive DG and DA performance in a cross-user classification manner. Significance. This work demonstrates the usability and feasibility of the proposed multi-source framework in cross-user myoelectric control.

Disambiguation-based partial label feature selection via feature dependency and label consistency

Partial label learning refers to the issue that each training sample corresponds to a candidate label set containing only one valid label. Feature selection can be viewed as an effective pre-processing technology to improve the generalization performance of learning models, while the partial label feature selection task is challenging due to ambiguous labeling information. To this end, this paper utilizes granular ball computing and neighborhood rough sets to put forward a disambiguation-based partial label feature selection algorithm via feature dependency and label consistency. Firstly, the proposed algorithm performs an adaptive neighbor aggregation based on granular balls to disambiguate candidate labels. Adaptive neighbors are flexible to make more efficient learning performance. Then, considering the labeling confidence induced by disambiguation, the significance of each feature is evaluated by fusing the feature dependency from the neighborhood granularity and the label consistency among the nearest neighbors. Neighborhood rough sets directly handle continuous features that can reduce the adverse impact of data discretization on feature selection. The label consistency among adjacent samples is important for measuring feature discrimination. The fusion of feature dependency and label consistency can facilitate performance improvement. Finally, experiments conducted on eight controlled UCI and five real-world partial label datasets demonstrate that the proposed algorithm can improve the generalization performance of partial label learning and achieve superior performance compared to the partial label feature selection methods.

Classification of Complicated Urban Forest Acoustic Scenes with Deep Learning Models

The use of passive acoustic monitoring (PAM) can compensate for the shortcomings of traditional survey methods on spatial and temporal scales and achieve all-weather and wide-scale assessment and prediction of environmental dynamics. Assessing the impact of human activities on biodiversity by analyzing the characteristics of acoustic scenes in the environment is a frontier hotspot in urban forestry. However, with the accumulation of monitoring data, the selection and parameter setting of the deep learning model greatly affect the content and efficiency of sound scene classification. This study compared and evaluated the performance of different deep learning models for acoustic scene classification based on the recorded sound data from Guangzhou urban forest. There are seven categories of acoustic scenes for classification: human sound, insect sound, bird sound, bird–human sound, insect–human sound, bird–insect sound, and silence. A dataset containing seven acoustic scenes was constructed, with 1000 samples for each scene. The requirements of the deep learning models on the training data volume and training epochs in the acoustic scene classification were evaluated through several sets of comparison experiments, and it was found that the models were able to achieve satisfactory accuracy when the training sample data volume for a single category was 600 and the training epochs were 100. To evaluate the generalization performance of different models to new data, a small test dataset was constructed, and multiple trained models were used to make predictions on the test dataset. All experimental results showed that the DenseNet_BC_34 model performs best among the comparison models, with an overall accuracy of 93.81% for the seven acoustic scenes on the validation dataset. This study provides practical experience for the application of deep learning techniques in urban sound monitoring and provides new perspectives and technical support for further exploring the relationship between human activities and biodiversity.

New Insights into Stroke from Continuous Passively Collected Temperature and Sleep Data Using Wrist-Worn Wearables.

Actigraphy may provide new insights into clinical outcomes and symptom management of patients through passive, continuous data collection. We used the GENEActiv smartwatch to passively collect actigraphy, wrist temperature, and ambient light data from 27 participants after stroke or probable brain transient ischemic attack (TIA) over 42 periods of device wear. We computed 323 features using established algorithms and proposed 25 novel features to characterize sleep and temperature. We investigated statistical associations between the extracted features and clinical outcomes evaluated using clinically validated questionnaires to gain insight into post-stroke recovery. We subsequently fitted logistic regression models to replicate clinical diagnosis (stroke or TIA) and disability due to stroke. The model generalization performance was assessed using a leave-one-subject-out cross validation method with the selected feature subsets, reporting the area under the curve (AUC). We found that several novel features were strongly correlated (|r|>0.3) with stroke symptoms and mental health measures. Using selected novel features, we obtained an AUC of 0.766 to estimate diagnosis and an AUC of 0.749 to estimate whether disability due to stroke was present. Collectively, these findings suggest that features extracted from the temperature smartwatch sensor may reveal additional clinically useful information over and above existing actigraphy-based features.

The segmentation effect of style transfer on fetal head ultrasound image: a study of multi-source data.

The generalization ability of the fetal head segmentation method is reduced due to the data obtained by different machines, settings, and operations. To keep the generalization ability, we proposed a Fourier domain adaptation (FDA) method based on amplitude and phase to achieve better multi-source ultrasound data segmentation performance. Given the source/target image, the Fourier domain information was first obtained using fast Fourier transform. Secondly, the target information was mapped to the source Fourier domain through the phase adjustment parameter α and the amplitude adjustment parameter β. Thirdly, the target image and the preprocessed source image obtained through the inverse discrete Fourier transform were used as the input of the segmentation network. Finally, the dice loss was computed to adjust α and β. In the existing transform methods, the proposed method achieved the best performance. The adaptive-FDA method provides a solution for the automatic preprocessing of multi-source data. Experimental results show that it quantitatively improves the segmentation results and model generalization performance.

Full-Memory Transformer for Image Captioning

The Transformer-based approach represents the state-of-the-art in image captioning. However, existing studies have shown Transformer has a problem that irrelevant tokens with overlapping neighbors incorrectly attend to each other with relatively large attention scores. We believe that this limitation is due to the incompleteness of the Self-Attention Network (SAN) and Feed-Forward Network (FFN). To solve this problem, we present the Full-Memory Transformer method for image captioning. The method improves the performance of both image encoding and language decoding. In the image encoding step, we propose the Full-LN symmetric structure, which enables stable training and better model generalization performance by symmetrically embedding Layer Normalization on both sides of the SAN and FFN. In the language decoding step, we propose the Memory Attention Network (MAN), which extends the traditional attention mechanism to determine the correlation between attention results and input sequences, guiding the model to focus on the words that need to be attended to. Our method is evaluated on the MS COCO dataset and achieves good performance, improving the result in terms of BLEU-4 from 38.4 to 39.3.

Study of MI-BCI classification method based on the Riemannian transform of personalized EEG spatiotemporal features.

Motor imagery (MI) is a traditional paradigm of brain-computer interface (BCI) and can assist users in creating direct connections between their brains and external equipment. The common spatial patterns algorithm is the most popular spatial filtering technique for collecting EEG signal features in MI-based BCI systems. Due to the defect that it only considers the spatial information of EEG signals and is susceptible to noise interference and other issues, its performance is diminished. In this study, we developed a Riemannian transform feature extraction method based on filter bank fusion with a combination of multiple time windows. First, we proposed the multi-time window data segmentation and recombination method by combining it with a filter group to create new data samples. This approach could capture individual differences due to the variation in time-frequency patterns across different participants, thereby improving the model's generalization performance. Second, Riemannian geometry was used for feature extraction from non-Euclidean structured EEG data. Then, considering the non-Gaussian distribution of EEG signals, the neighborhood component analysis (NCA) algorithm was chosen for feature selection. Finally, to meet real-time requirements and a low complexity, we employed a Support Vector Machine (SVM) as the classification algorithm. The proposed model achieved improved accuracy and robustness. In this study, we proposed an algorithm with superior performance on the BCI Competition IV dataset 2a, achieving an accuracy of 89%, a kappa value of 0.73 and an AUC of 0.9, demonstrating advanced capabilities. Furthermore, we analyzed data collected in our laboratory, and the proposed method achieved an accuracy of 77.4%, surpassing other comparative models. This method not only significantly improved the classification accuracy of motor imagery EEG signals but also bore significant implications for applications in the fields of brain-computer interfaces and neural engineering.

Importance-Aware Information Bottleneck Learning Paradigm for Lip Reading

Lip reading is the task of decoding text from speakers' mouth movements. Numerous deep learning-based methods have been proposed to address this task. However, these existing deep lip reading models suffer from poor generalization due to overfitting the training data. To resolve this issue, we present a novel learning paradigm that aims to improve the interpretability and generalization of lip reading models. In specific, a Variational Temporal Mask (VTM) module is customized to automatically analyze the importance of frame-level features. Furthermore, the prediction consistency constraints of global information and local temporal important features are introduced to strengthen the model generalization. We evaluate the novel learning paradigm with multiple lip reading baseline models on the LRW and LRW-1000 datasets. Experiments show that the proposed framework significantly improves the generalization performance and interpretability of lip reading models.

Toward Generalization of Deep Learning-Based Plant Disease Identification Under Controlled and Field Conditions

Identifying corn diseases under field conditions is crucial for implementing effective disease management systems. Deep learning (DL)-based plant disease identification using deep neural networks (DNN) has been successfully implemented in recent years. Recent work suggests DL models trained on lab-acquired image data do not generalize to similar accuracy levels for identifying diseases in the field. Additionally, most studies have not evaluated the generalizability of DL models for identifying plant diseases from various datasets and diverse imaging conditions. This study evaluates how well DL models generalize across different datasets and environmental conditions for identifying plant diseases using five datasets consisting of foliar disease images in corn: namely PlantVillage, PlantDoc, Digipathos, northern leaf blight (NLB) dataset, and a custom acquired CD&S dataset. Multiple DL-based image classification models were trained and evaluated using different dataset combinations. Transfer learning was utilized using five different pre-trained DNN architectures: InceptionV3, ResNet50, VGG16, DesneNet169, and Xception, for conducting four different experiments. After training the models, images for distinct corn diseases from different datasets were used as testing data for evaluating the generalization ability of each DL model. It was observed that the DenseNet169 model performed the best. The highest generalization accuracy of 81.60% was observed when the DenseNet169 model was trained using red, green, blue, and alpha (RGBA) images from CD&S corn disease dataset with removed backgrounds. In addition, 77.50% to 80.33% accuracy was achieved for the PlantVillage dataset when combined with field-acquired images from either the PlantDoc or the CD&S dataset. The results suggest that background removal using RGBA images from CD&S dataset or augmentation of field and lab data improves the generalization performance of DL models and could be used for developing field-deployable disease management systems.

FS-BAN: Born-Again Networks for Domain Generalization Few-Shot Classification

Conventional Few-shot classification (FSC) aims to recognize samples from novel classes given limited labeled data. Recently, domain generalization FSC (DG-FSC) has been proposed with the goal to recognize novel class samples from unseen domains. DG-FSC poses considerable challenges to many models due to the domain shift between base classes (used in training) and novel classes (encountered in evaluation). In this work, we make two novel contributions to tackle DG-FSC. Our first contribution is to propose Born-Again Network (BAN) episodic training and comprehensively investigate its effectiveness for DG-FSC. As a specific form of knowledge distillation, BAN has been shown to achieve improved generalization in conventional supervised classification with a closed-set setup. This improved generalization motivates us to study BAN for DG-FSC, and we show that BAN is promising to address the domain shift encountered in DG-FSC. Building on the encouraging findings, our second (major) contribution is to propose Few-Shot BAN (FS-BAN), a novel BAN approach for DG-FSC. Our proposed FS-BAN includes novel multi-task learning objectives: Mutual Regularization, Mismatched Teacher, and Meta-Control Temperature, each of these is specifically designed to overcome central and unique challenges in DG-FSC, namely overfitting and domain discrepancy. We analyze different design choices of these techniques. We conduct comprehensive quantitative and qualitative analysis and evaluation over six datasets and three baseline models. The results suggest that our proposed FS-BAN consistently improves the generalization performance of baseline models and achieves state-of-the-art accuracy for DG-FSC. Project Page: yunqing-me.github.io/Born-Again-FS/.

ConvSegNet: Automated Polyp Segmentation From Colonoscopy Using Context Feature Refinement With Multiple Convolutional Kernel Sizes

Colorectal cancer occurs in the rectal of humans, and early detection has been proved to reduce its mortality rate. Colonoscopy is the standard used in detecting the presence of polyps in the rectal, and accurate segmentation of the polyps from colonoscopy images often provides helpful information for early diagnosis and treatment. Although existing deep learning models often achieve high segmentation performance when tested on the same dataset used in model training; still, their performance often degrades when applied to out-of-distribution datasets, leading to low model generalization or overfitting. This challenge is often associated with the quality of the features learnt from the input images. In this work, a novel Context Feature Refinement (CFR) module is proposed to address the challenge of low model generalization and segmentation performance. The CFR module is built to extract contextual information from the incoming feature map by using multiple parallel convolutional layers with progressively increasing kernel sizes. Using multiple parallel convolutions with different kernel sizes helped to extract more efficient multi-scale contextual information and thus enabled the network to effectively identify and segment small and fine details, as well as larger and more complex structures in the input images. Extensive experiments on three public benchmark datasets in CVC-ClinicDB, Kvasir-SEG, and BKAI-NeoPolyp showed that the proposed ConvSegNet model achieved jaccard, dice and F2 scores of 0.8650, 0.9177, and 0.9328 on CVC-ClinicDB, 0.7936, 0.8618, and 0.8855 on Kvasir-SEG, and 0.8045, 0.8747 and 0.8909 on BKAI-NeoPolyp datasets respectively. Also, an improved generalization performance was achieved by the ConvSegNet model, compared to the benchmark polyp segmentation models.

The WuC-Adam algorithm based on joint improvement of Warmup and cosine annealing algorithms.

The Adam algorithm is a common choice for optimizing neural network models. However, its application often brings challenges, such as susceptibility to local optima, overfitting and convergence problems caused by unstable learning rate behavior. In this article, we introduce an enhanced Adam optimization algorithm that integrates Warmup and cosine annealing techniques to alleviate these challenges. By integrating preheating technology into traditional Adam algorithms, we systematically improved the learning rate during the initial training phase, effectively avoiding instability issues. In addition, we adopt a dynamic cosine annealing strategy to adaptively adjust the learning rate, improve local optimization problems and enhance the model's generalization ability. To validate the effectiveness of our proposed method, extensive experiments were conducted on various standard datasets and compared with traditional Adam and other optimization methods. Multiple comparative experiments were conducted using multiple optimization algorithms and the improved algorithm proposed in this paper on multiple datasets. On the MNIST, CIFAR10 and CIFAR100 datasets, the improved algorithm proposed in this paper achieved accuracies of 98.87%, 87.67% and 58.88%, respectively, with significant improvements compared to other algorithms. The experimental results clearly indicate that our joint enhancement of the Adam algorithm has resulted in significant improvements in model convergence speed and generalization performance. These promising results emphasize the potential of our enhanced Adam algorithm in a wide range of deep learning tasks.

PaCNN-LSTM: A Localization Scheme Based on Improved Contrastive Learning and Parallel Fusion Neural Network

The deep learning technique plays an important role in Wi-Fi localization systems as it could mine deep features of measurement data. The main challenges are to combat the signal fluctuation resulting in decrease of sample discrimination, and to leverage the broadest information of sample measurements during the training phase, since they are directly related to the location accuracy and robustness. Hence, to address the above issues, this paper proposes an indoor Wi-Fi localization scheme which mainly contains two modules. Firstly, an improved contrastive learning is introduced to handle the sample signal measurements to increase the discrimination. It is from the perspective of learning and encoding, and avoids the drawbacks brought by traditional processing methods. Then, we build a parallel fusion network named as PaCNN-LSTM based on convolutional neural network (CNN) and long short-term memory network (LSTM). Compared with existing networks, PaCNN-LSTM connects neural networks in parallel rather than serial, which improves the generalization performance of model when extracting the spatial and temporal features of signal measurements. In addition, it also considers the large amount of middle layer information that is always ignored. By adding a Flatten layer after the pooling layer, the available information of samples has been broadened. Extensive experimental results show that the localization performance of the proposed scheme is outperformed than others, where the location accuracy is improved by about 22%.

Language-Aware Domain Generalization Network for Cross-Scene Hyperspectral Image Classification

Text information including extensive prior knowledge about land cover classes has been ignored in hyperspectral image classification (HSI) tasks. It is necessary to explore the effectiveness of linguistic mode in assisting HSI classification. In addition, the large-scale pre-training image-text foundation models have demonstrated great performance in a variety of downstream applications, including zero-shot transfer. However, most domain generalization methods have never addressed mining linguistic modal knowledge to improve the generalization performance of model. To compensate for the inadequacies listed above, a Language-aware Domain Generalization Network (LDGnet) is proposed to learn cross-domain invariant representation from cross-domain shared prior knowledge. The proposed method only trains on the source domain (SD) and then transfers the model to the target domain (TD). The dual-stream architecture including image encoder and text encoder is used to extract visual and linguistic features, in which coarse-grained and fine-grained text representations are designed to extract two levels of linguistic features. Furthermore, linguistic features are used as cross-domain shared semantic space, and visual-linguistic alignment is completed by supervised contrastive learning in semantic space. Extensive experiments on three datasets demonstrate the superiority of the proposed method when compared with state-of-the-art techniques.

UNet segmentation network of COVID-19 CT images with multi-scale attention.

In recent years, the global outbreak of COVID-19 has posed an extremely serious life-safety risk to humans, and in order to maximize the diagnostic efficiency of physicians, it is extremely valuable to investigate the methods of lesion segmentation in images of COVID-19. Aiming at the problems of existing deep learning models, such as low segmentation accuracy, poor model generalization performance, large model parameters and difficult deployment, we propose an UNet segmentation network integrating multi-scale attention for COVID-19 CT images. Specifically, the UNet network model is utilized as the base network, and the structure of multi-scale convolutional attention is proposed in the encoder stage to enhance the network's ability to capture multi-scale information. Second, a local channel attention module is proposed to extract spatial information by modeling local relationships to generate channel domain weights, to supplement detailed information about the target region to reduce information redundancy and to enhance important information. Moreover, the network model encoder segment uses the Meta-ACON activation function to avoid the overfitting phenomenon of the model and to improve the model's representational ability. A large number of experimental results on publicly available mixed data sets show that compared with the current mainstream image segmentation algorithms, the pro-posed method can more effectively improve the accuracy and generalization performance of COVID-19 lesions segmentation and provide help for medical diagnosis and analysis.