Fine-tuning Stage Research Articles

A Bi-Party Engaged Modeling Framework for Renewable Power Predictions With Privacy-Preserving

This paper presents a pioneering study in developing data-driven models for predicting the future renewable power out-put sequence via using numerical weather predictions of multiple sites without breaching the data privacy. A novel bi-party engaged data-driven modeling framework (BEDMF) is developed to enable efficiently learning local and global latent features serving as de-centralized data for data-driven modeling with privacy-preserv-ing. The BEDMF contains two stages, the pretraining stage and fine-tuning. At the pretraining stage of the BEDMF, local latent features are learned via local models and then aggregated to pro-duce the global latent feature via a global model. At the fine-tuning stage, local latent features are learned using local data and global latent feature from the previous iteration. The proposed frame-work enables capturing spatial-temporal patterns among multiple sites to further benefit modeling in renewable power prediction tasks. Meanwhile, the framework preserves the data privacy via isolating data locally in the clients. To verify the advantage of the BEDMF, a comprehensive computational study is conducted to benchmark it against famous baselines. Results show that the BEDMF achieve at least 3% improvements on average.

Two-stage framework with improved U-Net based on self-supervised contrastive learning for pavement crack segmentation

After the deep learning method emerged, the automated detection technology of pavement crack images has significantly progressed. The dominant approach is supervised deep learning, which relies on large-scale labeled ground truth. However, the problems are mostly unlabeled original crack images, which are difficult to fully utilize by the supervised deep learning network model. As a representative method of self-supervised learning, contrast learning can learn feature representations from unlabeled data, thus improving the accuracy of downstream tasks. This paper proposes a two-stage framework with improved U-Net based on self-supervised contrastive learning for pavement crack image segmentation. The framework takes improved U-Net as the basic architecture to highlight the significant features of the target segment of fine cracks. U-Net is improved by integrating the residual structure and attention mechanism in the typical U-Net architecture. The framework includes two learning stages: pre-training and fine-tuning. In the pre-training stage, the potential feature representation is learned from the unlabeled crack image. Crack images and pavement background images are used in the training data so that the model learns the distinguishable mapping relationship between crack and its background in the high-dimensional vector space without supervision comparison. In the fine-tuning stage, the network loads the parameters after the pre-training and uses the labeled training data for the retraining. Experimental results show that the proposed two-stage framework significantly improves the performance of crack segmentation accuracy without increasing the number of existing training samples and their labeling.

Multi-scale Transformer Network with Edge-aware Pre-training for Cross-Modality MR Image Synthesis.

Cross-modality magnetic resonance (MR) image synthesis can be used to generate missing modalities from given ones. Existing (supervised learning) methods often require a large number of paired multi-modal data to train an effective synthesis model. However, it is often challenging to obtain sufficient paired data for supervised training. In reality, we often have a small number of paired data while a large number of unpaired data. To take advantage of both paired and unpaired data, in this paper, we propose a Multi-scale Transformer Network (MT-Net) with edge-aware pre-training for cross-modality MR image synthesis. Specifically, an Edge-preserving Masked AutoEncoder (Edge-MAE) is first pre-trained in a self-supervised manner to simultaneously perform 1) image imputation for randomly masked patches in each image and 2) whole edge map estimation, which effectively learns both contextual and structural information. Besides, a novel patch-wise loss is proposed to enhance the performance of Edge-MAE by treating different masked patches differently according to the difficulties of their respective imputations. Based on this proposed pre-training, in the subsequent fine-tuning stage, a Dual-scale Selective Fusion (DSF) module is designed (in our MT-Net) to synthesize missing-modality images by integrating multi-scale features extracted from the encoder of the pre-trained Edge-MAE. Furthermore, this pre-trained encoder is also employed to extract high-level features from the synthesized image and corresponding ground-truth image, which are required to be similar (consistent) in the training. Experimental results show that our MT-Net achieves comparable performance to the competing methods even using 70% of all available paired data. Our code will be released at https://github.com/lyhkevin/MT-Net.

Anti-Rounding Image Steganography With Separable Fine-Tuned Network

Image steganographic methods based on encoder-decoder model with end-to-end network architecture recently have been proposed. However, in steganographic applications, the feature map (called stego matrix) generated by the encoder needs to be rounded as a real stego image for the receiver. The loss of precision by rounding stego matrix leads to the decline in the accuracy of extracted secret messages. The challenge of using end-to-end network to preserve robustness against rounding operation is that it is non-differentiable. In this paper, we propose an anti-rounding image steganography method with separable fine-tuning network architecture which includes the joint training stage (JT-stage) and the separable fine-tuning stage (SF-stage). Firstly, in JT-stage, an embedded generator and a stego matrix extractor are jointly learned without rounding operation. Utilizing concatenation in embedded generator can realistically fuse cover image and secret messages. And the multi-scale fusion block and residual dense block in stego matrix extractor can make secret messages more correctly decoded. Moreover, the discriminator is constructed by generative adversarial nets (GAN) in JT-stage to effectively improve the authenticity and steganalysis security. Then, in SF-stage, the embedded generator is frozen, and the stego matrix is obtained and rounded as a stego image. A stego image extractor is constructed by fine-tuning the layers of the stego matrix extractor to improve the accuracy of message extraction. As the loss will not backpropagate in the embedded generator, the non-differentiability of rounding operation can be offset. Experiments show that the proposed separation fine-tuning network is robust to rounding operation, and effectively reduces the degradation of the image quality and steganalysis performance.

Coarse-to-fine prior-guided attention network for multi-structure segmentation on dental panoramic radiographs

Objective. Accurate segmentation of various anatomical structures from dental panoramic radiographs is essential for the diagnosis and treatment planning of various diseases in digital dentistry. In this paper, we propose a novel deep learning-based method for accurate and fully automatic segmentation of the maxillary sinus, mandibular condyle, mandibular nerve, alveolar bone and teeth on panoramic radiographs. Approach. A two-stage coarse-to-fine prior-guided segmentation framework is proposed to segment multiple structures on dental panoramic radiographs. In the coarse stage, a multi-label segmentation network is used to generate the coarse segmentation mask, and in the fine-tuning stage, a prior-guided attention network with an encoder-decoder architecture is proposed to precisely predict the mask of each anatomical structure. First, a prior-guided edge fusion module is incorporated into the network at the input of each convolution level of the encode path to generate edge-enhanced image feature maps. Second, a prior-guided spatial attention module is proposed to guide the network to extract relevant spatial features from foreground regions based on the combination of the prior information and the spatial attention mechanism. Finally, a prior-guided hybrid attention module is integrated at the bottleneck of the network to explore global context from both spatial and category perspectives. Main results. We evaluated the segmentation performance of our method on a testing dataset that contains 150 panoramic radiographs collected from real-world clinical scenarios. The segmentation results indicate that our proposed method achieves more accurate segmentation performance compared with state-of-the-art methods. The average Jaccard scores are 87.91%, 85.25%, 63.94%, 93.46% and 88.96% for the maxillary sinus, mandibular condyle, mandibular nerve, alveolar bone and teeth, respectively. Significance. The proposed method was able to accurately segment multiple structures on panoramic radiographs. This method has the potential to be part of the process of automatic pathology diagnosis from dental panoramic radiographs.

Decomposed deep multi-view subspace clustering with self-labeling supervision

Most deep multi-view subspace clustering (DMVSC) methods usually employ pipeline optimization by learning the self-expression with a deep model first and then applying spectral clustering to group multi-view data. Subsequently, end-to-end DMVSC methods have been proposed by integrating these two steps into a unified optimization framework. However, the pipeline methods may suffer from misaligned clustering accumulation due to the noise or outlying entries, and the end-to-end methods often constitute a relatively complex parameter optimization. In this paper, we propose a novel method named decomposed deep multi-view subspace clustering with self-labeling supervision (D2MVSC) that runs with a decomposed optimization strategy by three-stage training. Specifically, multi-scale features are extracted by autoencoder in the pre-training stage. According to the discriminative contribution of each view, consensus self-expression is learned from these features by adaptive fusion and structure supervision to generate high-quality pseudo-labels in the fine-tuning stage. Finally, the pseudo-labels are used to retrain the model in a self-labeling supervision manner for robust clustering. Exciting, the self-labeling supervision can be used as an add-on module for other DMVSC methods to improve clustering performance. Extensive experiments on six datasets verify the effectiveness and superiority of our method over other state-of-the-art methods.

Mathematical model of the stress-strain state of the protective device of the cabin of a wheeled skidder

A technique for assessing the stress-strain state of ROPS under lateral loading is presented, which allows at the design and fine-tuning stage to evaluate the energy-absorbing properties of ROPS with minimal time, which, in turn, allows us to judge the quality of the protective device being developed and the degree of ensuring safe working conditions for the operator during the operation of the machine being created in the future. The methodology for assessing the stress-strain state of the ROPS cab of a wheeled skidding tractor under lateral loading, taking into account the influence of technological defects, allows us to estimate the maximum length of the defect and the load.

SSL-ProtoNet: Self-supervised Learning Prototypical Networks for few-shot learning

Few-shot learning is seeking to generalize well to unseen tasks with insufficient labeled samples. Existing works have achieved generalization by exploring inter-class discrimination. However, their performance is limited because sample discrimination is neglected. In this work, we propose a metric-based few-shot approach that leverages self-supervised learning, Prototypical networks, and knowledge distillation, referred to as SSL-ProtoNet, to utilize sample discrimination. The proposed SSL-ProtoNet consists of three stages: pre-training stage, fine-tuning stage, and self-distillation stage. In the pre-training stage, self-supervised learning is leveraged to cluster the samples with their augmented variants to enhance the sample discrimination. The learned representation is then served as an initial point for the next stage. In the fine-tuning stage, the model weights transferred from the pre-training stage are fine-tuned to the target few-shot tasks. A self-supervised loss and a few-shot loss are integrated to prevent overfitting during few-shot task adaptation and to maintain the embedding diversity. In the self-distillation stage, the model is arranged in a teacher–student architecture. The teacher model will serve as a guidance in student model training to reduce overfitting and further improve the performance. The experimental results show that the proposed SSL-ProtoNet outshines the state-of-the-art few-shot image classification methods on three benchmark few-shot datasets, namely, miniImageNet, tieredImageNet, and CIFAR-FS. The source code for the proposed method is available at https://github.com/Jityan/sslprotonet.

Enhancing Code Summarization with Graph Embedding and Pre-Trained Model

Code summarization is a task that aims at automatically producing descriptions of source code. Recently many deep-learning-based approaches have been proposed to generate accurate code summaries, among which pre-trained models (PTMs) for programming languages have achieved promising results. It is well known that source code written in programming languages is highly structured and unambiguous. Though previous work pre-trained the model with well-design tasks to learn universal representation from a large scale of data, they have not considered structure information during the fine-tuning stage. To make full use of both the pre-trained programming language model and the structure information of source code, we utilize Flow-Augmented Abstract Syntax Tree (FA-AST) of source code for structure information and propose GraphPLBART — Graph-augmented Programming Language and Bi-directional Auto-Regressive Transformer, which can effectively introduce structure information to a well PTM through a cross attention layer. Compared with the best-performing baselines, GraphPLBART still improves by 3.2%, 7.1%, and 1.2% in terms of BLEU, METEOR, and ROUGE-L, respectively, on Java dataset, and also improves by 4.0%, 6.3%, and 2.1% on Python dataset. Further experiment shows that the structure information from FA-AST has significant benefits for the performance of GraphPLBART. In addition, our meticulous manual evaluation experiment further reinforces the superiority of our proposed approach. This demonstrates its remarkable abstract quality and solidifies its position as a promising solution in the field of code summarization.

Pre-Training Across Different Cities for Next POI Recommendation

The Point-of-Interest (POI) transition behaviors could hold absolute sparsity and relative sparsity very differently for different cities. Hence, it is intuitive to transfer knowledge across cities to alleviate those data sparsity and imbalance problems for next POI recommendation. Recently, pre-training over a large-scale dataset has achieved great success in many relevant fields, like computer vision and natural language processing. By devising various self-supervised objectives, pre-training models can produce more robust representations for downstream tasks. However, it is not trivial to directly adopt such existing pre-training techniques for next POI recommendation, due to the lacking of common semantic objects (users or items) across different cities . Thus in this paper, we tackle such a new research problem of pre-training across different cities for next POI recommendation. Specifically, to overcome the key challenge that different cities do not share any common object, we propose a novel pre-training model named CATUS , by transferring the cat egory-level u niversal tran s ition knowledge over different cities. Firstly, we build two self-supervised objectives in CATUS : next category prediction and next POI prediction , to obtain the universal transition-knowledge across different cities and POIs. Then, we design a category-transition oriented sampler on the data level and an implicit and explicit transfer strategy on the encoder level to enhance this transfer process. At the fine-tuning stage, we propose a distance oriented sampler to better align the POI representations into the local context of each city. Extensive experiments on two large datasets consisting of four cities demonstrate the superiority of our proposed CATUS over the state-of-the-art alternatives. The code and datasets are available at https://github.com/NLPWM-WHU/CATUS.

SONets: Sub-operator learning enhanced neural networks for solving parametric partial differential equations

Complex nonlinear partial differential equations (PDEs) can be decomposed into subsystems comprising certain linear and nonlinear sub-operators. We usually have fundamental solutions to the linear subsystems and have a good knowledge of their uniqueness and regularity. The nonlinear subsystems have more straightforward formulas than the original complex nonlinear PDEs, even with reduced dimensions. Solutions of the complex nonlinear PDEs based on simpler subsystems are investigated in this study. We derive a proper subsystem decomposition of the nonlinear PDEs into linear and nonlinear subsystems to get unique solutions. We propose the sub-operator learning enhanced neural networks (SONets) based on this proper subsystem decomposition to solve nonlinear PDEs. The solutions of nonlinear subsystems with generated force terms are modeled via operator learning. In contrast, only the cumbersome integration and series parts of fundamental solutions of linear subsystems are modeled. The complex nonlinear PDEs are then solved by finding the force terms to minimize the physics-informed loss functions. It is also convenient to generalize SONets for a new PDE parameter, e.g., the diffusion coefficient. Next, we extend the SONets to a meta version (meta-SONets) for parametric PDEs by transfer learning. In meta-SONets, the initial conditions are explicitly encoded via operator learning neural networks (ENNs) to output force terms, followed by the whole SONets to solve PDEs. In the pre-training stage of meta-SONets, parametric PDEs given randomly generated initial conditions are solved by training ENNs to minimize the physics-informed loss functions. At the fine-tuning stage, the trainable parameters of ENNs are fine-tuned to solve the PDE given a new initial condition. Numerical experiments are conducted to solve the Burgers equation, the nonlinear diffusion-reaction system, and the Allen-Cahn equation. The numerical results indicate that SONets with neural network-parameterized force terms obtain solutions with good accuracy. With transfer learning, the computational cost of meta-SONets for solving the nonlinear PDEs with a new initial condition is dramatically reduced.

2V-CBCT: Two-Orthogonal-Projection Based CBCT Reconstruction and Dose Calculation from Real CBCT Projection Data

Not all radiation therapy (RT) treatments/fractions have CBCT acquired, but two orthogonal projections (i.e., KV radiography) are always available. This work demonstrates the feasibility of two-orthogonal-projection-based CBCT (2V-CBCT) reconstruction and dose calculation for RT from real CBCT projection data, which is the first 2V-CBCT feasibility study using real projection data, to the best of our knowledge. 2V-CBCT is a severely ill-posed inverse problem for which we propose a coarse-to-fine learning strategy. First, a 3D deep neural network that can extract and exploit the inter-slice and intra-slice information is adopted to predict the initial 3D volumes. Then, a 2D deep neural network is utilized to fine-tune the initial 3D volumes slice-by-slice. During the fine-tuning stage, a perceptual loss based on multi-frequency features is employed to enhance the image reconstruction. Dose calculation results from both photon and proton RT demonstrate that 2V-CBCT provides comparable accuracy with full-view CBCT based on real projection data. The proposed method was evaluated on real HN data acquired from on-board CBCT scanners rather than the low-resolution simulated data or down-sampled data. Both visual assessment and quantitative analysis demonstrate that the proposed coarse-to-fine learning strategy has the potential to produce satisfactory volumetric images from two orthogonal projections. Furthermore, we assessed the utility of 2V-CBCT in RT. The results show that the dose distribution maps, dose-volume histograms, and dose parameters calculated using 2V-CBCT have comparable accuracy with the counterparts calculated using the corresponding full-view CBCT for both photon and proton RT. In the table, the methods under comparison are pCT (planning CT), FV-CBCT (CBCT reconstructed with full-view projection data), and 2V-CBCT (CBCT reconstructed with two orthogonal projections). A new effective 2V-CBCT reconstruction method is proposed and validated using real CBCT projection data, which can potentially provide comparable dose calculation accuracy for both photon and proton RT.

Sequence pre-training-based graph neural network for predicting lncRNA-miRNA associations.

MicroRNAs (miRNAs) silence genes by binding to messenger RNAs, whereas long non-coding RNAs (lncRNAs) act as competitive endogenous RNAs (ceRNAs) that can relieve miRNA silencing effects and upregulate target gene expression. The ceRNA association between lncRNAs and miRNAs has been a research hotspot due to its medical importance, but it is challenging to verify experimentally. In this paper, we propose a novel deep learning scheme, i.e. sequence pre-training-based graph neural network (SPGNN), that combines pre-training and fine-tuning stages to predict lncRNA-miRNA associations from RNA sequences and the existing interactions represented as a graph. First, we utilize a sequence-to-vector technique to generate pre-trained embeddings based on the sequences of all RNAs during the pre-training stage. In the fine-tuning stage, we use Graph Neural Network to learn node representations from the heterogeneous graph constructed using lncRNA-miRNA association information. We evaluate our proposed scheme SPGNN on our newly collected animal lncRNA-miRNA association dataset and demonstrate that combining the $k$-mer technique and Doc2vec model for pre-training with the Simple Graph Convolution Network for fine-tuning is effective in predicting lncRNA-miRNA associations. Our approach outperforms state-of-the-art baselines across various evaluation metrics. We also conduct an ablation study and hyperparameter analysis to verify the effectiveness of each component and parameter of our scheme. The complete code and dataset are available on GitHub: https://github.com/zixwang/SPGNN.

Saliency-Enhanced Content-Based Image Retrieval for Diagnosis Support in Dermatology Consultation: Reader Study.

Previous research studies have demonstrated that medical content image retrieval can play an important role by assisting dermatologists in skin lesion diagnosis. However, current state-of-the-art approaches have not been adopted in routine consultation, partly due to the lack of interpretability limiting trust by clinical users. This study developed a new image retrieval architecture for polarized or dermoscopic imaging guided by interpretable saliency maps. This approach provides better feature extraction, leading to better quantitative retrieval performance as well as providing interpretability for an eventual real-world implementation. Content-based image retrieval (CBIR) algorithms rely on the comparison of image features embedded by convolutional neural network (CNN) against a labeled data set. Saliency maps are computer vision-interpretable methods that highlight the most relevant regions for the prediction made by a neural network. By introducing a fine-tuning stage that includes saliency maps to guide feature extraction, the accuracy of image retrieval is optimized. We refer to this approach as saliency-enhanced CBIR (SE-CBIR). A reader study was designed at the University Hospital Zurich Dermatology Clinic to evaluate SE-CBIR's retrieval accuracy as well as the impact of the participant's confidence on the diagnosis. SE-CBIR improved the retrieval accuracy by 7% (77% vs 84%) when doing single-lesion retrieval against traditional CBIR. The reader study showed an overall increase in classification accuracy of 22% (62% vs 84%) when the participant is provided with SE-CBIR retrieved images. In addition, the overall confidence in the lesion's diagnosis increased by 24%. Finally, the use of SE-CBIR as a support tool helped the participants reduce the number of nonmelanoma lesions previously diagnosed as melanoma (overdiagnosis) by 53%. SE-CBIR presents better retrieval accuracy compared to traditional CBIR CNN-based approaches. Furthermore, we have shown how these support tools can help dermatologists and residents improve diagnosis accuracy and confidence. Additionally, by introducing interpretable methods, we should expect increased acceptance and use of these tools in routine consultation.

Unsupervised Learning-Based Spectrum Sensing Algorithm with Defending Adversarial Attacks

Although the spectrum sensing algorithms based on deep learning have achieved remarkable detection performance, the sensing performance is easily affected by adversarial attacks due to the fragility of neural networks. Even slight adversarial perturbations lead to a sharp deterioration of the model detection performance. To enhance the defense capability of the spectrum sensing model against such attacks, an unsupervised learning-based spectrum sensing algorithm with defending adversarial attacks (USDAA) is proposed, which is divided into two stages: adversarial pre-training and fine-tuning. In the adversarial pre-training stage, encoders are used to extract the features of adversarial samples and clean samples, respectively, and then decoders are used to reconstruct the samples, and comparison loss and reconstruction loss are designed to optimize the network parameters. It can reduce the dependence of model training on labeled samples and improve the robustness of the model to attack perturbations. In the fine-tuning stage, a small number of adversarial samples are used to fine-tune the pre-trained encoder and classification layer to obtain the spectrum sensing defense model. The experimental results show that the USDAA algorithm is better than the denoising autoencoder and distillation defense algorithm (DAED) against FGSM and PGD adversarial attacks. The number of labeled samples used in USDAA is only 11% of the DAED. When the false alarm probability is 0.1 and the SNR is −10 dB, the detection probability of the USDAA algorithm for the fast gradient sign method (FGSM) and the projected gradient descent (PGD) attack samples with random perturbations is above 88%, while the detection probability of the DAED algorithm for both attack samples is lower than 69%. Additionally, the USDAA algorithm has better robustness to attack with unknown perturbations.

Prompting and Tuning: A Two-Stage Unsupervised Domain Adaptive Person Re-identification Method on Vision Transformer Backbone

This paper explores the Vision Transformer (ViT) backbone for Unsupervised Domain Adaptive (UDA) person Re-Identification (Re-ID). While some recent studies have validated ViT for supervised Re-ID, no study has yet to use ViT for UDA Re-ID. We observe that the ViT structure provides a unique advantage for UDA Re-ID, i.e., it has a prompt (the learnable class token) at its bottom layer, that can be used to efficiently condition the deep model for the underlying domain. To utilize this advantage, we propose a novel two-stage UDA pipeline named Prompting And Tuning (PAT) which consists of a prompt learning stage and a subsequent fine-tuning stage. In the first stage, PAT roughly adapts the model from source to target domain by learning the prompts for two domains, while in the second stage, PAT fine-tunes the entire backbone for further adaption to increase the accuracy. Although these two stages both adopt the pseudo labels for training, we show that they have different data preferences. With these two preferences, prompt learning and fine-tuning integrated well with each other and jointly facilitated a competitive PAT method for UDA Re-ID.

Molecular Descriptors Property Prediction Using Transformer-Based Approach.

In this study, we introduce semi-supervised machine learning models designed to predict molecular properties. Our model employs a two-stage approach, involving pre-training and fine-tuning. Particularly, our model leverages a substantial amount of labeled and unlabeled data consisting of SMILES strings, a text representation system for molecules. During the pre-training stage, our model capitalizes on the Masked Language Model, which is widely used in natural language processing, for learning molecular chemical space representations. During the fine-tuning stage, our model is trained on a smaller labeled dataset to tackle specific downstream tasks, such as classification or regression. Preliminary results indicate that our model demonstrates comparable performance to state-of-the-art models on the chosen downstream tasks from MoleculeNet. Additionally, to reduce the computational overhead, we propose a new approach taking advantage of 3D compound structures for calculating the attention score used in the end-to-end transformer model to predict anti-malaria drug candidates. The results show that using the proposed attention score, our end-to-end model is able to have comparable performance with pre-trained models.

Unsupervised Vehicle Re-Identification Based on Cross-Style Semi-Supervised Pre-Training and Feature Cross-Division

Vehicle Re-Identification (Re-ID) based on Unsupervised Domain Adaptation (UDA) has shown promising performance. However, two main issues still exist: (1) existing methods that use Generative Adversarial Networks (GANs) for domain gap alleviation combine supervised learning with hard labels of the source domain, resulting in a mismatch between style transfer data and hard labels; (2) pseudo label assignment in the fine-tuning stage is solely determined by similarity measures of global features using clustering algorithms, leading to inevitable label noise in generated pseudo labels. To tackle these issues, this paper proposes an unsupervised vehicle re-identification framework based on cross-style semi-supervised pre-training and feature cross-division. The framework consists of two parts: cross-style semi-supervised pre-training (CSP) and feature cross-division (FCD) for model fine-tuning. The CSP module generates style transfer data containing source domain content and target domain style using a style transfer network, and then pre-trains the model in a semi-supervised manner using both source domain and style transfer data. A pseudo-label reassignment strategy is designed to generate soft labels assigned to the style transfer data. The FCD module obtains feature partitions through a novel interactive division to reduce the dependence of pseudo-labels on global features, and the final similarity measurement combines the results of partition features and global features. Experimental results on the VehicleID and VeRi-776 datasets show that the proposed method outperforms existing unsupervised vehicle re-identification methods. Compared with the last best method on each dataset, the method proposed in this paper improves the mAP by 0.63% and the Rank-1 by 0.73% on the three sub-datasets of VehicleID on average, and it improves mAP by 0.9% and Rank-1 by 1% on VeRi-776 dataset.

Self-supervised learning–based underwater acoustical signal classification via mask modeling

The classification of underwater acoustic signals has garnered a great deal of attention in recent years due to its potential applications in military and civilian contexts. While deep neural networks have emerged as the preferred method for this task, the representation of the signals plays a crucial role in determining the performance of the classification. However, the representation of underwater acoustic signals remains an under-explored area. In addition, the annotation of large-scale datasets for the training of deep networks is a challenging and expensive task. To tackle these challenges, we propose a novel self-supervised representation learning method for underwater acoustic signal classification. Our approach consists of two stages: a pretext learning stage using unlabeled data and a downstream fine-tuning stage using a small amount of labeled data. The pretext learning stage involves randomly masking the log Mel spectrogram and reconstructing the masked part using the Swin Transformer architecture. This allows us to learn a general representation of the acoustic signal. Our method achieves a classification accuracy of 80.22% on the DeepShip dataset, outperforming or matching previous competitive methods. Furthermore, our classification method demonstrates good performance in low signal-to-noise ratio or few-shot settings.

Enhancing text representations separately with entity descriptions

Several studies have focused on incorporating language models with entity descriptions to facilitate the model with a better understanding of knowledge. Existing methods usually either integrate descriptions in the pre-training stage by designing description-related tasks, or in the fine-tuning stage by directly appending description strings to the original input, this paper falls into the latter group. We separate entity descriptions from the original text and process them by another lighter module. Specifically, we use the original large model to encode the original input, while the lighter module processes the entity descriptions. We also propose a layer-wise fusion strategy to deeply couple the representations of the input and descriptions. To further improve the fusion of the two representations, we explore two auxiliary tasks: the entity-description enhancement task and the entity contrastive task. Experiments on (Open Entity, FIGER, FewRel, TACRED, SST) datasets yield respective improvements of (0.9, 1.4, 0.6, 0.5, 0.3). Utilizing ChatGPT as the description embedding method holds the potential for even more promising results.