Cross-domain self-supervised contrastive learning with multi-scale feature fusion for bearing fault diagnosis under limited labels

Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the schema and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used datasets for images, videos, audios, and 3D data, as well as the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning.

Self-supervised reinforcement learning-based energy management for a hybrid electric vehicle

Spatiotemporal self-supervised predictive learning for atmospheric variable prediction via multi-group multi-attention

CLA: A self-supervised contrastive learning method for leaf disease identification with domain adaptation

Deep learning (DL) finds rich applications in the wireless domain to improve spectrum awareness. Typically, DL models are either randomly initialized following a statistical distribution or pretrained on tasks from other domains in the form of transfer learning without accounting for the unique characteristics of wireless signals. Self-supervised learning (SSL) enables the learning of useful representations from Radio Frequency (RF) signals themselves even when only limited training data samples with labels are available. We present a self-supervised RF signal representation learning method and apply it to the automatic modulation recognition (AMR) task by specifically formulating a set of transformations to capture the wireless signal characteristics. We show that the sample efficiency (the number of labeled samples needed to achieve a certain performance) of AMR can be significantly increased (almost an order of magnitude) by learning signal representations with SSL. This translates to substantial time and cost savings. Furthermore, SSL increases the model accuracy compared to the state-of-the-art DL methods and maintains high accuracy when limited training data is available.

CS-CO: A Hybrid Self-Supervised Visual Representation Learning Method for H&E-stained Histopathological Images.

The rise of self-supervised learning (SSL) methods in recent years presents an opportunity to leverage unlabeled and domain-specific datasets generated by image-based plant phenotyping platforms to accelerate plant breeding programs. Despite the surge of research on SSL, there has been a scarcity of research exploring the applications of SSL to image-based plant phenotyping tasks, particularly detection and counting tasks. We address this gap by benchmarking the performance of 2 SSL methods—momentum contrast (MoCo) v2 and dense contrastive learning (DenseCL)—against the conventional supervised learning method when transferring learned representations to 4 downstream (target) image-based plant phenotyping tasks: wheat head detection, plant instance detection, wheat spikelet counting, and leaf counting. We studied the effects of the domain of the pretraining (source) dataset on the downstream performance and the influence of redundancy in the pretraining dataset on the quality of learned representations. We also analyzed the similarity of the internal representations learned via the different pretraining methods. We find that supervised pretraining generally outperforms self-supervised pretraining and show that MoCo v2 and DenseCL learn different high-level representations compared to the supervised method. We also find that using a diverse source dataset in the same domain as or a similar domain to the target dataset maximizes performance in the downstream task. Finally, our results show that SSL methods may be more sensitive to redundancy in the pretraining dataset than the supervised pretraining method. We hope that this benchmark/evaluation study will guide practitioners in developing better SSL methods for image-based plant phenotyping.

Many heads are better than one: A multiscale neural information feature fusion framework for spatial route selections decoding from multichannel neural recordings of pigeons

Data-driven fault diagnosis of wind turbines has gained popularity, and various deep learning models have been developed accordingly with massive amounts of data and achieved an excellent diagnosis performance. However, most existing deep learning models require a similar distribution of both training and testing data, thus the trained model cannot generalize new wind turbines with different data distributions. In addition, there are insufficient fault data in practice, and therefore the cost of training a new model from scratch is extremely high. To solve these problems, a cross-turbine fault diagnosis method based on deep transfer learning is proposed for wind turbines with the available supervisory control and data acquisition (SCADA) data. To better capture the spatial features of SCADA data, a deep multi-scale residual attention convolutional neural network (DMRACNN) is first designed. Then, the distribution differences between the source and target domain data are aligned at feature level. Specifically, we investigate the transfer performance of four different domain adaptation metrics. We evaluate our proposed method using SCADA data from two wind turbines to compare the diagnostic performance of four basic networks combined with four transfer metrics. Compared with traditional deep learning methods, our proposed DMRACNN achieved significant performance improvements. A cross-validation experiment using two turbines demonstrates the strong generalization ability of the proposed method.

LiM-Net: Lightweight multi-level multiscale network with deep residual learning for automatic liver segmentation in CT images

Deep learning, due to its excellent feature-adaptive capture ability, has been widely utilized in the fault diagnosis field. However, there are two common problems in deep-learning-based fault diagnosis methods: (1) many researchers attempt to deepen the layers of deep learning models for higher diagnostic accuracy, but degradation problems of deep learning models often occur; and (2) the use of multiscale features can easily be ignored, which makes the extracted data features lack diversity. To deal with these problems, a novel multiscale feature fusion deep residual network is proposed in this paper for the fault diagnosis of rolling bearings, one which contains multiple multiscale feature fusion blocks and a multiscale pooling layer. The multiple multiscale feature fusion block is designed to automatically extract the multiscale features from raw signals, and further compress them for higher dimensional feature mapping. The multiscale pooling layer is constructed to fuse the extracted multiscale feature mapping. Two famous rolling bearing datasets are adopted to evaluate the diagnostic performance of the proposed model. The comparison results show that the diagnostic performance of the proposed model is superior to not only several popular models, but also other advanced methods in the literature.

Deep learning in bioinformatics is often limited to problems where extensive amounts of labeled data are available for supervised classification. By exploiting unlabeled data, self-supervised learning techniques can improve the performance of machine learning models in the presence of limited labeled data. Although many self-supervised learning methods have been suggested before, they have failed to exploit the unique characteristics of genomic data. Therefore, we introduce Self-GenomeNet, a self-supervised learning technique that is custom-tailored for genomic data. Self-GenomeNet leverages reverse-complement sequences and effectively learns short- and long-term dependencies by predicting targets of different lengths. Self-GenomeNet performs better than other self-supervised methods in data-scarce genomic tasks and outperforms standard supervised training with ~10 times fewer labeled training data. Furthermore, the learned representations generalize well to new datasets and tasks. These findings suggest that Self-GenomeNet is well suited for large-scale, unlabeled genomic datasets and could substantially improve the performance of genomic models.

Microbially induced corrosion (MIC) of metal surfaces caused by biofilms has wide-ranging consequences. Analysis of biofilm images to understand the distribution of morphological components in images such as microbial cells, MIC byproducts, and metal surfaces non-occluded by cells can provide insights into assessing the performance of coatings and developing new strategies for corrosion prevention. We present an automated approach based on self-supervised deep learning methods to analyze Scanning Electron Microscope (SEM) images and detect cells and MIC byproducts. The proposed approach develops models that can successfully detect cells, MIC byproducts, and non-occluded surface areas in SEM images with a high degree of accuracy using a low volume of data while requiring minimal expert manual effort for annotating images. We develop deep learning network pipelines involving both contrastive (Barlow Twins) and non-contrastive (MoCoV2) self-learning methods and generate models to classify image patches containing three labels-cells, MIC byproducts, and non-occluded surface areas. Our experimental results based on a dataset containing seven grayscale SEM images show that both Barlow Twin and MoCoV2 models outperform the state-of-the-art supervised learning models achieving prediction accuracy increases of approximately 8 and 6%, respectively. The self-supervised pipelines achieved this superior performance by requiring experts to annotate only ~10% of the input data. We also conducted a qualitative assessment of the proposed approach using experts and validated the classification outputs generated by the self-supervised models. This is perhaps the first attempt toward the application of self-supervised learning to classify biofilm image components and our results show that self-supervised learning methods are highly effective for this task while minimizing the expert annotation effort.

A water body is a common object in remote sensing images and high-quality water body extraction is important for some further applications. With the development of deep learning (DL) in recent years, semantic segmentation technology based on deep convolution neural network (DCNN) brings a new way for automatic and high-quality body extraction from remote sensing images. Although several methods have been proposed, there exist two major problems in water body extraction, especially for high resolution remote sensing images. One is that it is difficult to effectively detect both large and small water bodies simultaneously and accurately predict the edge position of water bodies with DCNN-based methods, and the other is that DL methods need a large number of labeled samples which are often insufficient in practical application. In this paper, a novel SFnet-DA network based on the domain adaptation (DA) embedding selective self-attention (SSA) mechanism and multi-scale feature fusion (MFF) module is proposed to deal with these problems. Specially, the SSA mechanism is used to increase or decrease the space detail and semantic information, respectively, in the bottom-up branches of the network by selective feature enhancement, thus it can improve the detection capability of water bodies with drastic scale change and can prevent the prediction from being affected by other factors, such as roads and green algae. Furthermore, the MFF module is used to accurately acquire edge information by changing the number of the channel of advanced feature branches with a unique fusion method. To skip the labeling work, SFnet-DA reduces the difference in feature distribution between labeled and unlabeled datasets by building an adversarial relationship between the feature extractor and the domain classifier, so that the trained parameters of the labeled datasets can be directly used to predict the unlabeled images. Experimental results demonstrate that the proposed SFnet-DA has better performance on water body segmentation than state-of-the-art methods.

Deep learning methods have attracted much attention in the field of polarimetric synthetic aperture radar (PolSAR) image classification over the past few years. However, for supervised learning based methods, it is quite difficult to get large amounts of high-quality and labeled PolSAR data in real applications. In addition, there is a problem of poor generalization for the method of specific supervision labels. To solve the above issue, we explore how to learn representations from unlabeled data from a new perspective. In this letter, a self-supervised PolSAR representation learning (SSPRL) method is proposed. Different from supervised learning based methods, SSPRL aims to learn PolSAR image representations via unsupervised learning approach. Specifically, a self-supervised learning (SSL) method without negative samples is explored and a positive sample generation approach and a novel encoder architecture designed for PolSAR images are proposed. Moreover, mixup is implemented as a regularization strategy. Further the convolutional encoder is utilized to transfer the feature representation from the unlabeled PolSAR data to the downstream task, that is, to achieve the few-shot PolSAR classification. Comparative experimental results on two widely-used PolSAR benchmark datasets verify the effectiveness of proposed method and demonstrate that SSPRL produces impressive performance on few-shot classification task compared with state-of-the-art algorithms.