Two-stage Training Process Research Articles

A generation-storage coordination dispatch strategy for power system based on causal reinforcement learning

In the backdrop of global energy transformation, power systems integrating high proportions of renewable energy sources are facing unprecedented challenges in operational stability and dispatch efficiency. To address these challenges, this study introduces a generation-storage coordination real-time dispatch strategy based on Causal Power System Dynamic Reinforcement Learning (CPSDRL). Diverging from traditional reinforcement learning approaches, CPSDRL innovatively incorporates causal inference within the state prediction model – the crux of model-based reinforcement learning – thereby establishing the Power Causal Dynamic Model (PCDM). Assisted by the prior knowledge of power systems, the model significantly enhances prediction accuracy and reliability through a two-stage training process. Utilizing PCDM, this study further applies a direct policy search algorithm to optimize the real-time dispatch strategy. Experimental results indicate that the proposed method improves the stability of generation-storage coordination real-time dispatch and exhibits competitive advantages in sample efficiency and computational speed, compared to traditional model-based and model-free reinforcement learning algorithms. This method is expected to enhance the practicality and adaptability of causal reinforcement learning techniques in power system scheduling and control.

BarlowTwins-CXR: enhancing chest X-ray abnormality localization in heterogeneous data with cross-domain self-supervised learning.

Chest X-ray imaging based abnormality localization, essential in diagnosing various diseases, faces significant clinical challenges due to complex interpretations and the growing workload of radiologists. While recent advances in deep learning offer promising solutions, there is still a critical issue of domain inconsistency in cross-domain transfer learning, which hampers the efficiency and accuracy of diagnostic processes. This study aims to address the domain inconsistency problem and improve autonomic abnormality localization performance of heterogeneous chest X-ray image analysis, particularly in detecting abnormalities, by developing a self-supervised learning strategy called "BarlwoTwins-CXR". We utilized two publicly available datasets: the NIH Chest X-ray Dataset and the VinDr-CXR. The BarlowTwins-CXR approach was conducted in a two-stage training process. Initially, self-supervised pre-training was performed using an adjusted Barlow Twins algorithm on the NIH dataset with a Resnet50 backbone pre-trained on ImageNet. This was followed by supervised fine-tuning on the VinDr-CXR dataset using Faster R-CNN with Feature Pyramid Network (FPN). The study employed mean Average Precision (mAP) at an Intersection over Union (IoU) of 50% and Area Under the Curve (AUC) for performance evaluation. Our experiments showed a significant improvement in model performance with BarlowTwins-CXR. The approach achieved a 3% increase in mAP50 accuracy compared to traditional ImageNet pre-trained models. In addition, the Ablation CAM method revealed enhanced precision in localizing chest abnormalities. The study involved 112,120 images from the NIH dataset and 18,000 images from the VinDr-CXR dataset, indicating robust training and testing samples. BarlowTwins-CXR significantly enhances the efficiency and accuracy of chest X-ray image-based abnormality localization, outperforming traditional transfer learning methods and effectively overcoming domain inconsistency in cross-domain scenarios. Our experiment results demonstrate the potential of using self-supervised learning to improve the generalizability of models in medical settings with limited amounts of heterogeneous data. This approach can be instrumental in aiding radiologists, particularly in high-workload environments, offering a promising direction for future AI-driven healthcare solutions.

Tackling Vision Language Tasks through Learning Inner Monologues

Visual language tasks such as Visual Question Answering (VQA) or Visual Entailment (VE) require AI models to comprehend and reason with both visual and textual content. Driven by the power of Large Language Models (LLMs), two prominent methods have emerged: (1) the hybrid integration between LLMs and Vision-Language Models (VLMs), where visual inputs are firstly converted into language descriptions by VLMs, serving as inputs for LLMs to generate final answer(s); (2) visual feature alignment in language space, where visual inputs are encoded as embeddings and projected to LLMs' language space via further supervised fine-tuning. The first approach provides light training costs and interpretability but is hard to be optimized in an end-to-end fashion. The second approach presents decent performance, but feature alignment usually requires large amounts of training data and lacks interpretability. To tackle this dilemma, we propose a novel approach, Inner Monologue Multi-Modal Optimization (IMMO), to solve complex vision language problems by simulating Inner Monologue, a cognitive process in which an individual engages in silent verbal communication with themselves. More specifically, we enable LLMs and VLMs to interact through natural language conversation (i.e., Inner Monologue) and propose to use a two-stage training process to learn how to do Inner Monologue (self-asking questions and answering questions). IMMO is evaluated on two popular tasks and achieves competitive performance with less training data when compared with state-of-the-art models while concurrently keeping the interpretability. The results suggest that by emulating the cognitive phenomenon of internal dialogue, our approach can enhance reasoning and explanation abilities, contributing to the more effective fusion of vision and language models. More importantly, instead of using predefined human-crafted monologues, IMMO learns this process within the deep learning models, broadening its potential applications across various AI challenges beyond vision and language tasks.

KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning

Large Language Models (LLMs) have demonstrated impressive performance in natural language processing tasks by leveraging chain of thought (CoT) that enables step-by-step thinking. Extending LLMs with multimodal capabilities is the recent interest, but incurs computational cost and requires substantial hardware resources. To address these challenges, we propose KAM-CoT a framework that integrates CoT reasoning, Knowledge Graphs (KGs), and multiple modalities for a comprehensive understanding of multimodal tasks. KAM-CoT adopts a two-stage training process with KG grounding to generate effective rationales and answers. By incorporating external knowledge from KGs during reasoning, the model gains a deeper contextual understanding reducing hallucinations and enhancing the quality of answers. This knowledge-augmented CoT reasoning empowers the model to handle questions requiring external context, providing more informed answers. Experimental findings show KAM-CoT outperforms the state-of-the-art methods. On the ScienceQA dataset, we achieve an average accuracy of 93.87%, surpassing GPT-3.5 (75.17%) by 18% and GPT-4 (83.99%) by 10%. Remarkably, KAM-CoT achieves these results with only 280M trainable parameters at a time, demonstrating its cost-efficiency and effectiveness.

Task-based transferable deep-learning scatter correction in cone beam computed tomography: a simulation study.

X-ray scatter significantly affects the image quality of cone beam computed tomography (CBCT). Although convolutional neural networks (CNNs) have shown promise in correcting x-ray scatter, their effectiveness is hindered by two main challenges: the necessity for extensive datasets and the uncertainty regarding model generalizability. This study introduces a task-based paradigm to overcome these obstacles, enhancing the application of CNNs in scatter correction. Using a CNN with U-net architecture, the proposed methodology employs a two-stage training process for scatter correction in CBCT scans. Initially, the CNN is pre-trained on approximately 4000 image pairs from geometric phantom projections, then fine-tuned using transfer learning (TL) on 250 image pairs of anthropomorphic projections, enabling task-specific adaptations with minimal data. 2D scatter ratio (SR) maps from projection data were considered as CNN targets, and such maps were used to perform the scatter prediction. The fine-tuning process for specific imaging tasks, like head and neck imaging, involved simulating scans of an anthropomorphic phantom and pre-processing the data for CNN retraining. For the pre-training stage, it was observed that SR predictions were quite accurate (). The accuracy of SR predictions was further improved after TL, with a relatively short retraining time ( times faster than pre-training) and using considerably fewer samples compared to the pre-training dataset ( times smaller). A fast and low-cost methodology to generate task-specific CNN for scatter correction in CBCT was developed. CNN models trained with the proposed methodology were successful to correct x-ray scatter in anthropomorphic structures, unknown to the network, for simulated data.

Enhanced Out-of-Stock Detection in Retail Shelf Images Based on Deep Learning.

The term out-of-stock (OOS) describes a problem that occurs when shoppers come to a store and the product they are seeking is not present on its designated shelf. Missing products generate huge sales losses and may lead to a declining reputation or the loss of loyal customers. In this paper, we propose a novel deep-learning (DL)-based OOS-detection method that utilizes a two-stage training process and a post-processing technique designed for the removal of inaccurate detections. To develop the method, we utilized an OOS detection dataset that contains a commonly used fully empty OOS class and a novel class that represents the frontal OOS. We present a new image augmentation procedure in which some existing OOS instances are enlarged by duplicating and mirroring themselves over nearby products. An object-detection model is first pre-trained using only augmented shelf images and, then, fine-tuned on the original data. During the inference, the detected OOS instances are post-processed based on their aspect ratio. In particular, the detected instances are discarded if their aspect ratio is higher than the maximum or lower than the minimum instance aspect ratio found in the dataset. The experimental results showed that the proposed method outperforms the existing DL-based OOS-detection methods and detects fully empty and frontal OOS instances with 86.3% and 83.7% of the average precision, respectively.

ReFs: A hybrid pre-training paradigm for 3D medical image segmentation

Self-supervised learning (SSL) has achieved remarkable progress in medical image segmentation. The application of an SSL algorithm often follows a two-stage training process: using unlabeled data to perform label-free representation learning and fine-tuning the pre-trained model on the downstream tasks. One issue of this paradigm is that the SSL step is unaware of the downstream task, which may lead to sub-optimal feature representation for a target task. In this paper, we propose a hybrid pre-training paradigm that is driven by both self-supervised and supervised objectives. To achieve this, a supervised reference task is involved in self-supervised learning, aiming to improve the representation quality. Specifically, we employ the off-the-shelf medical image segmentation task as reference, and encourage learning a representation that (1) incurs low prediction loss on both SSL and reference tasks and (2) leads to a similar gradient when updating the feature extractor from either task. In this way, the reference task pilots SSL in the direction beneficial for the downstream segmentation. To this end, we propose a simple but effective gradient matching method to optimize the model towards a consistent direction, thus improving the compatibility of both SSL and supervised reference tasks. We call this hybrid pre-training paradigm reference-guided self-supervised learning (ReFs), and perform it on a large-scale unlabeled dataset and an additional reference dataset. The experimental results demonstrate its effectiveness on seven downstream medical image segmentation benchmarks.

GraphHop: An Enhanced Label Propagation Method for Node Classification.

A scalable semisupervised node classification method on graph-structured data, called GraphHop, is proposed in this work. The graph contains all nodes' attributes and link connections but labels of only a subset of nodes. Graph convolutional networks (GCNs) have provided superior performance in node label classification over the traditional label propagation (LP) methods for this problem. Nevertheless, current GCN algorithms suffer from a considerable amount of labels for training because of high model complexity or cannot be easily generalized to large-scale graphs due to the expensive cost of loading the entire graph and node embeddings. Besides, nonlinearity makes the optimization process a mystery. To this end, an enhanced LP method, called GraphHop, is proposed to tackle these problems. GraphHop can be viewed as a smoothening LP algorithm, in which each propagation alternates between two steps: label aggregation and label update. In the label aggregation step, multihop neighbor embeddings are aggregated to the center node. In the label update step, new embeddings are learned and predicted for each node based on aggregated results from the previous step. The two-step iteration improves the graph signal smoothening capacity. Furthermore, to encode attributes, links, and labels on graphs effectively under one framework, we adopt a two-stage training process, i.e., the initialization stage and the iteration stage. Thus, the smooth attribute information extracted from the initialization stage is consistently imposed in the propagation process in the iteration stage. Experimental results show that GraphHop outperforms state-of-the-art graph learning methods on a wide range of tasks in graphs of various sizes (e.g., multilabel and multiclass classification on citation networks, social graphs, and commodity consumption graphs).

ScMultiGAN: cell-specific imputation for single-cell transcriptomes with multiple deep generative adversarial networks.

The emergence of single-cell RNA sequencing (scRNA-seq) technology has revolutionized the identification of cell types and the study of cellular states at a single-cell level. Despite its significant potential, scRNA-seq data analysis is plagued by the issue of missing values. Many existing imputation methods rely on simplistic data distribution assumptions while ignoring the intrinsic gene expression distribution specific to cells. This work presents a novel deep-learning model, named scMultiGAN, for scRNA-seq imputation, which utilizes multiple collaborative generative adversarial networks (GAN). Unlike traditional GAN-based imputation methods that generate missing values based on random noises, scMultiGAN employs a two-stage training process and utilizes multiple GANs to achieve cell-specific imputation. Experimental results show the efficacy of scMultiGAN in imputation accuracy, cell clustering, differential gene expression analysis and trajectory analysis, significantly outperforming existing state-of-the-art techniques. Additionally, scMultiGAN is scalable to large scRNA-seq datasets and consistently performs well across sequencing platforms. The scMultiGAN code is freely available at https://github.com/Galaxy8172/scMultiGAN.

Keypoint-Based Automated Component Placement Inspection for Printed Circuit Boards

This study aims to develop novel automated computer vision algorithms and systems for component replacement inspection for printed circuit boards (PCBs). The proposed algorithms are able to identify the locations and sizes of different components. They are object detection algorithms based on key points of the target components. The algorithms can be implemented as neural networks consisting of two portions: frontend networks and backend networks. The frontend networks are used for the feature extractions of input images. The backend networks are adopted to produce component inspection results. Each component class can has its own frontend and backend networks. In this way, the neural model for the component class can be effectively reused for different PCBs. To reduce the computation time for the inference of the networks, different component classes can share the same frontend networks. A two-stage training process is proposed to effectively explore features of different components for accurate component inspection. The proposed algorithm has the advantages of simplicity in training for data collection, high accuracy in defect detection, and high reusability and flexibility for online inspection. The algorithm is an effective alternative for automated inspection in smart factories, with growing demand for product quality and diversification.

Adaptive evidential K-NN classification: Integrating neighborhood search and feature weighting

The number of nearest neighbors K and the utilized distance measure considerably impact the performance of the K-nearest neighbor (K-NN) algorithm. The information provided by neighbors may be perplexing due to the set presumption of K for every test sample without any prior knowledge, which might result in incorrect classification results. The hypothesis of locally constant class conditional probabilities relied on by K-NN, is violated by high dimensions, which also bring out the dimensionality curse. Notably, in the case of imperfect data, training samples may be noise-corrupted or located in substantially overlapping regions, thus impairing the effectiveness of the K-NN rule. In contrast to existing research, which frequently focuses on just one of the problems above, this paper presents an adaptive evidential K-NN classification (AEK-NN) algorithm that synchronizes neighborhood search with feature weighting. In order to jointly learn the adaptive neighborhood for each sample and various feature weights, AEK-NN maps the Euclidean space into a rebuilt space consisting of similarity coefficients through a two-stage training process. Within the evidence theory framework, AEK-NN produces classification results for data with imperfect labels. The advantages of applying evidence theory, incorporating adaptive neighborhood search, and feature weighting are shown through ablation studies. We also use simulated and real-world datasets to demonstrate the state-of-the-art performance of AEK-NN.

Respiratory Sound Classification by Applying Deep Neural Network with a Blocking Variable

Respiratory diseases are leading causes of death worldwide, and failure to detect diseases at an early stage can threaten people’s lives. Previous research has pointed out that deep learning and machine learning are valid alternative strategies to detect respiratory diseases without the presence of a doctor. Thus, it is worthwhile to develop an automatic respiratory disease detection system. This paper proposes a deep neural network with a blocking variable, namely Blnet, to classify respiratory sound, which integrates the strength of the ResNet, GoogleNet, and the self-attention mechanism. To solve the non-IID data problem, a two-stage training process with the blocking variable was developed. In addition, the mix-up data augmentation within the clusters was used to address the imbalanced data problem. The performance of the Blnet was tested on the ICBHI 2017 data, and the model achieved 79.13% specificity and 66.31% sensitivity, with an average score of 72.72%, which is a 4.22% improvement in the average score and a 12.61% improvement in sensitivity over the state-of-the-art results.

Unsupervised person re-identification by patch-based nearest neighbor mining

Unsupervised person re-identification (re-ID) is still challenging due to the difficulty in learning discriminative features without labeling information. Some existing methods use the instance specificity learning to learn image features without direct supervision signals. However, this kind of instance specificity learning treats each sample as an individual instance and can not mine the inter-sample relationships, which limits the ability to explore the category boundaries. To address the unsupervised person re-ID, this paper proposes a patch-based nearest neighbor mining (PNNM) method. The proposed PNNM utilizes a patch-based feature extraction model to capture patch features from person images. Then the patch discrimination mining (PDM) is applied to supervise the model to learn a discriminative patch feature space. Meanwhile, the nearest neighbor mining (NNM) is introduced to guide the model to mine the image-level discrimination from unlabeled person images. The NNM aims to explore nearest neighbor sets with high category consistency on the whole training dataset. A two-stage training process is built with the PDM and NNM to enable the model to capture both patch-level and image-level visual clues. Extensive experiments are conducted to show the effectiveness of the proposed PNNM method in the unsupervised person re-ID task.

Unknown Security Attack Detection of Industrial Control System by Deep Learning

With the rapid development of network technologies, the network security of industrial control systems has aroused widespread concern. As a defense mechanism, an ideal intrusion detection system (IDS) can effectively detect abnormal behaviors in a system without affecting the performance of the industrial control system (ICS). Many deep learning methods are used to build an IDS, which rely on massive numbers of variously labeled samples for model training. However, network traffic is imbalanced, and it is difficult for researchers to obtain sufficient attack samples. In addition, the attack variants are rich, and constructing all possible attack types in advance is impossible. In order to overcome these challenges and improve the performance of an IDS, this paper presents a novel intrusion detection approach which integrates a one-dimensional convolutional autoencoder (1DCAE) and support vector data description (SVDD) for the first time. For the two-stage training process, 1DCAE fails to retain the key features of intrusion detection and SVDD has to add restrictions, so a joint optimization solution is introduced. A three-stage optimization process is proposed to obtain better performance. Experiments on the benchmark intrusion detection dataset NSL-KDD show that the proposed method can effectively detect various unknown attacks, learning with only normal traffic. Compared with the recent state-of-art intrusion detection baselines, the proposed method is improved in most metrics.

Interpretable deep learning approach for oral cancer classification using guided attention inference network.

.Significance: Convolutional neural networks (CNNs) show the potential for automated classification of different cancer lesions. However, their lack of interpretability and explainability makes CNNs less than understandable. Furthermore, CNNs may incorrectly concentrate on other areas surrounding the salient object, rather than the network’s attention focusing directly on the object to be recognized, as the network has no incentive to focus solely on the correct subjects to be detected. This inhibits the reliability of CNNs, especially for biomedical applications.Aim: Develop a deep learning training approach that could provide understandability to its predictions and directly guide the network to concentrate its attention and accurately delineate cancerous regions of the image.Approach: We utilized Selvaraju et al.’s gradient-weighted class activation mapping to inject interpretability and explainability into CNNs. We adopted a two-stage training process with data augmentation techniques and Li et al.’s guided attention inference network (GAIN) to train images captured using our customized mobile oral screening devices. The GAIN architecture consists of three streams of network training: classification stream, attention mining stream, and bounding box stream. By adopting the GAIN training architecture, we jointly optimized the classification and segmentation accuracy of our CNN by treating these attention maps as reliable priors to develop attention maps with more complete and accurate segmentation.Results: The network’s attention map will help us to actively understand what the network is focusing on and looking at during its decision-making process. The results also show that the proposed method could guide the trained neural network to highlight and focus its attention on the correct lesion areas in the images when making a decision, rather than focusing its attention on relevant yet incorrect regions.Conclusions: We demonstrate the effectiveness of our approach for more interpretable and reliable oral potentially malignant lesion and malignant lesion classification.

Automatic Personality Identification using Machine Learning

Machine Learning has made significant changes in the world making our life more easier and comfortable .One of the most exciting applications is the prediction of Personality automatically using different algorithms. Personality computing and emotive computing, where the popularity of temperament traits is important, have gained increasing interest and a spotlight in several analysis areas recently. These applications can powerfully predict the personality of a Person. The aim of this paper is to use a more rigorous construct Validation system to extend the potential of machine learning approaches to personality assessment. We have reviewed multiple recent applications of Machine Learning to recognize personality, thus providing a broader context of fundamental principles of constructing, validating, and then providing recommendations on how to use Machine Learning to advance the level of our understanding and applying our learnings to develop advanced personality recognition applications. araphrased Text Output text rewrite / rewrite We use deep neural network learning to recognize characteristics independently and, through feature-level fusion of these networks, we obtain final predictions of obvious personalities. We use a previously trained long-term and short-term memory network to integrate time information. We train large-scale models comprised of specific subnetworks- modalities through a two-stage training process. We first train the subnets separately for and then use these trained networks to fit the overall model. We used the ChaLearn First Impressions V2 challenge dataset to evaluate the proposed method. Our method achieves the most effective overall "medium precision" score, with an average score of for 5 personality characteristics, which is compared to the state-of-the-art method.

Accuracy of New Deep Learning Model-Based Segmentation and Key-Point Multi-Detection Method for Ultrasonographic Developmental Dysplasia of the Hip (DDH) Screening

Hip joint ultrasonographic (US) imaging is the golden standard for developmental dysplasia of the hip (DDH) screening. However, the effectiveness of this technique is subject to interoperator and intraobserver variability. Thus, a multi-detection deep learning artificial intelligence (AI)-based computer-aided diagnosis (CAD) system was developed and evaluated. The deep learning model used a two-stage training process to segment the four key anatomical structures and extract their respective key points. In addition, the check angle of the ilium body balancing level was set to evaluate the system’s cognitive ability. Hence, only images with visible key anatomical points and a check angle within ±5° were used in the analysis. Of the original 921 images, 320 (34.7%) were deemed appropriate for screening by both the system and human observer. Moderate agreement (80.9%) was seen in the check angles of the appropriate group (Cohen’s κ = 0.525). Similarly, there was excellent agreement in the intraclass correlation coefficient (ICC) value between the measurers of the alpha angle (ICC = 0.764) and a good agreement in beta angle (ICC = 0.743). The developed system performed similarly to experienced medical experts; thus, it could further aid the effectiveness and speed of DDH diagnosis.

An interpretable data augmentation scheme for machine fault diagnosis based on a sparsity-constrained generative adversarial network

Vibration signal-based methods have been widely utilized in machine fault diagnosis. Usually, a lack of sufficient training data can prevent these methods from achieving satisfactory performance. The generative adversarial network (GAN) is a feasible solution to this problem. However, existing GAN-based methods struggle to stably generate raw vibration signals. To achieve vibration signal generation, a novel sparsity-constrained GAN (SC-GAN) method containing a two-stage training process is developed, which can perform data augmentation for machine fault diagnosis with a simple structure. Autoencoder (AE)-based pretraining and sparsity regularization constraints are implemented in the proposed method. Furthermore, to understand the internal mechanisms of vibration signal generation, we propose a method for analyzing the network’s weight matrix to interpret the generation mechanism. In a case study on rolling element bearings, the SC-GAN is verified to be able to generate raw vibration signals under 10 different health conditions with a more stable training process than other models. In a fault diagnosis task, the data augmentation by SC-GAN significantly improves the diagnostic accuracy by 7.44%. An analysis of the well-trained SC-GAN shows that the model captures key frequency components, which provides a credible interpretation for the generation mechanism. Another case study on the gearbox illustrates the good generalization ability of SC-GAN to other machines and more complicated signals.

Labeling lateral prefrontal sulci using spherical data augmentation and context-aware training

The inference of cortical sulcal labels often focuses on deep (primary and secondary) sulcal regions, whereas shallow (tertiary) sulcal regions are largely overlooked in the literature due to the scarcity of manual/well-defined annotations and their large neuroanatomical variability. In this paper, we present an automated framework for regional labeling of both primary/secondary and tertiary sulci of the dorsal portion of lateral prefrontal cortex (LPFC) using spherical convolutional neural networks. We propose two core components that enhance the inference of sulcal labels to overcome such large neuroanatomical variability: (1) surface data augmentation and (2) context-aware training. (1) To take into account neuroanatomical variability, we synthesize training data from the proposed feature space that embeds intermediate deformation trajectories of spherical data in a rigid to non-rigid fashion, which bridges an augmentation gap in conventional rotation data augmentation. (2) Moreover, we design a two-stage training process to improve labeling accuracy of tertiary sulci by informing the biological associations in neuroanatomy: inference of primary/secondary sulci and then their spatial likelihood to guide the definition of tertiary sulci. In the experiments, we evaluate our method on 13 deep and shallow sulci of human LPFC in two independent data sets with different age ranges: pediatric (N=60) and adult (N=36) cohorts. We compare the proposed method with a conventional multi-atlas approach and spherical convolutional neural networks without/with rotation data augmentation. In both cohorts, the proposed data augmentation improves labeling accuracy of deep and shallow sulci over the baselines, and the proposed context-aware training offers further improvement in the labeling of shallow sulci over the proposed data augmentation. We share our tools with the field and discuss applications of our results for understanding neuroanatomical-functional organization of LPFC and the rest of cortex (https://github.com/ilwoolyu/SphericalLabeling).

SeSe-Net: Self-Supervised deep learning for segmentation

Deep convolutional networks have become ubiquitous for image segmentation. But relatively little is known on how to leverage unlabeled data to further improve their performance. Here we introduce a new approach named as Self-supervised deep learning for Segmentation (SeSe-Net) that uses a ‘Worker’ (W) neural network to segment input images and a ‘Supervisor’ (S) neural network to evaluate the quality of the segmentation results. We further propose a two-stage training process. In the first stage, W learns to segment with standard dataset and generates training sets for S. Based on this, S learns how well the W performs for the segmentation task. In the second stage, S transfers the knowledge from the first stage to supervise the learning process of W on unlabeled datasets towards better segmentation performance. We show that SeSe-Net leads to a significant performance boosting with additional unlabeled data. Furthermore, with less than 5% of labeled training dataset, SeSe-Net performs on par with challenging baseline trained on full training dataset in terms of Dice Metric (DM) on three different datasets. Theoretical analysis is also given to establish the convergence of SeSe-Net.