Large Annotated Datasets Research Articles

KaIDA: a modular tool for assisting image annotation in deep learning.

Deep learning models achieve high-quality results in image processing. However, to robustly optimize parameters of deep neural networks, large annotated datasets are needed. Image annotation is often performed manually by experts without a comprehensive tool for assistance which is time- consuming, burdensome, and not intuitive. Using the here presented modular Karlsruhe Image Data Annotation (KaIDA) tool, for the first time assisted annotation in various image processing tasks is possible to support users during this process. It aims to simplify annotation, increase user efficiency, enhance annotation quality, and provide additional useful annotation-related functionalities. KaIDA is available open-source at https://git.scc.kit.edu/sc1357/kaida.

Longitudinal detection of new MS lesions using deep learning.

The detection of new multiple sclerosis (MS) lesions is an important marker of the evolution of the disease. The applicability of learning-based methods could automate this task efficiently. However, the lack of annotated longitudinal data with new-appearing lesions is a limiting factor for the training of robust and generalizing models. In this study, we describe a deep-learning-based pipeline addressing the challenging task of detecting and segmenting new MS lesions. First, we propose to use transfer-learning from a model trained on a segmentation task using single time-points. Therefore, we exploit knowledge from an easier task and for which more annotated datasets are available. Second, we propose a data synthesis strategy to generate realistic longitudinal time-points with new lesions using single time-point scans. In this way, we pretrain our detection model on large synthetic annotated datasets. Finally, we use a data-augmentation technique designed to simulate data diversity in MRI. By doing that, we increase the size of the available small annotated longitudinal datasets. Our ablation study showed that each contribution lead to an enhancement of the segmentation accuracy. Using the proposed pipeline, we obtained the best score for the segmentation and the detection of new MS lesions in the MSSEG2 MICCAI challenge.

Active multitask learning with uncertainty-weighted loss for coronary calcium scoring.

The coronary artery calcification (CAC) score is an independent marker for the risk of cardiovascular events. Automatic methods for quantifying CAC could reduce workload and assist radiologists in clinical decision-making. However, large annotated datasets are needed for training to achieve very good model performance, which is an expensive process and requires expert knowledge. The number of training data required can be reduced in an active learning scenario, which requires only the most informative samples to be labeled. Multitask learning techniques can improve model performance by joint learning of multiple related tasks and extraction of shared informativefeatures. We propose an uncertainty-weighted multitask learning model for coronary calcium scoring in electrocardiogram-gated (ECG-gated), noncontrast-enhanced cardiac calcium scoring CT. The model was trained to solve the two tasks of coronary artery region segmentation (weak labels) and coronary artery calcification segmentation (strong labels) simultaneously in an active learning scenario to improve model performance and reduce the number of samples needed for training. We compared our model with a single-task U-Net and a sequential-task model as well as other state-of-the-art methods. The model was evaluated on 1275 individual patients in three different datasets (DISCHARGE, CADMAN, orCaScore), and the relationship between model performance and various influencing factors (image noise, metal artifacts, motion artifacts, image quality) wasanalyzed. Joint learning of multiclass coronary artery region segmentation and binary coronary calcium segmentation improved calcium scoring performance. Since shared information can be learned from both tasks for complementary purposes, the model reached optimal performance with only 12% of the training data and one-third of the labeling time in an active learning scenario. We identified image noise as one of the most important factors influencing model performance along with anatomical abnormalities and metalartifacts. Our multitask learning approach with uncertainty-weighted loss improves calcium scoring performance by joint learning of shared features and reduces labeling costs when trained in an active learningscenario.

Detection of Diabetic Retinopathy Using Deep Learning

Abstract: Diabetic retinopathy is one of the most dangerous complications of diabetes, leading to permanent blindness if left untreated. One of the major challenges is early detection, which is very important for the success of treatment. Unfortunately, accurate identification of the stage of diabetic retinopathy is notoriously tricky and requires expert human interpretation of fundus images. Simplifying the detection step is essential and can help millions of people. Convolutional Neural Networks (CNNs) have been successfully used in many neighboring subjects and for the diagnosis of diabetic retinopathy itself. However, the high cost of large annotated datasets as well as inconsistencies between different clinicians hinders the implementation of these methods. In this paper, we propose an automatic method based on deep learning to detect the stage of diabetic retinopathy using a single human fundus image. In addition, we propose a multi-stage transfer learning approach that uses similar datasets with different labels. The presented method can be used as a screening method for the early detection of diabetic retinopathy with a sensitivity and specificity of 0.99 and is ranked 54 out of 2943 competing methods (quadratic weighted kappa score 0.925466) on the APTOS 2019 Blindness Detection Dataset (13,000 images).

Bounding‐box deep calibration for high performance face detection

Modern convolutional neural networks (CNNs)-based face detectors have achieved tremendous strides due to large annotated datasets. However, misaligned results with high detection confidence but low localization accuracy restrict the further improvement of detection performance. In this paper, the authors first predict high confidence detection results on the training set itself. Surprisingly, a considerable part of them exist in the same misalignment problem. Then, the authors carefully examine these cases and point out that annotation misalignment is the main reason. Later, a comprehensive discussion is given for the replacement rationality between predicted and annotated bounding-boxes. Finally, the authors propose a novel Bounding-Box Deep Calibration (BDC) method to reasonably replace misaligned annotations with model predicted bounding-boxes and offer calibrated annotations for the training set. Extensive experiments on multiple detectors and two popular benchmark datasets show the effectiveness of BDC on improving models' precision and recall rate, without adding extra inference time and memory consumption. Our simple and effective method provides a general strategy for improving face detection, especially for light-weight detectors in real-time situations.

FootApp: An AI-powered system for football match annotation

In the last years, scientific and industrial research has experienced a growing interest in acquiring large annotated data sets to train artificial intelligence algorithms for tackling problems in different domains. In this context, we have observed that even the market for football data has substantially grown. The analysis of football matches relies on the annotation of both individual players’ and team actions, as well as the athletic performance of players. Consequently, annotating football events at a fine-grained level is a very expensive and error-prone task. Most existing semi-automatic tools for football match annotation rely on cameras and computer vision. However, those tools fall short in capturing team dynamics and in extracting data of players who are not visible in the camera frame. To address these issues, in this manuscript we present FootApp, an AI-based system for football match annotation. First, our system relies on an advanced and mixed user interface that exploits both vocal and touch interaction. Second, the motor performance of players is captured and processed by applying machine learning algorithms to data collected from inertial sensors worn by players. Artificial intelligence techniques are then used to check the consistency of generated labels, including those regarding the physical activity of players, to automatically recognize annotation errors. Notably, we implemented a full prototype of the proposed system, performing experiments to show its effectiveness in a real-world adoption scenario.

Cloth manipulation based on category classification and landmark detection

Cloth manipulation remains a challenging problem for the robotic community. Recently, there has been an increased interest in applying deep learning techniques to problems in the fashion industry. As a result, large annotated data sets for cloth category classification and landmark detection were created. In this work, we leverage these advances in deep learning to perform cloth manipulation. We propose a full cloth manipulation framework that, performs category classification and landmark detection based on an image of a garment, followed by a manipulation strategy. The process is performed iteratively to achieve a stretching task where the goal is to bring a crumbled cloth into a stretched out position. We extensively evaluate our learning pipeline and show a detailed evaluation of our framework on different types of garments in a total of 140 recorded and available experiments. Finally, we demonstrate the benefits of training a network on augmented fashion data over using a small robotic-specific data set.

Label Hallucination for Few-Shot Classification

Few-shot classification requires adapting knowledge learned from a large annotated base dataset to recognize novel unseen classes, each represented by few labeled examples. In such a scenario, pretraining a network with high capacity on the large dataset and then finetuning it on the few examples causes severe overfitting. At the same time, training a simple linear classifier on top of ``frozen'' features learned from the large labeled dataset fails to adapt the model to the properties of the novel classes, effectively inducing underfitting. In this paper we propose an alternative approach to both of these two popular strategies. First, our method pseudo-labels the entire large dataset using the linear classifier trained on the novel classes. This effectively ``hallucinates'' the novel classes in the large dataset, despite the novel categories not being present in the base database (novel and base classes are disjoint). Then, it finetunes the entire model with a distillation loss on the pseudo-labeled base examples, in addition to the standard cross-entropy loss on the novel dataset. This step effectively trains the network to recognize contextual and appearance cues that are useful for the novel-category recognition but using the entire large-scale base dataset and thus overcoming the inherent data-scarcity problem of few-shot learning. Despite the simplicity of the approach, we show that that our method outperforms the state-of-the-art on four well-established few-shot classification benchmarks. The code is available at https://github.com/yiren-jian/LabelHalluc.

Abstract 1930: Unsupervised sub-cellular segmentation of complex tissue and cell samples using highly multiplexed imaging-derived a priori knowledge

Abstract Spatial context of heterocellular interactions within solid tumor microenvironments (TMEs) is important for deciphering the mechanistic underpinnings of malignant TME phenotypes and leveraging that knowledge to improve personalized and precision medicine. Recent development of highly multiplexed imaging approaches, such as co-detection by indexing (CODEX), cyclic immunofluorescence (cycif), and imaging mass cytometry (IMC) has, for the first time, allowed deep interrogation of this heterocellular spatial complexity. Proper quantitation of this complexity, however, requires the ability to easily and accurately segment and localize cells and their sub-cellular compartments within the spatial context of the tumor microenvironment. Seminal approaches based on semi-supervised and supervised learning methods, including deep learning techniques, have been developed to segment cells and their nuclei. However, generalization of these methods to segmenting heterogenous and complex cell and tissue samples with varying resolution, magnification, and dynamic range, remains a persistent bottleneck. In case of deep learning, the requirement of large and accurate annotated datasets further adds to the challenge of seamless integration of systems-based methods in pathology and cancer research. Here, we demonstrate that by leveraging cell-compartment specific a priori knowledge captured by these imaging modalities, we can segment cells in complex tissue and cell samples in an unsupervised manner, without requiring model training. We specifically show that using nucleus and cell-membrane markers, we can accurately segment sub-cellular compartments of a diversity of tissue samples imaged at different resolutions, magnifications and dynamic ranges, and cell samples at varying levels of confluency. We also demonstrate that our method is fast. Given its ease of use, accuracy, robustness, and no requirement of large, annotated datasets, our unsupervised segmentation method fills a much-needed gap toward integration of spatial systems biology and cancer research within the convergence science paradigm. Citation Format: Bogdan Kochetov, Phoenix D. Bell, Rebecca Raphael, Benjamin J. Raymond, Brian J. Leibowitz, Jingshan Tong, Brenda Diergaarde, Jian Yu, Reetesh K. Pai, Robert E. Schoen, Lin Zhang, Aatur Singhi, Shikhar Uttam. Unsupervised sub-cellular segmentation of complex tissue and cell samples using highly multiplexed imaging-derived a priori knowledge [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 1930.

Effective multiscale deep learning model for COVID19 segmentation tasks: A further step towards helping radiologist

Infection by the SARS-CoV-2 leading to COVID-19 disease is still rising and techniques to either diagnose or evaluate the disease are still thoroughly investigated. The use of CT as a complementary tool to other biological tests is still under scrutiny as the CT scans are prone to many false positives as other lung diseases display similar characteristics on CT scans. However, fully investigating CT images is of tremendous interest to better understand the disease progression and therefore thousands of scans need to be segmented by radiologists to study infected areas. Over the last year, many deep learning models for segmenting CT-lungs were developed. Unfortunately, the lack of large and shared annotated multicentric datasets led to models that were either under-tested (small dataset) or not properly compared (own metrics, none shared dataset), often leading to poor generalization performance. To address, these issues, we developed a model that uses a multiscale and multilevel feature extraction strategy for COVID19 segmentation and extensively validated it on several datasets to assess its generalization capability for other segmentation tasks on similar organs. The proposed model uses a novel encoder and decoder with a proposed kernel-based atrous spatial pyramid pooling module that is used at the bottom of the model to extract small features with a multistage skip connection concatenation approach. The results proved that our proposed model could be applied on a small-scale dataset and still produce generalizable performances on other segmentation tasks. The proposed model produced an efficient Dice score of 90% on a 100 cases dataset, 95% on the NSCLC dataset, 88.49% on the COVID19 dataset, and 97.33 on the StructSeg 2019 dataset as compared to existing state-of-the-art models. The proposed solution could be used for COVID19 segmentation in clinic applications. The source code is publicly available at https://github.com/RespectKnowledge/Mutiscale-based-Covid-_segmentation-usingDeep-Learning-models.

Deep Learning-Based Total Kidney Volume Segmentation in Autosomal Dominant Polycystic Kidney Disease Using Attention, Cosine Loss, and Sharpness Aware Minimization.

Early detection of the autosomal dominant polycystic kidney disease (ADPKD) is crucial as it is one of the most common causes of end-stage renal disease (ESRD) and kidney failure. The total kidney volume (TKV) can be used as a biomarker to quantify disease progression. The TKV calculation requires accurate delineation of kidney volumes, which is usually performed manually by an expert physician. However, this is time-consuming and automated segmentation is warranted. Furthermore, the scarcity of large annotated datasets hinders the development of deep learning solutions. In this work, we address this problem by implementing three attention mechanisms into the U-Net to improve TKV estimation. Additionally, we implement a cosine loss function that works well on image classification tasks with small datasets. Lastly, we apply a technique called sharpness aware minimization (SAM) that helps improve the generalizability of networks. Our results show significant improvements (p-value < 0.05) over the reference kidney segmentation U-Net. We show that the attention mechanisms and/or the cosine loss with SAM can achieve a dice score (DSC) of 0.918, a mean symmetric surface distance (MSSD) of 1.20 mm with the mean TKV difference of −1.72%, and R of 0.96 while using only 100 MRI datasets for training and testing. Furthermore, we tested four ensembles and obtained improvements over the best individual network, achieving a DSC and MSSD of 0.922 and 1.09 mm, respectively.

Self-Supervised Representation Learning: Introduction, advances, and challenges

Self-supervised representation learning methods aim to provide powerful deep feature learning without the requirement of large annotated datasets, thus alleviating the annotation bottleneck that is one of the main barriers to practical deployment of deep learning today. These methods have advanced rapidly in recent years, with their efficacy approaching and sometimes surpassing fully supervised pre-training alternatives across a variety of data modalities including image, video, sound, text and graphs. This article introduces this vibrant area including key concepts, the four main families of approach and associated state of the art, and how self-supervised methods are applied to diverse modalities of data. We further discuss practical considerations including workflows, representation transferability, and compute cost. Finally, we survey the major open challenges in the field that provide fertile ground for future work.

Nucleus segmentation: towards automated solutions.

Single nucleus segmentation is a frequent challenge of microscopy image processing, since it is the first step of many quantitative data analysis pipelines. The quality of tracking single cells, extracting features or classifying cellular phenotypes strongly depends on segmentation accuracy. Worldwide competitions have been held, aiming to improve segmentation, and recent years have definitely brought significant improvements: large annotated datasets are now freely available, several 2D segmentation strategies have been extended to 3D, and deep learning approaches have increased accuracy. However, even today, no generally accepted solution and benchmarking platform exist. We review the most recent single-cell segmentation tools, and provide an interactive method browser to select the most appropriate solution.

Temporally coherent person matting trained on fake-motion dataset

We propose a novel neural-network-based method to perform matting of videos depicting people that does not require additional user input such as trimaps. Our architecture achieves temporal stability of the resulting alpha mattes by using motion-estimation-based smoothing of image-segmentation algorithm outputs, combined with convolutional-LSTM modules on U-Net skip connections.We also propose a fake-motion algorithm that generates training clips for the video-matting network given photos with ground-truth alpha mattes and background videos. We apply random motion to photos and their mattes to simulate movement one would find in real videos and composite the result with the background clips. It lets us train a deep neural network operating on videos in an absence of a large annotated video dataset and provides ground-truth training-clip foreground optical flow for use in loss functions.

Transfer learning for data-efficient abdominal muscle segmentation with convolutional neural networks.

BackgroundSkeletal muscle segmentation is an important procedure for assessing sarcopenia, an emerging imaging biomarker of patient frailty. Data annotation remains the bottleneck for training deep learning auto‐segmentation models.PurposeThere is a need to define methodologies for applying models to different domains (e.g., anatomical regions or imaging modalities) without dramatically increasing data annotation.MethodsTo address this problem, we empirically evaluate the generalizability of various source tasks for transfer learning: natural image classification, natural image segmentation, unsupervised image reconstruction, and self‐supervised jigsaw solving. Axial CT slices at L3 were extracted from PET‐CT scans for 204 oesophago‐gastric cancer patients and the skeletal muscle manually delineated by an expert. Features were transferred and segmentation models trained on subsets (n=5,10,25,50,75,100,125) of the manually annotated training set. Four‐fold cross‐validation was performed to evaluate model generalizability. Human‐level performance was established by performing an inter‐observer study consisting of ten trained radiographers.ResultsWe find that accurate segmentation models can be trained on a fraction of the data required by current approaches. The Dice similarity coefficient and root mean square distance‐to‐agreement were calculated for each prediction and used to assess model performance. Models pre‐trained on a segmentation task and fine‐tuned on 10 images produce delineations that are comparable to those from trained observers and extract reliable measures of muscle health.ConclusionsAppropriate transfer learning can generate convolutional neural networks for abdominal muscle segmentation that achieve human‐level performance while decreasing the required data by an order of magnitude, compared to previous methods (n=160→10). This work enables the development of future models for assessing skeletal muscle at other anatomical sites where large annotated data sets are scarce and clinical needs are yet to be addressed.

Fine-grained Human Analysis under Occlusions and Perspective Constraints in Multimedia Surveillance

Human detection in the wild is a research topic of paramount importance in computer vision, and it is the starting step for designing intelligent systems oriented to human interaction that work in complete autonomy. To achieve this goal, computer vision and machine learning should aim at superhuman capabilities. In this work, we address the problem of fine-grained human analysis under occlusions and perspective constraints. More specifically, we discuss some issues and some possible solutions to effectively detect people using pose estimation methods and to detect humans under occlusions both in the two-dimensional (2D) image plane and in the 3D space exploiting single monocular cameras. Dealing with occlusion can be done at the joint level or pixel level: We discuss two different solutions, the former based on a supervised neural network architecture for detecting occluded joints and the latter based on a semi-supervised specialized GAN that exploits both appearance and human shape attributes to determine the missing parts of the visible shape. To deal with perspective constraints, we further discuss a neural approach based on a double architecture that learns to create an optimal neural representation, which is useful to reconstruct the 3D position of human keypoints starting with simple RGB images. All these approaches have a critical point in common: the need for large annotated datasets. To have large, fair, consistent, transparent, and ethical datasets, we propose the adoption of synthetic datasets as, for example, JTA and MOTSynth. In this article, we discuss the pros and cons of using synthetic datasets while tackling several human-centered AI issues with respect to European GDPR rules for privacy. We further explore and discuss an application in the field of risk assessment by space occupancy estimation during the COVID-19 pandemic called Inter-Homines.

Segmentation and classification on chest radiography: a systematic survey.

Chest radiography (X-ray) is the most common diagnostic method for pulmonary disorders. A trained radiologist is required for interpreting the radiographs. But sometimes, even experienced radiologists can misinterpret the findings. This leads to the need for computer-aided detection diagnosis. For decades, researchers were automatically detecting pulmonary disorders using the traditional computer vision (CV) methods. Now the availability of large annotated datasets and computing hardware has made it possible for deep learning to dominate the area. It is now the modus operandi for feature extraction, segmentation, detection, and classification tasks in medical imaging analysis. This paper focuses on the research conducted using chest X-rays for the lung segmentation and detection/classification of pulmonary disorders on publicly available datasets. The studies performed using the Generative Adversarial Network (GAN) models for segmentation and classification on chest X-rays are also included in this study. GAN has gained the interest of the CV community as it can help with medical data scarcity. In this study, we have also included the research conducted before the popularity of deep learning models to have a clear picture of the field. Many surveys have been published, but none of them is dedicated to chest X-rays. This study will help the readers to know about the existing techniques, approaches, and their significance.

Efficient Cancer Classification by Coupling Semi Supervised and Multiple Instance Learning

The annotation of large datasets is often the bottleneck in the successful application of artificial intelligence in computational pathology. For this reason recently Multiple Instance Learning (MIL) and Semi Supervised Learning (SSL) approaches are gaining popularity because they require fewer annotations. In this work we couple SSL and MIL to train a deep learning classifier that combines the advantages of both methods and overcomes their limitations. Our method is able to learn from the global WSI diagnosis and a combination of labeled and unlabeled patches. Furthermore, we propose and evaluate an efficient labeling paradigm that guarantees a strong classification performance when combined with our learning framework. We compare our method to SSL and MIL baselines, the state-of-the-art and completely supervised training. With only a small percentage of patch labels our proposed model achieves a competitive performance on SICAPv2 (Cohen’s kappa of 0.801 with 450 patch labels), PANDA (Cohen’s kappa of 0.794 with 22,023 patch labels) and Camelyon16 (ROC AUC of 0.913 with 433 patch labels). Our code is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/arneschmidt/ssl_and_mil_cancer_classification</uri> .

AUTOMATIC EFFUSION MEASUREMENT WITH LIMITED LABELED DATA

Accurate quantification of effusion volume is becoming more important as OA is increasingly recognized to have inflammatory components including joint effusion. Effective clinical management requires accurate quantification of features related to inflammation; particularly synovitis and joint effusion which represents a target for therapy. Rather than relying on highly variable subjective assessment of joint effusions, with the development of deep learning techniques we have witnessed increased automation of radiological assessments. However, most of these techniques use supervised learning and are trained on large annotated datasets. Acquiring “ground truth” labels for large datasets is expensive and time-consuming. This becomes even more complicated and challenging when it comes to detecting medical image features like effusion. Detecting effusions accurately requires a high level of expertise. Because pockets of joint fluid can be in unexpected locations, human experts frequently need to step back to look at the entire image stack multiple times to prevent inattentional error. Therefore, proposing a method for learning from limited labeled data is crucial for medical image analysis. To perform automated measurement of knee effusion volume using minimal labeled data. In this method, we aim to develop an alternative deep learning training strategy based on self-supervised pretraining on unlabeled knee MRI scans. This study requires two sets of data: 1) a large unlabeled dataset, and 2) a small labeled dataset. For the large unlabeled dataset, we have used 4 different OAI MRI sequence data (64k slices have been selected randomly). For the labeled dataset we have used low-resolution sagittal Turbo Spin Echo (TSE) MR sequences from the OAI dataset. Effusion regions were extracted by a trained musculoskeletal radiologist (DC) for 23 participants (a total of 31 scans) using an interactive software developed in-house. We have used 5 scans for the test and 26 scans (a total of 700 slices) for training. In this method, IMaskRCNN deep learning architecture with the ResNet-101 backbone have been used. The training process has been performed in two-phase, self-supervised pretraining, and fine-tuning. The self-supervised pretraining using the fill-blank method was performed on unlabeled images to learn visual representations in musculoskeletal MRIs. After pretraining, we fined-tuned last layers of the network (backbone weights have been frozen) using the training division of the small TSE dataset. The dice similarity coefficient, Intersection over union (IoU) was used to compare the result of the segmentation between the predicted output and the ground truth. Qualitative results of the proposed method is presented in the figure. As shown the network was able to provide a good prediction of effusion location and extracting it. The segmentation model using the proposed pretraining method was successful in finding the location of effusion for 90% of the slices with IoU = 60%. The dice score for the detected effusion is 0.70 indicates a high agreement between the automatic segmentation and the labels provided by the expert. Self-supervised pretraining on the OAI dataset, can be a solution to the scarcity of pixel-wise ground truth (GT) for effusion quantification. Alberta Innovates, AHS Chair in Diagnostic Imaging, Medical Imaging Consultants, CIHR. CORRESPONDENCE ADDRESS: banafshe.felfeliyan@ucalgary.ca

Semi-Supervised Skin Lesion Segmentation With Coupling CNN and Transformer Features

An automatic skin lesion segmentation algorithm not only facilitates the dermatologist’s workload on skin cancer analysis but also provides a platform for early cancer prediction. Over the years, several deep learning methods have been proposed to address the skin lesion segmentation problem. However, training deep models usually requires a large-scale annotated dataset, which is not feasible in the medical domain due to the annotation burden. In addition, the low data regime highly increases the overfitting potential for the neural network. To address these limitations in an end-to-end manner, we propose to incorporate unlabelled samples during the training process. Our network offers a semi-supervised training schema, wherein the first stage performs a supervised training strategy to learn semantic segmentation map while the second step focuses on the unsupervised technique to enrich the encoder module. Specifically, unlike the literature work on skin lesion segmentation, we design a surrogate task on top of the convolutional and Transformer representations to learn data-driven features from the image itself to alleviate the requirement of the large annotated dataset. The effectiveness of the proposed method is demonstrated using three different skin lesion segmentation datasets, namely ISIC 2018 (dice score 0.905), ISIC 2017 (dice score 0.898) and PH2 (dice score 0.940). Particularly we observed that including the unsupervised samples can increase the dice score by 2%.