Manual Labeling Research Articles

Automated segmentation and classification of supraspinatus fatty infiltration in shoulder magnetic resonance image using a convolutional neural network.

Goutallier's fatty infiltration of the supraspinatus muscle is a critical condition in degenerative shoulder disorders. Deep learning research primarily uses manual segmentation and labeling to detect this condition. Employing unsupervised training with a hybrid framework of segmentation and classification could offer an efficient solution. To develop and assess a two-step deep learning model for detecting the region of interest and categorizing the magnetic resonance image (MRI) supraspinatus muscle fatty infiltration according to Goutallier's scale. A retrospective study was performed from January 1, 2019 to September 20, 2020, using 900 MRI T2-weighted images with supraspinatus muscle fatty infiltration diagnoses. A model with two sequential neural networks was implemented and trained. The first sub-model automatically detects the region of interest using a U-Net model. The second sub-model performs a binary classification using the VGG-19 architecture. The model's performance was computed as the average of five-fold cross-validation processes. Loss, accuracy, Dice coefficient (CI. 95%), AU-ROC, sensitivity, and specificity (CI. 95%) were reported. Six hundred and six shoulders MRIs were analyzed. The Goutallier distribution was presented as follows: 0 (66.50%); 1 (18.81%); 2 (8.42%); 3 (3.96%); 4 (2.31%). Segmentation results demonstrate high levels of accuracy (0.9977 ± 0.0002) and Dice score (0.9441 ± 0.0031), while the classification model also results in high levels of accuracy (0.9731 ± 0.0230); sensitivity (0.9000 ± 0.0980); specificity (0.9788 ± 0.0257); and AUROC (0.9903 ± 0.0092). The two-step training method proposed using a deep learning model demonstrated strong performance in segmentation and classification tasks.

Automatic substantia nigra segmentation with Swin-Unet in susceptibility- and T2-weighted imaging: application to Parkinson disease diagnosis.

Accurately distinguishing between Parkinson disease (PD) and healthy controls (HCs) through reliable imaging method is crucial for appropriate therapeutic intervention. However, PD diagnosis is hindered by the subjective nature of the evaluation. We aimed to develop an automatic deep-learning method that can segment the substantia nigra areas on susceptibility-weighted imaging (SWI) and T2-weighted imaging (T2WI) and further differentiate patients with PD from HCs using a machine learning algorithm. Magnetic resonance imaging (MRI) data from 83 patients with PD and 83 age- and sex-matched HCs were obtained on the same 3.0-T MRI scanner. A deep learning method with Swin-Unet was developed to segment volumes of interest (VOIs) on SWI and then map the VOIs on SWI to the corresponding T2WI; features were then extracted from the VOIs on SWI and T2WI. Three machine learning models were developed and compared to differentiate those with PD from HCs. Swin-Unet achieved a better Dice coefficient than did U-Net in SWI segmentation (0.832 vs. 0.712). Machine learning models outperformed visual analysis (P>0.05), and logistic regression (LR) achieved the best performance [area under the curve (AUC) ≥0.819] and the most stable (relative standard deviations in AUC ≤0.05). The test results showed that the AUC of the LR model based on SWI segmentation was 0.894 while that of the LR model based on T2WI segmentation was 0.876. There was no significant difference in VOIs based on manual labeling or automatic segmentation across T2WI, SWI, or a combination of the two (P>0.05). The AUCs of the LR model based on automatic segmentation were close to those of the model based on manual labeling (P>0.05). Our approach could provide a powerful and useful method for automatically and rapidly diagnosing PD in the clinic with only T2WI.

A Self-Supervised Network-Based Smoke Removal and Depth Estimation for Monocular Endoscopic Videos.

In minimally invasive surgery videos, label-free monocular laparoscopic depth estimation is challenging due to smoke. For this reason, we propose a self-supervised collaborative network-based depth estimation method with smoke-removal for monocular endoscopic video, which is decomposed into two steps of smoke-removal and depth estimation. In the first step, we develop a de-endoscopic smoke for cyclic GAN (DS-cGAN) to mitigate the smoke components at different concentrations. The designed generator network comprises sharpened guide encoding module (SGEM), residual dense bottleneck module (RDBM) and refined upsampling convolution module (RUCM), which restores more detailed organ edges and tissue structures. In the second step, high resolution residual U-Net (HRR-UNet) consisting of a DepthNet and two PoseNets is designed to improve the depth estimation accuracy, and adjacent frames are used for camera self-motion estimation. In particular, the proposed method requires neither manual labeling nor patient computed tomography scans during the training and inference phases. Experimental studies on the laparoscopic data set of the Hamlyn Centre show that our method can effectively achieve accurate depth information after net smoking in real surgical scenes while preserving the blood vessels, contours and textures of the surgical site. The experimental results demonstrate that the proposed method outperforms existing state-of-the-art methods in effectiveness and achieves a frame rate of 94.45fps in real time, making it a promising clinical application.

BEIT-based and feature extraction unsupervised clustering algorithm for wafer patterns

Abstract In the wafer manufacturing process, a multitude of electrical tests are conducted, with each test generating a measurement value at the corresponding location. The distribution of different consecutive test values on a wafer constitutes a wafer pattern. Accurate classification of wafer patterns is essential for tracing the root causes of issues and optimizing the manufacturing process. Traditional pattern classification algorithms rely on supervised learning, which necessitates extensive manual labeling of samples for model training. To optimize the consumption of human resources, we propose an unsupervised clustering algorithm for wafer patterns based on BEIT and feature extraction. This method not only meets the requirements for analyzing and tracing problems but also eliminates the need for manual labeling, thus conserving resources. The final experimental results show that our method, compared to the BEIT model, has increased the JC (Jaccard Coefficient) index by 16.2 percentage points when the clustering sample size reaches 50%, and has improved by 14.4 percentage points on the FM (Fowlkes and Mallows) index.

Research on hybrid segmentation technologies for postprocessing the lung and trachea CT images

This paper introduces a systematic method for segmenting the main trachea and bronchioles in lung computed tomography scans. It begins with a stack-based three-dimensional region growth algorithm to outline the main trachea, which is then refined using morphological techniques to improve accuracy. The segmentation of bronchioles is achieved through domain labeling, lung tissue segmentation, adaptive binarization, and inner product analysis. The main trachea and bronchioles are integrated using an operating room (OR) operation and a novel splicing algorithm to form a complete tracheal tree. The method’s accuracy is validated against manual labeling, showing a Dice coefficient of about 0.99, on average, in lung parenchyma segmentation and a segmentation overlap with expert results ranging from 79.89% to 93.31% in lung trachea tree segmentation. This robust methodology is thoroughly tested and validated.

Self-supervised learning for efficient seismic facies classification

Seismic facies classification is an important task in seismic interpretation that allows the identification of rock bodies with similar physical characteristics. Manual labeling of seismic data is immensely time consuming, given the recent surge in data volumes. Self-supervised learning (SSL) enables models to learn powerful representations from unlabeled data, thereby improving performance in downstream tasks using limited labeled data. We investigate the effectiveness of SSL for efficient facies classification by evaluating various convolutional and vision transformer-based models. We pretrain the models on image reconstruction and fine-tune them on facies segmentation. Results on the southern North Sea F3 seismic block in the Netherlands and the Penobscot seismic volume in the Sable Subbasin, offshore Nova Scotia, Canada, show that SSL has comparable performance to supervised learning using only 5%–10% labeled data. Further, SSL exhibits stable domain adaptation on the Penobscot data set even with 5% labeled data, indicating an improved generalization compared with the supervised learning setup. The findings demonstrate that SSL significantly enhances model accuracy and data efficiency for seismic facies classification.

Human-in-the-loop: Using classifier decision boundary maps to improve pseudo labels

For classification tasks, several strategies aim to tackle the problem of not having sufficient labeled data, usually by automatic labeling or by fully passing this task to a user. Automatic labeling is simple to apply but can fail handling complex situations where human insights may be required to decide the correct labels. Conversely, manual labeling leverages the expertise of specialists but may waste precious effort which could be handled by automatic methods. More specifically, automatic solutions could be improved by combining an active learning loop with manual labeling assisted by visual depictions of a classifier’s behavior. We propose to include the human in the labeling loop by using manual labeling in feature spaces produced by a deep feature annotation (DeepFA) technique. To assist manual labeling, we provide users with visual insights on the classifier’s decision boundaries. Finally, we use the manual and automatically computed labels jointly to retrain the classifier in an active learning (AL) loop scheme. Experiments using a toy and a real-world application dataset show that our proposed combination of manual labeling supported by visualization of decision boundaries and automatic labeling can yield a significant increase in classifier performance with a quite limited user effort.

Automatic Road Crack Detection Using Convolutional Neural Network Based on Semi-Supervised Learning

Crack detection in pavements is a critical task for infrastructure maintenance, but it often requires extensive manual labeling of training samples, which is both time-consuming and labor-intensive. To address this challenge, this paper proposes a semi-supervised learning approach based on a DenseNet classification model to detect pavement cracks more efficiently. The primary objective is to leverage a small set of labeled samples to improve the model's performance by incorporating a large number of unlabeled samples through semi-supervised learning. This method enhances the DenseNet model's ability to generalize by iteratively learning from new unlabeled datasets. As a result, the proposed approach not only reduces the need for extensive manual labeling but also mitigates issues related to label inconsistency and errors in the original labels. The experimental results demonstrate that the semi-supervised DenseNet model achieves a prediction precision of 96.77% and a recall of 94.17%, with an F1 score of 95.45% and an Intersectidn over Union (IoU) of 91.30%. These metrics highlight the model's high accuracy and effectiveness in crack detection. The proposed method not only improves label quality and model performance but also offers practical value for engineering applications in the field of pavement maintenance, making it a valuable tool for infrastructure management.

The Human Cerebellum: A Digital Anatomical Atlas at the Level of Individual Folia.

Scientific interest in the cerebellum has surged in the last few decades with an emerging consensus on a multifaceted functionality and intricate, but not yet fully understood, functional topography over the cerebellar cortex. To further refine this structure-function relationship and quantify its inter-subject variability, a high-resolution digital anatomical atlas is fundamental. Using a combination of manual labeling and image processing, we turned a recently published reconstruction of the human cerebellum, the first such reconstruction fine enough to resolve the individual folia, into a digital atlas with both surface and volumetric representations. Its unprecedented granularity (0.16 mm) and detailed expert labeling make the atlas usable as an anatomical ground truth, enabling new ways of analyzing and visualizing cerebellar data through its digital format.

High-resolution segmentations of the hypothalamus and its subregions for training of segmentation models

Segmentation of brain structures on magnetic resonance imaging (MRI) is a highly relevant neuroimaging topic, as it is a prerequisite for different analyses such as volumetry or shape analysis. Automated segmentation facilitates the study of brain structures in larger cohorts when compared with manual segmentation, which is time-consuming. However, the development of most automated methods relies on large and manually annotated datasets, which limits the generalizability of these methods. Recently, new techniques using synthetic images have emerged, reducing the need for manual annotation. Here we provide a dataset composed of label maps built from publicly available ultra-high resolution ex vivo MRI from 10 whole hemispheres, which can be used to develop segmentation methods using synthetic data. The label maps are obtained with a combination of manual labels for the hypothalamic regions and automated segmentations for the rest of the brain, and mirrored to simulate entire brains. We also provide the pre-processed ex vivo scans, as this dataset can support future projects to include other structures after these are manually segmented.

A Global Database of Pitted Cones on Mars for Research on Martian Volcanism

This study presents a global database of pitted cones on Mars. This database was created using an active deep-learning approach based on a global mosaic of Mars Context Camera (CTX) images, followed by extensive manual verification. The global database contains 250,547 pitted cones along with their central coordinates and diameters. 94.2% of the pitted cones have diameters ranging from 100 m to 1 km. Most of them are in the northern lowlands, while some are scattered in the southern highlands. Systematic evaluation of the extracted pitted cones is performed in Isidis Planitia, Acidalia Planitia, and Cerberus Palus on Mars. The results reveal that the precision ranges from 79.2% to 85.6%, and the recall rate ranges from 62.4% to 87.7%, indicating a varying among regions but satisfactory performance. Manual digitizing evaluation indicates that over 94.1% of manual labels by different annotators have a differentiation rate of 10%, showing promising precision and consistency. This global database can facilitate further research on the origin and evolution of pitted cones on Mars.

Recognition of dispersed organic matter macerals using YOLOv5m model with convolutional block attention module

The identification of dispersed organic matter (DOM) macerals plays a crucial role in petroleum geology. However, the conventional manual statistical methods are both time-consuming and laborious. With advancements in deep learning, automatic computerized identification of macerals has become the prevailing trend. In this study, we present the first application of the YOLOv5m lightweight semantic segmentation model with convolutional block attention module (CBAM) for classifying DOM macerals into five distinct categories, namely vitrinite, inertinite, solid bitumen, sporinite, and alginite. The results demonstrate that the YOLOv5m + CBAM model exhibits a superior convergence rate, as evidenced by the loss and precision-recall curves. It also achieves a 2 % increase in mean average precision at 0.5 intersection over union threshold over the YOLOv5m model. Moreover, the YOLOv5m + CBAM model shows robustness and efficiency in DOM macerals segmentation, achieving a mean intersection over union of 53.4 % and a mean pixel accuracy of 72.2 % (excluding the background category), while maintaining similar parameters and floating-point operations comparable to those of YOLOv5m. Comparative analysis against models such as DeepLab v3 + and U-net reveals that YOLOv5m + CBAM achieves superior accuracy with fewer parameters, particularly on smaller datasets. Statistical analysis confirms its consistency with manual labeling, resulting in a mean absolute percentage error of 7.34 %. Consequently, the proposed YOLOv5m + CBAM framework represents an ideal approach for pixel segmentation of DOM macerals.

PolyMeme: Fine-Grained Internet Meme Sensing.

Internet memes are a special type of digital content that is shared through social media. They have recently emerged as a popular new format of media communication. They are often multimodal, combining text with images and aim to express humor, irony, sarcasm, or sometimes convey hatred and misinformation. Automatically detecting memes is important since it enables tracking of social and cultural trends and issues related to the spread of harmful content. While memes can take various forms and belong to different categories, such as image macros, memes with labeled objects, screenshots, memes with text out of the image, and funny images, existing datasets do not account for the diversity of meme formats, styles and content. To bridge this gap, we present the PolyMeme dataset, which comprises approximately 27 K memes from four categories. This was collected from Reddit and a part of it was manually labelled into these categories. Using the manual labels, deep learning networks were trained to classify the unlabelled images with an estimated error rate of 7.35%. The introduced meme dataset in combination with existing datasets of regular images were used to train deep learning networks (ResNet, ViT) on meme detection, exhibiting very high accuracy levels (98% on the test set). In addition, no significant gains were identified from the use of regular images containing text.

DART: An automated end-to-end object detection pipeline with data Diversification, open-vocabulary bounding box Annotation, pseudo-label Review, and model Training

Accurate real-time object detection is vital across numerous industrial applications, from safety monitoring to quality control. Traditional approaches, however, are hindered by arduous manual annotation and data collection, struggling to adapt to ever-changing environments and novel target objects. To address these limitations, this paper presents DART, an innovative automated end-to-end pipeline that revolutionizes object detection workflows from data collection to model evaluation. It eliminates the need for laborious human labeling and extensive data collection while achieving outstanding accuracy across diverse scenarios. DART encompasses four key stages: (1) Data Diversification using subject-driven image generation (DreamBooth with SDXL), (2) Annotation via open-vocabulary object detection (Grounding DINO) to generate bounding box and class labels, (3) Review of generated images and pseudo-labels by large multimodal models (InternVL-1.5 and GPT-4o) to guarantee credibility, and (4) Training of real-time object detectors (YOLOv8 and YOLOv10) using the verified data. We apply DART to a self-collected dataset of construction machines named Liebherr Product, which contains over 15K high-quality images across 23 categories. The current instantiation of DART significantly increases average precision (AP) from 0.064 to 0.832. Its modular design ensures easy exchangeability and extensibility, allowing for future algorithm upgrades, seamless integration of new object categories, and adaptability to customized environments without manual labeling and additional data collection. The code and dataset are released at https://github.com/chen-xin-94/DART.

Validation of an Improved Vision-Based Web Page Parsing Pipeline

In this article, we present a novel approach to quantitative evaluation of a model for parsing web pages as visual images, intended to provide improvements for users with assistive needs (cognitive or visual deficits, enabling decluttering or zooming and supporting more effective screen reader output). This segmentation-classification pipeline is tested in stages: We first discuss the validation of the segmentation algorithm, showing that our approach produces automated segmentations that are very similar to those produced by real users when making use of a drawing interface to designate edges and regions. We also examine the properties of these ground truth segmentations produced under different conditions. We then describe our Hidden Markov tree approach for classification and present results which serve provide important validation for this model. The analysis is set against effective choices for dataset and pruning options, measured with respect to manual ground truth labelling of regions. In all, we offer a detailed quantitative validation (focused on complex news pages) of a fully pipelined approach for interpreting web pages as visual images, an approach which enables important advances for users with assistive needs.

A training-free non-intrusive air conditioning load monitoring method based on fuzzy comprehensive evaluation

Air conditioner (AC) is a grid friendly load, monitoring its operating status plays a key role in evaluating user side potential for participation in demand response. With continuous innovations in AC technology, the AC operation characteristics have shown increasing complexity, which poses significant challenges for non-intrusive AC load monitoring. Furthermore, obtaining labeled training data for target scenarios is often challenging, which hinders deployment of many Non-Intrusive Load Monitoring (NILM) methods in practice. In this regard, this article proposes a training-free non-intrusive AC load monitoring method based on fuzzy comprehensive evaluation (FCE), which is very easy to deploy and has good robustness. Firstly, considering various influence factors, a FCE model is constructed based on common knowledge about AC. This model simulates the manual labeling process to locate AC operation periods. Secondly, an AC power estimation method based on weighted filtering is established, in which weighted filter is used to eliminate the non-AC load components that may overlap during AC operation periods. Specifically, an optimization model is constructed to make the cumulative power of the AC events in its working cycle approach zero, with the filter weight vector as the objective variable, in which a heuristic greedy algorithm is used to find the optimal filter weight vector. When multiple feasible solutions with similar objective function values are derived in the solving process, the probability of load events appearing in AC and non-AC operation periods are compared. The final solution is selected, taking into account the likelihood of each feasible solution corresponding to a load event sequence belonging to AC load. The testing results on the PecanStreet dataset and multiple real-world measured datasets prove the effectiveness of the proposed method in monitoring complex ACs with poor waveform consistency.

Accurate neuron segmentation method for one-photon calcium imaging videos combining convolutional neural networks and clustering

One-photon fluorescent calcium imaging helps understand brain functions by recording large-scale neural activities in freely moving animals. Automatic, fast, and accurate active neuron segmentation algorithms are essential to extract and interpret information from these videos. One-photon imaging videos’ low resolution, high noise, and high background fluctuation pose significant challenges. Here, we develop a software pipeline to address the challenges of processing one-photon calcium imaging videos. We extend our previous two-photon active neuron segmentation algorithm, Shallow U-Net Neuron Segmentation (SUNS), to better suppress background fluctuations in one-photon videos. We also develop additional neuron extraction (ANE) to locate small or dim neurons missed by SUNS. To train our segmentation method, we create ground truth neurons by developing a manual labeling pipeline assisted with semi-automatic refinement. Our method is more accurate and faster than state-of-the-art techniques when processing simulated videos and multiple experimental datasets acquired over various brain regions with different imaging conditions.

Intelligent Classification and Segmentation of Sandstone Thin Section Image Using a Semi-Supervised Framework and GL-SLIC

This study presents the development and validation of a robust semi-supervised learning framework specifically designed for the automated segmentation and classification of sandstone thin section images from the Yanchang Formation in the Ordos Basin. Traditional geological image analysis methods encounter significant challenges due to the labor-intensive and error-prone nature of manual labeling, compounded by the diversity and complexity of rock thin sections. Our approach addresses these challenges by integrating the GL-SLIC algorithm, which combines Gabor filters and Local Binary Patterns for effective superpixel segmentation, laying the groundwork for advanced component identification. The primary innovation of this research is the semi-supervised learning model that utilizes a limited set of manually labeled samples to generate high-confidence pseudo labels, thereby significantly expanding the training dataset. This methodology effectively tackles the critical challenge of insufficient labeled data in geological image analysis, enhancing the model’s generalization capability from minimal initial input. Our framework improves segmentation accuracy by closely aligning superpixels with the intricate boundaries of mineral grains and pores. Additionally, it achieves substantial improvements in classification accuracy across various rock types, reaching up to 96.3% in testing scenarios. This semi-supervised approach represents a significant advancement in computational geology, providing a scalable and efficient solution for detailed petrographic analysis. It not only enhances the accuracy and efficiency of geological interpretations but also supports broader hydrocarbon exploration efforts.

An automated lightweight approach for detecting dead fish in a recirculating aquaculture system

Accurate methods for detecting dead fish are of great significance for fisheries because of their potential to improve production and reduce water pollution. Small-scale fisheries typically apply manual observation, but this approach is labor-intensive and time-consuming, which limits its application on large-scale farms. Advances in computer vision tools have provided the foundation for revolutionary methods of automated identification of individual-animal behavior. However, computer vision systems often suffer from unstable accuracy, low efficiency, and limited detection capability. Thus, the aim of this study was to design a deep learning system, designated “Deadfish-YOLO”, to detect dead fish based solely on standard underwater camera images with no additional hardware. First, eight selected rearing tanks were monitored by a self-developed computer version system. From these tank images, a dataset containing 18,114 frames was built by manual labeling. Next, a lightweight backbone network was generated with YOLOv4 to ensure fast computations, and an attention mechanism was introduced into the model to suppress unimportant features. Finally, the ReLU-memristor-like activation function was adopted to improve neural-network performance. The accuracy and processing speed of Deadfish-YOLO were superior to those of other state-of-the-art single- and two-stage detection models; Deadfish-YOLO ran at 85 frames per second with a mean average precision of 0.946 and an average ratio of 0.924 between intersection and union. These results demonstrate that Deadfish-YOLO may be used to automatically monitor dead fish in real circulating aquaculture systems. In addition, the results of this study should facilitate the wider application of artificial-intelligence-based animal monitoring in aquaculture.

Novel identification algorithm for plasma boundary gap based on visible endoscope diagnostic on EAST tokamak

The precise plasma boundary gap identification at the midplane is a prerequisite for achieving controlled plasma positioning and holds a significant importance for the stable operation of tokamak devices. This study proposes a plasma boundary gap at the midplane recognition algorithm based on visual endoscopy diagnostic. The model is an end-to-end one that uses a convolutional neural network that does not require manual data labeling. The model performance is improved by experimentally comparing different convolutional layers and input image sizes. The model is validated using a testing dataset comprising 400 plasma discharge moments. The model has average errors of 3.7 and 4 mm for gap-in and -out, respectively, when compared to those obtained by equilibrium fitting. The proposed approach offers a convenient and effective means of obtaining the boundary gap value and is particularly suited for future fusion experimental devices, such as BEST and ITER tokamak.