Expert Labels Research Articles

Deep Learning-Based In Situ Micrograph Synthesis and Augmentation for Crystallization Process Image Analysis

Deep learning-based in situ imaging and analysis for crystallization process are essential for optimizing product qualities, reducing experimental costs through real-time monitoring, and controlling the process. However, large and high-quality annotated datasets are required to train accurate models, which are time consuming. Therefore, we proposed a novel methodology that applied image synthesis neural networks to generate virtual information-rich images, enabling efficient and rapid dataset expansion while simultaneously reducing annotation costs. Experiments were conducted on the L-alanine crystallization process to obtain process images and to validate the proposed workflow. The proposed method, aided by interpolation augmentation and data warping augmentation to enhance data richness, utilized only 25% of the training annotations, consistently segmenting crystallization process images comparable to those models utilizing 100% of the training data annotations, achieving an average precision of nearly 98%. Additionally, based on the analysis of Kullback–Leibler divergence, the proposed method demonstrated excellent performance in extracting in situ information regarding aspect ratios and crystal size distributions during the crystallization process. Moreover, its ability to leverage expert labels with a four-fold enhanced efficiency holds great potential for advancing various applications in crystallization processes.

Validation of diagnostic wearable ECG data from high-risk patients and integration into the atrial fibrillation catheter ablation pathway

Abstract Background Wearables like the Apple Watch (AW) offer non-invasive, long-term monitoring and patient-led, single-lead ECG recordings. Although validated for Atrial Fibrillation (AF) detection in the general population, the accuracy and clinical utility of the single-lead ECG recording feature for the diagnosis of AF has not been evaluated in patients at high risk of atrial arrhythmias.(1) An evaluation of the accuracy of the Wearable-derived labels is important to determine the feasibility of integrating AW-based remote monitoring into clinical pathways. Objective To evaluate the accuracy and data burden from AW-based, patient-led ECG monitoring in patients undergoing AF catheter ablation. Methods Patients referred for first-time AF catheter ablation to the Study Institution between January 2022 and March 2023 were invited to enrol in the AF Follow-up with Apple Watch (AFFU-AW) Trial. Patients randomised to the active arm of the study received 12 months of post-ablation follow-up using a loaned Apple Watch. Participants underwent an education session and were asked to perform daily ECGs; sending through any labelled as AF, inconclusive, or in the context of symptoms. The accuracy of the automated ECG labelling was determined against blinded, independent labelling by two experienced Electrophysiologists. Results 1093 remote transmissions were emailed in for evaluation from 52 patients to the Study team. This was a median of 6 [2, 17] per patient and a receiver burden of 2 [1, 4] ECGs per day. 475 were categorised by the AW as AF with a sensitivity of 0.71 and specificity of 0.96 compared to expert labelling. 342 (31%) were not categorised by the wearable, with an ‘inconclusive’ interpretation (161 [47%]) or ‘high heart rate’ (108 [32%]) as the most cited reasons. 178 (55%) of the uncategorised ECGs were subsequently overread as AF. Accompanying symptoms were reported with 324 (30%) ECGs, most commonly palpitations. Conclusion This is the first reported evaluation of the AF diagnostic feature based on AW ECG recordings as part of a remote patient rhythm surveillance pathway after catheter ablation. The high positive predictive value of ‘Sinus Rhythm’ and ‘AF’ labels and the high specificity of the ‘AF’ label are comparable to diagnostic algorithms in implantable loop recorders.(2) The source code to automate data curation and structured databasing of wearable ECGs has been made available for care providers to integrate the service into device monitoring workflows.(3)

Point-supervised Brain Tumor Segmentation with Box-prompted Medical Segment Anything Model.

Delineating lesions and anatomical structure is important for image-guided interventions. Point-supervised medical image segmentation (PSS) has great potential to alleviate costly expert delineation labeling. However, due to the lack of precise size and boundary guidance, the effectiveness of PSS often falls short of expectations. Although recent vision foundational models, such as the medical segment anything model (MedSAM), have made significant advancements in bounding-box-prompted segmentation, it is not straightforward to utilize point annotation, and is prone to semantic ambiguity. In this preliminary study, we introduce an iterative framework to facilitate semantic-aware point-supervised MedSAM. Specifically, the semantic box-prompt generator (SBPG) module has the capacity to convert the point input into potential pseudo bounding box suggestions, which are explicitly refined by the prototype-based semantic similarity. This is then succeeded by a prompt-guided spatial refinement (PGSR) module that harnesses the exceptional generalizability of MedSAM to infer the segmentation mask, which also updates the box proposal seed in SBPG. Performance can be progressively improved with adequate iterations. We conducted an evaluation on BraTS2018 for the segmentation of whole brain tumors and demonstrated its superior performance compared to traditional PSS methods and on par with box-supervised methods.

A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts

ABSTRACTDeep‐learning (DL) models have become increasingly beneficial for the detection of retrogressive thaw slumps (RTS) in the permafrost domain. However, comparing accuracy metrics is challenging due to unstandardized labeling guidelines. To address this, we conducted an experiment with 12 international domain experts from a broad range of scientific backgrounds. Using 3 m PlanetScope multispectral imagery, they digitized RTS footprints in two sites. We evaluated label uncertainty by comparing manually outlined RTS labels using Intersection‐over‐Union (IoU) and F1 metrics. At the Canadian Peel Plateau site, we see good agreement, particularly in the active parts of RTS. Differences were observed in the interpretation of the debris tongue and the stable vegetated sections of RTS. At the Russian Bykovsky site, we observed a larger mismatch. Here, the same differences were documented, but several participants mistakenly identified non‐RTS features. This emphasizes the importance of site‐specific knowledge for reliable label creation. The experiment highlights the need for standardized labeling procedures and definition of their scientific purpose. The most similar expert labels outperformed the accuracy metrics reported in the literature, highlighting human labeling capabilities with proper training, site knowledge, and clear guidelines. These findings lay the groundwork for DL‐based RTS monitoring in the pan‐Arctic.

Evaluating Synthetic Data Generation from User Generated Text

Abstract User-generated content provides a rich resource to study social and behavioral phenomena. Although its application potential is currently limited by the paucity of expert labels and the privacy risks inherent in personal data, synthetic data can help mitigate this bottleneck. In this work, we introduce an evaluation framework to facilitate research on synthetic language data generation for user-generated text. We define a set of aspects for assessing data quality, namely style preservation, meaning preservation, and divergence, as a proxy for privacy.We introduce metrics corresponding to each aspect. Moreover, through a set of generation strategies and representative tasks and baselines across domains, we demonstrate the relation between the quality aspects of synthetic user generated content, generation strategies, metrics and downstream performance. To our knowledge, our work is the first unified evaluation framework for user-generated text in relation to the specified aspects, offering both intrinsic and extrinsic evaluation. We envisage it will facilitate developments towards shareable, high-quality synthetic language data.

Expert label for explainable fault diagnosis and for unknown fault generalization

This study proposes the expert label that incorporates specific process knowledge into sample labels to enhance the explainability of diagnostic results. Traditional sample label is a one-hot encoding vector without inherent physical meaning, limiting the diagnostic scope to predefined fault types without explainability. In contrast, expert label as multi-label provides multi-dimensional information including disturbance type, unit location, and associated variable type. Since multi-label can accommodate an exponential number of fault types, the diagnosis models based on expert label have the potential to generalize unknown faults. Meanwhile, we develop the process multi-scale feature mixture network (PMFMN), which excels in deep feature extraction through modules including multi-scale decomposition, multiple pattern representation, and Fourier attention mechanism. The effectiveness of PMFMN with expert label is validated through case studies in two chemical processes, which further improves the fault diagnosis rate, provides reliable diagnostic results with explainability, and can diagnose unknown faults.

The Human Cerebellum: A Digital Anatomical Atlas at the Level of Individual Folia.

Scientific interest in the cerebellum has surged in the last few decades with an emerging consensus on a multifaceted functionality and intricate, but not yet fully understood, functional topography over the cerebellar cortex. To further refine this structure-function relationship and quantify its inter-subject variability, a high-resolution digital anatomical atlas is fundamental. Using a combination of manual labeling and image processing, we turned a recently published reconstruction of the human cerebellum, the first such reconstruction fine enough to resolve the individual folia, into a digital atlas with both surface and volumetric representations. Its unprecedented granularity (0.16 mm) and detailed expert labeling make the atlas usable as an anatomical ground truth, enabling new ways of analyzing and visualizing cerebellar data through its digital format.

Expert-Level Detection of Referable Glaucoma from Fundus Photographs in a Safety Net Population: The AI and Teleophthalmology in Los Angeles Initiative.

To develop and test a deep learning (DL) algorithm for detecting referable glaucoma in the Los Angeles County (LAC) Department of Health Services (DHS) teleretinal screening program. Fundus photographs and patient-level labels of referable glaucoma (defined as cup-to-disc ratio [CDR] ≥ 0.6) provided by 21 trained optometrist graders were obtained from the LAC DHS teleretinal screening program. A DL algorithm based on the VGG-19 architecture was trained using patient-level labels generalized to images from both eyes. Area under the receiver operating curve (AUC), sensitivity, and specificity were calculated to assess algorithm performance using an independent test set that was also graded by 13 clinicians with one to 15 years of experience. Algorithm performance was tested using reference labels provided by either LAC DHS optometrists or an expert panel of 3 glaucoma specialists. 12,098 images from 5,616 patients (2,086 referable glaucoma, 3,530 non-glaucoma) were used to train the DL algorithm. In this dataset, mean age was 56.8 ± 10.5 years with 54.8% females and 68.2% Latinos, 8.9% Blacks, 2.7% Caucasians, and 6.0% Asians. 1,000 images from 500 patients (250 referable glaucoma, 250 non-glaucoma) with similar demographics (p ≥ 0.57) were used to test the DL algorithm. Algorithm performance matched or exceeded that of all independent clinician graders in detecting patient-level referable glaucoma based on LAC DHS optometrist (AUC = 0.92) or expert panel (AUC = 0.93) reference labels. Clinician grader sensitivity (range: 0.33-0.99) and specificity (range: 0.68-0.98) ranged widely and did not correlate with years of experience (p ≥ 0.49). Algorithm performance (AUC = 0.93) also matched or exceeded the sensitivity (range: 0.78-1.00) and specificity (range: 0.32-0.87) of 6 LAC DHS optometrists in the subsets of the test dataset they graded based on expert panel reference labels. A DL algorithm for detecting referable glaucoma developed using patient-level data provided by trained LAC DHS optometrists approximates or exceeds performance by ophthalmologists and optometrists, who exhibit variable sensitivity and specificity unrelated to experience level. Implementation of this algorithm in screening workflows could help reallocate eye care resources and provide more reproducible and timely glaucoma care.

Constructing a Clinical Patient Similarity Network of Gastric Cancer.

Clinical molecular genetic testing and molecular imaging dramatically increase the quantity of clinical data. Combined with the extensive application of electronic health records, a medical data ecosystem is forming, which calls for big-data-based medicine models. We tried to use big data analytics to search for similar patients in a cancer cohort, showing how to apply artificial intelligence (AI) algorithms to clinical data processing to obtain clinically significant results, with the ultimate goal of improving healthcare management. In order to overcome the weaknesses of most data processing algorithms that rely on expert labeling and annotation, we uniformly adopted one-hot encoding for all types of clinical data, calculating the Euclidean distance to measure patient similarity and subgrouping via an unsupervised learning model. Overall survival (OS) was investigated to assess the clinical validity and clinical relevance of the model. We took gastric cancers (GCs) as an example to build a high-dimensional clinical patient similarity network (cPSN). When performing the survival analysis, we found that Cluster_2 had the longest survival rates, while Cluster_5 had the worst prognosis among all the subgroups. As patients in the same subgroup share some clinical characteristics, the clinical feature analysis found that Cluster_2 harbored more lower distal GCs than upper proximal GCs, shedding light on the debates. Overall, we constructed a cancer-specific cPSN with excellent interpretability and clinical significance, which would recapitulate patient similarity in the real-world. The constructed cPSN model is scalable, generalizable, and performs well for various data types.

Two-stage deep neural network for diagnosing fungal keratitis via in vivo confocal microscopy images

Timely and effective diagnosis of fungal keratitis (FK) is necessary for suitable treatment and avoiding irreversible vision loss for patients. In vivo confocal microscopy (IVCM) has been widely adopted to guide the FK diagnosis. We present a deep learning framework for diagnosing fungal keratitis using IVCM images to assist ophthalmologists. Inspired by the real diagnostic process, our method employs a two-stage deep architecture for diagnostic predictions based on both image-level and sequence-level information. To the best of our knowledge, we collected the largest dataset with 96,632 IVCM images in total with expert labeling to train and evaluate our method. The specificity and sensitivity of our method in diagnosing FK on the unseen test set achieved 96.65% and 97.57%, comparable or better than experienced ophthalmologists. The network can provide image-level, sequence-level and patient-level diagnostic suggestions to physicians. The results show great promise for assisting ophthalmologists in FK diagnosis.

Clustering with Minimum Spanning Trees: How Good Can It Be?

AbstractMinimum spanning trees (MSTs) provide a convenient representation of datasets in numerous pattern recognition activities. Moreover, they are relatively fast to compute. In this paper, we quantify the extent to which they are meaningful in low-dimensional partitional data clustering tasks. By identifying the upper bounds for the agreement between the best (oracle) algorithm and the expert labels from a large battery of benchmark data, we discover that MST methods can be very competitive. Next, we review, study, extend, and generalise a few existing, state-of-the-art MST-based partitioning schemes. This leads to some new noteworthy approaches. Overall, the Genie and the information-theoretic methods often outperform the non-MST algorithms such as K-means, Gaussian mixtures, spectral clustering, Birch, density-based, and classical hierarchical agglomerative procedures. Nevertheless, we identify that there is still some room for improvement, and thus the development of novel algorithms is encouraged.

Efficient deep learning-based automated diagnosis from echocardiography with contrastive self-supervised learning

BackgroundAdvances in self-supervised learning (SSL) have enabled state-of-the-art automated medical image diagnosis from small, labeled datasets. This label efficiency is often desirable, given the difficulty of obtaining expert labels for medical image recognition tasks. However, most efforts toward SSL in medical imaging are not adapted to video-based modalities, such as echocardiography.MethodsWe developed a self-supervised contrastive learning approach, EchoCLR, for echocardiogram videos with the goal of learning strong representations for efficient fine-tuning on downstream cardiac disease diagnosis. EchoCLR pretraining involves (i) contrastive learning, where the model is trained to identify distinct videos of the same patient, and (ii) frame reordering, where the model is trained to predict the correct of video frames after being randomly shuffled.ResultsWhen fine-tuned on small portions of labeled data, EchoCLR pretraining significantly improves classification performance for left ventricular hypertrophy (LVH) and aortic stenosis (AS) over other transfer learning and SSL approaches across internal and external test sets. When fine-tuning on 10% of available training data (519 studies), an EchoCLR-pretrained model achieves 0.72 AUROC (95% CI: [0.69, 0.75]) on LVH classification, compared to 0.61 AUROC (95% CI: [0.57, 0.64]) with a standard transfer learning approach. Similarly, using 1% of available training data (53 studies), EchoCLR pretraining achieves 0.82 AUROC (95% CI: [0.79, 0.84]) on severe AS classification, compared to 0.61 AUROC (95% CI: [0.58, 0.65]) with transfer learning.ConclusionsEchoCLR is unique in its ability to learn representations of echocardiogram videos and demonstrates that SSL can enable label-efficient disease classification from small amounts of labeled data.

Contrastive Self-supervised Learning for Neurodegenerative Disorder Classification.

Neurodegenerative diseases such as Alzheimer's disease (AD) or frontotemporal lobar degeneration (FTLD) involve specific loss of brain volume, detectable in vivo using T1-weighted MRI scans. Supervised machine learning approaches classifying neurodegenerative diseases require diagnostic-labels for each sample. However, it can be difficult to obtain expert labels for a large amount of data. Self-supervised learning (SSL) offers an alternative for training machine learning models without data-labels. We investigated if the SSL models can applied to distinguish between different neurodegenerative disorders in an interpretable manner. Our method comprises a feature extractor and a downstream classification head. A deep convolutional neural network trained in a contrastive self-supervised way serves as the feature extractor, learning latent representation, while the classifier head is a single-layer perceptron. We used N=2694 T1-weighted MRI scans from four data cohorts: two ADNI datasets, AIBL and FTLDNI, including cognitively normal controls (CN), cases with prodromal and clinical AD, as well as FTLD cases differentiated into its sub-types. Our results showed that the feature extractor trained in a self-supervised way provides generalizable and robust representations for the downstream classification. For AD vs. CN, our model achieves 82% balanced accuracy on the test subset and 80% on an independent holdout dataset. Similarly, the Behavioral variant of frontotemporal dementia (BV) vs. CN model attains an 88% balanced accuracy on the test subset. The average feature attribution heatmaps obtained by the Integrated Gradient method highlighted hallmark regions, i.e., temporal gray matter atrophy for AD, and insular atrophy for BV. In conclusion, our models perform comparably to state-of-the-art supervised deep learning approaches. This suggests that the SSL methodology can successfully make use of unannotated neuroimaging datasets as training data while remaining robust and interpretable.

Understanding the impact of climate change on critical infrastructure through nlp analysis of scientific literature

ABSTRACT Climate change is intensifying natural hazards, putting critical infrastructure systems at risk. The effects of climate change on critical infrastructure can be significant, and communities need to consider these risks when planning and designing infrastructure systems for the future. To that end, natural language processing (NLP) is a promising approach for analyzing large volumes of climate change and infrastructure-related scientific literature. To train a supervised model using NLP techniques, a significant subset of the corpus must be labeled into categories based on user-defined criteria, which is a time-consuming process. To expedite this process, we developed a weak supervision-based approach that leverages semantic similarity between categories and documents to generate category labels for the domain-specific corpus. In comparison with a months-long process of subject-matter expert labeling, we assign category labels to the whole corpus using weak supervision and supervised learning in 13 hours.

A comprehensive methodology to construct standardised datasets for Science and Technology Parks

This work presents a standardised approach to create datasets for Science and Technology Parks (STPs), facilitating future analysis of STP characteristics, trends and performance. STPs are the most representative examples of innovation ecosystems. The ETL (extraction-transformation-load) structure was adapted to a global field study of STPs. A selection stage and quality check were incorporated, and the methodology was applied to Spanish STPs. This study applies diverse techniques such as expert labelling and information extraction which uses language technologies. A novel methodology for building quality and standardised STP datasets was designed and applied to a Spanish STP case study with 49 STPs. An updatable dataset and a list of the main features impacting STPs are presented. Twenty-one (n = 21) core features were refined and selected, with fifteen of them (71.4 %) being robust enough for developing further quality analysis. The methodology presented integrates different sources with heterogeneous information that is often decentralised, disaggregated and in different formats: excel files, and unstructured information in HTML or PDF format. The existence of this updatable dataset and the defined methodology will enable powerful AI tools to be applied that focus on more sophisticated analysis, such as taxonomy, monitoring, and predictive and prescriptive analytics in the innovation ecosystems field.

Use of an Artificial Intelligence-Generated Vascular Severity Score Improved Plus Disease Diagnosis in Retinopathy of Prematurity

To evaluate whether providing clinicians with an artificial intelligence (AI)-based vascular severity score (VSS) improves consistency in the diagnosis of plus disease in retinopathy of prematurity (ROP). Multireader diagnostic accuracy imaging study. Eleven ROP experts, 9 of whom had been in practice for 10 years or more. RetCam (Natus Medical Incorporated) fundus images were obtained from premature infants during routine ROP screening as part of the Imaging and Informatics in ROP study between January 2012 and July 2020. From all available examinations, a subset of 150 eye examinations from 110 infants were selected for grading. An AI-based VSS was assigned to each set of images using the i-ROP DL system (Siloam Vision). The clinicians were asked to diagnose plus disease for each examination and to assign an estimated VSS (range, 1-9) at baseline, and then again 1 month later with AI-based VSS assistance. A reference standard diagnosis (RSD) was assigned to each eye examination from the Imaging and Informatics in ROP study based on 3 masked expert labels and the ophthalmoscopic diagnosis. Mean linearly weighted κ value for plus disease diagnosis compared with RSD. Area under the receiver operating characteristic curve (AUC) and area under the precision-recall curve (AUPR) for labels 1 through 9 compared with RSD for plus disease. Expert agreement improved significantly, from substantial (κ value, 0.69 [0.59, 0.75]) to near perfect (κ value, 0.81 [0.71, 0.86]), when AI-based VSS was integrated. Additionally, a significant improvement in plus disease discrimination was achieved as measured by mean AUC (from 0.94 [95% confidence interval (CI), 0.92-0.96] to 0.98 [95% CI, 0.96-0.99]; difference, 0.04 [95% CI, 0.01-0.06]) and AUPR (from 0.86 [95% CI, 0.81-0.90] to 0.95 [95% CI, 0.91-0.97]; difference, 0.09 [95% CI, 0.03-0.14]). Providing ROP clinicians with an AI-based measurement of vascular severity in ROP was associated with both improved plus disease diagnosis and improved continuous severity labeling as compared with an RSD for plus disease. If implemented in practice, AI-based VSS could reduce interobserver variability and could standardize treatment for infants with ROP. Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Deep learning and machine learning classification technique for integrated forecasting

Smart fisheries are increasingly using artificial intelligence (AI) technologies to increase their sustainability. The potential fishing zone (PFZ) forecasts several fish aggregation zones throughout the duration of the prediction in any sea. The autoregressive integrated moving average (ARIMA) and random forest model are used in the current study to provide a technique for locating viable fishing zones in deep marine seas. A significant amount of data was gathered for the database's creation, including monitoring information for Indian fishing fleets from 2017 to 2019. Using expert label datasets for validation, it was discovered that the model's detection accuracy was 98%. Our method uses salinity and dissolved oxygen, two crucial markers of water quality, to identify suitable fishing zones for the first time. In the current research, a system was created to identify and map the quantity of fishing activity. The tests use a number of parameter measurements to evaluate the contrast-enhanced computed tomography (CECT) approach to machine learning (ML) and deep learning (DL) methodologies. The findings showed that the CECT had a 94% accuracy rate compared to a convolutional neural network's 92% accuracy rate for the 80% training data and 20% testing data.

Improving algorithmic trading consistency via human alignment and imitation learning

Research on algorithmic trading using reinforcement learning has become increasingly popular in recent years. Although most of the current reinforcement learning methods are employed to train the agent for some kind of modeling or data problem, it is worthwhile to explore in aligning agents with human behavior in applications as crucial as financial trading. Achieving such consistency by incorporating human expert experience into agent behavior is a key for potential improvements in this field. Imitation learning learns directly from examples of humans or other agents performing tasks. However, using imitation learning alone suffers from the problem of transitionally fitting expert example data. By combining the advantages of imitation learning and the Advantage Actor–Critic method, the Human Alignment Advantage Actor–Critic (HA3C) algorithm is proposed, to enhance single-asset trading strategy. First, by adding daily and weekly frequency trading data as input features to TimesNet, which is specifically designed to extract correlated temporal patterns from time-series data, it can capture both short-term and long-term features, thus capturing time-series features more comprehensively. Second, an expert action labeling method is proposed to train a strategy prediction network through supervised learning of behavior imitation. Third, a pre-trained strategy network is transferred to balance the exploration and exploitation of the agent’s behavior. Imitation learning techniques leverage finance-specific knowledge to enhance algorithmic trading consistency. This approach enables algorithms to mimic and adapt human decision-making patterns in finance, ultimately improving overall performance. This paper introduces a novel return-based function that efficiently balances short-term and long-term returns over flexible time horizons. It considers the maximum return from different positions and uses flexible time windows to capture trends while maximizing returns. Finally, evaluation on six commonly used datasets, such as DJI and SP500, demonstrates the advantages of the proposed HA3C algorithm compared with other classical and reinforcement learning-based strategies. Notably, on the HSI dataset, the HA3C strategy significantly outperforms other methods, achieving an impressive cumulative return of 681.55% and a Sharpe ratio of 5.07. These results show the superior performance of the HA3C algorithm in enhancing stock trading strategies and its potential to impact algorithmic trading consistency through aligning agent behavior with human expertise.

Enhancing Semi-Supervised Few-Shot Hyperspectral Image Classification via Progressive Sample Selection

Hyperspectral images (HSIs) provide valuable spatial–spectral information for ground analysis. However, in few-shot (FS) scenarios, the limited availability of training samples poses significant challenges in capturing the sample distribution under diverse environmental conditions. Semi-supervised learning has shown promise in exploring the distribution of unlabeled samples through pseudo-labels. Nonetheless, FS HSI classification encounters the issue of high intra-class spectral variability and inter-class spectral similarity, which often lead to the diffusion of unreliable pseudo-labels during the iterative process. In this paper, we propose a simple yet effective progressive pseudo-label selection strategy that leverages the spatial–spectral consistency of HSI pixel samples. By leveraging spatially aligned ground materials as connected regions with the same semantic and similar spectrum, pseudo-labeled samples were selected based on round-wise confidence scores. Samples within both spatially and semantically connected regions of FS samples were assigned pseudo-labels and joined subsequent training rounds. Moreover, considering the spatial positions of FS samples that may appear in diverse patterns, to fully utilize unlabeled samples that fall outside the neighborhood of FS samples but still belong to certain connected regions, we designed a matching active learning approach for expert annotation based on the temporal confidence difference. We identified samples with the highest training value in specific regions, utilizing the consistency between predictive labels and expert labels to decide whether to include the region or the sample itself in the subsequent semi-supervised iteration. Experiments on both classic and more recent HSI datasets demonstrated that the proposed base model achieved SOTA performance even with extremely rare labeled samples. Moreover, the extended version with active learning further enhances performance by involving limited additional annotation.

Lessons Learned in Building Expertly Annotated Multi-Institution Datasets and Hosting the RSNA AI Challenges.

The Radiological Society of North America (RSNA) has held artificial intelligence competitions to tackle real-world medical imaging problems at least annually since 2017. This article examines the challenges and processes involved in organizing these competitions, with a specific emphasis on the creation and curation of high-quality datasets. The collection of diverse and representative medical imaging data involves dealing with issues of patient privacy and data security. Furthermore, ensuring quality and consistency in data, which includes expert labeling and accounting for various patient and imaging characteristics, necessitates substantial planning and resources. Overcoming these obstacles requires meticulous project management and adherence to strict timelines. The article also highlights the potential of crowdsourced annotation to progress medical imaging research. Through the RSNA competitions, an effective global engagement has been realized, resulting in innovative solutions to complex medical imaging problems, thus potentially transforming health care by enhancing diagnostic accuracy and patient outcomes. Keywords: Use of AI in Education, Artificial Intelligence © RSNA, 2024.