Annotation Process Research Articles

Design, creation, and use of the Test Us Bank (TUB) COVID-19 sample biorepository.

Shortly after the first case of SARS-CoV-2 was diagnosed a public health emergency (PHE) was declared and a multi-agency response was initiated within the US federal government to create and propagate testing capacity. As part of this response, an unprecedented program designated Rapid Acceleration of Diagnostics (RADx) Tech was established by the National Institutes of Health (NIH) to facilitate the development of point-of-care tests for the COVID-19. The RADx Tech Clinical Studies Core (CSC), located at the University of Massachusetts Chan Medical School (UMass Chan), with partnering academic, private, and non-governmental organizations around the country, was tasked with developing clinical studies to support this work. This manuscript details development of a biorepository specifically focused on the collection and storage of samples designed for diagnostic platform development. It highlights the unified collection and annotation process that enabled gathering a diverse set of samples. This diversity encompasses the geography and backgrounds of the participants as well as sample characteristics such as variant type and RT-PCR cycle threshold (CT) value of the corresponding reference sample on a uniform clinical reference platform.

Deciphering disagreement in the annotation of EU legislation

AbstractThe topic of annotating legal data has received surprisingly little attention. A key challenge of the annotation process is reaching a sufficient agreement between annotators and filtering mistakes from genuine disagreement. This study presents an approach that provides insights into and resolves potential disagreement amongst annotators. It (1) introduces different strategies to calculate agreement levels and compares (2) agreement levels between annotators (inter-annotator agreement) before and after a revision round and (3) agreement levels for annotators who annotate the same texts twice (intra-annotator agreement). The inter-annotator agreement levels are compared to a revision round in which an arbiter corrected the annotator’s labels. The analysis is based on the annotation of EU legislative provisions at two stages (initial annotations, after annotator revisions) and for various tasks (Definitions, References, Quantities, IF-THEN statements, Exceptions, Scope, Hierarchy, Deontic Clauses, Active and Passive Role) by multiple annotators. The results reveal that agreement levels vary based on the stage of measurement (before/after revisions), the nature of the task, the method of assessment, and the annotator combination. The agreement scores - along with some initial measurements—align with those reported in previous research but increase after each revision round. This suggests that annotator revisions can substantially reduce disagreement. Additionally, disagreements were found not only between but also among annotators. This inconsistency does not appear to stem from a lack of understanding of the guidelines or a lack of seriousness in task execution, as evidenced by moderate to substantial inter-annotator agreement scores. These findings suggest that annotators identified multiple valid interpretations, which highlights the complexity of annotating legislative provisions. The results underscore the significance of embracing, addressing, and reporting about (dis)agreement in different ways and at the various stages of an annotation task.

NCBI RefSeq: reference sequence standards through 25years of curation and annotation.

Reference sequences and annotations serve as the foundation for many lines of research today, from organism and sequence identification to providing a core description of the genes, transcripts and proteins found in an organism's genome. Interpretation of data including transcriptomics, proteomics, sequence variation and comparative analyses based on reference gene annotations informs our understanding of gene function and possible disease mechanisms, leading to new biomedical discoveries. The Reference Sequence (RefSeq) resource created at the National Center for Biotechnology Information (NCBI) leverages both automatic processes and expert curation to create a robust set of reference sequences of genomic, transcript and protein data spanning the tree of life. RefSeq continues to refine its annotation and quality control processes and utilize better quality genomes resulting from advances in sequencing technologies as well as RNA-Seq data to produce high-quality annotated genomes, ortholog predictions across more organisms and other products that are easily accessible through multiple NCBI resources. This report summarizes the current status of the eukaryotic, prokaryotic and viral RefSeq resources, with a focus on eukaryotic annotation, the increase in taxonomic representation and the effect it will have on comparative genomics. The RefSeq resource is publicly accessible at https://www.ncbi.nlm.nih.gov/refseq.

Constructing a Classification Scheme - and its Consequences: A Field Study of Learning to Label Data for Computer Vision in a Hospital Intensive Care Unit

Research on data annotation for artificial intelligence (AI) has demonstrated that biases, power, and culture impact the ways that annotators apply labels to data and subsequently affect downstream AI systems. However, annotators can only apply labels that are available to them in the annotation classification scheme. Drawing on a 3-year ethnographic study of an R&D collaboration between medical and AI researchers, we argue that the construction of the classification schema itself -- decisions about what kinds of data can and cannot be collected, what activities can and cannot be detected in the data, what the possible annotation classes ought to be, and the rules by which an item ought to be classified into each class -- dramatically shape the annotation process, and through it, the AI. We draw on Bowker and Star's [9] classification theory to detail how the creation of a training data codebook for a computer vision algorithm in hospital intensive care units (ICUs) evolved from its original, clinically-driven goal of classifying complex clinical activities into a narrower goal of identifying physical objects and simpler activities in the ICU. This work reinforces how trade-offs and decisions made long before annotators begin labeling data are highly consequential to the resulting AI system.

Metapragmatic markers and the instantiation of pragmatic frames: A cognitive-linguistic approach to the problem of current discourse

Abstract Reference to current discourse is often cited as a definitional criterion for classifying linguistic units as metapragmatic. However, there is no consensus in research on the precise meaning and limits of this criterion. In this paper, a cognitive-linguistic perspective is adopted to address this issue by focusing on how interactants conceptualize discourse. A synthesis of Langacker’s (2001) current-discourse-space model (CDS model) and the notion of pragmatic frames is used to analyze metapragmatic practices in a television discussion. The analytical annotation process shows that metapragmatic markers (MPMs) evoke specific kinds of frame structures including the instantiation of pragmatic frames. Based on the annotation of the entire transcribed discussion, it will be argued that the visualization of discourse sequences in the CDS model and the examination of the specific frame structures evoked by MPMs contribute to a better understanding of the core of metapragmaticity. The paper concludes with a pilot study that uses quantitative methods to demonstrate the analytical potential of a cognition-centered categorization of MPMs. It is shown that the use of MPMs reflects different interactional roles (i.e., host or discussant) and the conversational strategies used to shape the conceptualization of argumentative discourse to pursue specific interactional goals.

Deep learning for surgical instrument recognition and segmentation in robotic-assisted surgeries: a systematic review

Applying deep learning (DL) for annotating surgical instruments in robot-assisted minimally invasive surgeries (MIS) represents a significant advancement in surgical technology. This systematic review examines 48 studies that utilize advanced DL methods and architectures. These sophisticated DL models have shown notable improvements in the precision and efficiency of detecting and segmenting surgical tools. The enhanced capabilities of these models support various clinical applications, including real-time intraoperative guidance, comprehensive postoperative evaluations, and objective assessments of surgical skills. By accurately identifying and segmenting surgical instruments in video data, DL models provide detailed feedback to surgeons, thereby improving surgical outcomes and reducing complication risks. Furthermore, the application of DL in surgical education is transformative. The review underscores the significant impact of DL on improving the accuracy of skill assessments and the overall quality of surgical training programs. However, implementing DL in surgical tool detection and segmentation faces challenges, such as the need for large, accurately annotated datasets to train these models effectively. The manual annotation process is labor-intensive and time-consuming, posing a significant bottleneck. Future research should focus on automating the detection and segmentation process and enhancing the robustness of DL models against environmental variations. Expanding the application of DL models across various surgical specialties will be essential to fully realize this technology’s potential. Integrating DL with other emerging technologies, such as augmented reality (AR), also offers promising opportunities to further enhance the precision and efficacy of surgical procedures.

Can GPT-4 learn to analyse moves in research article abstracts?

Abstract One of the most powerful and enduring ideas in written discourse analysis is that genres can be described in terms of the moves which structure a writer’s purpose. Considerable research has sought to identify these distinct communicative acts, but analyses have been beset by problems of subjectivity, reliability, and the time-consuming need for multiple coders to confirm analyses. In this article, we employ the affordances of Generative Pre-trained Transformer 4 (GPT-4) to automate the annotation process by using natural language prompts. Focusing on abstracts from articles in four applied linguistics journals, we devise prompts which enable the model to identify moves effectively. The annotated outputs of these prompts were evaluated by two assessors with a third addressing disagreements. The results show that an eight-shot prompt was more effective than one using two, confirming that the inclusion of examples illustrating areas of variability can enhance GPT-4’s ability to recognize multiple moves in a single sentence and reduce bias related to textual position. We suggest that GPT-4 offers considerable potential in automating this annotation process, when human actors with domain-specific linguistic expertise inform the prompting process.

Information mismatch in PHH3-assisted mitosis annotation leads to interpretation shifts in H&E slide analysis

The count of mitotic figures (MFs) observed in hematoxylin and eosin (H&E)-stained slides is an important prognostic marker, as it is a measure for tumor cell proliferation. However, the identification of MFs has a known low inter-rater agreement. In a computer-aided setting, deep learning algorithms can help to mitigate this, but they require large amounts of annotated data for training and validation. Furthermore, label noise introduced during the annotation process may impede the algorithms’ performance. Unlike H&E, where identification of MFs is based mainly on morphological features, the mitosis-specific antibody phospho-histone H3 (PHH3) specifically highlights MFs. Counting MFs on slides stained against PHH3 leads to higher agreement among raters and has therefore recently been used as a ground truth for the annotation of MFs in H&E. However, as PHH3 facilitates the recognition of cells indistinguishable from H&E staining alone, the use of this ground truth could potentially introduce an interpretation shift and even label noise into the H&E-related dataset, impacting model performance. This study analyzes the impact of PHH3-assisted MF annotation on inter-rater reliability and object level agreement through an extensive multi-rater experiment. Subsequently, MF detectors, including a novel dual-stain detector, were evaluated on the resulting datasets to investigate the influence of PHH3-assisted labeling on the models’ performance. We found that the annotators’ object-level agreement significantly increased when using PHH3-assisted labeling (F1: 0.53 to 0.74). However, this enhancement in label consistency did not translate to improved performance for H&E-based detectors, neither during the training phase nor the evaluation phase. Conversely, the dual-stain detector was able to benefit from the higher consistency. This reveals an information mismatch between the H&E and PHH3-stained images as the cause of this effect, which renders PHH3-assisted annotations not well-aligned for use with H&E-based detectors. Based on our findings, we propose an improved PHH3-assisted labeling procedure.

Enhancing emerging technology discovery in nanomedicine by integrating innovative sentences using BERT and NLDA

Abstract Purpose Nanomedicine has significant potential to revolutionize biomedicine and healthcare through innovations in diagnostics, therapeutics, and regenerative medicine. This study aims to develop a novel framework that integrates advanced natural language processing, noise-free topic modeling, and multidimensional bibliometrics to systematically identify emerging nanomedicine technology topics from scientific literature. Design/methodology/approach The framework involves collecting full-text articles from PubMed Central and nanomedicine-related metrics from the Web of Science for the period 2013–2023. A fine-tuned BERT model is employed to extract key informative sentences. Noiseless Latent Dirichlet Allocation (NLDA) is applied to model interpretable topics from the cleaned corpus. Additionally, we develop and apply metrics for novelty, innovation, growth, impact, and intensity to quantify the emergence of novel technological topics. Findings By applying this methodology to nanomedical publications, we identify an increasing emphasis on research aligned with global health priorities, particularly inflammation and biomaterial interactions in disease research. This methodology provides deeper insights through full-text analysis and leading to a more robust discovery of emerging technologies. Research limitations One limitation of this study is its reliance on the existing scientific literature, which may introduce publication biases and language constraints. Additionally, manual annotation of the dataset, while thorough, is subject to subjectivity and can be time-consuming. Future research could address these limitations by incorporating more diverse data sources, and automating the annotation process. Practical implications The methodology presented can be adapted to explore emerging technologies in other scientific domains. It allows for tailored assessment criteria based on specific contexts and objectives, enabling more precise analysis and decision-making in various fields. Originality/value This study offers a comprehensive framework for identifying emerging technologies in nanomedicine, combining theoretical insights and practical applications. Its potential for adaptation across scientific disciplines enhances its value for future research and decision-making in technology discovery.

RepeatsDB in 2025: expanding annotations of structured tandem repeats proteins on AlphaFoldDB.

RepeatsDB (URL: https://repeatsdb.org) stands as a key resource for the classification and annotation of Structured Tandem Repeat Proteins (STRPs), incorporating data from both the Protein Data Bank (PDB) and AlphaFoldDB. This latest release features substantial advancements, including annotations for over 34000 unique protein sequences from >2000 organisms, representing a fifteenfold increase in coverage. Leveraging state-of-the-art structural alignment tools, RepeatsDB now offers faster and more precise detection of STRPs across both experimental and predicted structures. Key improvements also include a redesigned user interface and enhanced web server, providing an intuitive browsing experience with improved data searchability and accessibility. A new statistics page allows users to explore database metrics based on repeat classifications, while API enhancements support scalability to manage the growing volume of data. These advancements not only refine the understanding of STRPs but also streamline annotation processes, further strengthening RepeatsDB's role in advancing our understanding of STRP functions.

Do gestures have meaning? An empirical analysis on the gestures performed by Fidel Castro at the United Nations

In everyday conversations, there are said to be unconscious meanings that we convey through our body. For example, if a story is being told, our hands move according to the message, following a rhythm and forming shapes that we are unaware of. According to McNeill (1992), gestures are a form of language different from speech, but even though they have been regarded as individual entities, they are, in fact, parallel and work in synch with each other. This is the scope of the analysis of gestures. In the following paper, the objective was to test whether there is an association between gestures and meaning. Consequently, the gestures of former Cuban president, Fidel Castro, were extracted and analyzed from his intervention at the United Nations in 1979. The process of annotation was carried out on the program ELAN, based on a series of categories: the type of gesture and topic, the position and movement of both arms and hands as well as the orientation. Additionally, a series of quantitative and qualitative analysis were performed through the statistical program R. A Chi-squared test showed a significant relationship between the type of gestures and the topics which was further corroborated through a conditional inference tree and a multiple correspondence analysis, reaffirming the idea that these are form-meaning pairs. Moreover, there was a strong connection between the type of gesture and arm movement, suggesting that this is due to the intention of the message while the orientation of the hand was a clear indicator of arm movement.

Keypoint-Based Bee Orientation Estimation and Ramp Detection at the Hive Entrance for Bee Behavior Identification System

This paper addresses the challenge of accurately estimating bee orientations on beehive landing boards, which is crucial for optimizing beekeeping practices and enhancing agricultural productivity. The research utilizes YOLOv8 pose models, trained on a dataset created using an open-source computer vision annotation tool. The annotation process involves associating bounding boxes with keypoints to represent bee orientations, with each bee annotated using two keypoints: one for the head and one for the stinger. The YOLOv8-pose models demonstrate high precision, achieving 98% accuracy for both bounding box and keypoint detection in 1024×576 px images. However, trade-offs between model size and processing speed are addressed, with the smaller nano model reaching 67 frames per second on 640×384 px images. The entrance ramp detection model achieves 91.7% intersection over union across four keypoints, making it effective for detecting the hive’s landing board. The paper concludes with plans for future research, including the behavioral analysis of bee colonies and model optimization for real-time applications.

Understanding complex casual leisure information needs: an analysis of search requests for books, games, movies and music

PurposeIn this paper, we introduce the CRISPS (CRoss-domaIn relevance aSPects Scheme) coding scheme for complex information needs in the four leisure domains of books, games, movies and music. It categorizes the relevance aspects people consider when searching for these resources. The coding scheme and findings help search engines to better support complex information needs, both by prioritizing which aspects are easier to classify automatically and by determining which information sources should be considered.Design/methodology/approachA cross-domain classification scheme for relevance aspects and information needs in casual leisure domains (CRISPS) is developed and applied. The paper provides the documentation of the scheme development and annotation process as well as a detailed, large-scale analysis of 2000 requests (500 per domain) and relevance aspects for four domains as expressed in complex search requests in everyday life information seeking posted to online forums.FindingsWe identify and discuss relevance aspect frequencies, information need types and the described search process of the requests. Furthermore, the coding scheme development and the annotation process are documented and reflected on.Originality/valueThis is the first categorization and analysis of complex information needs in these four leisure domains combined. The coding scheme and findings can be used to develop new types of search interfaces that incorporate the kinds of relevance aspects identified in the scheme, allowing to express complex needs in the form of structured queries.

Effectiveness and Genetic Control of Trichoderma spp. as a Biological Control of Wheat Powdery Mildew Disease.

Wheat powdery mildew (WPM) is one of the most devasting diseases that affects wheat yield worldwide. Few efforts have been made to control such a serious disease. An effective way to control WPM is urgently needed. Biological control is an effective way to control plant diseases worldwide. In this study, the efficiency of three different Trichoderma spp. in controlling WPM at the seedling growth stage was tested using 35 highly diverse wheat genotypes. Highly significant differences were found in WPM resistance among the four treatments, confirming the efficiency of Trichoderma in controlling WPM. Of the three species, T. asperellum T34 (T34) was the most effective species in controlling WPM, as it reduced the symptoms by 50.56%. A set of 196 wheat genotypes was used to identify the genetic control of the WPM resistance induced by T34. A total of 39, 27, and 18 gene models were identified to contain the significant markers under Pm, T34, and the improvement in powdery mildew resistance due to T34 (T34_improvement) conditions. Furthermore, no gene model was common between T34 and Pm, suggesting the presence of completely different genetic systems controlling the resistance under T34 and Pm. The functional annotation and biological process pathways of the detected gene models confirm their association with the normal and induced resistance. This study, for the first time, confirms the efficiency of T34 in controlling WPM and provides a deep understanding of the genetic control of induced and normal resistance to WPM.

A Low-Measurement-Cost-Based Multi-Strategy Hyperspectral Image Classification Scheme.

The cost of hyperspectral image (HSI) classification primarily stems from the annotation of image pixels. In real-world classification scenarios, the measurement and annotation process is both time-consuming and labor-intensive. Therefore, reducing the number of labeled pixels while maintaining classification accuracy is a key research focus in HSI classification. This paper introduces a multi-strategy triple network classifier (MSTNC) to address the issue of limited labeled data in HSI classification by improving learning strategies. First, we use the contrast learning strategy to design a lightweight triple network classifier (TNC) with low sample dependence. Due to the construction of triple sample pairs, the number of labeled samples can be increased, which is beneficial for extracting intra-class and inter-class features of pixels. Second, an active learning strategy is used to label the most valuable pixels, improving the quality of the labeled data. To address the difficulty of sampling effectively under extremely limited labeling budgets, we propose a new feature-mixed active learning (FMAL) method to query valuable samples. Fine-tuning is then used to help the MSTNC learn a more comprehensive feature distribution, reducing the model's dependence on accuracy when querying samples. Therefore, the sample quality is improved. Finally, we propose an innovative dual-threshold pseudo-active learning (DSPAL) strategy, filtering out pseudo-label samples with both high confidence and uncertainty. Extending the training set without increasing the labeling cost further improves the classification accuracy of the model. Extensive experiments are conducted on three benchmark HSI datasets. Across various labeling ratios, the MSTNC outperforms several state-of-the-art methods. In particular, under extreme small-sample conditions (five samples per class), the overall accuracy reaches 82.97% (IP), 87.94% (PU), and 86.57% (WHU).

Concordance in the estimation of tumor percentage in non-small cell lung cancer using digital pathology

The incorporation of digital pathology in clinical practice will require the training of pathologists in digital skills. Our study aimed to assess the reliability among pathologists in determining tumor percentage in whole slide images (WSI) of non-small cell lung cancer (NSCLC) using digital image analysis, and study how the results correlate with the molecular findings. Pathologists from nine centers were trained to quantify epithelial tumor cells, tumor-associated stromal cells, and non-neoplastic cells from NSCLC WSI using QuPath. Then, we conducted two consecutive ring trials. In the first trial, analyzing four WSI, reliability between pathologists in the assessment of tumor cell percentage was poor (intraclass correlation coefficient (ICC) 0.09). After performing the first ring trial pathologists received feedback. The second trial, comprising 10 WSI with paired next-generation sequencing results, also showed poor reliability (ICC 0.24). Cases near the recommended 20% visual threshold for molecular techniques exhibited higher values with digital analysis. In the second ring trial reliability slightly improved and human errors were reduced from 5.6% to 1.25%. Most discrepancies arose from subjective tasks, such as the annotation process, suggesting potential improvement with future artificial intelligence solutions.

DR-GPT: A large language model for medical report analysis of diabetic retinopathy patients.

Diabetic retinopathy (DR) is a sight-threatening condition caused by diabetes. Screening programmes for DR include eye examinations, where the patient's fundi are photographed, and the findings, including DR severity, are recorded in the medical report. However, statistical analyses based on DR severity require structured labels that calls for laborious manual annotation process if the report format is unstructured. In this work, we propose a large language model DR-GPT for classification of the DR severity from unstructured medical reports. On a clinical set of medical reports, DR-GPT reaches 0.975 quadratic weighted Cohen's kappa using truncated Early Treatment Diabetic Retinopathy Study scale. When DR-GPT annotations for unlabeled data are paired with corresponding fundus images, the additional data improves image classifier performance with statistical significance. Our analysis shows that large language models can be applied for unstructured medical report databases to classify diabetic retinopathy with a variety of applications.

Leveraging Large Language Model ChatGPT for enhanced understanding of end-user emotions in social media feedbacks

For software evolution, user feedback has become a meaningful way to improve applications. Recent studies show a significant increase in analyzing end-user feedback from various social media platforms for software evolution. However, less attention has been given to the end-user feedback for low-rating software applications. Also, such approaches are developed mainly on the understanding of human annotators who might have subconsciously tried for a second guess, questioning the validity of the methods. For this purpose, we proposed an approach that analyzes end-user feedback for low-rating applications to identify the end-user opinion types associated with negative reviews (an issue or bug). Also, we utilized Generative Artificial Intelligence (AI), i.e., ChatGPT, as an annotator and negotiator when preparing a truth set for the deep learning (DL) classifiers to identify end-user emotion. For the proposed approach, we first used the ChatGPT Application Programming Interface (API) to identify negative end-user feedback by processing 71853 reviews collected from 45 apps in the Amazon store. Next, a novel grounded theory is developed by manually processing end-user negative feedback to identify frequently associated emotion types, including anger, confusion, disgust, distrust, disappointment, fear, frustration, and sadness. Next, two datasets were developed, one with human annotators using a content analysis approach and the other using ChatGPT API with the identified emotion types. Next, another round is conducted with ChatGPT to negotiate over the conflicts with the human-annotated dataset, resulting in a conflict-free emotion detection dataset. Finally, various DL classifiers, including LSTM, BILSTM, CNN, RNN, GRU, BiGRU and BiRNN, are employed to identify their efficacy in detecting end-users emotions by preprocessing the input data, applying feature engineering, balancing the data set, and then training and testing them using a cross-validation approach. We obtained an average accuracy of 94%, 94%, 93%, 92%, 91%, 91%, and 85%, with LSTM, BILSTM, RNN, CNN, GRU, BiGRU and BiRNN, respectively, showing improved results with the truth set curated with human and ChatGPT. Using ChatGPT as an annotator and negotiator can help automate and validate the annotation process, resulting in better DL performances.

Accelerated Data Engine: A faster dataset construction workflow for computer vision applications in commercial livestock farms

Large-scale, high-quality dataset was the foundation of developing advanced artificial intelligence applications. However, creating such a benchmark dataset in a professional field, such as precision management of animals, was always a challenge because of the costly and labor-intensive process of annotation and review. This study introduced a novel workflow named Accelerated Data Engine (ADE), designed to efficiently produce representative and high-quality computer vision datasets from raw animal surveillance footage. By incorporating referring and grounding models (R&G models) as auto-annotators, along with a distillation mechanism for dataset-auditors, ADE significantly speeded up the dataset construction process. The new workflow received natural language inputs as referrals to identify animal instances, delineated their body shapes, and then refined the auto-annotated data through a selection process. To demonstrate the efficacy of ADE, three 30-minute surveillance video samples featuring pigs, sheep, and cattle were discussed in this study. The results indicated the R&G models effectively annotated animals across various farms, while distillation mechanisms could identify various detection errors, balance the data representations, refine annotations, and verify the data quality. Two high-quality cattle datasets (6.5 k and 486 frames), including 26 k and 2.5 k cattle instances, were generated through the ADE workflow from 24-hour surveillance videos on a commercial cattle farm and made publicly available. The proposed dataset has achievable performance between 74.6 %∼84.1 %. The ADE workflow saved 78.4 % of manual work compared to the traditional dataset construction workflow (approximately 141 h). This pioneering approach empowered the fast creation of benchmark animal datasets and would enhance computer vision applications in the livestock production industry in the future.

Integrating Automated Labeling Framework for Enhancing Deep Learning Models to Count Corn Plants Using UAS Imagery.

Plant counting is a critical aspect of crop management, providing farmers with valuable insights into seed germination success and within-field variation in crop population density, both of which are key indicators of crop yield and quality. Recent advancements in Unmanned Aerial System (UAS) technology, coupled with deep learning techniques, have facilitated the development of automated plant counting methods. Various computer vision models based on UAS images are available for detecting and classifying crop plants. However, their accuracy relies largely on the availability of substantial manually labeled training datasets. The objective of this study was to develop a robust corn counting model by developing and integrating an automatic image annotation framework. This study used high-spatial-resolution images collected with a DJI Mavic Pro 2 at the V2-V4 growth stage of corn plants from a field in Wooster, Ohio. The automated image annotation process involved extracting corn rows and applying image enhancement techniques to automatically annotate images as either corn or non-corn, resulting in 80% accuracy in identifying corn plants. The accuracy of corn stand identification was further improved by training four deep learning (DL) models, including InceptionV3, VGG16, VGG19, and Vision Transformer (ViT), with annotated images across various datasets. Notably, VGG16 outperformed the other three models, achieving an F1 score of 0.955. When the corn counts were compared to ground truth data across five test regions, VGG achieved an R2 of 0.94 and an RMSE of 9.95. The integration of an automated image annotation process into the training of the DL models provided notable benefits in terms of model scaling and consistency. The developed framework can efficiently manage large-scale data generation, streamlining the process for the rapid development and deployment of corn counting DL models.