Annotation Process Research Articles

Processing of Condition Monitoring Annotations with BERT and Technical Language Substitution: A Case Study

Annotations in condition monitoring systems contain information regarding asset history and fault characteristics in the form of unstructured text that could, if unlocked, be used for intelligent fault diagnosis. However, processing these annotations with pre-trained natural language models such as BERT is problematic due to out-of-vocabulary (OOV) technical terms, resulting in inaccurate language embeddings. Here we investigate the effect of OOV technical terms on BERT and SentenceBERT embeddings by substituting technical terms with natural language descriptions. The embeddings were computed for each annotation in a pre-processed corpus, with and without substitution. The K-Means clustering score was calculated on sentence embeddings, and a Long Short-Term Memory (LSTM) network was trained on word embeddings with the objective to recreate the output from a keywordbased annotation classifier. The K-Means score for SentenceBERT annotation embeddings improved by 40% at seven clusters by technical language substitution, and the labelling capacity of the BERT-LSTM model was improved from 88.3 to 94.2%. These results indicate that the substitution of OOV technical terms can improve the representation accuracy of the embeddings of the pre-trained BERT and SentenceBERT models, and that pre-trained language models can be used to process technical language.

Exploiting temporal coherence for self-supervised visual tracking by using vision transformer

Deep learning based fully-supervised visual trackers entail the requirement of large-scale and frame-wise annotation that needs a laborious and tedious data annotation process. To reducing the amount of labeled efforts, a self-supervised learning framework, the ETC, is proposed in this work that exploits temporal coherence as a self-supervised signal and uses visual transformer to capture the relationship among the unlabeled video frames. We design a cycle-consistent transformer architecture to cast self-supervised tracking as cycle prediction problems. With carefully-designed and targeted configurations for cycle-consistent transformer including temporal sampling strategies, tracking initialization and data augmentation, our approach is applicable for two tracking settings, i.e., the unlabeled sample (ULS) scene and the few labeled sample (FLS) scene. To learn richer and more discriminative representations, we not only utilize the inter-frame correspondence, but also conduct the intra-frame correspondence to effectively model the target-to-frame and long-range correspondence. Extensive experiments are conducted on the popular benchmark datasets OTB2015, VOT2018, UAV123, TColor-128, NFS and LaSOT, and the results show that our approach achieves competitive results in the ULS setting, and supplies a trade-off between performance and annotation cost in the FLS setting.

A multi camera unsupervised domain adaptation pipeline for object detection in cultural sites through adversarial learning and self-training

Object detection algorithms allow to enable many interesting applications which can be implemented in different devices, such as smartphones and wearable devices. In the context of a cultural site, implementing these algorithms in a wearable device, such as a pair of smart glasses, allow to enable the use of augmented reality (AR) to show extra information about the artworks and enrich the visitors’ experience during their tour. However, object detection algorithms require to be trained on many well annotated examples to achieve reasonable results. This brings a major limitation since the annotation process requires human supervision which makes it expensive in terms of time and costs. A possible solution to reduce these costs consist in exploiting tools to automatically generate synthetic labeled images from a 3D model of the site. However, models trained with synthetic data do not generalize on real images acquired in the target scenario in which they are supposed to be used. Furthermore, object detectors should be able to work with different wearable devices or different mobile devices, which makes generalization even harder. In this paper, we present a new dataset collected in a cultural site to study the problem of domain adaptation for object detection in the presence of multiple unlabeled target domains corresponding to different cameras and a labeled source domain obtained considering synthetic images for training purposes. We present a new domain adaptation method which outperforms current state-of-the-art approaches combining the benefits of aligning the domains at the feature and pixel level with a self-training process. We release the dataset at the following link https://iplab.dmi.unict.it/OBJ-MDA/ and the code of the proposed architecture at https://github.com/fpv-iplab/STMDA-RetinaNet.

Evaluating Plant Gene Models Using Machine Learning.

Gene models are regions of the genome that can be transcribed into RNA and translated to proteins, or belong to a class of non-coding RNA genes. The prediction of gene models is a complex process that can be unreliable, leading to false positive annotations. To help support the calling of confident conserved gene models and minimize false positives arising during gene model prediction we have developed Truegene, a machine learning approach to classify potential low confidence gene models using 14 gene and 41 protein-based characteristics. Amino acid and nucleotide sequence-based features were calculated for conserved (high confidence) and non-conserved (low confidence) annotated genes from the published Pisum sativum Cameor genome. These features were used to train eXtreme Gradient Boost (XGBoost) classifier models to predict whether a gene model is likely to be real. The optimized models demonstrated a prediction accuracy ranging from 87% to 90% and an F-1 score of 0.91–0.94. We used SHapley Additive exPlanations (SHAP) and feature importance plots to identify the features that contribute to the model predictions, and we show that protein and gene-based features can be used to build accurate models for gene prediction that have applications in supporting future gene annotation processes.

Agreement process of ECG annotations using thesaurus (list of typical phrases) of ECG conclusions

A thesaurus (logically ordered list) of statements for the ECG annotation has been developed. 549 ECGs from the PTB Diagnostic ECG Database were independently annotated by 3 clinicians using this thesaurus and web interface. The resulting independent annotations did not match (the proportion of matches was from 3 to 100% for different groups of conclusions in pairwise comparison of annotators). The annotations were agreed, for which a special procedure was developed. The procedure includes participation of moderators (highly qualified physicians) and using videoconferencing. Annotations are published in the public domain and can be used both for the purposes of knowledge control and for testing the accuracy of the automatic ECG analysis algorithms and software.

Abstract 1924: PULS-AI: A multimodal artificial intelligence model to predict survival of solid tumor patients treated with antiangiogenics

Abstract The need for developing new biomarkers is increasing with the emergence of many targeted therapies. In this study, we used artificial intelligence (AI) to develop a multimodal model (PULS-AI) predicting the survival of solid tumor patients treated with antiangiogenic treatments. Our retrospective, multicentric study included 616 patients with 7 different cancer types: renal cell carcinoma, colorectal carcinoma, hepatocellular carcinoma, gastrointestinal carcinoma, melanoma, breast cancer, and sarcoma. A set of 196 patients was left out of the study for validation. Clinical data including patient, treatment, and cancer metadata were collected at baseline for all patients, as well as computed tomography (CT) and ultrasound (US) images. Radiologists annotated all metastases on the CT images and the visible tumor lesion on the US images. AI models were used to extract relevant features from the regions of interest on CT and US images. In addition, handcrafted features related to the tumor burden were extracted from the annotations of all lesions on CT such as the number of lesions and the tumor burden volume per organ (lungs, liver, skull, bone, other). Finally, a Cox regression model was fitted to the set of imaging features and clinical features. The annotation process led to 1147 annotated US images with lesions delineation and 4564 reviewed CTs, of which 989 were selected and fully annotated with a total of 9516 annotated lesions.The developed model reaches an average concordance index of 0.71 (0.67-0.75, 95% CI). Using a risk threshold of 50%, PULS-AI model is able to significantly isolate (log-rank test P-value < 0.001) high-risk patients from low-risk patients (respective median OS of 12 and 32 months) with a hazard ratio of 3.52 (2.35-5.28, 95% CI). The results of this study show that AI algorithms are able to extract relevant information from radiology images and to aggregate data from multiple modalities to build powerful prognostic tools. Such tools may provide assistance to oncology clinicians in therapeutic decision-making. Citation Format: Kathryn Schutte, Fabien Brulport, Sana Harguem-Zayani, Jean-Baptiste Schiratti, Ridouane Ghermi, Paul Jehanno, Alexandre Jaeger, Talal Alamri, Raphael Naccache, Leila Haddag-Miliani, Teresa Orsi, Jean-Philippe Lamarque, Isaline Hoferer, Littisha Lawrance, Baya Benatsou, Imad Bousaid, Mickael Azoulay, Antoine Verdon, François Bidault, Corinne Balleyguier, Victor Aubert, Etienne Bendjebbar, Charles Maussion, Nicolas Loiseau, Benoit Schmauch, Meriem Sefta, Gilles Wainrib, Thomas Clozel, Samy Ammari, Nathalie Lassau. PULS-AI: A multimodal artificial intelligence model to predict survival of solid tumor patients treated with antiangiogenics [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 1924.

Video annotation with metadata: A case study of soccer game video annotation with players

In recent years, the use of convolutional neural networks for video processing has become very attractive. The reason lies in the computational power for data processing which is available today. There are many well-defined research areas where neural networks have brought higher reliability than other conventional approaches; for example, traffic sign recognition and isolated number recognition. In this paper, we will describe the architecture and the implementation of the process of soccer game annotation. The game is annotated with data about players. The technology of convolutional neural networks is used for number recognition. The process runs in real-time on a streaming video. Content enriched with metadata is given to the user in parallel with the real-time video. In the paper, we will describe in some detail the following modules: Image binarization, shot localization, the selection and recognition of numbers on players` jerseys.

TERJEMAHAN BERANOTASI DONGENG LA SORCIÈRE DE LA RUE MOUFFETARD dalam BAHASA INDONESIA

An annotated translation is a translation with notes or annotations of the equivalent selected in the target language with the aim of being accountable for the equivalent. This study examines a tale entitled “la Sorcière de la Rue Mouffetard” written by Pierre Gripari. This study employs qualitative methods and a comparative model to discuss translation problems and their solutions using translation strategies from Molina and Albir (2002). The use of interjection, idioms, and words containing culture is the problem in this fairy tale translation of language and cultural elements. Various dictionaries and websites are used both in the process of translating and the annotation process. The results of this study are expected to become one of the Indonesian-language fairy tales that will enrich children’s literature, especially from France.

PeakForest: a multi-platform digital infrastructure for interoperable metabolite spectral data and metadata management

IntroductionAccuracy of feature annotation and metabolite identification in biological samples is a key element in metabolomics research. However, the annotation process is often hampered by the lack of spectral reference data in experimental conditions, as well as logistical difficulties in the spectral data management and exchange of annotations between laboratories.ObjectivesTo design an open-source infrastructure allowing hosting both nuclear magnetic resonance (NMR) and mass spectra (MS), with an ergonomic Web interface and Web services to support metabolite annotation and laboratory data management.MethodsWe developed the PeakForest infrastructure, an open-source Java tool with automatic programming interfaces that can be deployed locally to organize spectral data for metabolome annotation in laboratories. Standardized operating procedures and formats were included to ensure data quality and interoperability, in line with international recommendations and FAIR principles.ResultsPeakForest is able to capture and store experimental spectral MS and NMR metadata as well as collect and display signal annotations. This modular system provides a structured database with inbuilt tools to curate information, browse and reuse spectral information in data treatment. PeakForest offers data formalization and centralization at the laboratory level, facilitating shared spectral data across laboratories and integration into public databases.ConclusionPeakForest is a comprehensive resource which addresses a technical bottleneck, namely large-scale spectral data annotation and metabolite identification for metabolomics laboratories with multiple instruments. PeakForest databases can be used in conjunction with bespoke data analysis pipelines in the Galaxy environment, offering the opportunity to meet the evolving needs of metabolomics research. Developed and tested by the French metabolomics community, PeakForest is freely-available at https://github.com/peakforest.

An Improved Visual Attention Model for Automated Vehicle License Plate Number Recognition Using Computer Vision

The role of an automatic licensed plate detection system (ALPDS) cannot be over-emphasized in the world today. The need for an automated system for vehicle license plate number recognition is important for security challenges. Therefore, this paper provides a smart system for vehicle license number recognition using Computer Vision. The system was trained using images of vehicles license numbers as training data. The training images were first annotated using the Visual Graphic Generator (VGG) annotation tool, after the annotation process, the trained images were pre-processed using the OpenCV library for conversion and masking of images. TesseractOCR was then used in extracting just texts from the images. The pre-processed and segmented images were then used in training the Mask R-CNN from a pre-trained weight. The result of the proposed system shows how the Mask R-CNN model was trained in ten training steps. The mask R-CNN model obtained accuracy and a loss value for each training step. The mask R-CNN model was evaluated using both training and test data. For the training and testing data, the Mask R-CNN was evaluated in terms of accuracy and loss. The evaluation was done using graphs. The results from the graph show that the Mask R-CNN had a better accuracy result in both training and testing data. The accuracy for training data was that of 95.25% and the accuracy for the testing data was 97.69%. For real-time vehicle license plate number recognition, we deployed our proposed model to the web. Here, we built a web application that allows real-time surveillance video. Our model was tested on different vehicles in the car park. The result of the mask R-CNN on the test shows how the Mask R-CNN model was used in not just capturing and extracting the vehicle’s license plate number but predicting the characters that appeared on the vehicle’s license plate number. We also compared our proposed system with another existing system. The comparison was done in terms of accuracy, loss, and precision. The result of our proposed model gave us an accuracy of 97.69%, which is higher than the existing system (85%). This study can further be improved by using the Internet of Things in performing live video streaming and also providing a database system that will be storing the predicted vehicle numbers for cars that are detected.

An unsupervised annotation of Arabic texts using multi-label topic modeling and genetic algorithm

Every day the world produces an enormous amount of textual data. This unstructured text is of little use unless it is labeled using a combination of categories, keywords, tags. Humans can never annotate such massive data, and with a growing divide between the daily produced data and those annotated, the only alternative is to mechanize it. Automatic annotation process helps in saving resources in terms of time and cost. The process of multi-label annotation involves associating a document with multiple relevant labels. This paper proposes an unsupervised model to annotate corpus using multi-labels automatically. The model is based on multi-label topic modeling and genetic algorithm (GA). Topic modeling is a technique to extract the hidden topics from text, and the GA is used to find the optimal number of topics. We hyper-tuned the parameters of the topic modeling using two different training methods: variational Bayes and Gibbs sampling. The class imbalance in a corpus can affect the result of topic modeling, where the majority class dominates the minority class. We overcome this problem using the partitioning method. Though the proposed model was developed for the Arabic dataset, it is language neutral. We tested our model on three large Arabic corpora and three large English social media datasets. For the Arabic language, our work being the first work that tackles multi-label annotation, we needed a reference to compare our model. For the Arabic corpus, we compared the result of automatic annotation against humans using crowdsourcing (whose labeling was checked for quality). The analysis of the annotation shows an agreement among models (machine vs. human) of 79.30%. Moreover, for the English dataset, the results are quite competitive.

Computer-Aided Depth Video Stream Masking Framework for Human Body Segmentation in Depth Sensor Images.

The identification of human activities from videos is important for many applications. For such a task, three-dimensional (3D) depth images or image sequences (videos) can be used, which represent the positioning information of the objects in a 3D scene obtained from depth sensors. This paper presents a framework to create foreground–background masks from depth images for human body segmentation. The framework can be used to speed up the manual depth image annotation process with no semantics known beforehand and can apply segmentation using a performant algorithm while the user only adjusts the parameters, or corrects the automatic segmentation results, or gives it hints by drawing a boundary of the desired object. The approach has been tested using two different datasets with a human in a real-world closed environment. The solution has provided promising results in terms of reducing the manual segmentation time from the perspective of the processing time as well as the human input time.

Automated, Score‐Based Pharmacophore Generation Using Multiple Copy Simultaneous Search

Pharmacophores are three‐dimensional arrangements of molecular features required for biological activity that are often used in virtual screening efforts to prioritize ligands for experimental screening. G protein‐coupled receptors (GPCR) are integral membrane proteins of considerable interest as targets for ligand discovery and drug development. Pharmacophore models are most typically ligand‐based and constructed via the identification of structural commonalities between known bioactive ligands. However, structure‐based pharmacophore models (requiring only a target protein’s structure) provide an alternative to ligand‐based pharmacophore models. Although both ligand‐based and structure‐based pharmacophore models have exhibited success in prior virtual screening studies, most pharmacophore modeling efforts are not applicable to GPCR targets lacking known ligands. Consequently, the development of a purely structure‐based pharmacophore modeling protocol for GPCR has become an increasingly attractive approach as the number of publicly available, high‐resolution GPCR structures has increased (113 unique GPCR represented as of November 24, 2021). Thus, the aim of this work was to develop a structure‐based pharmacophore modeling approach that can generate well‐performing pharmacophore models in GPCR crystal structures and homology models. Pharmacophore models were generated in crystal structures and homology models of 13 class A GPCR via automated feature annotation of differing subsets of functional group fragments placed with Multiple Copy Simultaneous Search (MCSS). During the feature annotation process, distance cutoffs were enforced to generate combinations of pharmacophore features that reflect spatial arrangements of interactions typically observed in GPCR pharmacophore models. Resulting pharmacophore models for each target GPCR were used to search a database containing active and decoy/inactive compounds for 30 class A GPCR and scored using enrichment factor (EF) and goodness‐of‐hit (GH) metrics to assess performance. To identify well‐performing pharmacophore models for cases where active ligands are unknown, pharmacophore models were classified using cluster‐then‐predict logistic regression. As a proof‐of principle for this method, pharmacophore models were generated for the orphan GPCR GPR101 to elucidate candidate ligands that may possess activity for GPR101. Application of this method to the set of 13 class A GPCR targets resulted in the generation of pharmacophore models possessing EF scores ≥ 2 in both crystal structures (12 of 13 cases) and homology models (9 of 13 cases). In addition, classification of pharmacophore models with cluster‐then‐predict logistic regression resulted in positive predictive values (PPV) of at least 0.67 for pharmacophore models generated with selected MCSS fragment subsets. Lastly, implementation of generated pharmacophore models in a virtual screening workflow identified 33 candidate ligands for GPR101.

CMAL: Cost-Effective Multi-Label Active Learning by Querying Subexamples

Multi-label active learning (MAL) aims to learn an accurate multi-label classifier by selecting which examples (or example-label pairs) will be annotated and reducing query effort. MAL is a more complicated and expensive process than single-label active learning, due to one example can be associated with a set of non-exclusive labels and the annotator has to scrutinize the whole example and label space to provide correct annotations. Instead of scrutinizing the whole example for annotation, we may just examine some of its subexamples with respect to a label for annotation. In this way, we can not only save the annotation cost but also speedup the annotation process. Given this observation, we introduce CMAL, a two-stage Cost-effective MAL strategy (CMAL) by querying subexamples. CMAL first selects the most informative example-label pairs by leveraging uncertainty, label correlation and label space sparsity. Specifically, the uncertainty of a label to an example can be reduced if its correlated labels already annotated to the example, and its uncertainty can be reduced also if more examples annotated to this label. Next, CMAL greedily queries the most probable positive subexample-label pairs of the selected example-label pair. In addition, we propose rCMAL to account for the representative of examples to more reliably select example-label pairs in the first stage. Extensive experiments on multi-label datasets from diverse domains show that our proposed CMAL and rCMAL can better save the query cost than state-of-the-art MAL methods. The contribution of leveraging label correlation, label sparsity, and representative for saving cost is also confirmed.

Image collection and annotation platforms to establish amulti-source database of oral lesions.

To describe the development of a platform for image collection and annotation that resulted in a multi-sourced international image dataset of oral lesions to facilitate the development of automated lesion classification algorithms. We developed a web-interface, hosted on a web server to collect oral lesions images from international partners. Further, we developed a customised annotation tool, also a web-interface for systematic annotation of images to build a rich clinically labelled dataset. We evaluated the sensitivities comparing referral decisions through the annotation process with the clinical diagnosis of the lesions. The image repository hosts 2474 images of oral lesions consisting of oral cancer, oral potentially malignant disorders and other oral lesions that were collected through MeMoSA® UPLOAD. Eight-hundred images were annotated by seven oral medicine specialists on MeMoSA® ANNOTATE, to mark the lesion and to collect clinical labels. The sensitivity in referral decision for all lesions that required a referral for cancer management/surveillance was moderate to high depending on the type of lesion (64.3%-100%). This is the first description of a database with clinically labelled oral lesions. This database could accelerate the improvement of AI algorithms that can promote the early detection of high-risk oral lesions.

Non-destructive Leaf Area Index estimation via guided optical imaging for large scale greenhouse environments

This paper presents a financially viable and non-destructive rail-based video monitoring method that utilizes optical image segmentation to estimate the canopy leaf area index (LAI) of greenhouse tomato plants. The LAI is directly related to the time-dependent crop growth and indicates plant health and potential crop yields. A rail-guided mobile camera system was commissioned that records continuous images by scanning multiple rows of two tomato plant species for over two years. UNET semantic image segmentation of the individual image frames was performed to compute the relative leaf area over time. This study also describes the image annotation process necessary to train the neural network and evaluate the segmentation results. The results are calibrated and compared to the defoliation-based (destructive) LAI estimation performed by the grower. This UNET segmentation performs well, which is enabled through the controlled environment and the well-defined boundary conditions provided by the greenhouse environment and the managed measurement conditions. Our results deviate from the manual LAI estimation by less than ten percent. Further, we are able to minimize confusion between foreground and background plants and other obstructions with an estimated error smaller than three percent, which is strictly necessary to produce reproducible results.

From a Dynamic Image Annotation Process within the Humanities to a Canonical Workflow

Abstract One idea of the Canonical Workflow Framework for Research (CWFR) is to improve the reusability and automation in research. In this paper, we aim to deliver a concrete view on the application of CWFRs to a use case of the arts and humanities to enrich further discussions on the practical realization of canonical workflows and the benefits that come with it. This use case involves context dependent data transformation and feature extraction, ingests into multiple repositories as well as a “human-in-the-loop” workflow step, which introduces a certain complexity into the mapping to a canonical workflow.

Hard Sample Aware Noise Robust Learning for Histopathology Image Classification

Deep learning-based histopathology image classification is a key technique to help physicians in improving the accuracy and promptness of cancer diagnosis. However, the noisy labels are often inevitable in the complex manual annotation process, and thus mislead the training of the classification model. In this work, we introduce a novel hard sample aware noise robust learning method for histopathology image classification. To distinguish the informative hard samples from the harmful noisy ones, we build an easy/hard/noisy (EHN) detection model by using the sample training history. Then we integrate the EHN into a self-training architecture to lower the noise rate through gradually label correction. With the obtained almost clean dataset, we further propose a noise suppressing and hard enhancing (NSHE) scheme to train the noise robust model. Compared with the previous works, our method can save more clean samples and can be directly applied to the real-world noisy dataset scenario without using a clean subset. Experimental results demonstrate that the proposed scheme outperforms the current state-of-the-art methods in both the synthetic and real-world noisy datasets. The source code and data are available at https://github.com/bupt-ai-cz/HSA-NRL/.

Eliciting Confidence for Improving Crowdsourced Audio Annotations

In this work we explore confidence elicitation methods for crowdsourcing "soft" labels, e.g., probability estimates, to reduce the annotation costs for domains with ambiguous data. Machine learning research has shown that such "soft" labels are more informative and can reduce the data requirements when training supervised machine learning models. By reducing the number of required labels, we can reduce the costs of slow annotation processes such as audio annotation. In our experiments we evaluated three confidence elicitation methods: 1) "No Confidence" elicitation, 2) "Simple Confidence" elicitation, and 3) "Betting" mechanism for confidence elicitation, at both individual (i.e., per participant) and aggregate (i.e., crowd) levels. In addition, we evaluated the interaction between confidence elicitation methods, annotation types (binary, probability, and z-score derived probability), and "soft" versus "hard" (i.e., binarized) aggregate labels. Our results show that both confidence elicitation mechanisms result in higher annotation quality than the "No Confidence" mechanism for binary annotations at both participant and recording levels. In addition, when aggregating labels at the recording level, results indicate that we can achieve comparable results to those with 10-participant aggregate annotations using fewer annotators if we aggregate "soft" labels instead of "hard" labels. These results suggest that for binary audio annotation using a confidence elicitation mechanism and aggregating continuous labels we can obtain higher annotation quality, more informative labels, with quality differences more pronounced with fewer participants. Finally, we propose a way of integrating these confidence elicitation methods into a two-stage, multi-label annotation pipeline.

Impact of gene annotation choice on the quantification of RNA-seq data

BackgroundRNA sequencing is currently the method of choice for genome-wide profiling of gene expression. A popular approach to quantify expression levels of genes from RNA-seq data is to map reads to a reference genome and then count mapped reads to each gene. Gene annotation data, which include chromosomal coordinates of exons for tens of thousands of genes, are required for this quantification process. There are several major sources of gene annotations that can be used for quantification, such as Ensembl and RefSeq databases. However, there is very little understanding of the effect that the choice of annotation has on the accuracy of gene expression quantification in an RNA-seq analysis.ResultsIn this paper, we present results from our comparison of Ensembl and RefSeq human annotations on their impact on gene expression quantification using a benchmark RNA-seq dataset generated by the SEQC consortium. We show that the use of RefSeq gene annotation models led to better quantification accuracy, based on the correlation with ground truths including expression data from >800 real-time PCR validated genes, known titration ratios of gene expression and microarray expression data. We also found that the recent expansion of the RefSeq annotation has led to a decrease in its annotation accuracy. Finally, we demonstrated that the RNA-seq quantification differences observed between different annotations were not affected by the use of different normalization methods.ConclusionIn conclusion, our study found that the use of the conservative RefSeq gene annotation yields better RNA-seq quantification results than the more comprehensive Ensembl annotation. We also found that, surprisingly, the recent expansion of the RefSeq database, which was primarily driven by the incorporation of sequencing data into the gene annotation process, resulted in a reduction in the accuracy of RNA-seq quantification.