Annotation Method Research Articles

Pose recognition of dancing images using fuzzy deep learning technique in anIoTenvironment

AbstractWith the development of smart technologies (e.g., Internet of Things (IoT), Artificial Intelligence (AI), and Big Data), people have start using them for various purposes. The real‐time IoT devices generate an enormous amount of video and imaging data, leading to the concept of complex data structure. And people face major challenges in mining and extraction of useful features and information from such data. How to efficiently analyse and process video data to obtain valuable information has become a key research topic. The traditional manual annotation methods are unable to meet the current demand for the growing number of videos. Therefore, a more convenient method for processing video data needs to be developed. The research objective of this paper is dance videos, and the goal is to realize automatic recognition of dance movements. In this paper, a Dual Convolutional Neural Network Algorithm (DCNNA) is proposed for the automatic recognition of different dance movements in live and remote videos. DCNNA can extract video information more comprehensively and efficiently. It can simultaneously extract the light flow features corresponding to the action changes and the information contained in each frame of the video. Therefore, the dance movements can be more accurately identified. In the experiments, the performance of DCNNA is evaluated based on dance videos and compared with Inception V3 and 3D‐CNN. All the experiments illustrate the superior performance of the proposed DCNN algorithm. From the experimental results, it is quite obvious that the F1 score of the proposed DCNNA is 11% and 6% higher than that of the Inception V3 and 3D‐CNN, respectively.

Fully automated annotation of mitochondrial genomes using a cluster-based approach with de Bruijn graphs.

A wide range of scientific fields, such as forensics, anthropology, medicine, and molecular evolution, benefits from the analysis of mitogenomic data. With the development of new sequencing technologies, the amount of mitochondrial sequence data to be analyzed has increased exponentially over the last few years. The accurate annotation of mitochondrial DNA is a prerequisite for any mitogenomic comparative analysis. To sustain with the growth of the available mitochondrial sequence data, highly efficient automatic computational methods are, hence, needed. Automatic annotation methods are typically based on databases that contain information about already annotated (and often pre-curated) mitogenomes of different species. However, the existing approaches have several shortcomings: 1) they do not scale well with the size of the database; 2) they do not allow for a fast (and easy) update of the database; and 3) they can only be applied to a relatively small taxonomic subset of all species. Here, we present a novel approach that does not have any of these aforementioned shortcomings, (1), (2), and (3). The reference database of mitogenomes is represented as a richly annotated de Bruijn graph. To generate gene predictions for a new user-supplied mitogenome, the method utilizes a clustering routine that uses the mapping information of the provided sequence to this graph. The method is implemented in a software package called DeGeCI (De Bruijn graph Gene Cluster Identification). For a large set of mitogenomes, for which expert-curated annotations are available, DeGeCI generates gene predictions of high conformity. In a comparative evaluation with MITOS2, a state-of-the-art annotation tool for mitochondrial genomes, DeGeCI shows better database scalability while still matching MITOS2 in terms of result quality and providing a fully automated means to update the underlying database. Moreover, unlike MITOS2, DeGeCI can be run in parallel on several processors to make use of modern multi-processor systems.

Efficient Annotation and Learning for 3D Hand Pose Estimation: A Survey

AbstractIn this survey, we present a systematic review of 3D hand pose estimation from the perspective of efficient annotation and learning. 3D hand pose estimation has been an important research area owing to its potential to enable various applications, such as video understanding, AR/VR, and robotics. However, the performance of models is tied to the quality and quantity of annotated 3D hand poses. Under the status quo, acquiring such annotated 3D hand poses is challenging, e.g., due to the difficulty of 3D annotation and the presence of occlusion. To reveal this problem, we review the pros and cons of existing annotation methods classified as manual, synthetic-model-based, hand-sensor-based, and computational approaches. Additionally, we examine methods for learning 3D hand poses when annotated data are scarce, including self-supervised pretraining, semi-supervised learning, and domain adaptation. Based on the study of efficient annotation and learning, we further discuss limitations and possible future directions in this field.

Relative-Breakpoint-Based Crack Annotation Method for Lightweight Crack Identification Using Deep Learning Methods

After years of service, bridges could lose their expected functions. Considering the significant number of bridges and the adverse inspecting environment, the urgent requirement for timely and efficient inspection solutions, such as computer vision techniques, have been attractive in recent years, especially for those bridge components with poor accessibility. In this paper, a lightweight procedure for bridge apparent-defect detection is proposed, including a crack annotation method and crack detection. First of all, in order to save computational costs and improve generalization performance, we propose herein a relative-breakpoint annotation method to build a crack instance segmentation dataset, a critical process for a supervised vision-based crack detection method. Then, the trained models based on classic Mask RCNN and Yolact are transferred to evaluate the effectiveness of the proposed method. To verify the correctness, universality and generality of the proposed crack-detection framework, approximately 800 images are used for model training, while nearly 100 images are saved for validation. Results show that the crack instance segmentation model can achieve a level of 90% for both accuracy and recall values, with a limited dataset.

Three-dimensional branch segmentation and phenotype extraction of maize tassel based on deep learning

BackgroundThe morphological structure phenotype of maize tassel plays an important role in plant growth, reproduction, and yield formation. It is an important step in the distinctness, uniformity, and stability (DUS) testing to obtain maize tassel phenotype traits. Plant organ segmentation can be achieved with high-precision and automated acquisition of maize tassel phenotype traits because of the advances in the point cloud deep learning method. However, this method requires a large number of data sets and is not robust to automatic segmentation of highly adherent organ components; thus, it should be combined with point cloud processing technology.ResultsAn innovative method of incomplete annotation of point cloud data was proposed for easy development of the dataset of maize tassels,and an automatic maize tassel phenotype analysis system: MaizeTasselSeg was developed. The tip feature of point cloud is trained and learned based on PointNet + + network, and the tip point cloud of tassel branch was automatically segmented. Complete branch segmentation was realized based on the shortest path algorithm. The Intersection over Union (IoU), precision, and recall of the segmentation results were 96.29, 96.36, and 93.01, respectively. Six phenotypic traits related to morphological structure (branch count, branch length, branch angle, branch curvature, tassel volume, and dispersion) were automatically extracted from the segmentation point cloud. The squared correlation coefficients (R2) for branch length, branch angle, and branch count were 0.9897, 0.9317, and 0.9587, respectively. The root mean squared error (RMSE) for branch length, branch angle, and branch count were 0.529 cm, 4.516, and 0.875, respectively.ConclusionThe proposed method provides an efficient scheme for high-throughput organ segmentation of maize tassels and can be used for the automatic extraction of phenotypic traits of maize tassel. In addition, the incomplete annotation approach provides a new idea for morphology-based plant segmentation.

ScDeepInsight: a supervised cell-type identification method for scRNA-seq data with deep learning.

Annotation of cell-types is a critical step in the analysis of single-cell RNA sequencing (scRNA-seq) data that allows the study of heterogeneity across multiple cell populations. Currently, this is most commonly done using unsupervised clustering algorithms, which project single-cell expression data into a lower dimensional space and then cluster cells based on their distances from each other. However, as these methods do not use reference datasets, they can only achieve a rough classification of cell-types, and it is difficult to improve the recognition accuracy further. To effectively solve this issue, we propose a novel supervised annotation method, scDeepInsight. The scDeepInsight method is capable of performing manifold assignments. It is competent in executing data integration through batch normalization, performing supervised training on the reference dataset, doing outlier detection and annotating cell-types on query datasets. Moreover, it can help identify active genes or marker genes related to cell-types. The training of the scDeepInsight model is performed in a unique way. Tabular scRNA-seq data are first converted to corresponding images through the DeepInsight methodology. DeepInsight can create a trainable image transformer to convert non-image RNA data to images by comprehensively comparing interrelationships among multiple genes. Subsequently, the converted images are fed into convolutional neural networks such as EfficientNet-b3. This enables automatic feature extraction to identify the cell-types of scRNA-seq samples. We benchmarked scDeepInsight with six other mainstream cell annotation methods. The average accuracy rate of scDeepInsight reached 87.5%, which is more than 7% higher compared with the state-of-the-art methods.

AtacAnnoR: a reference-based annotation tool for single cell ATAC-seq data.

Here, we present AtacAnnoR, a two-round annotation method for scATAC-seq data using well-annotated scRNA-seq data as reference. We evaluate AtacAnnoR's performance against six competing methods on 11 benchmark datasets. Our results show that AtacAnnoR achieves the highest mean accuracy and the highest mean balanced accuracy and performs particularly well when unpaired scRNA-seq data are used as the reference. Furthermore, AtacAnnoR implements a 'Combine and Discard' strategy to further improve annotation accuracy when annotations of multiple references are available. AtacAnnoR has been implemented in an R package and can be directly integrated into currently popular scATAC-seq analysis pipelines.

Highly Flexible Deep-Learning-Based Automatic Analysis for Graphically Encoded Hydrogel Microparticles.

Graphically encoded hydrogel microparticle (HMP)-based bioassay is a diagnostic tool characterized by exceptional multiplex detectability and robust sensitivity and specificity. Specifically, deep learning enables highly fast and accurate analyses of HMPs with diverse graphical codes. However, previous related studies have found the use of plain particles as data to be disadvantageous for accurate analyses of HMPs loaded with functional nanomaterials. Furthermore, the manual data annotation method used in existing approaches is highly labor-intensive and time-consuming. In this study, we present an efficient deep-learning-based analysis of encoded HMPs with diverse graphical codes and functional nanomaterials, utilizing the auto-annotation and synthetic data mixing methods for model training. The auto-annotation enhanced the throughput of dataset preparation up to 0.11 s/image. Using synthetic data mixing, a mean average precision of 0.88 was achieved in the analysis of encoded HMPs with magnetic nanoparticles, representing an approximately twofold improvement over the standard method. To evaluate the practical applicability of the proposed automatic analysis strategy, a single-image analysis was performed after the triplex immunoassay for the preeclampsia-related protein biomarkers. Finally, we accomplished a processing throughput of 0.353 s per sample for analyzing the result image.

A Multi-Scale Rebar Detection Network with an Embedded Attention Mechanism

With the rapid development of computer vision and machine vision, methods based on deep learning have achieved good results in the field of object detection, identification, and tracking. However, for the detection and identification of rebars in smart construction sites, it is very difficult to perform accurate real-time detection of rebars by using object detection technology on the equipment in the field because of the problems of the dense cross-section between bundled bars, the mutual adhesion of cross-section boundaries, and mutual occlusion between cross-sections. To address the above problems, we propose a multi-scale rebar detection network RebarNet with an embedded attention mechanism based on YOLOv5, combining the K-means++ algorithm, attention mechanism, a newly designed SD_IoU Loss, and multi-scale feature fusion, aiming to solve the problems of missed and false detection in dense small object detection. Due to the problems of scarce rebar cross-section datasets, no publicly available large datasets, and weak rebar cross-sectional features, we constructed a new rebar cross-sectional dataset, used a semi-automatic annotation method to annotate part of the dataset, and then used the data enhancement algorithm to expand the rebar dataset. The experimental results show that the average accuracy (mAP) of our proposed RebarNet network is 97.9%, which is comparable to mainstream target detection algorithms such as Faster R-CNN, SSD, RetinaNet, CenterNet, CornerNet, YOLOv3, YOLOv4, and YOLOv5s. mAP0.5 is improved by 8.1%, 13%, 26.4%, 25.8%, 26.2%, 11.7%, 7.6%, and 9%, respectively. In addition, the frames per second (FPS) transmission reaches 89 frames per second, the model weight is only 17.0 MB. In summary, the proposed RebarNet can effectively reduce missed and false detections in the rebar counting detection task based on real-time detection.

Automated voice pathology discrimination from audio recordings benefits from phonetic analysis of continuous speech

In this paper we evaluate the hypothesis that automated methods for diagnosis of voice disorders from speech recordings would benefit from contextual information found in continuous speech. Rather than basing a diagnosis on how disorders affect the average acoustic properties of the speech signal, the idea is to exploit the possibility that different disorders will cause different acoustic changes within different phonetic contexts. Any differences in the pattern of effects across contexts would then provide additional information for discrimination of pathologies. We evaluate this approach using two complementary studies: the first uses a short phrase which is automatically annotated using a phonetic transcription, the second uses a long reading passage which is automatically annotated from text. The first study uses a single sentence recorded from 597 speakers in the Saarbrucken Voice Database to discriminate structural from neurogenic disorders. The results show that discrimination performance for these broad pathology classes improves from 59% to 67% unweighted average recall when classifiers are trained for each phone-label and the results fused. Although the phonetic contexts improved discrimination, the overall sensitivity and specificity of the method seems insufficient for clinical application. We hypothesise that this is because of the limited contexts in the speech audio and the heterogeneous nature of the disorders. In the second study we address these issues by processing recordings of a long reading passage obtained from clinical recordings of 60 speakers with either Spasmodic Dysphonia or Vocal fold Paralysis. We show that discrimination performance increases from 80% to 87% unweighted average recall if classifiers are trained for each phone-labelled region and predictions fused. We also show that the sensitivity and specificity of a diagnostic test with this performance is similar to other diagnostic procedures in clinical use. In conclusion, the studies confirm that the exploitation of contextual differences in the way disorders affect speech improves automated diagnostic performance, and that automated methods for phonetic annotation of reading passages are robust enough to extract useful diagnostic information.

Lexicon annotation in sentiment analysis for dialectal Arabic: Systematic review of current trends and future directions

Due to the vast volumes of newly streamed data on the Internet and social media, the use of sentiment analysis (SA) to extract information and analyze people's opinions has become a trendy topic. Yet, the majority of research are attributed to the English language, despite the fact that other languages, such as Arabic, are among the most popular on the Internet. Considering the availability of numerous dialects of this language and how their data were annotated and processed, the absence of research in this field is evident. Understanding these initiatives merits a great deal of attention in Arabic SA research. To the best of our knowledge, this domain has not been considered before, and thus the aim of this study is to perform a systematic review with regard to SA and data annotations for Arabic dialects published between 2015 and 2023. The outcomes of this research offer a refined taxonomy of data annotation methods classified into three categories: (1) manual, (2) automatic, and (3) hybrid methods. In addition, a discussion of the research challenges, motivations, and recommendations is presented with detailed taxonomy analysis of current research trends, and from this, we identify new research gaps and propose new research implications and future directions that will encourage more scholars to contribute to Arabic SA research, facilitate more successful multilingual SA applications, and provide insights regarding Arabic SA in different contexts.

Semantic automatic annotation method based on artificial intelligence for electric power internet of things

AbstractThe development of the power Internet of Things is currently underway, and a proposal for a semantic Internet of Things based on artificial intelligence algorithms is made to address the challenges in obtaining prior knowledge for heterogeneous data fusion, improving the real‐time performance of the ontology library, and enhancing the efficiency of manual labeling of instance object data in the power field. This proposal introduces an Automatic Semantic Annotation Method to provide an effective knowledge organization model for sensor systems. Data mining knowledge is utilized to drive ontology update and improvement, resulting in more accurate semantic annotation and enhanced machine understanding. Experimental results show that artificial intelligence algorithms can automatically extract concepts from sensory data and achieve automatic semantic annotation during ontology instantiation.

The State of Squamate Genomics: Past, Present, and Future of Genome Research in the Most Speciose Terrestrial Vertebrate Order.

Squamates include more than 11,000 extant species of lizards, snakes, and amphisbaenians, and display a dazzling diversity of phenotypes across their over 200-million-year evolutionary history on Earth. Here, we introduce and define squamates (Order Squamata) and review the history and promise of genomic investigations into the patterns and processes governing squamate evolution, given recent technological advances in DNA sequencing, genome assembly, and evolutionary analysis. We survey the most recently available whole genome assemblies for squamates, including the taxonomic distribution of available squamate genomes, and assess their quality metrics and usefulness for research. We then focus on disagreements in squamate phylogenetic inference, how methods of high-throughput phylogenomics affect these inferences, and demonstrate the promise of whole genomes to settle or sustain persistent phylogenetic arguments for squamates. We review the role transposable elements play in vertebrate evolution, methods of transposable element annotation and analysis, and further demonstrate that through the understanding of the diversity, abundance, and activity of transposable elements in squamate genomes, squamates can be an ideal model for the evolution of genome size and structure in vertebrates. We discuss how squamate genomes can contribute to other areas of biological research such as venom systems, studies of phenotypic evolution, and sex determination. Because they represent more than 30% of the living species of amniote, squamates deserve a genome consortium on par with recent efforts for other amniotes (i.e., mammals and birds) that aim to sequence most of the extant families in a clade.

IpaPy2: Integrated Probabilistic Annotation (IPA) 2.0 - an improved Bayesian-based method for the annotation of LC-MS/MS untargeted metabolomics data.

The Integrated Probabilistic Annotation (IPA) is an automated annotation method for LC-MS-based untargeted metabolomics experiments, that provides statistically rigorous estimates of the probabilities associated with each annotation. Here we introduce ipaPy2, a substantially improved and completely refactored Python implementation of the IPA method. The revised method is now able to integrate tandem MS fragmentation data, which increases the accuracy of the identifications. Moreover, ipaPy2 provides a much more user-friendly interface, and isotope peaks are no longer treated as individual features, but integrated into isotope fingerprints, greatly speeding up the calculations. The method has also been fully integrated with the mzMatch pipeline, so that the results of the annotation can be explored through the newly developed PeakMLViewerPy tool available at https://github.com/UoMMIB/PeakMLViewerPy. The source code, extensive documentation and tutorials are freely available on GitHub at https://github.com/francescodc87/ipaPy2. Supplementary data are available at Bioinformatics online.

Generalized Cell Type Annotation and Discovery for Single-Cell RNA-Seq Data

The rapid development of single-cell RNA sequencing (scRNA-seq) technology allows us to study gene expression heterogeneity at the cellular level. Cell annotation is the basis for subsequent downstream analysis in single-cell data mining. Existing methods rarely explore the fine-grained semantic knowledge of novel cell types absent from the reference data and usually susceptible to batch effects on the classification of seen cell types. Taking into consideration these limitations, this paper proposes a new and practical task called generalized cell type annotation and discovery for scRNA-seq data. In this task, cells of seen cell types are given class labels, while cells of novel cell types are given cluster labels instead of a unified “unassigned” label. To address this problem, we carefully design a comprehensive evaluation benchmark and propose a novel end-to-end algorithm framework called scGAD. Specifically, scGAD first builds the intrinsic correspondence across the reference and target data by retrieving the geometrically and semantically mutual nearest neighbors as anchor pairs. Then we introduce an anchor-based self-supervised learning module with a connectivity-aware attention mechanism to facilitate model prediction capability on unlabeled target data. To enhance the inter-type separation and intra-type compactness, we further propose a confidential prototypical self-supervised learning module to uncover the consensus category structure of the reference and target data. Extensive results on massive real datasets demonstrate the superiority of scGAD over various state-of-the-art clustering and annotation methods.

Advances in machine learning for predicting protein functions

Proteins play a variety of functional roles in cellular activities and are indispensable for life. Understanding the functions of proteins is crucial in many fields such as medicine and drug development. In addition, the application of enzymes in green synthesis has been of great interest, but the high cost of obtaining specific functional enzymes as well as the variety of enzyme types and functions hamper their application. At present, the specific functions of proteins are mainly determined through tedious and time-consuming experimental characterization. With the rapid development of bioinformatics and sequencing technologies, the number of protein sequences that have been sequenced is much larger than those can be annotated, thus developing efficient methods for predicting protein functions becomes crucial. With the rapid development of computer technology, data-driven machine learning methods have become a promising solution to these challenges. This review provides an overview of protein function and its annotation methods as well as the development history and operation process of machine learning. In combination with the application of machine learning in the field of enzyme function prediction, we present an outlook on the future direction of efficient artificial intelligence-assisted protein function research.

A context-aware approach for vessels’ trajectory prediction

An accurate vessel trajectory prediction can improve safety management at sea. Although the vessels' trajectories can be affected by complex and high dimensional context factors such as wind, wave size, and current, the research in incorporating them in trajectory prediction is still burgeoning. This paper proposes a context-aware data-driven framework for vessel trajectory prediction. In this regard, trajectories are enriched with context information at the first stage. This includes evaluating methods of trajectory annotation. Next, feature selection techniques are applied to solve the problems caused by high dimensionality, i.e., incensement in the complexity and the computational load. Finally, selected factors are used in a Context-aware Long Short-Term Memory (CLSTM) network. The implementations on the Automatic Identification System (AIS), weather, and ocean condition dataset around the English Channel collected during February 2016, indicated that the spline interpolation is suitable for trajectory annotation. Water depth, wave direction, and height are the most influencing context in vessels’ data-driven movement prediction. The predictions made by the proposed framework are 15.31% more accurate than the prediction of a Long Short-Term Memory (LSTM) network. Therefore, the proposed context-aware trajectory prediction framework might suit maritime systems and applications like collision avoidance, vessel route planning, and anomaly detection.

PLPose: An efficient framework for detecting power lines via key points-based pose estimation

The inspection and maintenance of electrical power lines (PL) via unmanned aerial vehicles (UAV) require fast and accurate PL detection to ensure smooth and secure electrical operations. However, the detection of PLs from aerial images is a highly challenging task due to the thin nature of PLs and the inherent noisy image backgrounds. Traditional line and edge detection methods do not detect the PLs accurately due to the cluttered backgrounds while the more recent deep learning (DL) CNNs are also not feasible for efficient PL detection due to the coarse bounding boxes and the computationally expensive pixel-based segmentations. Hence, in this study we propose PLPose, a novel framework for detecting the PLs via key points-based pose estimation technique and adapt the MobileNetV3 CNN for this task (kMobileNetV3), by adding a simple key point detection head to predict the PL key points. We also introduce a novel data-centric architecture (kMobileNetV3 + UDP), by adding the unbiased data processing (UDP) module to our kMobileNetV3, for faster and more accurate key point-based PL detection along with novel methods for data annotations and performance evaluation. Evaluations of PLPose on three benchmark PL datasets (PLDM, PLDU and the Mendeley Powerline Dataset) reveal that our proposed framework outperforms the state-of-the-art top-down pose estimation networks (HRNet-w32, HRNet-w32 + UDP and Resnet-50 Simple Baselines) in processing speed (∼29 FPS) and model size (5.23 M) for PL detection. Thus, the comprehensive experimental results demonstrate the effectiveness of our proposed framework. Our code is available from Github (https://www.github.com/rubeea/pl_mmpose).

Named Entity Identification in the Power Dispatch Domain Based on RoBERTa-Attention-FL Model

Named entity identification is an important step in building a knowledge graph of the grid domain, which contains a certain number of nested entities. To address the issue of nested entities in the Chinese power dispatching domain’s named entity recognition, we propose a RoBERTa-Attention-FL model. This model effectively recognizes nested entities using the span representation annotation method. We extract the output values from RoBERTa’s middle 4–10 layers, obtain syntactic information from the Transformer Encoder layers via the multi-head self-attention mechanism, and integrate it with deep semantic information output from RoBERTa’s last layer. During training, we use Focal Loss to mitigate the sample imbalance problem. To evaluate the model’s performance, we construct named entity recognition datasets for flat and nested entities in the power dispatching domain annotated with actual power operation data, and conduct experiments. The results indicate that compared to the baseline model, the RoBERTa-Attention-FL model significantly improves recognition performance, increasing the F1-score by 4.28% to 90.35%, with an accuracy rate of 92.53% and a recall rate of 88.12%.

S-KMN: Integrating semantic features learning and knowledge mapping network for automatic quiz question annotation

Quiz question annotation aims to assign the most relevant knowledge point to a question, which is a key technology to support intelligent education applications. However, the existing methods only extract the explicit semantic information that reveals the literal meaning of a question, and ignore the implicit knowledge information that highlights the knowledge intention. To this end, an innovative dual-channel model, the Semantic-Knowledge Mapping Network (S-KMN) is proposed to enrich the question representation from two perspectives, semantic and knowledge, simultaneously. It integrates semantic features learning and knowledge mapping network (KMN) to extract explicit semantic features and implicit knowledge features of questions,respectively. Designing KMN to extract implicit knowledge features is the focus of this study. First, the context-aware and sequence information of knowledge attribute words in the question text is integrated into the knowledge attribute graph to form the knowledge representation of each question. Second, learning a projection matrix, which maps the knowledge representation to the latent knowledge space based on the scene base vectors, and the weighted summations of these base vectors serve as knowledge features. To enrich the question representation, an attention mechanism is introduced to fuse explicit semantic features and implicit knowledge features, which realizes further cognitive processing on the basis of understanding semantics. The experimental results on 19,410 real-world physics quiz questions in 30 knowledge points demonstrate that the S-KMN outperforms the state-of-the-art text classification-based question annotation method. Comprehensive analysis and ablation studies validate the superiority of our model in selecting knowledge-specific features.