Annotation Problem Research Articles

Impact of Annotation Noise on Histopathology Nucleus Segmentation

Abstract Deep learning is often used for automated diagnosis support in biomedical image processing scenarios. Annotated datasets are essential for the supervised training of deep neural networks. The problem of consistent and noise-free annotation remains for experts such as pathologists. The variability within an annotator (intra) and the variability between annotators (inter) are current challenges. In clinical practice or biology, instance segmentation is a common task, but a comprehensive and quantitative study regarding the impact of noisy annotations lacks. In this paper, we present a concept to categorize and simulate various types of annotation noise as well as an evaluation of the impact on deep learning pipelines. Thereby, we use the multi-organ histology image dataset MoNuSeg to discuss the influence of annotator variability. We provide annotation recommendations for clinicians to achieve high-quality automated diagnostic algorithms.

Predicting long-term stock movements with fused textual features of Chinese research reports

By shaping investors’ perceptions and assessments of the stock, research reports have significant impacts on the stock market. Due to the limitations of text mining technology, it is difficult for researchers to effectively utilize long research reports, and most studies mainly focus on investor sentiment. However, due to the lack of appropriate open-domain toolkits, the annotations of sentiment often require expensive manual labeling. In addition, most existing studies have shown the success of using textual data as a supplement to historical price data in short-term forecasting, but not in long-term forecasting. To cover this gap and solve the problem of difficult annotations, we introduce a novel knowledge-driven approach for long-term stock movement prediction based on Chinese research reports. In detail, a new long-term Stock Movement Prediction dataset composed of Research Reports is proposed, namely SMPRR. It is mainly composed of long, formal, and professional research reports and historical prices. Furthermore, we propose the Multi-module Feature Fusion method based on the pre-trained language model FinBERT (MFF-FinBERT), which can effectively fuse textual features from research reports. The experiment results show that the proposed model has achieved better performance than existing methods in the forecasting of one-year stock movements, and the accuracy reaches 79.2%. The results also indicate that the basic information of stocks plays an important role in long-term forecasting, which is in line with the theory of value investing.

Detecting tiny objects in aerial images: A normalized Wasserstein distance and a new benchmark

Tiny object detection (TOD) in aerial images is challenging since a tiny object only contains a few pixels. State-of-the-art object detectors do not provide satisfactory results on tiny objects due to the lack of supervision from discriminative features. Our key observation is that the Intersection over Union (IoU) metric and its extensions are very sensitive to the location deviation of the tiny objects, which drastically deteriorates the quality of label assignment when used in anchor-based detectors. To tackle this problem, we propose a new evaluation metric dubbed Normalized Wasserstein Distance (NWD) and a new RanKing-based Assigning (RKA) strategy for tiny object detection. The proposed NWD-RKA strategy can be easily embedded into all kinds of anchor-based detectors to replace the standard IoU threshold-based one, significantly improving label assignment and providing sufficient supervision information for network training. Tested on four datasets, NWD-RKA can consistently improve tiny object detection performance by a large margin. Besides, observing prominent noisy labels in the Tiny Object Detection in Aerial Images (AI-TOD) dataset, we are motivated to meticulously relabel it and release AI-TOD-v2 and its corresponding benchmark. In AI-TOD-v2, the missing annotation and location error problems are considerably mitigated, facilitating more reliable training and validation processes. Embedding NWD-RKA into DetectoRS, the detection performance achieves 4.3 AP points improvement over state-of-the-art competitors on AI-TOD-v2. Datasets, codes, and more visualizations are available at: https://chasel-tsui.github.io/AI-TOD-v2/.

Image retrieval based on weighted nearest neighbor tag prediction

Abstract With the development of communication and computer technology, the application of big data technology has become increasingly widespread. Reasonable, effective, and fast retrieval methods for querying information from massive data have become an important content of current research. This article provides an image retrieval method based on the weighted nearest neighbor label prediction for the problem of automatic image annotation and keyword image retrieval. In order to improve the performance of the test method, scientific experimental verification was implemented. The nearest neighbor weights are determined by maximizing the training image annotation, and experiments are carried out from multiple angles based on the Mahalanobis metric learning integration model. The experimental results show that the proposed tag correlation prediction propagation model has obvious improvements in accuracy, recall rate, break-even point, and overall average accuracy performance compared with other widely used algorithm models.

Two- and three-dimensional deep human detection by generating orthographic top view image from dense point cloud

Human detection still suffers from occlusion, complex backgrounds, and scale-variant problems. Projecting three-dimensional (3D) points onto the ground to generate an orthographic top view (OTV) image for detection can effectively alleviate these problems. However, depth sensors may be placed arbitrarily, making it difficult to create OTV images by the dense point cloud converted from a depth image. We focus on the generation of OTV images and human detection via the constructed OTV image. First, we propose a ground plane extraction method that is well suitable for various camera positions and orientations in complex scenes. Next, points are converted to a uniform coordinate system by ground parameters and encoded to generate a three-channel OTV image. Then, the mainstream two-dimensional (2D) network is employed to detect the human directly on OTV images and further obtain the 3D bounding box by computing the mapping from the OTV image. Besides, we propose a semiautomated annotation method to solve the problem of few OTV image annotations. The proposed method is evaluated on the EPFL dataset, including two subsets, and achieves state-of-the-art performance compared with the existing approaches. Moreover, our 2D and 3D human detection method can run more than 26FPS on the CPU.

Feature extraction with spectral clustering for gene function prediction using hierarchical multi-label classification

Gene annotation addresses the problem of predicting unknown associations between gene and functions (e.g., biological processes) of a specific organism. Despite recent advances, the cost and time demanded by annotation procedures that rely largely on in vivo biological experiments remain prohibitively high. This paper presents a novel in silico approach for to the annotation problem that combines cluster analysis and hierarchical multi-label classification (HMC). The approach uses spectral clustering to extract new features from the gene co-expression network (GCN) and enrich the prediction task. HMC is used to build multiple estimators that consider the hierarchical structure of gene functions. The proposed approach is applied to a case study on Zea mays, one of the most dominant and productive crops in the world. The results illustrate how in silico approaches are key to reduce the time and costs of gene annotation. More specifically, they highlight the importance of: (1) building new features that represent the structure of gene relationships in GCNs to annotate genes; and (2) taking into account the structure of biological processes to obtain consistent predictions.

Clustering-Based Semi-Supervised Cross-Modal Retrieval Using Scene Graph

In this paper, we propose a clustering-based semi-supervised cross-modal retrieval method to relieve the problem of insufficient annotation in cross-modal datasets. First, we reconstruct cross-modal data as scene graph structure to filter meaningless information. Second, we extract embedding representation features of images and texts to put them into a common space. Finally, we propose a clustering-based classification method with modality-independent constraint to discriminate samples. According to our experimental results, significant improvement on performance shows the accuracy of our method in terms of three widely used cross-modal datasets compared with the state-of-the-art methods.

Analysis of the Reform Effectiveness of Vocational Skill Identification Based on the New Social Training Model in the New Era under Deep Learning Assessment.

In order to be able to make full use of domain knowledge to improve the performance of skill word extraction, this paper proposes a skill word extraction method based on a combination of deep learning and corpus features. Skill word extraction is transformed into a sequence annotation problem, and based on the basic model of sequence annotation, Bi-LSTM-CRF, corpus features are added to the input layer, and the output of the input layer is connected with the Bi-LSTM output as the input of the CRF layer. The experimental results show timely updating of the question bank, paying attention to the quality of vocational skill identification, and strictly managing the issuance of vocational qualification false certificates.

Analysis of Distributed English Chunk Recognition under Correlation Evaluation Based on Deep Belief Network Model

Most of the traditional methods of English text chunk recognition are solved by setting the corresponding phrase identifier numbers and eventually transforming the chunk recognition problem into a lexical annotation problem. In language recognition, the traditional MFCC features are easily contaminated by noise and have weak noise immunity due to the insufficient amount of information on each frame of the signal. At the same time, SDC feature extraction methods commonly used today require artificial settings in parameter selection, which increases the uncertainty of recognition results. The method of identifying English text chunks by association evaluation of central word extensions identifies English text chunks from a different perspective. It has the following features: (i) each phase is considered as a cluster with the central word as the core, and the internal composition pattern of each phrase is fully considered; (ii) the results are dynamically evaluated using association and confidence. The results show that the proposed method can achieve higher recognition rate than traditional feature extraction methods. The recognition rate is faster, and the F -measure value of English block recognition reaches 94.05%, which is comparable to the best results so far.

Context-Dependent Annotation Method in Emergency Monitoring Information Systems

The article presents the method of context-dependent annotations used in solving problems of emergency monitoring on the basis of information systems. The method is based on a spectral language model that allows solving various information search problems taking into account the specific features of the applied area. The functional model of emergency monitoring task in IDEF0 notation is presented. The task of context-dependent annotating operational summaries as a basis for generating preliminary reports is formulated. The main problems that arise in solving this problem on a large volume of initial data and can be critical in conditions of fast-developing emergencies are identified. The problem of context-dependent annotations in the conditions of existing restrictions was set, the main language units used in solving it are described. A flowchart of solving the problem of context-dependent annotating taking into account the specifics of the subject area is presented, and the implementation of each stage of this algorithm is described in detail. Described is a method of determining relevance of a text fragment to a target query based on a spectral language model. Basic assessments of annotation quality are described. A comparative analysis of the quality and speed of constructing annotations manually and on the basis of the presented method using assessory estimates was carried out. The effectiveness of the context-dependent annotation method in processing a large number of documents in conditions of fast-developing emergency situations requiring emergency decision-making is shown.

From pairwise to multiple spliced alignment.

Alternative splicing is a ubiquitous process in eukaryotes that allows distinct transcripts to be produced from the same gene. Yet, the study of transcript evolution within a gene family is still in its infancy. One prerequisite for this study is the availability of methods to compare sets of transcripts while accounting for their splicing structure. In this context, we generalize the concept of pairwise spliced alignments (PSpAs) to multiple spliced alignments (MSpAs). MSpAs have several important purposes in addition to empowering the study of the evolution of transcripts. For instance, it is a key to improving the prediction of gene models, which is important to solve the growing problem of genome annotation. Despite its essentialness, a formal definition of the concept and methods to compute MSpAs are still lacking. We introduce the MSpA problem and the SplicedFamAlignMulti (SFAM) method, to compute the MSpA of a gene family. Like most multiple sequence alignment (MSA) methods that are generally greedy heuristic methods assembling pairwise alignments, SFAM combines all PSpAs of coding DNA sequences and gene sequences of a gene family into an MSpA. It produces a single structure that represents the superstructure and models of the gene family. Using real vertebrate and simulated gene family data, we illustrate the utility of SFAM for computing accurate gene family superstructures, MSAs, inferring splicing orthologous groups and improving gene-model annotations. The supporting data and implementation of SFAM are freely available at https://github.com/UdeS-CoBIUS/SpliceFamAlignMulti. Supplementary data are available at Bioinformatics Advances online.

Self-Supervised Matting-Specific Portrait Enhancement and Generation.

We resolve the ill-posed alpha matting problem from a completely different perspective. Given an input portrait image, instead of estimating the corresponding alpha matte, we focus on the other end, to subtly enhance this input so that the alpha matte can be easily estimated by any existing matting models. This is accomplished by exploring the latent space of GAN models. It is demonstrated that interpretable directions can be found in the latent space and they correspond to semantic image transformations. We further explore this property in alpha matting. Particularly, we invert an input portrait into the latent code of StyleGAN, and our aim is to discover whether there is an enhanced version in the latent space which is more compatible with a reference matting model. We optimize multi-scale latent vectors in the latent spaces under four tailored losses, ensuring matting-specificity and subtle modifications on the portrait. We demonstrate that the proposed method can refine real portrait images for arbitrary matting models, boosting the performance of automatic alpha matting by a large margin. In addition, we leverage the generative property of StyleGAN, and propose to generate enhanced portrait data which can be treated as the pseudo GT. It addresses the problem of expensive alpha matte annotation, further augmenting the matting performance of existing models.

Analysis of Underwater Image Processing Methods for Annotation in Deep Learning Based Fish Detection

With the advent of deep-learning (DL) techniques, image annotation has become a fundamental part of the research process. In the case of underwater image annotation, the human in charge of the task is faced whith the inherent quality problems of this kind of images. A large number of underwater image enhancement (UIE) methods have been developed aimed at improving the colors and contrast of these images. However, no attention has been paid to the specific problem of image annotation. In this case the global image quality of the processed image is less important than the fact that the objects to be annotated stand out and that their contours are easy to delineate. In this paper we evaluate seven state-of-the-art UIE techniques and rank them, through a subjective approach, according to their utility for the annotation process. The conclusion of our study is that, in general, the model-free Multiscale Retinex algorithm is preferred over more complex techniques that try to model the formation of the underwater images.

Method of automatIc text annoting based on hybrid intelligent technologies

In the article, the analysis of modern methods of automatic annotation and abstracting of texts is carried out, the disadvantages and advantages of existing methods are noted. The problem of automatic annotation is formulated. The main stages of automatic processing of information during annotation are determined. A model for processing text sources using multi-agent organizations is proposed. A method based on the hypothesis of the existence of meaning at the level of syntactic collocations has been developed. A model for solving the problem based on the extraction of N-grams and their subsequent grammatical processing is proposed.

Chinese Address Recognition Method Based on Multi-Feature Fusion

A place name is a textual identification of a specific spatial location by people and is an important carrier of geographical information. The recognition of Chinese place names is of great importance in information retrieval and event extraction. The traditional approach is to transform the recognition of Chinese place names into a sequential annotation problem, with commonly used classification models such as support vector machines and conditional random fields. In this paper, Chinese address recognition is converted into a sequential annotation task, and a multi-feature fusion approach to Chinese address recognition is proposed. A deep learning network architecture model based on the fusion of word, word and address features is constructed to convert word and word tokens and their features into vector representations; finally, the sequential annotation of sentences is performed by CRF to achieve the recognition and extraction of address information. On the autonomously constructed dataset, the proposed method MFBL (Multi-Feature-BiLSTM) improves in accuracy by 4 to 10 percentage points compared to other methods, demonstrating that the MFBL model has better performance in the address recognition task.

Linguistic Annotation of Translated Chinese Texts: Coordinating Theory, Algorithms and Data

Abstract The article tackles the problems of linguistic annotation in the Chinese texts presented in the Ruzhcorp – Russian-Chinese Parallel Corpus of RNC, and the ways to solve them. Particular attention is paid to the processing of Russian loanwords. On the one hand, we present the theoretical comparison of the widespread standards of Chinese text processing. On the other hand, we describe our experiments in three fields: word segmentation, grapheme-to-phoneme conversion, and PoS-tagging, on the specific corpus data that contains many transliterations and loanwords. As a result, we propose the preprocessing pipeline of the Chinese texts, that will be implemented in Ruzhcorp.

An LSTM Network for Apnea and Hypopnea Episodes Detection in Respiratory Signals.

One of the most common sleep disorders is sleep apnea. It manifests itself by episodes of shallow breathing or pauses in breathing during the night. Diagnosis of this disease involves polysomnography examination, which is expensive. Alternatively, diagnostic doctors can be supported with recordings from the in-home polygraphy sensors. Furthermore, numerous attempts for providing an automated apnea episodes annotation algorithm have been made. Most of them, however, do not distinguish between apnea and hypopnea episodes. In this work, a novel solution for epoch-based annotation problem is presented. Utilizing an architecture based on the long short-term memory (LSTM) networks, the proposed model provides locations of sleep disordered breathing episodes and identifies them as either apnea or hypopnea. To achieve this, special pre- and postprocessing steps have been designed. The obtained labels can be then used for calculation of the respiratory event index (REI), which serves as a disease severity indicator. The input for the model consists of the oronasal airflow along with the thoracic and abdominal respiratory effort signals. Performance of the proposed architecture was verified on the SHHS-1 and PhysioNet Sleep databases, obtaining mean REI classification error of 9.24/10.52 with standard deviation of 11.61/7.92 (SHHS-1/PhysioNet). Normal breathing, hypopnea and apnea differentiation accuracy is assessed on both databases, resulting in the correctly classified samples percentage of 86.42%/84.35%, 49.30%/58.28% and 68.20%/69.50% for normal breathing, hypopnea and apnea classes, respectively. Overall accuracies are 80.66%/82.04%. Additionally, the effect of wake periods is investigated. The results show that the proposed model can be successfully used for both episode classification and REI estimation tasks.

Front Cover: Synthetic Image Rendering Solves Annotation Problem in Deep Learning Nanoparticle Segmentation (Small Methods 7/2021)

In article number 2100223, Mill and co-workers demonstrated a novel methodology that tackles the data annotation problem for the deep learning-based segmentation of complex nanoparticle agglomerates in helium ion microscopy images. The study paves the way toward automated and high-throughput nanoparticle image analysis for a variety of applications.

Virtual-sample-based defect detection algorithm for aluminum tube surface

A surface defect is an important factor that affects product quality. However, due to the large differences in area of different surface defects, and noise on various surfaces, defect detection is challenging. The convolutional neural network (CNN)-based methods recently developed for defect detection produced higher recognition rates than traditional methods. However, they are typically trained using a supervised learning strategy and large defect sample sets which limits the practical use of these algorithms. This study proposes a novel virtual sample generation algorithm to solve the problem of insufficient defective samples and time-consuming manual annotation in current CNN-based defect detection algorithms. Next, an improved domain-adversarial neural network is proposed, which is trained on virtual and actual datasets to achieve unsupervised learning. Considering the imbalance in actual dataset, algorithm accuracy is improved by changing the proportions of defective and non-defective samples in the virtual sample set, and this strategy is experimentally verified. The performance of the proposed algorithm is compared with several top-performing defect inspection algorithms. The experimental results show that the proposed algorithm exhibits superior performance when compared to other algorithms.

Comparative Analysis of the Circular Transcriptome in Muscle, Liver, and Testis in Three Livestock Species.

Circular RNAs have been observed in a large number of species and tissues and are now recognized as a clear component of the transcriptome. Our study takes advantage of functional datasets produced within the FAANG consortium to investigate the pervasiveness of circular RNA transcription in farm animals. We describe here the circular transcriptional landscape in pig, sheep and bovine testicular, muscular and liver tissues using total 66 RNA-seq datasets. After an exhaustive detection of circular RNAs, we propose an annotation of exonic, intronic and sub-exonic circRNAs and comparative analyses of circRNA content to evaluate the variability between individuals, tissues and species. Despite technical bias due to the various origins of the datasets, we were able to characterize some features (i) (ruminant) liver contains more exonic circRNAs than muscle (ii) in testis, the number of exonic circRNAs seems associated with the sexual maturity of the animal. (iii) a particular class of circRNAs, sub-exonic circRNAs, are produced by a large variety of multi-exonic genes (protein-coding genes, long non-coding RNAs and pseudogenes) and mono-exonic genes (protein-coding genes from mitochondrial genome and small non-coding genes). Moreover, for multi-exonic genes there seems to be a relationship between the sub-exonic circRNAs transcription level and the linear transcription level. Finally, sub-exonic circRNAs produced by mono-exonic genes (mitochondrial protein-coding genes, ribozyme, and sno) exhibit a particular behavior. Caution has to be taken regarding the interpretation of the unannotated circRNA proportion in a given tissue/species: clusters of circRNAs without annotation were characterized in genomic regions with annotation and/or assembly problems of the respective animal genomes. This study highlights the importance of improving genome annotation to better consider candidate circRNAs and to better understand the circular transcriptome. Furthermore, it emphasizes the need for considering the relative “weight” of circRNAs/parent genes for comparative analyses of several circular transcriptomes. Although there are points of agreement in the circular transcriptome of the same tissue in two species, it will be not possible to do without the characterization of it in both species.