Annotation Speed Research Articles

Is Boundary Annotation Necessary? Evaluating Boundary-Free Approaches to Improve Clinical Named Entity Annotation Efficiency: Case Study.

Named entity recognition (NER) is a fundamental task in natural language processing. However, it is typically preceded by named entity annotation, which poses several challenges, especially in the clinical domain. For instance, determining entity boundaries is one of the most common sources of disagreements between annotators due to questions such as whether modifiers or peripheral words should be annotated. If unresolved, these can induce inconsistency in the produced corpora, yet, on the other hand, strict guidelines or adjudication sessions can further prolong an already slow and convoluted process. The aim of this study is to address these challenges by evaluating 2 novel annotation methodologies, lenient span and point annotation, aiming to mitigate the difficulty of precisely determining entity boundaries. We evaluate their effects through an annotation case study on a Japanese medical case report data set. We compare annotation time, annotator agreement, and the quality of the produced labeling and assess the impact on the performance of an NER system trained on the annotated corpus. We saw significant improvements in the labeling process efficiency, with up to a 25% reduction in overall annotation time and even a 10% improvement in annotator agreement compared to the traditional boundary-strict approach. However, even the best-achieved NER model presented some drop in performance compared to the traditional annotation methodology. Our findings demonstrate a balance between annotation speed and model performance. Although disregarding boundary information affects model performance to some extent, this is counterbalanced by significant reductions in the annotator's workload and notable improvements in the speed of the annotation process. These benefits may prove valuable in various applications, offering an attractive compromise for developers and researchers.

On-the-fly point annotation for fast medical video labeling.

In medical research, deep learning models rely on high-quality annotated data, a process often laborious and time-consuming. This is particularly true for detection tasks where bounding box annotations are required. The need to adjust two corners makes the process inherently frame-by-frame. Given the scarcity of experts' time, efficient annotation methods suitable for clinicians are needed. We propose an on-the-fly method for live video annotation to enhance the annotation efficiency. In this approach, a continuous single-point annotation is maintained by keeping the cursor on the object in a live video, mitigating the need for tedious pausing and repetitive navigation inherent in traditional annotation methods. This novel annotation paradigm inherits the point annotation's ability to generate pseudo-labels using a point-to-box teacher model. We empirically evaluate this approach by developing a dataset and comparing on-the-fly annotation time against traditional annotation method. Using our method, annotation speed was faster than the traditional annotation technique. We achieved a mean improvement of AP@50 over conventional method at equivalent annotation budgets on the developed dataset. Without bells and whistles, our approach offers a significant speed-up in annotation tasks. It can be easily implemented on any annotation platform to accelerate the integration of deep learning in video-based medical research.

Empirical legal analysis simplified: reducing complexity through automatic identification and evaluation of legally relevant factors.

This paper investigates the potential for reducing the complexity of AI and Law and empirical legal studies projects through a novel annotation methodology that relies on GPT Family Models to assist human annotators. Improving the speed, cost and quality of annotation could greatly benefit such projects. In modelling types of legal claims, researchers in the fields of empirical legal studies and AI and Law have long relied on manually annotating factors in case texts. To demonstrate our methodology, we employ cases and factors regarding whether a police officer has constitutional authority to detain a motorist on the basis of the officer's suspicion that the motorist is trafficking drugs. Our results demonstrate how recent advances in text analytics can reduce the burden of identifying factors in large numbers of cases and improve machine learning models' predictions of case outcomes. This article is part of the theme issue 'A complexity science approach to law and governance'.

Visual Prompting Based Incremental Learning for Semantic Segmentation of Multiplex Immuno-Flourescence Microscopy Imagery.

Deep learning approaches are state-of-the-art for semantic segmentation of medical images, but unlike many deep learning applications, medical segmentation is characterized by small amounts of annotated training data. Thus, while mainstream deep learning approaches focus on performance in domains with large training sets, researchers in the medical imaging field must apply new methods in creative ways to meet the more constrained requirements of medical datasets. We propose a framework for incrementally fine-tuning a multi-class segmentation of a high-resolution multiplex (multi-channel) immuno-flourescence image of a rat brain section, using a minimal amount of labelling from a human expert. Our framework begins with a modified Swin-UNet architecture that treats each biomarker in the multiplex image separately and learns an initial "global" segmentation (pre-training). This is followed by incremental learning and refinement of each class using a very limited amount of additional labeled data provided by a human expert for each region and its surroundings. This incremental learning utilizes the multi-class weights as an initialization and uses the additional labels to steer the network and optimize it for each region in the image. In this way, an expert can identify errors in the multi-class segmentation and rapidly correct them by supplying the model with additional annotations hand-picked from the region. In addition to increasing the speed of annotation and reducing the amount of labelling, we show that our proposed method outperforms a traditional multi-class segmentation by a large margin.

ICNet: A Dual-Branch Instance Segmentation Network for High-Precision Pig Counting

A clear understanding of the number of pigs plays a crucial role in breeding management. Computer vision technology possesses several advantages, as it is harmless and labour-saving compared to traditional counting methods. Nevertheless, the existing methods still face some challenges, such as: (1) the lack of a substantial high-precision pig-counting dataset; (2) creating a dataset for instance segmentation can be time-consuming and labor-intensive; (3) interactive occlusion and overlapping always lead to incorrect recognition of pigs; (4) existing methods for counting such as object detection have limited accuracy. To address the issues of dataset scarcity and labor-intensive manual labeling, we make a semi-auto instance labeling tool (SAI) to help us to produce a high-precision pig counting dataset named Count1200 including 1220 images and 25,762 instances. The speed at which we make labels far exceeds the speed of manual annotation. A concise and efficient instance segmentation model built upon several novel modules, referred to as the Instances Counting Network (ICNet), is proposed in this paper for pig counting. ICNet is a dual-branch model ingeniously formed of a combination of several layers, which is named the Parallel Deformable Convolutions Layer (PDCL), which is trained from scratch and primarily composed of a couple of parallel deformable convolution blocks (PDCBs). We effectively leverage the characteristic of modeling long-range sequences to build our basic block and compute layer. Along with the benefits of a large effective receptive field, PDCL achieves a better performance for multi-scale objects. In the trade-off between computational resources and performance, ICNet demonstrates excellent performance and surpasses other models in Count1200, AP of 71.4% and AP50 of 95.7% are obtained in our experiments. This work provides inspiration for the rapid creation of high-precision datasets and proposes an accurate approach to pig counting.

Parsing Netlists of Integrated Circuits from Images via Graph Attention Network.

A massive number of paper documents that include important information such as circuit schematics can be converted into digital documents by optical sensors like scanners or digital cameras. However, extracting the netlists of analog circuits from digital documents is an exceptionally challenging task. This process aids enterprises in digitizing paper-based circuit diagrams, enabling the reuse of analog circuit designs and the automatic generation of datasets required for intelligent design models in this domain. This paper introduces a bottom-up graph encoding model aimed at automatically parsing the circuit topology of analog integrated circuits from images. The model comprises an improved electronic component detection network based on the Swin Transformer, an algorithm for component port localization, and a graph encoding model. The objective of the detection network is to accurately identify component positions and types, followed by automatic dataset generation through port localization, and finally, utilizing the graph encoding model to predict potential connections between circuit components. To validate the model's performance, we annotated an electronic component detection dataset and a circuit diagram dataset, comprising 1200 and 3552 training samples, respectively. Detailed experimentation results demonstrate the superiority of our proposed enhanced algorithm over comparative algorithms across custom and public datasets. Furthermore, our proposed port localization algorithm significantly accelerates the annotation speed of circuit diagram datasets.

Conserved unique peptide patterns (CUPP) online platform 2.0: implementation of+1000 JGI fungal genomes.

Carbohydrate-processing enzymes, CAZymes, are classified into families based on sequence and three-dimensional fold. Because many CAZyme families contain members of diverse molecular function (different EC-numbers), sophisticated tools are required to further delineate these enzymes. Such delineation is provided by the peptide-based clustering method CUPP, Conserved Unique Peptide Patterns. CUPP operates synergistically with the CAZy family/subfamily categorizations to allow systematic exploration of CAZymes by defining small protein groups with shared sequence motifs. The updated CUPP library contains 21,930 of such motif groups including 3,842,628 proteins. The new implementation of the CUPP-webserver, https://cupp.info/, now includes all published fungal and algal genomes from the Joint Genome Institute (JGI), genome resources MycoCosm and PhycoCosm, dynamically subdivided into motif groups of CAZymes. This allows users to browse the JGI portals for specific predicted functions or specific protein families from genome sequences. Thus, a genome can be searched for proteins having specific characteristics. All JGI proteins have a hyperlink to a summary page which links to the predicted gene splicing including which regions have RNA support. The new CUPP implementation also includes an update of the annotation algorithm that uses only a fourth of the RAM while enabling multi-threading, providing an annotation speed below 1 ms/protein.

Efficient end-to-end learning for cell segmentation with machine generated weak annotations

Automated cell segmentation from optical microscopy images is usually the first step in the pipeline of single-cell analysis. Recently, deep-learning based algorithms have shown superior performances for the cell segmentation tasks. However, a disadvantage of deep-learning is the requirement for a large amount of fully annotated training data, which is costly to generate. Weakly-supervised and self-supervised learning is an active research area, but often the model accuracy is inversely correlated with the amount of annotation information provided. Here we focus on a specific subtype of weak annotations, which can be generated programmably from experimental data, thus allowing for more annotation information content without sacrificing the annotation speed. We designed a new model architecture for end-to-end training using such incomplete annotations. We have benchmarked our method on a variety of publicly available datasets, covering both fluorescence and bright-field imaging modality. We additionally tested our method on a microscopy dataset generated by us, using machine-generated annotations. The results demonstrated that our models trained under weak supervision can achieve segmentation accuracy competitive to, and in some cases, surpassing, state-of-the-art models trained under full supervision. Therefore, our method can be a practical alternative to the established full-supervision methods.

Pilot study to evaluate tools to collect pathologist annotations for validating machine learning algorithms.

Purpose: Validation of artificial intelligence (AI) algorithms in digital pathology with a reference standard is necessary before widespread clinical use, but few examples focus on creating a reference standard based on pathologist annotations. This work assesses the results of a pilot study that collects density estimates of stromal tumor-infiltrating lymphocytes (sTILs) in breast cancer biopsy specimens. This work will inform the creation of a validation dataset for the evaluation of AI algorithms fit for a regulatory purpose. Approach: Collaborators and crowdsourced pathologists contributed glass slides, digital images, and annotations. Here, "annotations" refer to any marks, segmentations, measurements, or labels a pathologist adds to a report, image, region of interest (ROI), or biological feature. Pathologists estimated sTILs density in 640 ROIs from hematoxylin and eosin stained slides of 64 patients via two modalities: an optical light microscope and two digital image viewing platforms. Results: The pilot study generated 7373 sTILs density estimates from 29 pathologists. Analysis of annotations found the variability of density estimates per ROI increases with the mean; the root mean square differences were 4.46, 14.25, and 26.25 as the mean density ranged from 0% to 10%, 11% to 40%, and 41% to 100%, respectively. The pilot study informs three areas of improvement for future work: technical workflows, annotation platforms, and agreement analysis methods. Upgrades to the workflows and platforms will improve operability and increase annotation speed and consistency. Conclusions: Exploratory data analysis demonstrates the need to develop new statistical approaches for agreement. The pilot study dataset and analysis methods are publicly available to allow community feedback. The development and results of the validation dataset will be publicly available to serve as an instructive tool that can be replicated by developers and researchers.

CDCAT: A multi-language cross-document entity and event coreference annotation tool

A tool for the manual annotation of cross-document entity and event coreferences that helps annotators to label mention coreference relations in text is essential for the annotation of coreference corpora. To the best of our knowledge, CROss-document Main Events and entities Recognition (CROMER) is the only open-source manual annotation tool available for cross-document entity and event coreferences. However, CROMER lacks multi-language support and extensibility. Moreover, to label cross-document mention coreference relations, CROMER requires the support of another intra-document coreference annotation tool known as Content Annotation Tool, which is now unavailable. To address these problems, we introduce Cross-Document Coreference Annotation Tool (CDCAT), a new multi-language open-source manual annotation tool for cross-document entity and event coreference, which can handle different input/output formats, preprocessing functions, languages, and annotation systems. Using this new tool, annotators can label a reference relation with only two mouse clicks. Best practice analyses reveal that annotators can reach an annotation speed of 0.025 coreference relations per second on a corpus with a coreference density of 0.076 coreference relations per word. As the first multi-language open-source cross-document entity and event coreference annotation tool, CDCAT can theoretically achieve higher annotation efficiency than CROMER.

Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists

BackgroundMachine learning, especially deep learning, is becoming more and more relevant in research and development in the medical domain. For all the supervised deep learning applications, data is the most critical factor in securing successful implementation and sustaining the progress of the machine learning model. Especially gastroenterological data, which often involves endoscopic videos, are cumbersome to annotate. Domain experts are needed to interpret and annotate the videos. To support those domain experts, we generated a framework. With this framework, instead of annotating every frame in the video sequence, experts are just performing key annotations at the beginning and the end of sequences with pathologies, e.g., visible polyps. Subsequently, non-expert annotators supported by machine learning add the missing annotations for the frames in-between.MethodsIn our framework, an expert reviews the video and annotates a few video frames to verify the object’s annotations for the non-expert. In a second step, a non-expert has visual confirmation of the given object and can annotate all following and preceding frames with AI assistance. After the expert has finished, relevant frames will be selected and passed on to an AI model. This information allows the AI model to detect and mark the desired object on all following and preceding frames with an annotation. Therefore, the non-expert can adjust and modify the AI predictions and export the results, which can then be used to train the AI model.ResultsUsing this framework, we were able to reduce workload of domain experts on average by a factor of 20 on our data. This is primarily due to the structure of the framework, which is designed to minimize the workload of the domain expert. Pairing this framework with a state-of-the-art semi-automated AI model enhances the annotation speed further. Through a prospective study with 10 participants, we show that semi-automated annotation using our tool doubles the annotation speed of non-expert annotators compared to a well-known state-of-the-art annotation tool.ConclusionIn summary, we introduce a framework for fast expert annotation for gastroenterologists, which reduces the workload of the domain expert considerably while maintaining a very high annotation quality. The framework incorporates a semi-automated annotation system utilizing trained object detection models. The software and framework are open-source.

TagLab: AI‐assisted annotation for the fast and accurate semantic segmentation of coral reef orthoimages

AbstractSemantic segmentation is a widespread image analysis task; in some applications, it requires such high accuracy that it still has to be done manually, taking a long time. Deep learning‐based approaches can significantly reduce such times, but current automated solutions may produce results below expert standards. We propose agLab, an interactive tool for the rapid labelling and analysis of orthoimages that speeds up semantic segmentation. TagLab follows a human‐centered artificial intelligence approach that, by integrating multiple degrees of automation, empowers human capabilities. We evaluated TagLab's efficiency in annotation time and accuracy through a user study based on a highly challenging task: the semantic segmentation of coral communities in marine ecology. In the assisted labelling of corals, TagLab increased the annotation speed by approximately 90% for nonexpert annotators while preserving the labelling accuracy. Furthermore, human–machine interaction has improved the accuracy of fully automatic predictions by about 7% on average and by 14% when the model generalizes poorly. Considering the experience done through the user study, TagLab has been improved, and preliminary investigations suggest a further significant reduction in annotation times.

Construction Model and Evaluation of Dynamic Knowledge Map for Deep Learning

The related technologies of knowledge map have been widely applied in many fields, such as search engine, intelligent question and answer, language understanding, recommendation calculation, big data decision analysis and so on. Deep learning originates from the research of artificial neural network, which uses deep network to learn knowledge from massive data. Its advantage is that it can obtain features without manual work. It is one of the current research hotspots to integrate the method of deep learning into the application of knowledge map. In this paper, a dynamic knowledge map reasoning method based on multi-relationship cyclic events is proposed, and an improved multi-relationship proximity aggregator is used to fuse the neighborhood information of target entities to obtain a more accurate representation of entity neighborhood vectors. It improves the speed of manual annotation, reduces the time and labor cost of manual annotation, and improves the migration ability of event extraction system.

InstaDam: Open-Source Platform for Rapid Semantic Segmentation of Structural Damage

The tremendous success of automated methods for the detection of damage in images of civil infrastructure has been fueled by exponential advances in deep learning over the past decade. In particular, many efforts have taken place in academia and more recently in industry that demonstrate the success of supervised deep learning methods for semantic segmentation of damage (i.e., the pixel-wise identification of damage in images). However, in graduating from the detection of damage to applications such as inspection automation, efforts have been limited by the lack of large open datasets of real-world images with annotations for multiple types of damage, and other related information such as material and component types. Such datasets for structural inspections are difficult to develop because annotating the complex and amorphous shapes taken by damage patterns remains a tedious task (requiring too many clicks and careful selection of points), even with state-of-the art annotation software. In this work, InstaDam—an open source software platform for fast pixel-wise annotation of damage—is presented. By utilizing binary masks to aid user input, InstaDam greatly speeds up the annotation process and improves the consistency of annotations. The masks are generated by applying established image processing techniques (IPTs) to the images being annotated. Several different tunable IPTs are implemented to allow for rapid annotation of a wide variety of damage types. The paper first describes details of InstaDam’s software architecture and presents some of its key features. Then, the benefits of InstaDam are explored by comparing it to the Image Labeler app in Matlab. Experiments are conducted where two employed student annotators are given the task of annotating damage in a small dataset of images using Matlab, InstaDam without IPTs, and InstaDam. Comparisons are made, quantifying the improvements in annotation speed and annotation consistency across annotators. A description of the statistics of the different IPTs used for different annotated classes is presented. The gains in annotation consistency and efficiency from using InstaDam will facilitate the development of datasets that can help to advance research into automation of visual inspections.

Integrating Manual and Automatic Annotation for the Creation of Discourse Network Data Sets

This article investigates the integration of machine learning in the political claim annotation workflow with the goal to partially automate the annotation and analysis of large text corpora. It introduces the MARDY annotation environment and presents results from an experiment in which the annotation quality of annotators with and without machine learning based annotation support is compared. The design and setting aim to measure and evaluate: a) annotation speed; b) annotation quality; and c) applicability to the use case of discourse network generation. While the results indicate only slight increases in terms of annotation speed, the authors find a moderate boost in annotation quality. Additionally, with the help of manual annotation of the actors and filtering out of the false positives, the machine learning based annotation suggestions allow the authors to fully recover the core network of the discourse as extracted from the articles annotated during the experiment. This is due to the redundancy which is naturally present in the annotated texts. Thus, assuming a research focus not on the complete network but the network core, an AI-based annotation can provide reliable information about discourse networks with much less human intervention than compared to the traditional manual approach.

Comparing Targeted vs. Untargeted MS2 Data-Dependent Acquisition for Peak Annotation in LC-MS Metabolomics.

One of the most widely used strategies for metabolite annotation in untargeted LCMS is based on the analysis of MSn spectra acquired using data-dependent acquisition (DDA), where precursor ions are sequentially selected from MS scans based on user-selected criteria. However, the number of MSn spectra that can be acquired during a chromatogram is limited and a trade-off between analytical speed, sensitivity and coverage must be ensured. In this research, we compare four different strategies for automated MS2 DDA, which can be easily implemented in the frame of standard QA/QC workflows for untargeted LC–MS. These strategies consist of (i) DDA in the MS working range; (ii) iterated DDA split into several m/z intervals; (iii) dynamic iterated DDA of (pre)selected potentially informative features; and (iv) dynamic iterated DDA of (pre)annotated metabolic features using a reference database. Their performance was assessed using the analysis of human milk samples as model example by comparing the percentage of LC–MS features selected as the precursor ion for MS2, the number, and class of annotated features, the speed and confidence of feature annotation, and the number of LC runs required.

Benchmarking immunoinformatic tools for the analysis of antibody repertoire sequences.

SummaryAntibody repertoires reveal insights into the biology of the adaptive immune system and empower diagnostics and therapeutics. There are currently multiple tools available for the annotation of antibody sequences. All downstream analyses such as choosing lead drug candidates depend on the correct annotation of these sequences; however, a thorough comparison of the performance of these tools has not been investigated. Here, we benchmark the performance of commonly used immunoinformatic tools, i.e. IMGT/HighV-QUEST, IgBLAST and MiXCR, in terms of reproducibility of annotation output, accuracy and speed using simulated and experimental high-throughput sequencing datasets.We analyzed changes in IMGT reference germline database in the last 10 years in order to assess the reproducibility of the annotation output. We found that only 73/183 (40%) V, D and J human genes were shared between the reference germline sets used by the tools. We found that the annotation results differed between tools. In terms of alignment accuracy, MiXCR had the highest average frequency of gene mishits, 0.02 mishit frequency and IgBLAST the lowest, 0.004 mishit frequency. Reproducibility in the output of complementarity determining three regions (CDR3 amino acids) ranged from 4.3% to 77.6% with preprocessed data. In addition, run time of the tools was assessed: MiXCR was the fastest tool for number of sequences processed per unit of time. These results indicate that immunoinformatic analyses greatly depend on the choice of bioinformatics tool. Our results support informed decision-making to immunoinformaticians based on repertoire composition and sequencing platforms. Availability and implementationAll tools utilized in the paper are free for academic use.Supplementary information Supplementary data are available at Bioinformatics online.

A User Interface for Optimizing Radiologist Engagement in Image Data Curation for Artificial Intelligence.

To delineate image data curation needs and describe a locally designed graphical user interface (GUI) to aid radiologists in image annotation for artificial intelligence (AI) applications in medical imaging. GUI components support image analysis toolboxes, picture archiving and communication system integration, third-party applications, processing of scripting languages, and integration of deep learning libraries. For clinical AI applications, GUI components included two-dimensional segmentation and classification; three-dimensional segmentation and quantification; and three-dimensional segmentation, quantification, and classification. To assess radiologist engagement and performance efficiency associated with GUI-related capabilities, image annotation rate (studies per day) and speed (minutes per case) were evaluated in two clinical scenarios of varying complexity: hip fracture detection and coronary atherosclerotic plaque demarcation and stenosis grading. For hip fracture, 1050 radiographs were annotated over 7 days (150 studies per day; median speed: 10 seconds per study [interquartile range, 3-21 seconds per study]). A total of 294 coronary CT angiographic studies with 1843 arteries and branches were annotated for atherosclerotic plaque over 23 days (15.2 studies [80.1 vessels] per day; median speed: 6.08 minutes per study [interquartile range, 2.8-10.6 minutes per study] and 73 seconds per vessel [interquartile range, 20.9-155 seconds per vessel]). GUI-component compatibility with common image analysis tools facilitates radiologist engagement in image data curation, including image annotation, supporting AI application development and evolution for medical imaging. When complemented by other GUI elements, a continuous integrated workflow supporting formation of an agile deep neural network life cycle results.Supplemental material is available for this article.© RSNA, 2019.

Fast, versatile and quantitative annotation of complex images.

We report a generic smartphone app for quantitative annotation of complex images. The app is simple enough to be used by children, and annotation tasks are distributed across app users, contributing to efficient annotation. We demonstrate its flexibility and speed by annotating >30,000 images, including features of rice root growth and structure, stem cell aggregate morphology, and complex worm (Caenorhabditis elegans) postures, for which we show that the speed of annotation is >130-fold faster than state-of-the-art techniques with similar accuracy.

Greedy Annotation of Remote Sensing Image Scenes Based on Automatic Aggregation via Hierarchical Similarity Diffusion

As a basic and key problem in the remote sensing community, remote sensing image scene understanding (RSISU) has attracted increasing research interest. In recent years, deep learning has revolutionized RSISU. However, the great success of deep learning strongly depends on the availability of a large-scale data set with explicit labels. Although remote sensing image scene data sets have been publicly released for a limited number of remote sensing image types, data sets of many types of remote sensing images are still not available, which limits the applicability of deep learning. Generally, exhaustively labeling remote sensing image scene data sets via manual labor is time consuming, and it becomes impossible when the data set volume is very large. Hence, it is necessary to develop an intelligent annotation approach to efficiently and accurately label these data sets. Based on a prior assumption of consistency, namely, the assumption that samples within the same cluster are likely to have the same label, this paper proposes a novel annotation method for remote sensing image scene data sets called automatic aggregation via hierarchical similarity diffusion (AA-HSD). More specifically, each remote sensing image scene is represented by multiple features. To make full use of these complementary features, this paper proposes a new hierarchical similarity diffusion method for robustly measuring the similarity matrix of the scenes in the data set. Based on this similarity matrix, the scenes are automatically aggregated into clusters. Instead of annotating the data set scene by scene, as in the traditional manual annotation solution, we annotate the data set cluster by cluster, which dramatically increases the annotation speed while achieving a very high accuracy. Extensive experiments on two public remote sensing image scene data sets demonstrate the validity of our proposed AA-HSD method, which outperforms all competing baselines.