Annotated Data Research Articles

Multi-year soundscape recordings and automated call detection reveals varied impact of moonlight on calling activity of neotropical forest katydids.

Night-time light can have profound ecological effects, even when the source is natural moonlight. The impacts of light can, however, vary substantially by taxon, habitat and geographical region. We used a custom machine learning model built with the Python package Koogu to investigate the in situ effects of moonlight on the calling activity of neotropical forest katydids over multiple years. We prioritised species with calls that were commonly detected in human annotated data, enabling us to evaluate model performance. We focused on eight species of katydids that the model identified with high precision (generally greater than 0.90) and moderate-to-high recall (minimum 0.35), ensuring that detections were generally correct and that many calls were detected. These results suggest that moonlight has modest effects on the amount of calling, with the magnitude and direction of effect varying by species: half of the species showed positive effects of light and half showed negative. These findings emphasize the importance of understanding natural history for anticipating how biological communities respond to moonlight. The methods applied in this project highlight the emerging opportunities for evaluating large quantities of data with machine learning models to address ecological questions over space and time. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.

Pseudo-class part prototype networks for interpretable breast cancer classification

Interpretability in machine learning has become increasingly important as machine learning is being used in more and more applications, including those with high-stakes consequences such as healthcare where Interpretability has been regarded as a key to the successful adoption of machine learning models. However, using confounding/irrelevant information in making predictions by deep learning models, even the interpretable ones, poses critical challenges to their clinical acceptance. That has recently drawn researchers’ attention to issues beyond the mere interpretation of deep learning models. In this paper, we first investigate application of an inherently interpretable prototype-based architecture, known as ProtoPNet, for breast cancer classification in digital pathology and highlight its shortcomings in this application. Then, we propose a new method that uses more medically relevant information and makes more accurate and interpretable predictions. Our method leverages the clustering concept and implicitly increases the number of classes in the training dataset. The proposed method learns more relevant prototypes without any pixel-level annotated data. To have a more holistic assessment, in addition to classification accuracy, we define a new metric for assessing the degree of interpretability based on the comments of a group of skilled pathologists. Experimental results on the BreakHis dataset show that the proposed method effectively improves the classification accuracy and interpretability by respectively 8%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$8 \\%$$\\end{document} and 18%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$18 \\%$$\\end{document}. Therefore, the proposed method can be seen as a step toward implementing interpretable deep learning models for the detection of breast cancer using histopathology images.

Learning At a Glance: Towards Interpretable Data-Limited Continual Semantic Segmentation Via Semantic-Invariance Modelling.

Continual semantic segmentation (CSS) based on incremental learning (IL) is a great endeavour in developing human- like segmentation models. However, current CSS approaches encounter challenges in the trade-off between preserving old knowledge and learning new ones, where they still need large-scale annotated data for incremental training and lack interpretability. In this paper, we present Learning at a Glance (LAG), an efficient, robust, human- like and interpretable approach for CSS. Specifically, LAG is a simple and model-agnostic architecture, yet it achieves competitive CSS efficiency with limited incremental data. Inspired by human- like recognition patterns, we propose a semantic-invariance modelling approach via semantic features decoupling that simultaneously reconciles solid knowledge inheritance and new-term learning. Concretely, the proposed decoupling manner includes two ways, i.e., channel- wise decoupling and spatial-level neuron-relevant semantic consistency. Our approach preserves semantic-invariant knowledge as solid prototypes to alleviate catastrophic forgetting, while also constraining sample-specific contents through an asymmetric contrastive learning method to enhance model robustness during IL steps. Experimental results in multiple datasets validate the effectiveness of the proposed method. Furthermore, we introduce a novel CSS protocol that better reflects realistic data-limited CSS settings, and LAG achieves superior performance under multiple data-limited conditions.

Toward explainable AI in radiology: Ensemble-CAM for effective thoracic disease localization in chest X-ray images using weak supervised learning

Chest X-ray (CXR) imaging is widely employed by radiologists to diagnose thoracic diseases. Recently, many deep learning techniques have been proposed as computer-aided diagnostic (CAD) tools to assist radiologists in minimizing the risk of incorrect diagnosis. From an application perspective, these models have exhibited two major challenges: (1) They require large volumes of annotated data at the training stage and (2) They lack explainable factors to justify their outcomes at the prediction stage. In the present study, we developed a class activation mapping (CAM)-based ensemble model, called Ensemble-CAM, to address both of these challenges via weakly supervised learning by employing explainable AI (XAI) functions. Ensemble-CAM utilizes class labels to predict the location of disease in association with interpretable features. The proposed work leverages ensemble and transfer learning with class activation functions to achieve three objectives: (1) minimizing the dependency on strongly annotated data when locating thoracic diseases, (2) enhancing confidence in predicted outcomes by visualizing their interpretable features, and (3) optimizing cumulative performance via fusion functions. Ensemble-CAM was trained on three CXR image datasets and evaluated through qualitative and quantitative measures via heatmaps and Jaccard indices. The results reflect the enhanced performance and reliability in comparison to existing standalone and ensembled models.

Abstract PO2-15-12: Novel Transcriptomic Classification System Unveils HER2-Low Breast Tumors

Abstract Breast cancers (BCs) are segregable into subtypes based on their intrinsic molecular features. When evaluating HER2 status, clinicians often rely on IHC, which is plagued by reproducibility issues. The widely used PAM50 transcriptomic system also does not include the HER2-low subtype. Addressing these limitations is particularly important given the recent approval of targeted therapy for HER2-low BCs. Here, we developed a unique transcriptomic classification system that can identify HER2-low BCs. Accurate identification of HER2-low BCs can help physicians navigate treatment selection and, in turn, allow patients to benefit from the new targeted therapy. We analyzed 6,361 BC samples with clinical and pathological annotation, including FISH and IHC data, from three public datasets (TCGA BRCA and SCAN-B [total n = 4,457]; METABRIC [n = 1,904]) for gene expression, differential expression, gene enrichment, copy number alterations, and mutations. An algorithm was applied to categorize samples from TCGA and SCAN-B into subtypes by customizing a gene set, employing UMAP dimensionality reduction, and utilizing clustering with the HDBSCAN algorithm. Each resulting cluster that was associated with a specific gene set was named and excluded from the next analysis. After four iterations, five subtypes were sequentially identified: Basal, HER2-high, HER2-low, Luminal B (LumB), and Luminal A (LumA). The Light GBM-based hierarchical classifier model was trained on the TCGA and SCAN-B cohorts and validated on the METABRIC dataset. Samples with high ERBB2 expression and copy number amplification were classified as HER2-high, and samples with similar PAM50 expression profiles but lacking ERBB2 expression were considered HER2-low. Luminal samples were grouped into two subtypes: LumA, characterized by low proliferation rates, and LumB, characterized by high proliferation rates. HER2-high, Basal, LumA, and LumB subtypes exhibited expected mutational and gene expression profiles. We observed ERBB2 copy number amplification and high gene expression in HER2-high samples from the TCGA and METABRIC cohorts, as confirmed by FISH and IHC in 97% and 93% of samples, respectively. Conversely, we did not see high ERBB2 expression or copy number amplification in the HER2-low samples, which was concordant with an absence of HER2 3+ expression by IHC. HER2-low samples had high mutational frequencies in PTEN, KMT2C, and ERBB2 (12%, 14%, and 8% respectively, adj. p < 0.005, chi-square test). HER2-low samples also showed increased EGFR (logFC = 2.8, adj. p < 0.001) and CLDN8 (logFC = 3.4, adj. p < 0.001) expression levels. The novel classifier identified 55% of the HER2-low samples as the luminal androgen receptor (LAR) Burstein subtype, as a result of higher AR expression than in HER2-high samples (logFC = 2.9, p = 0.04). Classification of SCAN-B luminal samples into the LumA and LumB subtypes also better predicted Ki-67 positivity (≥ 20% of Ki-67+ malignant cells by IHC, F1-score = 0.77) than the previously reported PAM50 classification (F1-score = 0.62). Our refined classifier effectively defined five breast cancer subtypes, including the HER2-low subtype, based on their molecular profiles. By determining HER2 status with transcriptomic data, our classifier may help minimize IHC-related reproducibility issues and, in turn, facilitate more precise treatment decision-making. Given its concordance with copy number amplification, IHC, and FISH data, our classification system can distinguish between HER2-high and HER2-low BCs. Since HER2-low BCs may represent a unique group that requires different treatment approaches, further optimization of our classifier may assist in identifying effective therapies for patients with HER2-low BCs. Citation Format: Polina Turova, Vladimir Kushnarev, Oleg Baranov, Anna Butusova, Sofia Menshikova, Sheila Yong, Anna Love, Konstantin Chernyshov, Nikita Kotlov. Novel Transcriptomic Classification System Unveils HER2-Low Breast Tumors [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO2-15-12.

Predicting metabolic modules in incomplete bacterial genomes with MetaPathPredict.

The reconstruction of complete microbial metabolic pathways using 'omics data from environmental samples remains challenging. Computational pipelines for pathway reconstruction that utilize machine learning methods to predict the presence or absence of KEGG modules in incomplete genomes are lacking. Here, we present MetaPathPredict, a software tool that incorporates machine learning models to predict the presence of complete KEGG modules within bacterial genomic datasets. Using gene annotation data and information from the KEGG module database, MetaPathPredict employs deep learning models to predict the presence of KEGG modules in a genome. MetaPathPredict can be used as a command line tool or as a Python module, and both options are designed to be run locally or on a compute cluster. Benchmarks show that MetaPathPredict makes robust predictions of KEGG module presence within highly incomplete genomes.

Deep Learning Enabled Comprehensive Evaluation of Jumping-Droplet Condensation and Frosting.

Superhydrophobicity-enabled jumping-droplet condensation and frosting have great potential in various engineering applications, ranging from heat transfer processes to antifog/frost techniques. However, monitoring such droplets is challenging due to the high frequency of droplet behaviors, cross-scale distribution of droplet sizes, and diversity of surface morphologies. Leveraging deep learning, we develop a semisupervised framework that monitors the optical observable process of condensation and frosting. This system is adept at identifying transient droplet distributions and dynamic activities, such as droplet coalescence, jumping, and frosting, on a variety of superhydrophobic surfaces. Utilizing this transient and dynamic information, various physical properties, such as heat flux, jumping characteristics, and frosting rate, can be further quantified, conveying the heat transfer and antifrost performances of each surface perceptually and comprehensively. Furthermore, this framework relies on only a small amount of annotated data and can efficiently adapt to new condensation conditions with varying surface morphologies and illumination techniques. This adaptability is beneficial for optimizing surface designs to enhance condensation heat transfer and antifrosting performance.

Bilateral Supervision Network for Semi-Supervised Medical Image Segmentation.

Massive high-quality annotated data is required by fully-supervised learning, which is difficult to obtain for image segmentation since the pixel-level annotation is expensive, especially for medical image segmentation tasks that need domain knowledge. As an alternative solution, semi-supervised learning (SSL) can effectively alleviate the dependence on the annotated samples by leveraging abundant unlabeled samples. Among the SSL methods, mean-teacher (MT) is the most popular one. However, in MT, teacher model's weights are completely determined by student model's weights, which will lead to the training bottleneck at the late training stages. Besides, only pixel-wise consistency is applied for unlabeled data, which ignores the category information and is susceptible to noise. In this paper, we propose a bilateral supervision network with bilateral exponential moving average (bilateral-EMA), named BSNet to overcome these issues. On the one hand, both the student and teacher models are trained on labeled data, and then their weights are updated with the bilateral-EMA, and thus the two models can learn from each other. On the other hand, pseudo labels are used to perform bilateral supervision for unlabeled data. Moreover, for enhancing the supervision, we adopt adversarial learning to enforce the network generate more reliable pseudo labels for unlabeled data. We conduct extensive experiments on three datasets to evaluate the proposed BSNet, and results show that BSNet can improve the semi-supervised segmentation performance by a large margin and surpass other state-of-the-art SSL methods.

Blik is an extensible 3D visualisation tool for the annotation and analysis of cryo-electron tomography data.

Powerful, workflow-agnostic and interactive visualisation is essential for the ad hoc, human-in-the-loop workflows typical of cryo-electron tomography (cryo-ET). While several tools exist for visualisation and annotation of cryo-ET data, they are often integrated as part of monolithic processing pipelines, or focused on a specific task and offering limited reusability and extensibility. With each software suite presenting its own pros and cons and tools tailored to address specific challenges, seamless integration between available pipelines is often a difficult task. As part of the effort to enable such flexibility and move the software ecosystem towards a more collaborative and modular approach, we developed blik, an open-source napari plugin for visualisation and annotation of cryo-ET data (source code: https://github.com/brisvag/blik). blik offers fast, interactive, and user-friendly 3D visualisation thanks to napari, and is built with extensibility and modularity at the core. Data is handled and exposed through well-established scientific Python libraries such as numpy arrays and pandas dataframes. Reusable components (such as data structures, file read/write, and annotation tools) are developed as independent Python libraries to encourage reuse and community contribution. By easily integrating with established image analysis tools-even outside of the cryo-ET world-blik provides a versatile platform for interacting with cryo-ET data. On top of core visualisation features-interactive and simultaneous visualisation of tomograms, particle picks, and segmentations-blik provides an interface for interactive tools such as manual, surface-based and filament-based particle picking, and image segmentation, as well as simple filtering tools. Additional self-contained napari plugins developed as part of this work also implement interactive plotting and selection based on particle features, and label interpolation for easier segmentation. Finally, we highlight the differences with existing software and showcase blik's applicability in biological research.

AGSAM: Agent-Guided Segment Anything Model for Automatic Segmentation in Few-Shot Scenarios

Precise medical image segmentation of regions of interest (ROIs) is crucial for accurate disease diagnosis and progression assessment. However, acquiring high-quality annotated data at the pixel level poses a significant challenge due to the resource-intensive nature of this process. This scarcity of high-quality annotated data results in few-shot scenarios, which are highly prevalent in clinical applications. To address this obstacle, this paper introduces Agent-Guided SAM (AGSAM), an innovative approach that transforms the Segment Anything Model (SAM) into a fully automated segmentation method by automating prompt generation. Capitalizing on the pre-trained feature extraction and decoding capabilities of SAM-Med2D, AGSAM circumvents the need for manual prompt engineering, ensuring adaptability across diverse segmentation methods. Furthermore, the proposed feature augmentation convolution module (FACM) enhances model accuracy by promoting stable feature representations. Experimental evaluations demonstrate AGSAM’s consistent superiority over other methods across various metrics. These findings highlight AGSAM’s efficacy in tackling the challenges associated with limited annotated data while achieving high-quality medical image segmentation.

The National Sleep Research Resource: making data findable, accessible, interoperable, reusable and promoting sleep science.

This paper presents a comprehensive overview of the National Sleep Research Resource (NSRR), a National Heart Lung and Blood Institute-supported repository developed to share data from clinical studies focused on the evaluation of sleep disorders. The NSRR addresses challenges presented by the heterogeneity of sleep-related data, leveraging innovative strategies to optimize the quality and accessibility of available datasets. It provides authorized users with secure centralized access to a large quantity of sleep-related data including polysomnography, actigraphy, demographics, patient-reported outcomes, and other data. In developing the NSRR, we have implemented data processing protocols that ensure de-identification and compliance with FAIR (Findable, Accessible, Interoperable, Reusable) principles. Heterogeneity stemming from intrinsic variation in the collection, annotation, definition, and interpretation of data has proven to be one of the primary obstacles to efficient sharing of datasets. Approaches employed by the NSRR to address this heterogeneity include (1) development of standardized sleep terminologies utilizing a compositional coding scheme, (2) specification of comprehensive metadata, (3) harmonization of commonly used variables, and (3) computational tools developed to standardize signal processing. We have also leveraged external resources to engineer a domain-specific approach to data harmonization. We describe the scope of data within the NSRR, its role in promoting sleep and circadian research through data sharing, and harmonization of large datasets and analytical tools. Finally, we identify opportunities for approaches for the field of sleep medicine to further support data standardization and sharing.

Vision-language foundation model for echocardiogram interpretation.

The development of robust artificial intelligence models for echocardiography has been limited by the availability of annotated clinical data. Here, to address this challenge and improve the performance of cardiac imaging models, we developed EchoCLIP, a vision-language foundation model for echocardiography, that learns the relationship between cardiac ultrasound images and the interpretations of expert cardiologists across a wide range of patients and indications for imaging. After training on 1,032,975 cardiac ultrasound videos and corresponding expert text, EchoCLIP performs well on a diverse range of benchmarks for cardiac image interpretation, despite not having been explicitly trained for individual interpretation tasks. EchoCLIP can assess cardiac function (mean absolute error of 7.1% when predicting left ventricular ejection fraction in an external validation dataset) and identify implanted intracardiac devices (area under the curve (AUC) of 0.84, 0.92 and 0.97 for pacemakers, percutaneous mitral valve repair and artificial aortic valves, respectively). We also developed a long-context variant (EchoCLIP-R) using a custom tokenizer based on common echocardiography concepts. EchoCLIP-R accurately identified unique patients across multiple videos (AUC of 0.86), identified clinical transitions such as heart transplants (AUC of 0.79) and cardiac surgery (AUC 0.77) and enabled robust image-to-text search (mean cross-modal retrieval rank in the top 1% of candidate text reports). These capabilities represent a substantial step toward understanding and applying foundation models in cardiovascular imaging for preliminary interpretation of echocardiographic findings.

GestureFusion: A Gesture-based Interaction System

Abstract: In an era of increasingly sophisticated human-computer interaction, the ability to comprehend and interpret hand gestures has become a pivotal element for bridging the gap between humans and machines. This project, titled ”GestureFusion: A Gesture-based Interaction System,” seeks to contribute to this rapidly evolving field by developing a comprehensive system for recognizing and categorizing hand gestures through the application of computer vision techniques. The primary objective of this project is to create a versatile and extensible hand gesture recognition system that can be employed in various domains such as virtual reality, robotics, sign language interpretation, and human-computer interface design. To achieve this, the project employs state-of-the-art computer vision algorithms and deep learning techniques. Key components of this project include data collection and annotation, model training and optimization, the development of a user-friendly application programming interface (API) for integration into diverse applications.

A small target detection algorithm based on improved YOLOv5 in aerial image.

Uncrewed aerial vehicle (UAV) aerial photography technology is widely used in both industrial and military sectors, but remote sensing for small target detection still faces several challenges. Firstly, the small size of targets increases the difficulty of detection and recognition. Secondly, complex aerial environmental conditions, such as lighting changes and background noise, significantly affect the quality of detection. Rapid and accurate identification of target categories is also a key issue, requiring improvements in detection speed and accuracy. This study proposes an improved remote sensing target detection algorithm based on the YOLOv5 architecture. In the YOLOv5s model, the Distribution Focal Loss function is introduced to accelerate the convergence speed of the network and enhance the network's focus on annotated data. Simultaneously, adjustments are made to the Cross Stage Partial (CSP) network structure, modifying the convolution kernel size, adding a new stack-separated convolution module, and designing a new attention mechanism to achieve effective feature fusion between different hierarchical structure feature maps. Experimental results demonstrate a significant performance improvement of the proposed algorithm on the RSOD dataset, with a 3.5% increase in detection accuracy compared to the original algorithm. These findings indicate that our algorithm effectively enhances the precision of remote sensing target detection and holds potential application prospects.

One model to use them all: training a segmentation model with complementary datasets.

Understanding surgical scenes is crucial for computer-assisted surgery systems to provide intelligent assistance functionality. One way of achieving this is via scene segmentation using machine learning (ML). However, such ML models require large amounts of annotated training data, containing examples of all relevant object classes, which are rarely available. In this work, we propose a method to combine multiple partially annotated datasets, providing complementary annotations, into one model, enabling better scene segmentation and the use of multiple readily available datasets. Our method aims to combine available data with complementary labels by leveraging mutual exclusive properties to maximize information. Specifically, we propose to use positive annotations of other classes as negative samples and to exclude background pixels of these binary annotations, as we cannot tell if a positive prediction by the model is correct. We evaluate our method by training a DeepLabV3 model on the publicly available Dresden Surgical Anatomy Dataset, which provides multiple subsets of binary segmented anatomical structures. Our approach successfully combines 6 classes into one model, significantly increasing the overall Dice Score by 4.4% compared to an ensemble of models trained on the classes individually. By including information on multiple classes, we were able to reduce the confusion between classes, e.g. a 24% drop for stomach and colon. By leveraging multiple datasets and applying mutual exclusion constraints, we developed a method that improves surgical scene segmentation performance without the need for fully annotated datasets. Our results demonstrate the feasibility of training a model on multiple complementary datasets. This paves the way for future work further alleviating the need for one specialized large, fully segmented dataset but instead the use of already existing datasets.

Using Portuguese BRCA pathogenic variation as a model to study the impact of human admixture on human health

BackgroundAdmixture occurs between different ethnic human populations. The global colonization in recent centuries by Europeans led to the most significant admixture in human history. While admixture may enhance genetic diversity for better fitness, it may also impact on human health by transmitting genetic variants for disease susceptibility in the admixture population. The admixture by Portuguese global exploration initiated in the 15th century has reached over 20 million of Portuguese-heritage population worldwide. It provides a valuable model to study the impact of admixture on human health. BRCA1 and BRCA2 (BRCA) are two of the important tumor suppressor genes. The pathogenic variation (PV) in BRCA is well determined to cause high risk of hereditary breast and ovarian cancer. Tracing the distribution of Portuguese BRCA PV in Portuguese-heritage population will help to understand the impact of admixture on cancer susceptibility in modern humans. In this study, we analyzed the distribution of the Portuguese-originated BRCA variation in Brazilian population, which has high degree Portuguese-heritage.MethodsBy comprehensive data mining, standardization and annotation, we generated a Portuguese-derived BRCA variation dataset and a Brazilian-derived BRCA variation dataset. We compared the two BRCA variation datasets to identify the BRCA variants shared between the two populations.ResultsThe Portuguese-derived BRCA variation dataset consists of 220 BRCA variants including 78 PVs from 11,482 Portuguese cancer patients, 93 (42.2%) in BRCA1 and 127 (57.7%) in BRCA2. Of the 556 Portuguese BRCA PV carriers carrying the 78 PVs, 331 (59.5%) carried the three Portuguese-BRCA founder PVs of BRCA1 c.2037delinsCC, BRCA1 c.3331_3334del and BRCA2 c.156_157insAlu. The Brazilian-derived BRCA variation dataset consists of 255 BRCA PVs from 7,711 cancer patients, 136 (53.3%) in BRCA1 and 119 (46.6%) in BRCA2. We developed an open database named dbBRCA-Portuguese (https://genemutation.fhs.um.edu.mo/dbbrca-portuguese/) and an open database named dbBRCA-Brazilian (https://genemutation.fhs.um.edu.mo/dbbrca-brazilian) to host the BRCA variation data from Portuguese and Brazilian populations. We compared the BRCA PV datasets between Portuguese and Brazilian populations, and identified 29 Portuguese-specific BRCA PVs shared between Portuguese and Brazilian populations, 14 in BRCA1 including the Portuguese founder BRCA1 c.3331_3334del and BRCA1 c.2037delinsCC, and 15 in BRCA2 including the Portuguese founder BRCA2 c.156_157insAlu. Searching the 78 Portuguese BRCA PVs in over 5,000 ancient human genomes identified evolution origin for only 8 PVs in Europeans dated between 37,470 and 3,818 years before present, confirming the Portuguese-specificity of Portuguese BRCA PVs; comparing the 78 Portuguese BRCA PVs Portuguese, 255 Brazilian BRCA PVs, and 134 African BRCA PVs showed little overlapping, ruling out the possibility that the BRCA PVs shared between Portuguese and Brazilian may also be contributed by African.ConclusionOur study provides evidence that the admixture in recent human history contributed to cancer susceptibility in modern humans.

Predicting Perceived Gloss: Do Weak Labels Suffice?

AbstractEstimating perceptual attributes of materials directly from images is a challenging task due to their complex, not fully‐understood interactions with external factors, such as geometry and lighting. Supervised deep learning models have recently been shown to outperform traditional approaches, but rely on large datasets of human‐annotated images for accurate perception predictions. Obtaining reliable annotations is a costly endeavor, aggravated by the limited ability of these models to generalise to different aspects of appearance. In this work, we show how a much smaller set of human annotations (“strong labels”) can be effectively augmented with automatically derived “weak labels” in the context of learning a low‐dimensional image‐computable gloss metric. We evaluate three alternative weak labels for predicting human gloss perception from limited annotated data. Incorporating weak labels enhances our gloss prediction beyond the current state of the art. Moreover, it enables a substantial reduction in human annotation costs without sacrificing accuracy, whether working with rendered images or real photographs.

Automated stance detection in complex topics and small languages: The challenging case of immigration in polarizing news media.

Automated stance detection and related machine learning methods can provide useful insights for media monitoring and academic research. Many of these approaches require annotated training datasets, which limits their applicability for languages where these may not be readily available. This paper explores the applicability of large language models for automated stance detection in a challenging scenario, involving a morphologically complex, lower-resource language, and a socio-culturally complex topic, immigration. If the approach works in this case, it can be expected to perform as well or better in less demanding scenarios. We annotate a large set of pro- and anti-immigration examples to train and compare the performance of multiple language models. We also probe the usability of GPT-3.5 (that powers ChatGPT) as an instructable zero-shot classifier for the same task. The supervised models achieve acceptable performance, but GPT-3.5 yields similar accuracy. As the latter does not require tuning with annotated data, it constitutes a potentially simpler and cheaper alternative for text classification tasks, including in lower-resource languages. We further use the best-performing supervised model to investigate diachronic trends over seven years in two corpora of Estonian mainstream and right-wing populist news sources, demonstrating the applicability of automated stance detection for news analytics and media monitoring settings even in lower-resource scenarios, and discuss correspondences between stance changes and real-world events.

Validating predictions of burial mounds with field data: the promise and reality of machine learning

PurposeThis paper provides practical advice for archaeologists and heritage specialists wishing to use ML approaches to identify archaeological features in high-resolution satellite imagery (or other remotely sensed data sources). We seek to balance the disproportionately optimistic literature related to the application of ML to archaeological prospection through a discussion of limitations, challenges and other difficulties. We further seek to raise awareness among researchers of the time, effort, expertise and resources necessary to implement ML successfully, so that they can make an informed choice between ML and manual inspection approaches.Design/methodology/approachAutomated object detection has been the holy grail of archaeological remote sensing for the last two decades. Machine learning (ML) models have proven able to detect uniform features across a consistent background, but more variegated imagery remains a challenge. We set out to detect burial mounds in satellite imagery from a diverse landscape in Central Bulgaria using a pre-trained Convolutional Neural Network (CNN) plus additional but low-touch training to improve performance. Training was accomplished using MOUND/NOT MOUND cutouts, and the model assessed arbitrary tiles of the same size from the image. Results were assessed using field data.FindingsValidation of results against field data showed that self-reported success rates were misleadingly high, and that the model was misidentifying most features. Setting an identification threshold at 60% probability, and noting that we used an approach where the CNN assessed tiles of a fixed size, tile-based false negative rates were 95–96%, false positive rates were 87–95% of tagged tiles, while true positives were only 5–13%. Counterintuitively, the model provided with training data selected for highly visible mounds (rather than all mounds) performed worse. Development of the model, meanwhile, required approximately 135 person-hours of work.Research limitations/implicationsOur attempt to deploy a pre-trained CNN demonstrates the limitations of this approach when it is used to detect varied features of different sizes within a heterogeneous landscape that contains confounding natural and modern features, such as roads, forests and field boundaries. The model has detected incidental features rather than the mounds themselves, making external validation with field data an essential part of CNN workflows. Correcting the model would require refining the training data as well as adopting different approaches to model choice and execution, raising the computational requirements beyond the level of most cultural heritage practitioners.Practical implicationsImproving the pre-trained model’s performance would require considerable time and resources, on top of the time already invested. The degree of manual intervention required – particularly around the subsetting and annotation of training data – is so significant that it raises the question of whether it would be more efficient to identify all of the mounds manually, either through brute-force inspection by experts or by crowdsourcing the analysis to trained – or even untrained – volunteers. Researchers and heritage specialists seeking efficient methods for extracting features from remotely sensed data should weigh the costs and benefits of ML versus manual approaches carefully.Social implicationsOur literature review indicates that use of artificial intelligence (AI) and ML approaches to archaeological prospection have grown exponentially in the past decade, approaching adoption levels associated with “crossing the chasm” from innovators and early adopters to the majority of researchers. The literature itself, however, is overwhelmingly positive, reflecting some combination of publication bias and a rhetoric of unconditional success. This paper presents the failure of a good-faith attempt to utilise these approaches as a counterbalance and cautionary tale to potential adopters of the technology. Early-majority adopters may find ML difficult to implement effectively in real-life scenarios.Originality/valueUnlike many high-profile reports from well-funded projects, our paper represents a serious but modestly resourced attempt to apply an ML approach to archaeological remote sensing, using techniques like transfer learning that are promoted as solutions to time and cost problems associated with, e.g. annotating and manipulating training data. While the majority of articles uncritically promote ML, or only discuss how challenges were overcome, our paper investigates how – despite reasonable self-reported scores – the model failed to locate the target features when compared to field data. We also present time, expertise and resourcing requirements, a rarity in ML-for-archaeology publications.

An Explainable Deep Learning Classifier of Bovine Mastitis Based on Whole-Genome Sequence Data—Circumventing the p >> n Problem

The serious drawback underlying the biological annotation of whole-genome sequence data is the p >> n problem, which means that the number of polymorphic variants (p) is much larger than the number of available phenotypic records (n). We propose a way to circumvent the problem by combining a LASSO logistic regression with deep learning to classify cows as susceptible or resistant to mastitis, based on single nucleotide polymorphism (SNP) genotypes. Among several architectures, the one with 204,642 SNPs was selected as the best. This architecture was composed of two layers with, respectively, 7 and 46 units per layer implementing respective drop-out rates of 0.210 and 0.358. The classification of the test data resulted in AUC = 0.750, accuracy = 0.650, sensitivity = 0.600, and specificity = 0.700. Significant SNPs were selected based on the SHapley Additive exPlanation (SHAP). As a final result, one GO term related to the biological process and thirteen GO terms related to molecular function were significantly enriched in the gene set that corresponded to the significant SNPs. Our findings revealed that the optimal approach can correctly predict susceptibility or resistance status for approximately 65% of cows. Genes marked by the most significant SNPs are related to the immune response and protein synthesis.