Annotated Dataset Research Articles

A Methodology for Advanced Manufacturing Defect Detection through Self-Supervised Learning on X-ray Images

In industrial quality control, especially in the field of manufacturing defect detection, deep learning plays an increasingly critical role. However, the efficacy of these advanced models is often hindered by their need for large-scale, annotated datasets. Moreover, these datasets are mainly based on RGB images, which are very different from X-ray images. Addressing this limitation, our research proposes a methodology that incorporates domain-specific self-supervised pretraining techniques using X-ray imaging to improve defect detection capabilities in manufacturing products. We employ two pretraining approaches, SimSiam and SimMIM, to refine feature extraction from manufacturing images. The pretraining stage is carried out using an industrial dataset of 27,901 unlabeled X-ray images from a manufacturing production line. We analyze the performance of the pretraining against transfer-learning-based methods in a complex defect detection scenario using a Faster R-CNN model. We conduct evaluations on both a proprietary industrial dataset and the publicly available GDXray dataset. The findings reveal that models pretrained with domain-specific X-ray images consistently outperform those initialized with ImageNet weights. Notably, Swin Transformer models show superior results in scenarios rich in labeled data, whereas CNN backbones are more effective in limited-data environments. Moreover, we underscore the enhanced ability of the models pretrained with X-ray images in detecting critical defects, crucial for ensuring safety in industrial settings. Our study offers substantial evidence of the benefits of self-supervised learning in manufacturing defect detection, providing a solid foundation for further research and practical applications in industrial quality control.

Exploring dermoscopic structures for melanoma lesions' classification.

Melanoma is one of the deadliest skin cancers that originate from melanocytes due to sun exposure, causing mutations. Early detection boosts the cure rate to 90%, but misclassification drops survival to 15-20%. Clinical variations challenge dermatologists in distinguishing benign nevi and melanomas. Current diagnostic methods, including visual analysis and dermoscopy, have limitations, emphasizing the need for Artificial Intelligence understanding in dermatology. In this paper, we aim to explore dermoscopic structures for the classification of melanoma lesions. The training of AI models faces a challenge known as brittleness, where small changes in input images impact the classification. A study explored AI vulnerability in discerning melanoma from benign lesions using features of size, color, and shape. Tests with artificial and natural variations revealed a notable decline in accuracy, emphasizing the necessity for additional information, such as dermoscopic structures. The study utilizes datasets with clinically marked dermoscopic images examined by expert clinicians. Transformers and CNN-based models are employed to classify these images based on dermoscopic structures. Classification results are validated using feature visualization. To assess model susceptibility to image variations, classifiers are evaluated on test sets with original, duplicated, and digitally modified images. Additionally, testing is done on ISIC 2016 images. The study focuses on three dermoscopic structures crucial for melanoma detection: Blue-white veil, dots/globules, and streaks. In evaluating model performance, adding convolutions to Vision Transformers proves highly effective for achieving up to 98% accuracy. CNN architectures like VGG-16 and DenseNet-121 reach 50-60% accuracy, performing best with features other than dermoscopic structures. Vision Transformers without convolutions exhibit reduced accuracy on diverse test sets, revealing their brittleness. OpenAI Clip, a pre-trained model, consistently performs well across various test sets. To address brittleness, a mitigation method involving extensive data augmentation during training and 23 transformed duplicates during test time, sustains accuracy. This paper proposes a melanoma classification scheme utilizing three dermoscopic structures across Ph2 and Derm7pt datasets. The study addresses AI susceptibility to image variations. Despite a small dataset, future work suggests collecting more annotated datasets and automatic computation of dermoscopic structural features.

Convolutional Autoencoder-Based medical image compression using a novel annotated medical X-ray imaging dataset

With the ongoing development of machine learning techniques, it is now necessary to train and evaluate these algorithms to have access to high-quality medical X-ray datasets. This study unfolds on two critical axes within the realm of medical imaging. We introduce the proposed Medical X-ray Imaging Dataset (MXID), a meticulously curated resource featuring images spanning 18 body parts. It has been refined to include a classification based on body type within each gender category. This dataset addresses the limitations of existing datasets by offering comprehensive coverage, precise annotations, and optimal image quality. While numerous datasets often fall short in terms of encompassing a broad range of anatomical regions, particularly those that are designed for multi-body component analysis. Our primary contribution lies in bridging the gap in available resources, providing a foundation of developing robust machine learning algorithms in medical imaging. Moreover, we extend our investigation to the integration of the MXID dataset into the domain of medical image compression, evaluating the performance of techniques such as Principal Component Analysis (PCA), K-means clustering, Convolutional Neural Network (CNN), Deep Convolutional Autoencoders (DCAEs), Autoencoders (AEs), and Variational Autoencoder (VAE), using two scenarios, one is for single image compression, and the other is for MXID dataset, which offer a unique perspective that enables a comprehensive assessment of these techniques across diverse anatomical regions. Our study contributes to advancing machine learning and deep learning applications in medical imaging, emphasizing the creation of a dataset and its integration into image compression.

Exploring Deep Learning Models for Accurate Alzheimer's Disease Classification based on MRI Imaging

INTRODUCTION: Alzheimer's disease (AD), a complex neurodegenerative condition, presents significant challenges in early and accurate diagnosis. Early prediction of AD severity holds the potential for improved patient care and timely interventions. This research investigates the use of deep learning methodologies to forecast AD severity utilizing data extracted from Magnetic Resonance Imaging (MRI) scans.
 OBJECTIVES: This study aims to explore the efficacy of deep learning models in predicting the severity of Alzheimer's disease using MRI data. Traditional diagnostic methods for AD, primarily reliant on cognitive assessments, often lead to late-stage detection. MRI scans offer a non-invasive means to examine brain structure and detect pathological changes associated with AD. However, manual interpretation of these scans is labor-intensive and subject to variability.
 METHODS: Various deep learning models, including Convolutional Neural Networks (CNNs) and advanced architectures like DenseNet, VGG16, ResNet50, MobileNet, AlexNet, and Xception, are explored for MRI scan analysis. The performance of these models in predicting AD severity is assessed and compared. Deep learning models autonomously learn hierarchical features from the data, potentially recognizing intricate patterns associated with different AD stages that may be overlooked in manual analysis.
 RESULTS: The study evaluates the performance of different deep learning models in predicting AD severity using MRI scans. The results highlight the efficacy of these models in capturing subtle patterns indicative of AD progression. Moreover, the comparison underscores the strengths and limitations of each model, aiding in the selection of appropriate methodologies for AD prognosis.
 CONCLUSION: This research contributes to the growing field of AI-driven healthcare by showcasing the potential of deep learning in revolutionizing AD diagnosis and prognosis. The findings emphasize the importance of leveraging advanced technologies, such as deep learning, to enhance the accuracy and timeliness of AD diagnosis. However, challenges remain, including the need for large annotated datasets, model interpretability, and integration into clinical workflows. Continued efforts in this area hold promise for improving the management of AD and ultimately enhancing patient outcomes.

A General Model for Aggregating Annotations AcrossSimple, Complex, and Multi-object Annotation Tasks (Abstract Reprint)

Human annotations are vital to supervised learning, yet annotators often disagree on the correct label, especially as annotation tasks increase in complexity. A common strategy to improve label quality is to ask multiple annotators to label the same item and then aggregate their labels. To date, many aggregation models have been proposed for simple categorical or numerical annotation tasks, but far less work has considered more complex annotation tasks, such as those involving open-ended, multivariate, or structured responses. Similarly, while a variety of bespoke models have been proposed for specific tasks, our work is the first we are aware of to introduce aggregation methods that generalize across many, diverse complex tasks, including sequence labeling, translation, syntactic parsing, ranking, bounding boxes, and keypoints. This generality is achieved by applying readily available task-specific distance functions, then devising a task-agnostic method to model these distances between labels, rather than the labels themselves. This article presents a unified treatment of our prior work on complex annotation modeling and extends that work with investigation of three new research questions. First, how do complex annotation task and dataset properties impact aggregation accuracy? Second, how should a task owner navigate the many modeling choices in order to maximize aggregation accuracy? Finally, what tests and diagnoses can verify that aggregation models are specified correctly for the given data? To understand how various factors impact accuracy and to inform model selection, we conduct large-scale simulation studies and broad experiments on real, complex datasets. Regarding testing, we introduce the concept of unit tests for aggregation models and present a suite of such tests to ensure that a given model is not mis-specified and exhibits expected behavior. Beyond investigating these research questions above, we discuss the foundational concept and nature of annotation complexity, present a new aggregation model as a conceptual bridge between traditional models and our own, and contribute a new general semisupervised learning method for complex label aggregation that outperforms prior work.

Visual Instruction Tuning with Polite Flamingo

Recent research has demonstrated that the multi-task fine-tuning of multi-modal Large Language Models (LLMs) using an assortment of annotated downstream vision-language datasets significantly enhances their performance. Yet, during this process, a side effect, which we termed as the "multi-modal alignment tax", surfaces. This side effect negatively impacts the model's ability to format responses appropriately - for instance, its "politeness" - due to the overly succinct and unformatted nature of raw annotations, resulting in reduced human preference. In this paper, we introduce Polite Flamingo, a multi-modal response rewriter that transforms raw annotations into a more appealing, "polite" format. Polite Flamingo is trained to reconstruct high-quality responses from their automatically distorted counterparts and is subsequently applied to a vast array of vision-language datasets for response rewriting. After rigorous filtering, we generate the PF-1M dataset and further validate its value by fine-tuning a multi-modal LLM with it. Combined with novel methodologies including U-shaped multi-stage tuning and multi-turn augmentation, the resulting model, Clever Flamingo, demonstrates its advantages in both multi-modal understanding and response politeness according to automated and human evaluations. Code and dataset are available at https://github.com/ChenDelong1999/polite-flamingo

Harnessing the Power of Beta Scoring in Deep Active Learning for Multi-Label Text Classification

Within the scope of natural language processing, the domain of multi-label text classification is uniquely challenging due to its expansive and uneven label distribution. The complexity deepens due to the demand for an extensive set of annotated data for training an advanced deep learning model, especially in specialized fields where the labeling task can be labor-intensive and often requires domain-specific knowledge. Addressing these challenges, our study introduces a novel deep active learning strategy, capitalizing on the Beta family of proper scoring rules within the Expected Loss Reduction framework. It computes the expected increase in scores using the Beta Scoring Rules, which are then transformed into sample vector representations. These vector representations guide the diverse selection of informative sample, directly linking this process to the model's expected proper score. Comprehensive evaluations across both synthetic and real datasets reveal our method's capability to often outperform established acquisition techniques in multi-label text classification, presenting encouraging outcomes across various architectural and dataset scenarios.

AGS: Affordable and Generalizable Substitute Training for Transferable Adversarial Attack

In practical black-box attack scenarios, most of the existing transfer-based attacks employ pretrained models (e.g. ResNet50) as the substitute models. Unfortunately, these substitute models are not always appropriate for transfer-based attacks. Firstly, these models are usually trained on a largescale annotated dataset, which is extremely expensive and time-consuming to construct. Secondly, the primary goal of these models is to perform a specific task, such as image classification, which is not developed for adversarial attacks. To tackle the above issues, i.e., high cost and over-fitting on taskspecific models, we propose an Affordable and Generalizable Substitute (AGS) training framework tailored for transferbased adversarial attack. Specifically, we train the substitute model from scratch by our proposed adversary-centric constrastive learning. This proposed learning mechanism introduces another sample with slight adversarial perturbations as an additional positive view of the input image, and then encourages the adversarial view and two benign views to interact comprehensively with each other. To further boost the generalizability of the substitute model, we propose adversarial invariant learning to maintain the representations of the adversarial example invariants under augmentations with various strengths. Our AGS model can be trained solely with unlabeled and out-of domain data and avoid overfitting to any task-specific models, because of its inherently self-supervised nature. Extensive experiments demonstrate that our AGS achieves comparable or superior performance compared to substitute models pretrained on the complete ImageNet training set, when executing attacks across a diverse range of target models, including ViTs, robustly trained models, object detection and segmentation models. Our source codes are available at https://github.com/lwmming/AGS.

Relax Image-Specific Prompt Requirement in SAM: A Single Generic Prompt for Segmenting Camouflaged Objects

Camouflaged object detection (COD) approaches heavily rely on pixel-level annotated datasets. Weakly-supervised COD (WSCOD) approaches use sparse annotations like scribbles or points to reduce annotation efforts, but this can lead to decreased accuracy. The Segment Anything Model (SAM) shows remarkable segmentation ability with sparse prompts like points. However, manual prompt is not always feasible, as it may not be accessible in real-world application. Additionally, it only provides localization information instead of semantic one, which can intrinsically cause ambiguity in interpreting targets. In this work, we aim to eliminate the need for manual prompt. The key idea is to employ Cross-modal Chains of Thought Prompting (CCTP) to reason visual prompts using the semantic information given by a generic text prompt. To that end, we introduce a test-time instance-wise adaptation mechanism called Generalizable SAM (GenSAM) to automatically generate and optimize visual prompts from the generic task prompt for WSCOD. In particular, CCTP maps a single generic text prompt onto image-specific consensus foreground and background heatmaps using vision-language models, acquiring reliable visual prompts. Moreover, to test-time adapt the visual prompts, we further propose Progressive Mask Generation (PMG) to iteratively reweight the input image, guiding the model to focus on the targeted region in a coarse-to-fine manner. Crucially, all network parameters are fixed, avoiding the need for additional training. Experiments on three benchmarks demonstrate that GenSAM outperforms point supervision approaches and achieves comparable results to scribble supervision ones, solely relying on general task descriptions. Our codes is in https://github.com/jyLin8100/GenSAM.

Authorship Attribution in Less-Resourced Languages: A Hybrid Transformer Approach for Romanian

Authorship attribution for less-resourced languages like Romanian, characterized by the scarcity of large, annotated datasets and the limited number of available NLP tools, poses unique challenges. This study focuses on a hybrid Transformer combining handcrafted linguistic features, ranging from surface indices like word frequencies to syntax, semantics, and discourse markers, with contextualized embeddings from a Romanian BERT encoder. The methodology involves extracting contextualized representations from a pre-trained Romanian BERT model and concatenating them with linguistic features, selected using the Kruskal–Wallis mean rank, to create a hybrid input vector for a classification layer. We compare this approach with a baseline ensemble of seven machine learning classifiers for authorship attribution employing majority soft voting. We conduct studies on both long texts (full texts) and short texts (paragraphs), with 19 authors and a subset of 10. Our hybrid Transformer outperforms existing methods, achieving an F1 score of 0.87 on the full dataset of the 19-author set (an 11% enhancement) and an F1 score of 0.95 on the 10-author subset (an increase of 10% over previous research studies). We conduct linguistic analysis leveraging textual complexity indices and employ McNemar and Cochran’s Q statistical tests to evaluate the performance evolution across the best three models, while highlighting patterns in misclassifications. Our research contributes to diversifying methodologies for effective authorship attribution in resource-constrained linguistic environments. Furthermore, we publicly release the full dataset and the codebase associated with this study to encourage further exploration and development in this field.

Abstract 144: Sex bias in neuroblastoma: Worse outcomes for female patients are associated with dysregulation of DLK1 and H19

Abstract Neuroblastoma (NB) is a devastating neural crest-derived tumor primarily affects children under five. Forty percent of NB patients are classified as high-risk (HR), and less than half of them survive. HR disease is split into two equally sized groups: one exhibiting genetic amplification of the MYCN proto-oncogene (MA), and another where MYCN is non-amplified (MNon). Overall survival (OS) studies are often either poorly annotated or focused on outcomes across the entire patient group, which may mask differential outcomes or biases among NB subgroups. Here we report our assessment of OS among several genetic, histopathological, and sex-based subgroups in three different well annotated NB patient datasets. Our analysis reveals a previously unseen sex-bias in HR MNon patients, where female patients have significantly worse OS than males. This disparity carries into stage 4s NB, defined by high survival in infants despite the presence of broadly disseminated disease. In stage 4s patients across two studies, we observe no deaths among males (n=66) and 6 among females (n=48), again marking significantly worse OS in females. Expression analysis comparing male and female HR MNon patients identifies DLK1 and H19, two imprinted genes implicated in multiple malignancies, as frequently overexpressed in females. A similar pattern is seen in RNA seq analysis of human MA (n=4) and MNon (n=4) cell lines. In addition, both DLK1 and H19 are associated with worse prognosis in female but not male patients. We also report that depletion of either gene results in reduced growth in vitro. Moreover, the NB-related ATRX tumor suppressor has been shown to inhibit H19 via insertion of histone variant H3.3. We observe an inverse correlation between H3.3B and H19 expression in female HR MNon patients. Further, H19 levels increase upon ATRX depletion in human NB cells, suggesting that ATRX indeed regulates H19 in NB. These findings provide the first evidence of female-biased outcomes in NB and suggest functional involvement of DLK1, H19 and ATRX. Citation Format: Mariya Raleigh, Vishwa Patel, Lorraine R. Benhamou, Xin Shen, Dennis A. Sheeter, Erika C. Espinosa, Niharika Reddy Badi, Chase Calvert, Garrett Bourne, John T. Powers. Sex bias in neuroblastoma: Worse outcomes for female patients are associated with dysregulation of DLK1 and H19 [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 144.

Abstract 3528: Constructing knowledge graph of N6-methyladnosion regulations in cancer from literature using ChatGPT-4

Abstract N6-methyladenosine (m6A) is the most abundant mRNA modification with a crucial role in cellular processes. Its involvement in cancers is well-established. Dysregulation of m6A is linked to cancer initiation, progression, and therapy resistance, impacting the tumor microenvironment (TME) and promoting metastasis. Understanding m6A's regulatory role in cancer is essential for identifying new therapeutic targets. We considered the construction of a knowledge graph (KG) representing molecular regulatory mechanisms (MRMs) of m6A in healthy conditions and different cancers from PubMed papers. Such a KG consists of interconnected triplets of (head node, regulatory relationship, tail node), each denoting regulation between genes or a gene and a phenotype/process. Advances in natural language processing (NLP) have provided tools such as SemRep and GNBR to automate the extraction of these triplets from the literature. However, these methods don’t extract contextual information, such as the type of cancers or cells in which these MRMs occur, leading to contradictory regulations in the constructed KG. In addition, existing approaches were designed to extract generic regulatory relations and therefore struggled in properly capturing unique concepts associated with m6A regulatory mechanisms. To tackle this challenge, we designed a novel ChatGPT-4 prompt to extract relational graphs from papers tailored for m6A regulatory mechanisms. An annotated dataset of 400 titles on m6A-related MRMs were created to evaluate the performance. We then applied the prompt to 1023 papers to create a m6A Molecular Regulatory Mechanism Knowledge Graph (m6A-MRM-KG). This graph illuminates m6A's roles in gene expression regulation, especially in cancer and immunity. Anticipated to enhance our understanding of cancer development, it provides insights into potential therapeutic strategies. The m6A-MRM-KG not only organizes information but also empowers researchers to uncover novel insights in the dynamic relationships of m6A modifications in diverse biological contexts. Citation Format: Sumin Jo, Xidong Wu, Yiming Zeng, Arun Das, Ting-He Zhang, David Alexander Spellman, Adam Ferris, Shou-Jiang Gao, Jianqiu (Michelle) Zhang, Yu-Chiao Chiu, Yufei Huang. Constructing knowledge graph of N6-methyladnosion regulations in cancer from literature using ChatGPT-4 [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3528.

Abstract 1090: GPC2 directed CAR T cells are efficacious against fusion-positive and fusion-negative rhabdomyosarcomas

Abstract Background: Rhabdomyosarcoma (RMS) is an aggressive pediatric solid tumor with poor survival and new targeted therapies are needed. Aims: Using an integrative proteogenomic quantitative approach, we sought to identify candidate immunotherapeutic targets then validate top targets in RMS preclinical models. Methods: We first performed plasma membrane enrichment followed by mass spectrometry (MS) to define the surfaceome of 7 fusion-positive (FP) and 14 fusion-negative (FN) RMS patient-derived xenograft models. Surface proteins were scored by IMMUNOTAR, our custom pipeline which extracts and integrates quantitative features from diverse proteogenomic and annotation datasets to provide a score for each protein, using mean-average-precision for score optimization. Cell surface expression of IMMUNOTAR-prioritized proteins was validated using flow cytometry, focusing on targets with immunotherapies in development. CAR T cells targeting the lead protein were engineered and studied in RMS preclinical models. Results: A total of 847 and 882 high confidence cell surface proteins were detected in FP and FN RMS surfaceome, respectively, with 94% of proteins overlapping the phenotypes. IMMUNOTAR ranked anaplastic lymphoma kinase (ALK), a putative immunotherapeutic target in FP RMS, as the top FP target. Glypican 2 (GPC2), a differentially expressed cell surface proteoglycan identified in neuroblastoma and other neural-derived tumors, targetable with CAR T cells and antibody-drug conjugates [Bosse et al. Cancer Cell 2017], ranked as the top protein in FN and 5th protein in FP RMS. Overall, GPC2 was abundantly expressed in 16 of 21 RMS models profiled. Flow cytometry revealed expression of GPC2 comparable to neuroblastoma in 5 of 8 RMS cells lines. Additionally, both FP and FN GPC2+ RMS cell lines (FN: RH36, SMS-CTR; FP: RH41, RH30) activated GPC2 CAR T cells with either a CD28 or a 4-1BB costimulatory domain, inducing release of IFN-gamma and IL-2 upon co-incubation across different E:T ratios as quantified by ELISA. Further, GPC2 CAR T cells induced potent RMS cell cytotoxicity compared to control CD19 CAR T cells with near 100% specific cytotoxicity at 1:1 E:T ratio. In a pilot study, GPC2 CAR T cells with 4-1BB costimulatory domains induced significant tumor regression (p=0.03) and prolonged progression-free survival (p=0.01) compared to CD19 CAR T cells in an established murine RH41 flank xenograft model. Conclusion: In summary, we probed the RMS surfaceome using MS and identified candidate proteins amenable for immunotherapeutic targeting using our IMMUNOTAR algorithm. We validated GPC2 as a candidate target in FP and FN RMS and are corroborating our findings in additional pre-clinical RMS models. Demonstration of GPC2 CAR efficacy in these models could lead to inclusion of RMS in our ongoing phase 1 clinical trial (NCT05650749) in a planned expansion phase. Citation Format: Rawan Shraim, Patrick M. Schuerch, Anna Maria Giudice, Karina L. Conkrite, Amber K. Weiner, Alexander B. Radaoui, Brian Mooney, Raushan T. Kurmasheva, Peter J. Houghton, Gregg B. Morin, Yael B. Mosse, Poul H. Sorensen, Benjamin A. Garcia, Ahmet Sacan, John M. Maris, Kristopher R. Bosse, Sharon J. Diskin. GPC2 directed CAR T cells are efficacious against fusion-positive and fusion-negative rhabdomyosarcomas [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 1090.

A novel Deep Learning architecture for lung cancer detection and diagnosis from Computed Tomography image analysis

Timely identification of lung nodules, which are precursors to lung cancer, and their evaluation can significantly reduce the incidence rate. Computed Tomography (CT) is the primary technique used for lung cancer screening due to its high resolution. Identifying white, spherical shadows as lung nodules in CT images is essential for accurately detecting lung cancer. Convolutional Neural Network (CNN)-based methods have performed better than traditional techniques in various medical image applications. However, challenges still need to be addressed due to insufficient annotated datasets, significant intra-class variations, and substantial inter-class similarities, which hinder their practical use. Manually labeling the position of nodules on CT slices is critical for distinguishing between benign and malignant cases, but it is an unreliable and time-consuming process. Insufficient data and class imbalance are the primary factors that may result in overfitting and below-par performance. The paper presents a novel Deep Learning (DL) framework to detect and classify lung cancer in input CT images. It introduces a 3D-VNet architecture for accurate segmentation of pulmonary nodules and a 3D-ResNet architecture designed for their classification. The segmentation model achieves a Dice Similarity Coefficient (DSC) of 99.34% on the LUNA16 dataset while reducing false positives to 0.4%. The classification model shows performance metrics with accuracy, sensitivity, and specificity of 99.2%, 98.8%, and 99.6%, respectively. The 3D-VNet network outperforms previous segmentation methods by accurately calibrating lung nodules of various sizes and shapes with excellent robustness. The classification model’s metrics show that the suggested method outperforms current approaches regarding accuracy, specificity, sensitivity and F1-Score.

Leveraging enhanced BERT models for detecting suicidal ideation in Thai social media content amidst COVID-19

During the COVID-19 pandemic, people experienced major lifestyle changes including enforced isolation which resulted in an increase in suicidal ideation. In the face of isolation, individuals sought avenues to express themselves, and social media platforms have emerged as a primary choice. It is crucial to detect and analyze expressions of suicidal ideation and emotional distress on these platforms in order to monitor and prevent suicide. This study aims to fill the research gap on analyzing suicidal ideation and emotional distress expressed in the Thai language on social media, specifically, Twitter. We present a dataset of 2,400 manually annotated Thai tweets marked for suicidal ideation and emotions. We then designed a deep learning model using key features extracted from the tweets to predict these factors and compared its performance with other common machine learning models. Our model outperformed the baseline models, with an F1-score of 93 % for predicting suicidal ideation and an F1-score of 88 % for predicting emotions. Using this model, we analyzed 67,627 tweets from 2019 to 2020 and found a marked increase in tweets expressing suicidal thoughts and sadness, up 40.97 % and 21.28 % respectively from 2019 to 2020. The findings indicate that the pandemic had a significant impact on the mental health of the Thai population. Our study provides a tool for identifying and monitoring suicidal thinking in similar settings and offers insight into the COVID-19 impact on mental health in Thailand. The annotated dataset will serve as a valuable resource for further research in this field.

A multiscale enhanced pavement crack segmentation network coupling spectral and spatial information of UAV hyperspectral imagery

Road pavement cracks are a critical factor affecting the health conditions of road pavements. Accurate crack detection contributes to providing data support for road maintenance measures. Compared to conventional crack detection algorithms, deep learning based crack segmentation methods have practical significance for road maintenance and traffic safety management due to their high precision and automation. However, existing deep learning methods often suffer from segmentation accuracy loss due to the varying crack sizes and the presence of stains on the pavement with spatial characteristics similar to cracks, leading to the misclassification of cracks. Therefore, this study proposed a multiscale enhanced road pavement crack segmentation network (MS-CrackSeg) by coupling spectral and spatial information to detect pavement cracks from unmanned aerial vehicle (UAV) hyperspectral imagery. MS-CrackSeg can simultaneously learn the spatial and rich spectral features of cracks in the hyperspectral imagery, improving the discrimination of those targets with similar spatial features to cracks compared to previous approaches. Moreover, the Multiscale Self-Attention-like Feature Extraction Module (MSSA) is introduced to extract and fuse multiscale crack features to enhance the crack detection. Experiments on a dataset consisting of 1031 hyperspectral images demonstrated the superior crack segmentation of the proposed method compared to the comparative methods. In particular, the proposed method achieved the highest F1-score of 0.74 and mean Intersection over Union of 0.79, indicating an exceptional performance. The developed approach offers improved data support for road maintenance measures and has validated the advantages of UAV hyperspectral imagery in road crack segmentation. The annotated hyperspectral dataset and the code for MS-CrackSeg network are available at https://github.com/williamchen-x/MS-CrackSeg.

ВИЯВЛЕННЯ ОБ’ЄКТІВ НА ЗОБРАЖЕННІ В ПОТОКОВОМУ РЕЖИМІ ПРИ ВИКОРИСТАННІ YOLOv5 і FASTER R-CNN

The accuracy of the model can be one of the main indicators, on a basis of which it is possible to conclude about the suitability of the model for its practical operation. However, taking into account the specifics of the identified task, it is also worth paying attention to the speed of the model, since there is a need to process data in streaming mode. To investigate the possibilities of using machine learning in an applied problem, two groups of object recognition models considered: YOLOv5 and Faster R-CNN. The purpose of the study is to analyze the architectural solutions of the most common object detection models YOLOv5 and Faster R-CNN to build a model to improve the speed and accuracy of object detection in an applied task or further combine them. A total of 550 training images and 105 validation images collected. A dataset of 573 images from the new location also collected for final validation of the models. The use of Roboflow provided for image annotation, which allows not only to mark images, but also to export annotated data sets in various formats. Training and validation of the models carried out on the Google Colab platform. The platform uses the Python programming language and the PyTorch framework. The yolov5 and detecron2 libraries for YOLOv5 and Faster R-CNN, respectively, used for model training and validation. To determine whether the result belongs to one of the four groups, the IOU metric is used, which is the ratio of the intersection area to the area of the union of the correct and predicted bounding frames. The size of the trained YOLOv5 and Faster R-CNN models was 40.2 MB and 230.8 MB, respectively. The models tested on the second validation set. As result of the study, a set of data from video surveillance cameras collected and anno-tated using RoboFlow. The main representatives of two groups of object detection algorithms YOLOv5 and Faster R-CNN trained using the prepared data set. The results showed that both models have their advantages and disadvantages, both models are applicable for different tasks.

Visual tracking in camera-switching outdoor sport videos: Benchmark and baselines for skiing

Skiing is a globally popular winter sport discipline with a rich history of competitive events. This domain offers ample opportunities for the application of computer vision to enhance the understanding of athletes’ performances. However, this potential has remained relatively untapped in comparison to other sports, primarily due to the limited availability of dedicated research studies and datasets. The present paper takes a significant stride towards bridging these gaps. It conducts a comprehensive examination of skier appearance tracking in videos capturing their entire performance—an essential step for more advanced performance analyses. To implement this investigation, we introduce SkiTB, the largest and most annotated dataset tailored for computer vision applications in skiing. We subject a range of visual object tracking algorithms to rigorous testing, including both well-established methodologies and a novel skier-specific baseline algorithm. The results yield valuable insights into the suitability of various tracking techniques for vision-based skiing analysis and into the generalization of state-of-the-art algorithms to complex target behaviors and conditions set by winter outdoor environments. To foster further development, we make SkiTB, the associated code, and the obtained results accessible through https://machinelearning.uniud.it/datasets/skitb.

Generalizing Parkinson's disease detection using keystroke dynamics: a self-supervised approach.

Passive monitoring of touchscreen interactions generates keystroke dynamic signals that can be used to detect and track neurological conditions such as Parkinson's disease (PD) and psychomotor impairment with minimal burden on the user. However, this typically requires datasets with clinically confirmed labels collected in standardized environments, which is challenging, especially for a large subject pool. This study validates the efficacy of a self-supervised learning method in reducing the reliance on labels and evaluates its generalizability. We propose a new type of self-supervised loss combining Barlow Twins loss, which attempts to create similar feature representations with reduced feature redundancy for samples coming from the same subject, and a Dissimilarity loss, which promotes uncorrelated features for samples generated by different subjects. An encoder is first pre-trained using this loss on unlabeled data from an uncontrolled setting, then fine-tuned with clinically validated data. Our experiments test the model generalizability with controls and subjects with PD on 2 independent datasets. Our approach showed better generalization compared to previous methods, including a feature engineering strategy, a deep learning model pre-trained on Parkinsonian signs, and a traditional supervised model. The absence of standardized data acquisition protocols and the limited availability of annotated datasets compromise the generalizability of supervised models. In these contexts, self-supervised models offer the advantage of learning more robust patterns from the data, bypassing the need for ground truth labels. This approach has the potential to accelerate the clinical validation of touchscreen typing software for neurodegenerative diseases.

GDUI: Guided Diffusion Model for Unlabeled Images

The diffusion model has made progress in the field of image synthesis, especially in the area of conditional image synthesis. However, this improvement is highly dependent on large annotated datasets. To tackle this challenge, we present the Guided Diffusion model for Unlabeled Images (GDUI) framework in this article. It utilizes the inherent feature similarity and semantic differences in the data, as well as the downstream transferability of Contrastive Language-Image Pretraining (CLIP), to guide the diffusion model in generating high-quality images. We design two semantic-aware algorithms, namely, the pseudo-label-matching algorithm and label-matching refinement algorithm, to match the clustering results with the true semantic information and provide more accurate guidance for the diffusion model. First, GDUI encodes the image into a semantically meaningful latent vector through clustering. Then, pseudo-label matching is used to complete the matching of the true semantic information of the image. Finally, the label-matching refinement algorithm is used to adjust the irrelevant semantic information in the data, thereby improving the quality of the guided diffusion model image generation. Our experiments on labeled datasets show that GDUI outperforms diffusion models without any guidance and significantly reduces the gap between it and models guided by ground-truth labels.