Semantic Classification Research Articles

A three in one bottom-up framework for simultaneous semantic segmentation, instance segmentation and classification of multi-organ nuclei in digital cancer histology

Simultaneous segmentation and classification of nuclei in digital histology remains challenging. The highest achieved Panoptic Quality (PQ) remains low due to overlapping nuclei, higher staining and tissue variability, and rough clinical conditions. The generic deep-learning methods usually rely on end-to-end models, which fail to address these problems associated explicitly with digital histology. We resolve these issues using a dual attention-based model combined with post-processing in a bottom-up fashion. We use three attention decoder heads, which produce semantic segmentation, edge proposals, and classification maps. We use these outputs to apply post-processing, including controlled watershed and pixel grouping, to produce instance segmentation and classification. Our multi-stage approach utilizes edge proposals and semantic segmentations compared to direct segmentation and classification strategies followed by most generic state-of-the-art methods. Due to this, we demonstrate a significant performance improvement in producing high-quality instance segmentation and nuclei classification. We have achieved a 0.841 Dice score for semantic segmentation, 0.713 bPQ scores for instance segmentation, and 0.633 mPQ for nuclei classification. Furthermore, the framework is less complex compared to the state-of-the-art.

Wildfire response of forest species from multispectral LiDAR data. A deep learning approach with synthetic data

Forests play a crucial role as the lungs and life-support system of our planet, harbouring 80% of the Earth's biodiversity. However, we are witnessing an average loss of 480 ha of forest every hour because of destructive wildfires spreading across the globe. To effectively mitigate the threat of wildfires, it is crucial to devise precise and dependable approaches for forecasting fire dynamics and formulating efficient fire management strategies, such as the utilisation of fuel models.The objective of this study was to enhance forest fuel classification that considers only structural information, such as the Prometheus model, by integrating data on the fire responses of various tree species and other vegetation elements, such as ground litter and shrubs. This distinction can be achieved using multispectral (MS) Light Detection and Ranging (LiDAR) data in mixed forests. The methodology involves a novel approach in semantic classifications of forests by generating synthetic data with semantic labels regarding fire responses and reflectance information at different spectral bands, as a real MS scanner device would detect. Forests, which are highly intricate environments, present challenges in accurately classifying point clouds. To address this complexity, a deep learning (DL) model for semantic classification was trained on synthetic point clouds in different formats to achieve the best performance when leveraging MS data.Forest plots in the study region were scanned using different Terrestrial Laser Scanning sensors at wavelengths of 905 and 1550 nm. Subsequently, an interpolation process was applied to generate the MS point clouds of each plot, and the trained DL model was applied to classify them. These classifications surpassed the average thresholds of 90% and 75% for accuracy and intersection over union, respectively, resulting in a more precise categorisation of fuel models based on the distinct responses of forest elements to fire. The results of this study reveal the potential of MS LiDAR data and DL classification models for improving fuel model retrieval in forest ecosystems and enhancing wildfire management efforts.

Analysing lexical semantic changes in chinese educational texts by integrating machine learning methods

Analyzing Lexical Semantic Changes (LSC) in Educational Texts (ET) refers to examining how the meanings of words, terms, or phrases used in ET have evolved. It involves learning shifts in the semantic content, connotations, and language associations within educational resources such as textbooks, research articles, and instructional content. The analysis can reveal how educational models, pedagogical methods, and terminology have transformed in response to technological innovations, societal changes, and pedagogical developments. This analysis provides visions into the dynamic nature of educational discourse, helping researchers, educators, and policymakers understand how language has adapted to reflect changes in educational paradigms and the broader educational context. This research investigates the semantic analysis and classification performance within ET, employing the innovative Decision Tree + Feed Forward Neural Networks (DT + FFNNs) framework. This research shows the dynamic semantic relationships inherent in educational terminology by diverse semantic similarity measures and contextualized embeddings. It looks at how educational language changes to reflect changes in society, technology, and pedagogy. The study uses a DT + FFNN framework for semantic analysis and classification. The study uses several embeddings and semantic similarity metrics, and Spearman’s Correlation Coefficient (SCC) is employed to evaluate their effectiveness. This study highlights the DT + FFNN framework’s capacity to capture complex semantics in an educational setting and offers insights into the adaptive nature of educational discourse. SCC serves as a guiding metric, offering insights into the efficiency of several embeddings and measures. The findings show the pivotal role of fine-tuning in significantly enhancing the accuracy of DT + FFNNs across measures, revealing its remarkable potential in capturing semantics within an educational context.

Marcación media en español: afección, control y sujetos no canónicos

Abstract Middle markers (such as reflexives in Spanish) are used to encode both semantically connected situations and some semantic properties of the subject participants. Previous literature on the topic usually mentioned two of these properties: affectedness (Klaiman 1988 and 1991; Kemmer 1993; Maldonado 1999; Zúñiga/Kittilä 2019) and control (potential control, Klaiman 1991; non-canonical control, Kaufmann 2007). Middle marking in Spanish has been also related to both affected/patient subjects (García-Miguel 1985; Mendikoetxea 1999; Jiménez Juliá 2016) and agentive/intentional subjects (Martín Zorraquino 1979; Morimoto 2011; Di Tullio 2012). Nevertheless, there is not a comprehensive study on affectedness, control and middle marking in Spanish. This paper seeks to shade some light on the topic offering a detailed analysis of affectedness, control, and their correlations within a verbal sample from 13 semantic classes of the middle domain. In particular, we claim that middle marking changes the status of the subject entity, which can be [+affected], [+control], or [-control] depending on the semantics of the verb, the semantics of the sentence, the status of the entity in the active voice, and on our world knowledge. Lastly, we argue that the general function of middle marking is to encode non-canonical subjects on the control construct.

Per-class curriculum for Unsupervised Domain Adaptation in semantic segmentation

AbstractAccurate training of deep neural networks for semantic segmentation requires a large number of pixel-level annotations of real images, which are expensive to generate or not even available. In this context, Unsupervised Domain Adaptation (UDA) can transfer knowledge from unlimited synthetic annotations to unlabeled real images of a given domain. UDA methods are composed of an initial training stage with labeled synthetic data followed by a second stage for feature alignment between labeled synthetic and unlabeled real data. In this paper, we propose a novel approach for UDA focusing the initial training stage, which leads to increased performance after adaptation. We introduce a curriculum strategy where each semantic class is learned progressively. Thereby, better features are obtained for the second stage. This curriculum is based on: (1) a class-scoring function to determine the difficulty of each semantic class, (2) a strategy for incremental learning based on scoring and pacing functions that limits the required training time unlike standard curriculum-based training and (3) a training loss to operate at class level. We extensively evaluate our approach as the first stage of several state-of-the-art UDA methods for semantic segmentation. Our results demonstrate significant performance enhancements across all methods: improvements of up to 10% for entropy-based techniques and 8% for adversarial methods. These findings underscore the dependency of UDA on the accuracy of the initial training. The implementation is available at https://github.com/vpulab/PCCL.

Translation of English Verbs Belonging to the Lexico-Semantic Group of Human Behavior into Russian and German (Based on W. Thackeray’s Novel “Vanity Fair”)

This article examines the use of English verbs related to human behavior in W. Thackeray’s novel “Vanity Fair” and their translations into German and Russian. Based on the analysis of lexicographic sources, the structure of the lexico-semantic group of behavioral verbs in English, German, and Russian was analyzed. Common transformations employed to translate such verbs from English into German and Russian were identified. The most representative examples were picked from the novel. Particular attention was given to the relationship between the translation methods and each target language, as well as between the source language and the target languages. The Russian and German translations of the novel were compared for accuracy in conveying all details. Using componential analysis, a semantic classification of the selected verbs, depending on the type of behavior they describe, was compiled for the three languages. The results obtained led to the following conclusions. The behavioral verbs in W. Thackeray’s novel “Vanity Fair” are most often translated into Russian and German with literal equivalents of the original lexemes. However, lexical and grammatical substitutions were also revealed: their usage depends on grammatical structure, lexical compatibility, and linguistic standards. The German translation proved to be more plausible than the Russian one. This could be because English and German are both Germanic languages and belong to the same branch of the Indo-European language family. In the Russian translation, multiple transformations of the original text are used more often. They are meant to convey the author’s ideas rather than the exact lexicographic meanings of separate words. The findings of this study contribute to the development of the theory and practice of translation and provide a deeper insight into W. Thackeray’s worldview.

The Pragmatic Motivation of Atypical English Way into/to Construction

The English way construction exhibits distinct constructional meaning and semantic features; however, the prepositional slot in this construction has not received sufficient attention, and there remains a lack of comparative research regarding different prepositions in this field. This study set out to examine the semantic classification, mapping features, cognitive factors in the subcategory English way into/to construction, and to improve the algorithm of ΔPratio. The research adopted a bottom-up corpus-driven approach based on the BNC and used HAC (Hierarchical Cluster Analysis) to explore the semantic categories of the verb collexemes and compare the lexical richness in these two constructions. The results revealed that at the level of linguistic structure, coercion and metaphor, as well as family resemblance, served as legitimization methods for verb entry into the construction, while the principles of economy and expressivity disclosed the intentions and motivations behind the semantic extensions of constructions at the pragmatic level. Additionally, this research found that the latter two principles were competitive in the interpretation process of way into/to construction, providing insights into understanding the psychological and social factors in language communication.

КЫРГЫЗ БАЛДАР АДАБИЯТЫНДА ТУУРАНДЫ СӨЗДӨРДҮН ӨЗГӨЧӨЛҮГҮ

The article examines the phonetic features and stylistic nature ofimitative words presented in Kyrgyz children's works. In Kyrgyz children's literature, many issues of studying words as functionally significant units of language today are not considered in sufficient depth. Ourgoal is to establish linguistic units, phonetic features and stylistic ways of usingimitativewords in children's fiction of the Kyrgyz people. As children acquire language, comparisons include creativity as a recurring component. When describing a child’s linguistic world, it is important to determine the appropriate parameters: people, nature, linguistic world. Among these areas, we noted that the function of comparisons in the world of language is special, andwe examined their problem as a component of the sound system of the language and their stylistic nature using the example of the works of Kyrgyz children. The semantic classification of imitative words in children's literature is also one of the main issues. Therefore, they were divided into groups depending on their importance.

A deep inverse convolutional neural network-based semantic classification method for land cover remote sensing images

The imbalance of land cover categories is a common problem. Some categories appear less frequently in the image, while others may occupy the vast majority of the proportion. This imbalance can lead the classifier to tend to predict categories with higher frequency of occurrence, while the recognition effect on minority categories is poor. In view of the difficulty of land cover remote sensing image multi-target semantic classification, a semantic classification method of land cover remote sensing image based on depth deconvolution neural network is proposed. In this method, the land cover remote sensing image semantic segmentation algorithm based on depth deconvolution neural network is used to segment the land cover remote sensing image with multi-target semantic segmentation; Four semantic features of color, texture, shape and size in land cover remote sensing image are extracted by using the semantic feature extraction method of remote sensing image based on improved sequential clustering algorithm; The classification and recognition method of remote sensing image semantic features based on random forest algorithm is adopted to classify and identify four semantic feature types of land cover remote sensing image, and realize the semantic classification of land cover remote sensing image. The experimental results show that after this method classifies the multi-target semantic types of land cover remote sensing images, the average values of Dice similarity coefficient and Hausdorff distance are 0.9877 and 0.9911 respectively, which can accurately classify the multi-target semantic types of land cover remote sensing images.

A Bayesian probabilistic analysis of the use of English modal verbs in L2 writing: Focusing on L1 influence and topic effects

Modal verbs, with their multifaceted semantic nuances and varied grammatical configurations, present notable challenges for L2 learners and regularly intrigue L2 researchers. This study attempts to investigate and compare how English modal verbs are used by L2 learners from different L1 backgrounds. By exploring the Turkish and Chinese learners’ subcorpora of the International Corpus of Learner English (ICLE), this work scrutizes the overall frequencies of nine core English modal verbs as grouped into three major semantic classes along with the influential lexico-syntatic variables that are semantic classes of collocated verbs, grammatical patterns and subject pronominality. The results of a Bayesian probabilistic analysis show that both the Turkish and Chinese learners primed similar modal verbs and constructional preferences without topic as the normalizing factor. While the broader analysis reveals no statistically significant divergences between these two learner groups in English modal verb preferences, a pronounced contextual influence is evident when the dataset narrows to essays on a unified theme. This nuanced shift underscores the intricate relationship between essay topics and linguistic structures, thus emphasizing the pivotal role of context in modal verb usage.

MESED: A Multi-Modal Entity Set Expansion Dataset with Fine-Grained Semantic Classes and Hard Negative Entities

The Entity Set Expansion (ESE) task aims to expand a handful of seed entities with new entities belonging to the same semantic class. Conventional ESE methods are based on mono-modality (i.e., literal modality), which struggle to deal with complex entities in the real world such as (1) Negative entities with fine-grained semantic differences. (2) Synonymous entities. (3) Polysemous entities. (4) Long-tailed entities. These challenges prompt us to propose novel Multi-modal Entity Set Expansion (MESE), where models integrate information from multiple modalities to represent entities. Intuitively, the benefits of multi-modal information for ESE are threefold: (1) Different modalities can provide complementary information. (2) Multi-modal information provides a unified signal via common visual properties for the same semantic class or entity. (3) Multi-modal information offers robust alignment signals for synonymous entities. To assess model performance in MESE, we constructed the MESED dataset which is the first multi-modal dataset for ESE with large-scale and elaborate manual calibration. A powerful multi-modal model MultiExpan is proposed which is pre-trained on four multimodal pre-training tasks. The extensive experiments and analyses on MESED demonstrate the high quality of the dataset and the effectiveness of our MultiExpan, as well as pointing the direction for future research. The benchmark and code are public at https://github.com/THUKElab/MESED.

Any-Way Meta Learning

Although meta-learning seems promising performance in the realm of rapid adaptability, it is constrained by fixed cardinality. When faced with tasks of varying cardinalities that were unseen during training, the model lacks its ability. In this paper, we address and resolve this challenge by harnessing `label equivalence' emerged from stochastic numeric label assignments during episodic task sampling. Questioning what defines ``true" meta-learning, we introduce the ``any-way" learning paradigm, an innovative model training approach that liberates model from fixed cardinality constraints. Surprisingly, this model not only matches but often outperforms traditional fixed-way models in terms of performance, convergence speed, and stability. This disrupts established notions about domain generalization. Furthermore, we argue that the inherent label equivalence naturally lacks semantic information. To bridge this semantic information gap arising from label equivalence, we further propose a mechanism for infusing semantic class information into the model. This would enhance the model's comprehension and functionality. Experiments conducted on renowned architectures like MAML and ProtoNet affirm the effectiveness of our method.

Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptation

We consider the task of generating diverse and realistic videos guided by natural audio samples from a wide variety of semantic classes. For this task, the videos are required to be aligned both globally and temporally with the input audio: globally, the input audio is semantically associated with the entire output video, and temporally, each segment of the input audio is associated with a corresponding segment of that video. We utilize an existing text-conditioned video generation model and a pre-trained audio encoder model. The proposed method is based on a lightweight adaptor network, which learns to map the audio-based representation to the input representation expected by the text-to-video generation model. As such, it also enables video generation conditioned on text, audio, and, for the first time as far as we can ascertain, on both text and audio. We validate our method extensively on three datasets demonstrating significant semantic diversity of audio-video samples and further propose a novel evaluation metric (AV-Align) to assess the alignment of generated videos with input audio samples. AV-Align is based on the detection and comparison of energy peaks in both modalities. In comparison to recent state-of-the-art approaches, our method generates videos that are better aligned with the input sound, both with respect to content and temporal axis. We also show that videos produced by our method present higher visual quality and are more diverse. Code and samples are available at: https://pages.cs.huji.ac.il/adiyoss-lab/TempoTokens/.

Self-supervised learning for medical image data with anatomy-oriented imaging planes

Self-supervised learning has emerged as a powerful tool for pretraining deep networks on unlabeled data, prior to transfer learning of target tasks with limited annotation. The relevance between the pretraining pretext and target tasks is crucial to the success of transfer learning. Various pretext tasks have been proposed to utilize properties of medical image data (e.g., three dimensionality), which are more relevant to medical image analysis than generic ones for natural images. However, previous work rarely paid attention to data with anatomy-oriented imaging planes, e.g., standard cardiac magnetic resonance imaging views. As these imaging planes are defined according to the anatomy of the imaged organ, pretext tasks effectively exploiting this information can pretrain the networks to gain knowledge on the organ of interest. In this work, we propose two complementary pretext tasks for this group of medical image data based on the spatial relationship of the imaging planes. The first is to learn the relative orientation between the imaging planes and implemented as regressing their intersecting lines. The second exploits parallel imaging planes to regress their relative slice locations within a stack. Both pretext tasks are conceptually straightforward and easy to implement, and can be combined in multitask learning for better representation learning. Thorough experiments on two anatomical structures (heart and knee) and representative target tasks (semantic segmentation and classification) demonstrate that the proposed pretext tasks are effective in pretraining deep networks for remarkably boosted performance on the target tasks, and superior to other recent approaches.

Sentiment Analysis of Short Texts Using SVMs and VSMs-Based Multiclass Semantic Classification

ABSTRACT In our approach, a hybrid machine learning model is proposed which uses Enhanced Vector Space Model (EVSM) along with Hybrid Support Vector Machine (HSVM) classifier. Initially the social media-based information is retrieved using Enhanced Vector Space Model (EVSM). EVSMs are employed in order to characterize the text content by mapping them into high-dimensional vector spaces, capturing the relationships between words and their contextual meanings. Rigorous feature selection methods are employed to designate texts for review, and a multiclass semantic classification algorithm, specifically the HSVM classifier, is utilized for categorization. Decision tree algorithm is used along with SVM to refine the selection process. To enhance sentiment analysis accuracy, sentiment dictionaries are not only presented but also extended through the expansion of Stanford’s GloVE tool. To enhance precision, the proposed work introduces weight-enhancing methods for processing renowned text weights. Sentiments are classified into positive, negative, and neutral categories. Notably, the achieved results demonstrate improved accuracy, attributed to the incorporation of an emotional sentiment enhancement factor for determining weights and leveraging sentiment dictionaries for word availability. The accuracy is obtained to be 92.78% with 91.33% positive sentiment rate and 97.32% negative sentiment rate.

Correction to: Cancer hallmark analysis using semantic classification with enhanced topic modelling on biomedical literature

Correction to: Cancer hallmark analysis using semantic classification with enhanced topic modelling on biomedical literature

Drowning in the Information Flood: Machine-Learning-Based Relevance Classification of Flood-Related Tweets for Disaster Management

In the early stages of a disaster caused by a natural hazard (e.g., flood), the amount of available and useful information is low. To fill this informational gap, emergency responders are increasingly using data from geo-social media to gain insights from eyewitnesses to build a better understanding of the situation and design effective responses. However, filtering relevant content for this purpose poses a challenge. This work thus presents a comparison of different machine learning models (Naïve Bayes, Random Forest, Support Vector Machine, Convolutional Neural Networks, BERT) for semantic relevance classification of flood-related, German-language Tweets. For this, we relied on a four-category training data set created with the help of experts from human aid organisations. We identified fine-tuned BERT as the most suitable model, averaging a precision of 71% with most of the misclassifications occurring across similar classes. We thus demonstrate that our methodology helps in identifying relevant information for more efficient disaster management.

Ticino: A multi-modal remote sensing dataset for semantic segmentation

Multi-modal remote sensing (RS) involves the fusion of data from multiple sensors, such as RGB, Multispectral, Hyperspectral, Light Detection and Ranging, Synthetic Aperture Radar, etc., each capturing unique information across different regions of the electromagnetic spectrum. The fusion of different modalities can provide complementary information, allowing for a comprehensive understanding of the Earth’s surface.Multi-modal RS image segmentation leverages various RS modalities to achieve pixel-level semantics classification. While deep learning has demonstrated promise in this domain, the limited availability of labeled multi-modal data poses a constraint on leveraging data-intensive techniques like deep learning to their full potential. To address this gap, we present Ticino, a novel multi-modal remote sensing dataset tailored for semantic segmentation.Ticino includes five modalities, including RGB, Digital Terrain Model, Panchromatic, and Hyperspectral images within the visual-near and short-wave infrared spectrum. Specifically annotated for Land Cover and Soil Agricultural Use, the dataset serves as a valuable resource for researchers in the field. Additionally, we conduct a comparative analysis, comparing single-modality with multi-modality deep learning techniques and evaluating the effectiveness of early fusion versus middle fusion approaches.This work aims to facilitate future research efforts in the domain by providing a robust benchmark dataset and insights into the effectiveness of various segmentation approaches.

NERONE: The Fast Way to Efficiently Execute Your Deep Learning Algorithm at the Edge.

Semantic segmentation and classification are pivotal in many clinical applications, such as radiation dose quantification and surgery planning. While manually labeling images is highly time-consuming, the advent of Deep Learning (DL) has introduced a valuable alternative. Nowadays, DL models inference is run on Graphics Processing Units (GPUs), which are power-hungry devices, and, therefore, are not the most suited solution in constrained environments where Field Programmable Gate Arrays (FPGAs) become an appealing alternative given their remarkable performance per watt ratio. Unfortunately, FPGAs are hard to use for non-experts, and the creation of tools to open their employment to the computer vision community is still limited. For these reasons, we propose NERONE, which allows end users to seamlessly benefit from FPGA acceleration and energy efficiency without modifying their DL development flows. To prove the capability of NERONE to cover different network architectures, we have developed four models, one for each of the chosen datasets (three for segmentation and one for classification), and we deployed them, thanks to NERONE, on three different embedded FPGA-powered boards achieving top average energy efficiency improvements of 3.4× and 1.9× against a mobile and a datacenter GPU devices, respectively.

Brain Tumor Classification using Convolutional Neural Network

This study intuitively addresses a crucial need in the medical domain by introducing a customized Mobile-Net model for classifying brain cancer in medical imaging data. The paramount importance of accurate classification in cancer diagnosis and treatment planning cannot be emphasized further. This particular research primarily focuses on the practical application of semantic classification techniques to precisely identify and outline brain cancer zones in medical imaging data. By utilizing a Mobile-Net architecture, the developed model highlights outstanding performance with an accuracy score of 85%.