Macro F1 Research Articles

Deep Heterogeneous Contrastive Hyper-Graph Learning for In-the-Wild Context-Aware Human Activity Recognition

Human Activity Recognition (HAR) is a challenging, multi-label classification problem as activities may co-occur and sensor signals corresponding to the same activity may vary in different contexts (e.g., different device placements). This paper proposes a Deep Heterogeneous Contrastive Hyper-Graph Learning (DHC-HGL) framework that captures heterogenous Context-Aware HAR (CA-HAR) hypergraph properties in a message-passing and neighborhood-aggregation fashion. Prior work only explored homogeneous or shallow-node-heterogeneous graphs. DHC-HGL handles heterogeneous CA-HAR data by innovatively 1) Constructing three different types of sub-hypergraphs that are each passed through different custom HyperGraph Convolution (HGC) layers designed to handle edge-heterogeneity and 2) Adopting a contrastive loss function to ensure node-heterogeneity. In rigorous evaluation on two CA-HAR datasets, DHC-HGL significantly outperformed state-of-the-art baselines by 5.8% to 16.7% on Matthews Correlation Coefficient (MCC) and 3.0% to 8.4% on Macro F1 scores. UMAP visualizations of learned CA-HAR node embeddings are also presented to enhance model explainability. Our code is publicly available1 to encourage further research.

Rumor detection methodology based on sentiment analysis and the transformer model with decision-level fusion

The use of transformer models in natural language processing (NLP) has gained significant attention in recent years due to their exceptional performance in various language tasks. This paper explores the application of transformer models in rumor detection, the relevant research on rumor detection, the use of transformer models, and the techniques used to boost the models performance. Ultimately, the purpose of this paper is to provide insight into the potential of transformer models in detecting rumors on social media. Unlike other rumor-detecting models, the author adds a sentiment analysis model as a supplement to rumor detection. Also, to address the issue of insufficient information in early-stage comments on rumors, this paper proposes a decision-level fusion method before the output layer, which effectively utilizes information from different sources and minimizes the negative impact of insufficient data sources. The early-stage rumor detection accuracy of the model is greatly enhanced by this method, therefore, the articles main contributions can be regarded as follows: First, this paper proposes a combination of an aspect level text sentiment analysis method according to syntactic features, gated recurrent units, and a self-attention mechanism. Experimental findings demonstrate that, compared to the original model without taking the sentiment analysis method into account, the proposed network model has advantages in accuracy and Macro F1 evaluation indexes. Second, a cross-text rumor-detecting method based on Decision-level fusion is proposed. Its advantage is that when the cross-text data source is incomplete and a certain text is missing, another text can be used to continue the analysis. Experimental findings show the effectiveness of this method in improving the accuracy of emotion recognition by integrating data from different modes. Third, a comparison is conducted between the performance of the Transformer-sentiment model and other related models, Text-CNN, Bi-LSTM, etc. The result shows that this integrated Transformer-sentiment model can not only solve the rumor detection tasks at higher accuracy, but can also overcome the shortcomings of the lack of datasets, which means that the model is more robust, and is able to detect rumors at the early stage of the rumor spreading process.

Watchog: A Light-weight Contrastive Learning based Framework for Column Annotation

Relational Web tables provide valuable resources for numerous downstream applications, making table understanding, especially column annotation that identifies semantic types and relations of columns, a hot topic in the field of data management. Despite recent efforts to improve different tasks in table understanding by using the power of large pre-trained language models, existing methods heavily rely on large-scale and high-quality labeled instances, while they still suffer from the data sparsity problem due to the imbalanced data distribution among different classes. In this paper, we propose the Watchog framework, which employs contrastive learning techniques to learn robust representations for tables by leveraging a large-scale unlabeled table corpus with minimal overhead. Our approach enables the learned table representations to enhance fine tuning with much fewer additional labeled instances than in prior studies for downstream column annotation tasks. Besides, we further proposed optimization techniques for semi-supervised settings. Experimental results on popular benchmarking datasets illustrate the superiority of our proposed techniques in two column annotation tasks under different settings. In particular, our Watchog framework effectively alleviates the class imbalance issue caused by a long-tailed label distribution. In the semi-supervised setting, Watchog outperforms the best-known method by up to 26% and 41% in Micro and Macro F1 scores, respectively, on the task of semantic type detection.

DeepWalk with Reinforcement Learning (DWRL) for node embedding

DeepWalk is used to convert nodes in an original graph into equivalent vectors in a latent space for performing various predictive tasks. To ensure second-order structural similarity between nodes in the original graph and their vectors in the latent space, dot products are applied to each pair of nodes explored on the random walk (RW) in the latent space. However. dot products for graphs with millions of nodes and billions of edges are computationally expensive. To minimize the computation time required for calculating the second-order structural similarity, DeepWalk with reinforcement learning (DWRL) is proposed herein. In DWRL, a level pointer for each node in the original graph is prepared. By identifying common nodes between each pair of nodes in the original graph, the number of computations in the dot product in the latent space is reduced, thereby ensuring second-order structural similarity. Additionally, repeated selection of the same node during RWs produces redundant samples for training. Therefore, the subsampling technique is used to choose the next node based on its degree, which improves the generalization of node representations in the latent space and increases accuracy. The proposed techniques are applied to popular datasets to perform multilabel classification and link prediction tasks, and their efficiency in reducing the computation time is verified. The proposed DWRL minimizes the computation time 47% for large graphs to build latent vectors and improves the average micro and macro F1 scores up to 12%. The link prediction performance also increases up to 20%.

Sentiment Analysis on Household Audio Conversation with Automated Chatbot for Hospital Visits Prediction

AbstractBackgroundPeople living with dementia (PLwD) often experience difficulties describing their physical conditions and seeking medical attention timely, resulting in hospital visits that may cause frustration and exhaustion. Therefore, we aim to develop a machine learning model that helps identify potential hospital visit needs to provide early medical attention for PLwD.MethodPortable smart chatbots are deployed to PLwD households. We collected audio clips between households and chatbots and the hospitalisation records of PLwD. We manually labelled the sentiment scores of each audio clip based on its content and tone to between ‐3 and 3, where ‐3 indicates very negative, 0 indicates neutral, and 3 indicates very positive. We use 10 days of sentiment scores on a minute basis (0 when there’s no interaction) to predict the need for hospital visits in the following days. The positive outputs indicate that hospital visits happened within the 5 following days, while the negative outputs indicate no hospital visits occurred. A gradient boosting model (GBM) is trained to predict if hospital visits will happen in the following 5 days. We evaluated the trained model using the cross‐validation (cv = 5) method and reported Accuracy, Sensitivity, Specificity and macro F1 score.ResultBetween May 13th, 2021, to June 5th 2022, we collected 14 PLwDs’ hospital visit records and 12,278 audio clips between their households and their chatbots. Among all the participating households, we constructed a total of 3065 days as training and testing data (albeit no activities some days), with 37 hospital visit occurrences. This results in 139 days where hospital visits occurred in the following 5 days. The GBM model classified the need for hospital visits in the following 5 days with 0.682 Accuracy while obtaining 0.890 in Sensitivity, 0.669 in Specificity, and 0.522 in F1 score.ConclusionSentiment scores obtained by analysing audio clips between households of PLwD and their chatbots can be used to predict hospital visits in the following 5 days, with 0.890 in Sensitivity. This can help health professionals to provide more timely care to PLwD without frequent home visits. However, further research is needed to improve the Specificity and F1 of the predicting model.

A Joint Fake News Detection Model based on Multi-Features

Text analysis-based models have achieved outstanding results in fake news detection tasks in recent years, which is closely linked to the quantity and quality enhancement of feature information extracted from the text. Drawing upon the existing semantic detection frameworks, studies in this field concentrate on extracting various textual information through a solitary auxiliary feature, text stance feature or sentiment feature. However, it is challenging to depict the general attributes of the text using a single auxiliary feature, which frequently results in missing essential details and leaves problems with stance distortion and emotional resonance. To tackle the problem, this study proposes a joint model for identifying fake news, incorporating numerous textual characteristics. By extracting and blending various aspects of text features, i.e., semantic, stance and sentiment features, a more detailed and effective joint analysis of textual information is attained, resulting in improved performance in fake news detection. On the RumourEval-17 datasets, our model attains the Macro F1 Score of 0.891, surpassing current models for detecting rumors. Additionally, our model obtains a Macro F1 Score of 0.904 on the latest COVID-19 dataset, demonstrating strong competitiveness and promising prospects for fake news detection.

Covid-19 detection using modified xception transfer learning approach from computed tomography images

The significance of efficient and accurate diagnosis amidst the unique challenges posed by the COVID-19 pandemic underscores the urgency for innovative approaches. In response to these challenges, we propose a transfer learning-based approach using a recently annotated Computed Tomography (CT) image database. While many approaches propose an intensive data preprocessing and/or complex model architecture, our method focuses on offering an efficient solution with minimal manual engineering. Specifically, we investigate the suitability of a modified Xception model for COVID-19 detection. The method involves adapting a pre-trained Xception model, incorporating both the architecture and pre-trained weights from ImageNet. The output of the model was designed to make the final diagnosis decisions. The training utilized 128 batch sizes and 224x224 input image dimensions, downsized from standard 512x512. No further da processing was performed on the input data. Evaluation is conducted on the 'COV19-CT-DB' CT image dataset, containing labeled COVID-19 and non-COVID-19 cases. Results reveal the method's superiority in accuracy, precision, recall, and macro F1 score on the validation subset, outperforming the VGG-16 transfer model and thus offering enhanced precision with fewer parameters. Furthermore, compared to alternative methods for the COV19-CT-DB dataset, our approach exceeds the baseline approach and other alternatives on the same dataset. Finally, the adaptability of the modified Xception transfer learning-based model to the unique features of the COV19-CT-DB dataset showcases its potential as a robust tool for enhanced COVID-19 diagnosis from CT images.

Enhancing individual tree mortality mapping: The impact of models, data modalities, and classification taxonomy

Tree mortality is rapidly increasing as a result of more frequent and extensive droughts and forest fires across the globe. The increasing pace and scale of disturbance and resulting mortality necessitates the study of tree mortality at the scale at which it occurs (individual trees) to develop effective management strategies. Achieving this objective requires high-resolution aerial or satellite remote sensing data supported by field validation and efficient modeling. However, the optimal remote sensing data modalities and models for mapping overstory tree mortality remain uncertain and there is a lack of benchmark datasets for reproducible research, comparison, and model improvement. In this research, we propose a framework to generate tree mortality maps at the scale of individual trees and a labeled benchmark dataset using open-source high-resolution remote sensing data. Our benchmark dataset consists of over 15,000 image crops that have been manually labeled via multi-annotator majority for live, dead, mixed, and other classes in National Ecological Observatory Network (NEON) RGB, NEON hyperspectral, and National Agriculture Imagery Program (NAIP) RGBNIR imagery. Specifically, we present a multi-class classification framework that categorizes lidar-derived tree segments as live, dead, mixed, or other. We compared the performance of different machine learning models, remote sensing data modalities and their associated spatial resolution, spectral depth, and orthorectification using our benchmark dataset. Among various models, we found Convolutional Neural Network (CNN) models outperformed all other models with the highest accuracies and F1 scores across all data modalities. Similarly, among remote sensing data modalities, NEON RGB data at 0.1 m resolution performed best due to its high spatial resolution and high quality orthorectification. The overall accuracy of 87.27% with a macro F1 score of 0.76 was achieved using the RGB data with the CNN model. Our framework can be applied to any forest ecosystem in which lidar and raster imagery are available. We applied the framework using the best performing model and data modality to produce an individual tree mortality map over the Teakettle Experimental Forest in the southern Sierra Nevada. The framework and high-resolution mortality map can be valuable resources for forest managers and provide an invaluable basis for studying potential mortality drivers of dominant trees, forest inventory, selective logging, fire disturbance, and succession modeling.

Retrieval Contrastive Learning for Aspect-Level Sentiment Classification

Aspect-Level Sentiment Classification (ALSC) aims to assign specific sentiments to a sentence toward different aspects, which is one of the crucial challenges in the field of Natural Language Processing (NLP). Despite numerous approaches being proposed and obtaining prominent results, the majority of them focus on leveraging the relationships between the aspect and opinion words in a single instance while ignoring correlations with other instances, which will make models inevitably become trapped in local optima due to the absence of a global viewpoint. Instance representation derived from a single instance, on the one hand, the contained information is insufficient due to the lack of descriptions from other perspectives; on the other hand, its stored knowledge is redundant since the inability to filter extraneous content. To obtain a polished instance representation, we developed a Retrieval Contrastive Learning (RCL) framework to subtly extract intrinsic knowledge across instances. RCL consists of two modules: (a) obtaining retrieval instances by sparse retriever and dense retriever, and (b) extracting and learning the knowledge of the retrieval instances by using Contrastive Learning (CL). To demonstrate the superiority of RCL, five ALSC models are employed to conduct comprehensive experiments on three widely-known benchmarks. Compared with the baselines, ALSC models achieve substantial improvements when trained with RCL. Especially, ABSA-DeBERTa with RCL obtains new state-of-the-art results, which outperform the advanced methods by 0.92%, 0.23%, and 0.47% in terms of Macro F1 gains on Laptops, Restaurants, and Twitter, respectively.

Graph-based multi-modality integration for prediction of cancer subtype and severity

Personalised cancer screening before therapy paves the way toward improving diagnostic accuracy and treatment outcomes. Most approaches are limited to a single data type and do not consider interactions between features, leaving aside the complementary insights that multimodality and systems biology can provide. In this project, we demonstrate the use of graph theory for data integration via individual networks where nodes and edges are individual-specific. We showcase the consequences of early, intermediate, and late graph-based fusion of RNA-Seq data and histopathology whole-slide images for predicting cancer subtypes and severity. The methodology developed is as follows: (1) we create individual networks; (2) we compute the similarity between individuals from these graphs; (3) we train our model on the similarity matrices; (4) we evaluate the performance using the macro F1 score. Pros and cons of elements of the pipeline are evaluated on publicly available real-life datasets. We find that graph-based methods can increase performance over methods that do not study interactions. Additionally, merging multiple data sources often improves classification compared to models based on single data, especially through intermediate fusion. The proposed workflow can easily be adapted to other disease contexts to accelerate and enhance personalized healthcare.

Machine learning based phase prediction and powder metallurgy assisted experimental validation of medium entropy compositionally complex alloys

Medium entropy alloys (MEAs) are a subset of compositionally complex alloys whose mixing entropy lies between R and 1.5 R where R is the universal gas constant. The properties of MEAs largely depend on the phases present in the alloy such as solid solution (SS), solid solution + intermetallic (SS + IM) and amorphous (AM). Hence, the correct prediction of phases can enable the efficient selection of material compositions with anticipated properties. In this paper, three machine learning (ML) algorithms viz. k nearest neighbors (KNNs), artificial neural network (ANN), and random forest (RF) were employed for the ternary phase classification problem. An MEA dataset was constructed by utilizing all reported MEAs till February 2023 to the best of authors’ knowledge. The study implied that the use of only three features (mixing enthalpy, atomic size mismatch, and a strain energy related parameter) were sufficient for the phase prediction in MEAs. Among the three ML algorithms, ANN had the highest macro averaged F1 score (86.7%) and accuracy (87.3%) in predicting the phases in MEAs, while RF has the lowest macro F1 score (84.67%) and accuracy (84.8%). However, for phase prediction between single phase SS and multi-phase SS (binary classification), distance-based algorithm (KNN) was found to be suitable. The prediction performance of ML model over a completely unseen data was assessed in the case study section. The experimentally determined phase details of three new MEA compositions fabricated by powder metallurgy route was also included in the unseen dataset. The SS and AM phases were correctly labeled nine times out of eleven instances by using ANN model. However, the model prediction for SS + IM phase was found to be less reliable (three out of five correct) owing to its relatively poor F1 score.

Generative Adversarial Network-based Noncontrast CT Angiography for Aorta and Carotid Arteries.

Background Iodinated contrast agents (ICAs), which are widely used in CT angiography (CTA), may cause adverse effects in humans, and their use is time-consuming and costly. Purpose To develop an ICA-free deep learning imaging model for synthesizing CTA-like images and to assess quantitative and qualitative image quality as well as the diagnostic accuracy of synthetic CTA (Syn-CTA) images. Materials and Methods A generative adversarial network (GAN)-based CTA imaging model was trained, validated, and tested on retrospectively collected pairs of noncontrast CT and CTA images of the neck and abdomen from January 2017 to June 2022, and further validated on an external data set. Syn-CTA image quality was evaluated using quantitative metrics. In addition, two senior radiologists scored the visual quality on a three-point scale (3 = good) and determined the vascular diagnosis. The validity of Syn-CTA images was evaluated by comparing the visual quality scores and diagnostic accuracy of aortic and carotid artery disease between Syn-CTA and real CTA scans. Results CT scans from 1749 patients (median age, 60 years [IQR, 50-68 years]; 1057 male patients) were included in the internal data set: 1137 for training, 400 for validation, and 212 for testing. The external validation set comprised CT scans from 42 patients (median age, 67 years [IQR, 59-74 years]; 37 male patients). Syn-CTA images had high similarity to real CTA images (normalized mean absolute error, 0.011 and 0.013 for internal and external test set, respectively; peak signal-to-noise ratio, 32.07 dB and 31.58 dB; structural similarity, 0.919 and 0.906). The visual quality of Syn-CTA and real CTA images was comparable (internal test set, P = .35; external validation set, P > .99). Syn-CTA showed reasonable to good diagnostic accuracy for vascular diseases (internal test set: accuracy = 94%, macro F1 score = 91%; external validation set: accuracy = 86%, macro F1 score = 83%). Conclusion A GAN-based model that synthesizes neck and abdominal CTA-like images without the use of ICAs shows promise in vascular diagnosis compared with real CTA images. Clinical trial registration no. NCT05471869 © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Zhang and Turkbey in this issue.

CHP Engine Anomaly Detection Based on Parallel CNN-LSTM with Residual Blocks and Attention.

The extreme operating environment of the combined heat and power (CHP) engine is likely to cause anomalies and defects, which can lead to engine failure; thus, detecting engine anomalies is essential. In this study, we propose a parallel convolutional neural network-long short-term memory (CNN-LSTM) residual blocks attention (PCLRA) anomaly detection model with engine sensor data. To our knowledge, this is the first time that parallel CNN-LSTM-based networks have been used in the field of CHP engine anomaly detection. In PCLRA, spatiotemporal features are extracted via CNN-LSTM in parallel and the information loss is compensated using the residual blocks and attention mechanism. The performance of PCLRA is compared with various hybrid models for 15 cases. First, the performances of serial and parallel models are compared. In addition, we evaluated the contributions of the residual blocks and attention mechanism to the performance of the CNN-LSTM hybrid model. The results indicate that PCLRA achieves the best performance, with a macro f1 score (mean ± standard deviation) of 0.951 ± 0.033, an anomaly f1 score of 0.903 ± 0.064, and an accuracy of 0.999 ± 0.002. We expect that the energy efficiency and safety of CHP engines can be improved by applying the PCLRA anomaly detection model.

Episodic Thinking in Alzheimer's Disease Through the Lens of Language: Linguistic Analysis and Transformer-Based Classification.

Episodic memory decline is a hallmark of Alzheimer's disease (AD) and linked to deficits in episodic thinking directed to the future. We addressed the question whether a deficit in episodic thinking can be picked up directly from connected speech and its detection can be automatized. We linguistically classified 2,809 utterances (including embedded clauses in the utterances) from picture descriptions from 70 healthy older controls, 82 people with mild probable AD (pAD), and 46 people with moderate pAD for whether they were episodic, nonepisodic, or "other" (e.g., off-task). Generalized linear regression models were used to investigate how ratios of these categories change in AD, controlling for age, gender, and education. Finally, we applied deep learning technique to explore the feasibility of automating the episodicity analysis. Decline in episodicity significantly distinguished controls from both mild pAD and moderate pAD. Correlation analysis suggested this decline not to be an effect of age, gender, and education but of cognitive ability. The decline was not compensated by an increase of nonepisodic utterances but mainly of off-task expressions. A transformer-based classifier to explore the possibility of automatizing the classification of episodicity achieved a macro F1 score of 0.913 in the ternary classification. These results show that a loss of episodicity is an early effect in AD that is manifested in spontaneous speech and can be reliably measured by both humans and machines.

The method of grouping and classifying music curriculum teaching resources in the context of double reduction

Abstract A subsumption classification method is proposed to improve the classification accuracy of teaching resources in the music curriculum through a double reduction policy. Subject words are selected from high-similarity and high-frequency word sets, and a subject word tree is constructed using an automatic tree construction method based on a probabilistic latent semantic analysis algorithm. To complete the subsumption classification of teaching resources, an improved multi-graph kernel convolutional network is employed to group tree leaf nodes. According to the classification evaluation results, the recall, accuracy, and F1 values are 90%, 96.48%, and 88.81%, respectively, and the macro F1 value is as high as 81.59%. It can be seen that the method can effectively classify the teaching resources of music courses with the best effect of subsumption classification, which helps to improve the appropriateness and adequacy of teaching resources utilization.

Chinese legal judgment prediction via knowledgeable prompt learning

In recent years, applying AI techniques in the legal field has attracted researchers’ attention. In particular, Legal Judgment Prediction (LJP), which aims to predict accusations based on given case description texts, has attracted much attention from the natural language processing community. However, most of the existing LJP methods are data-intensive. As we know, data annotation in the legal field is expensive. Prompt learning is a recently prevalent methodology, which often achieves surprising results in few-shot or even zero-shot scenarios. We propose a novel method for Chinese LJP based on prompt learning called KnowPrompt4LJP. The method aligns the Chinese LJP task with the pre-training task of a Pre-trained Language Model (PLM) via a prompt template to stimulate the PLM’s recall of learned knowledge. In addition, the well-designed prompt template can enhance the PLM’s understanding of the Chinese LJP task. We also use an external knowledge base to extract keyword information from the Chinese case description texts and incorporate it into the prompt template, thus enhancing the guidance of the prompt template to the PLM. Experimental results on CAIL2018, a high-quality Chinese LJP competition dataset, show that KnowPrompt4LJP achieves far better results than the baselines in zero-shot, few-shot, and full-size training data scenarios. KnowPrompt4LJP can achieve a macro F1 value of 0.70 in the low-resource scenario, which is comparable to the baselines’ results in the data-rich scenario. In the scenario of using full-size training data, KnowPrompt4LJP can achieve a macro F1 value of 0.81.

Automatic Myocardial Contrast Echocardiography Image Quality Assessment Using Deep Learning: Impact on Myocardial Perfusion Evaluation.

The image quality of myocardial contrast echocardiography (MCE) is critical for precise myocardial perfusion evaluation but challenging for echocardiographers. Differences in quality may lead to diagnostic heterogeneity. This study was aimed at achieving automatic MCE image quality assessment using a deep neural network (DNN) and investigating its impact on myocardial perfusion evaluation. The Resnet-18 model was used for training and testing on internal and external data sets. Quality assessment involved three aspects: left ventricular opacification (LVO), shadowing, and flash adequacy; the quality score was calculated based on image quality. This study explored the impact of the DNN-based quality score on perfusion evaluation (normal, delay or obstruction) by echocardiographers (two seniors, one junior and one novice). Additionally, the effect of the score difference between re-scans on perfusion evaluation was investigated. The time cost for DNN prediction was 0.045 s/frame. In internal validation and external testing, the DNN achieved F1 and macro F1 scores >90% for quality assessment and had high intraclass correlation coefficients (0.954 and 0.892, respectively) in sequence quality scores. The proportion of segments deemed uninterpretable increased as the DNN-based quality score decreased. The agreement of perfusion assessment between one senior and others decreased as the quality score decreased. And the greater the score difference between the re-scans, the lower was the agreement on perfusion assessment by the same echocardiographer. This study determined the effectiveness of DNN for real-time automatic MCE quality assessment. It has the potential to reduce the variability in perfusion evaluation among echocardiographers.

HMGAN

Wearable Human Activity Recognition (WHAR) is an important research field of ubiquitous and mobile computing. Deep WHAR models suffer from the overfitting problem caused by the lack of a large amount and variety of labeled data, which is usually addressed by generating data to enlarge the training set, i.e., Data Augmentation (DA). Generative Adversarial Networks (GANs) have shown their excellent data generation ability, and the generalization ability of a classification model can be improved by GAN-based DA. However, existing GANs cannot make full use of the important modality information and fail to balance modality details and global consistency, which cannot meet the requirements of deep multi-modal WHAR. In this paper, a hierarchical multi-modal GAN model (HMGAN) is proposed for WHAR. HMGAN consists of multiple modal generators, one hierarchical discriminator, and one auxiliary classifier. Multiple modal generators can learn the complex multi-modal data distributions of sensor data. Hierarchical discriminator can provide discrimination outputs for both low-level modal discrimination losses and high-level overall discrimination loss to draw a balance between modality details and global consistency. Experiments on five public WHAR datasets demonstrate that HMGAN achieves the state-of-the-art performance for WHAR, outperforming the best baseline by an average of 3.4%, 3.8%, and 3.5% in accuracy, macro F1 score, and weighted F1 score, respectively.

T5 for Hate Speech, Augmented Data, and Ensemble

We conduct relatively extensive investigations of automatic hate speech (HS) detection using different State-of-The-Art (SoTA) baselines across 11 subtasks spanning six different datasets. Our motivation is to determine which of the recent SoTA models is best for automatic hate speech detection and what advantage methods, such as data augmentation and ensemble, may have on the best model, if any. We carry out six cross-task investigations. We achieve new SoTA results on two subtasks—macro F1 scores of 91.73% and 53.21% for subtasks A and B of the HASOC 2020 dataset, surpassing previous SoTA scores of 51.52% and 26.52%, respectively. We achieve near-SoTA results on two others—macro F1 scores of 81.66% for subtask A of the OLID 2019 and 82.54% for subtask A of the HASOC 2021, in comparison to SoTA results of 82.9% and 83.05%, respectively. We perform error analysis and use two eXplainable Artificial Intelligence (XAI) algorithms (Integrated Gradient (IG) and SHapley Additive exPlanations (SHAP)) to reveal how two of the models (Bi-Directional Long Short-Term Memory Network (Bi-LSTM) and Text-to-Text-Transfer Transformer (T5)) make the predictions they do by using examples. Other contributions of this work are: (1) the introduction of a simple, novel mechanism for correcting Out-of-Class (OoC) predictions in T5, (2) a detailed description of the data augmentation methods, and (3) the revelation of the poor data annotations in the HASOC 2021 dataset by using several examples and XAI (buttressing the need for better quality control). We publicly release our model checkpoints and codes to foster transparency.

A Novel Learning-Based PLST Algorithm for Multi-Label Classification

In Multi-Label Classification (MLC), each data sample is characterized by multiple labels. In ML, there is no restriction on the number of classes a data sample could belong to. Various methods are proposed to construct a model to map data samples to the class labels. Transforming the MLC problem into several binary classification problems is the baseline approach, also called the binary relevance technique. As an alternative solution, the label powerset approach transforms the MLC problem into a multi-class classification problem by building a binary classifier for every label composition. However, in Ensemble Methods (EMs), multiple classifiers are consolidated to form a multi-label ensemble classifier. In this case, each classifier predicts a class label. Thereupon, the predicted class labels are joined using an ensemble method. This paper presents a neoteric EM classifier using the Ridge Regression-based Principal Label Space Transformation (RR-PLST). In the proposed model, the PLST utilizes SVD to discover correlations between the labels and data samples. Dimensionality reduction is a well-known technique to enhance the model's precision and diminish execution time. Consequently, the OPLS technique is recruited to reduce data dimensions in the proposed model. Moreover, in the proposed model, a Q-Learning method is used to amend the initially predicted labels. To demonstrate the effectiveness of the proposed model, vast experiments are conducted on the various widely used multi-label datasets. The proposed algorithm results are compared with similar algorithms by the AP, EM, HS, Macro, and Micro F1 metrics. The experimental results demonstrate the superiority of the proposed model.