Multi-label Research Articles

An efficient class-dependent learning label approach using feature selection to improve multi-label classification algorithms

An efficient class-dependent learning label approach using feature selection to improve multi-label classification algorithms

Graph neural network-based subgraph analysis for predicting adverse drug events

PurposeAdverse drug events (ADEs) are a significant global public health concern, and they have resulted in high rates of hospital admissions, morbidity, and mortality. Prior to the use of machine learning and deep learning methods, ADEs may not become well recognized until long after a drug has been approved and is widely used, which poses a significant challenge for ensuring patient safety. Consequently, there is a need to develop computational approaches for earlier identification of ADEs not detected during pre-registration clinical trials. MethodsThis paper presents a state-of-the-art network-based approach that models patients as subgraphs composed of nodes of International Classification of Diseases (ICD) codes and directed edges illustrating disease progression. Four Graph Neural Network (GNN) variants were employed to make sub-graph level predictions that answer three Research Questions (RQ): 1) whether ADE(s) would occur given a patient's prior diagnoses history, 2) when an ADE would occur, and 3) which ADE would occur. The first and second RQs were addressed using a binary classification approach. The third RQ was addressed using a multi-label classification model. ResultsThe proposed network-based approach demonstrated superior performance in predicting ADEs, with the GraphSage model exhibiting the highest accuracy for both RQ 1 (0.8863) and RQ 3 (0.9367), while the Graph Attention Networks (GAT) model was found to perform best for RQ 2 (0.8769). Furthermore, an analysis segmented by ADE classification revealed that while RQs 1 and 3 exhibited minimal variance across different ADE categories, a distinct advantage was observed for categories B, C, and E in the context of RQ 2 when applying this sub-graph method. ConclusionThe network-based approach demonstrates the potential of GNNs in supporting the early detection and prevention of ADEs. Accurately predicting ADEs could enable healthcare professionals to make informed clinical decisions, take preventive measures and adjust medication regimens before serious adverse events occur. The proposed prediction method could also lead to optimized usage of healthcare resources by preventing hospital admissions and reducing the overall burden of adverse drug events on the healthcare systems.

Research on Abnormal State Detection of CZ Silicon Single Crystal Based on Multimodal Fusion.

The Czochralski method is the primary technique for single-crystal silicon production. However, anomalous states such as crystal loss, twisting, swinging, and squareness frequently occur during crystal growth, adversely affecting product quality and production efficiency. To address this challenge, we propose an enhanced multimodal fusion classification model for detecting and categorizing these four anomalous states. Our model initially transforms one-dimensional signals (diameter, temperature, and pulling speed) into time-frequency domain images via continuous wavelet transform. These images are then processed using a Dense-ECA-SwinTransformer network for feature extraction. Concurrently, meniscus images and inter-frame difference images are obtained from the growth system's meniscus video feed. These visual inputs are fused at the channel level and subsequently processed through a ConvNeXt network for feature extraction. Finally, the time-frequency domain features are combined with the meniscus image features and fed into fully connected layers for multi-class classification. The experimental results show that the method can effectively detect various abnormal states, help the staff to make a more accurate judgment, and formulate a personalized treatment plan for the abnormal state, which can improve the production efficiency, save production resources, and protect the extraction equipment.

MHA-DGCLN: multi-head attention-driven dynamic graph convolutional lightweight network for multi-label image classification of kitchen waste

MHA-DGCLN: multi-head attention-driven dynamic graph convolutional lightweight network for multi-label image classification of kitchen waste

A multi-label text classifier: application on an Italian public tender procedure, project ISCOL@

The main means of communication during the pre-design phase is natural language. Effective communication during the pre-design phase through text documents and reports is essential to the success of a design and construction project. The study develops and evaluates a Natural Language Processing (NLP) tool called ArchiBERTo to process textual data related to design tender documents in the Italian public procurement process. Documenti di Indirizzo alla Progettazione (DIPs) are key documents, as they outline the demands, needs, and objectives of the public appointing party. ArchiBERTo is developed to process and convert DIP quality objective sentences into a hierarchy of objectives and criteria. The performances are evaluated by comparing the tool’s rankings with those provided by a group of domain experts. The results demonstrate ArchiBERTo's capability to reflect the collective ability of a panel of experts and to properly reflect the different contents of the DIP in the objectives hierarchy. The proposed system aims to address the issue of information asymmetry and potential misunderstandings, or varying interpretations, among the key actors of the Italian tendering procedure, namely the public appointing party, the design teams, and the external committee, regarding the relative importance of quality objectives and evaluation criteria. The utilization of the NLP systems ArchiBERTo to establish a shared hierarchy of objectives is expected to enhance communication and promote consensus during the pre-design phase. The minimization of the consensus issue among the actors can have a positive impact on the overall quality of the design proposals and facilitate the evaluation process conducted by the external committee, bridging the gap between expected and actual quality, ensuring that design proposals align with the quality objectives and demands of the public actor.

Automatic authorship attribution in Albanian texts.

Automatic authorship identification is a challenging task that has been the focus of extensive research in natural language processing. Regardless of the progress made in attributing authorship, the need for corpora in under-resourced languages impedes advancing and examining present methods. To address this gap, we investigate the problem of authorship attribution in Albanian. We introduce a newly compiled corpus of Albanian newsroom columns and literary works and analyze machine-learning methods for detecting authorship. We create a set of hand-crafted features targeting various categories (lexical, morphological, and structural) relevant to Albanian and experiment with multiple classifiers using two different multiclass classification strategies. Furthermore, we compare our results to those obtained using deep learning models. Our investigation focuses on identifying the best combination of features and classification methods. The results reveal that lexical features are the most effective set of linguistic features, significantly improving the performance of various algorithms in the authorship attribution task. Among the machine learning algorithms evaluated, XGBoost demonstrated the best overall performance, achieving an F1 score of 0.982 on literary works and 0.905 on newsroom columns. Additionally, deep learning models such as fastText and BERT-multilingual showed promising results, highlighting their potential applicability in specific scenarios in Albanian writings. These findings contribute to the understanding of effective methods for authorship attribution in low-resource languages and provide a robust framework for future research in this area. The careful analysis of the different scenarios and the conclusions drawn from the results provide valuable insights into the potential and limitations of the methods and highlight the challenges in detecting authorship in Albanian. Promising results are reported, with implications for improving the methods used in Albanian authorship attribution. This study provides a valuable resource for future research and a reference for researchers in this domain.

Multi-modal EEG NEO-FFI with Trained Attention Layer (MENTAL) for mental disorder prediction.

Early detection and accurate diagnosis of mental disorders is difficult due to the complexity of the diagnostic process, resulting in conditions being left undiagnosed or misdiagnosed. Previous studies have demonstrated that features of Electroencephalogram (EEG) data, such as Power Spectral Density (PSD), are useful biomarkers for indicating the onset of various mental disorders. Existing models using EEG data are typically trained to distinguish between healthy and afflicted individuals, and they are unable to distinguish between individuals with different disorders. We propose MENTAL (Multi-modal EEG NEO-FFI with Trained Attention Layer) to predict an individual's mental state using both EEG and Neo-Five Factor Inventory (NEO-FFI) personality data. We include an attention layer that captures the interactions between personality traits and PSD features, and emphasizes the important PSD features. MENTAL features a Recurrent Neural Network (RNN) to model the temporal nature of EEG data. We train our model with the Two Decades Brainclinics Research Archive for Insights in Neuroscience (TDBRAIN) dataset, which consists of 1274 healthy and psychiatric individuals including over 30 different diagnoses. MENTAL is able to achieve 93.3% accuracy when trained to classify between healthy individuals and those with ADHD. When trained to identify individuals with ADHD from among 33 disorder classes, MENTAL improves accuracy from 70.5 to 81.7%. MENTAL also demonstrates over 20% improvement in predictive accuracy when trained for MDD prediction. For the multi-class classification task of three classes, MENTAL improves accuracy by 4%, and for five classes, by nearly 8%. MENTAL is the first multi-modal model that utilizes both EEG and NEO-FFI data for the task of mental disorder prediction. We are one of the first groups to utilize TDBRAIN for automated disorder classification. MENTAL is accessible and cost-effective, as EEG is more affordable than other neuroimaging methods such as MRI, and the NEO-FFI is a self- reported survey. Our model can lead to acceptance and support for individuals living with mental health challenges and improve quality of life in our society.

Is metadata of articles about COVID-19 enough for multilabel topic classification task?

The ever-increasing volume of COVID-19-related articles presents a significant challenge for the manual curation and multilabel topic classification of LitCovid. For this purpose, a novel multilabel topic classification framework is developed in this study, which considers both the correlation and imbalance of topic labels, while empowering the pretrained model. With the help of this framework, this study devotes to answering the following question: Do full texts, MeSH (Medical Subject Heading), and biological entities of articles about COVID-19 encode more discriminative information than metadata (title, abstract, keyword, and journal name)? From extensive experiments on our enriched version of the BC7-LitCovid corpus and Hallmarks of Cancer corpus, the following conclusions can be drawn. Our framework demonstrates superior performance and robustness. The metadata of scientific publications about COVID-19 carries valuable information for multilabel topic classification. Compared to biological entities, full texts and MeSH can further enhance the performance of our framework for multilabel topic classification, but the improved performance is very limited. Database URL: https://github.com/pzczxs/Enriched-BC7-LitCovid.

Towards superior android ransomware detection: An ensemble machine learning perspective

Ransomware remains a pervasive threat to Android devices, with its ability to encrypt critical data and demand ransoms causing significant disruptions to users and organizations alike. This research proposes a novel ensemble-based machine learning approach for the detection of Android ransomware, leveraging the strengths of multiple classifiers to enhance detection accuracy and robustness. Utilizing a comprehensive dataset comprising 203,556 network traffic records across 10 distinct ransomware types and benign traffic, we meticulously preprocess and feature-engineer the data to ensure optimal model performance. The methodology integrates various ensemble classifiers, evaluating each through rigorous cross-validation. Feature importance analysis using Random Forest identifies key indicators of ransomware activity, enabling us to refine our models and focus on the most predictive features. The results demonstrate that the ensemble models, particularly Bagging, achieve near-perfect detection rates, with precision, recall, and F1 scores consistently exceeding 99% for different binary attacks and multi-class classification. Finally, in-depth statistical analysis further validates the superiority of our approach, showcasing significant improvements over traditional machine learning methods. This research sets a new benchmark for Android ransomware detection, offering a robust, scalable, and highly accurate solution that enhances the security and resilience of mobile networks against evolving cyber threats.

MobVGG: Ensemble technique for birds and drones prediction

Detection of aerial activities, including drones and birds, has practical implications for automating bird surveys and developing radar systems for aerial object collision detection. Convolutional neural networks (CNNs) have been extensively utilized for image recognition and classification tasks, albeit prior research predominantly focuses on single-class 'drone' classification. However, a gap persists in achieving high accuracy for multi-class classification. To address the limitations of traditional CNNs, such as vanishing gradients and the necessity for numerous layers, this study introduces a novel model termed "MobVGG.” This model combines the architectures of MobileNetV2 and VGG16 to accurately classify images as either 'bird' or 'drone'. The dataset comprises 4212 images for each category of 'bird' and 'drone'. The stringent methodology was applied for dataset preparation and model training to ensure the reliability of the findings. Comparative analysis with previous research demonstrates that the proposed MobVGG model, trained on both 'bird' and 'drone' images, achieves superior accuracy (96 %) compared to benchmark studies. Our paper targets researchers and graduate students as its primary audience.

Ability of clinical data to predict readmission in Child and Adolescent Mental Health Services

This study addresses the challenge of predicting readmissions in Child and Adolescent Mental Health Services (CAMHS) by analyzing the predictability of readmissions over short, medium, and long term periods. Using health records spanning 35 years, which included 22,643 patients and 30,938 episodes of care, we focused on the episode of care as a central unit, defined as a referral-discharge cycle that incorporates assessments and interventions. Data pre-processing involved handling missing values, normalizing, and transforming data, while resolving issues related to overlapping episodes and correcting registration errors where possible. Readmission prediction was inferred from electronic health records (EHR), as this variable was not directly recorded. A binary classifier distinguished between readmitted and non-readmitted patients, followed by a multi-class classifier to categorize readmissions based on timeframes: short (within 6 months), medium (6 months - 2 years), and long (more than 2 years). Several predictive models were evaluated based on metrics like AUC, F1-score, precision, and recall, and the K-prototype algorithm was employed to explore similarities between episodes through clustering. The optimal binary classifier (Oversampled Gradient Boosting) achieved an AUC of 0.7005, while the multi-class classifier (Oversampled Random Forest) reached an AUC of 0.6368. The K-prototype resulted in three clusters as optimal (SI: 0.256, CI: 4473.64). Despite identifying relationships between care intensity, case complexity, and readmission risk, generalizing these findings proved difficult, partly because clinicians often avoid discharging patients likely to be readmitted. Overall, while this dataset offers insights into patient care and service patterns, predicting readmissions remains challenging, suggesting a need for improved analytical models that consider patient development, disease progression, and intervention effects.

Towards imagined speech: Identification of brain states from EEG signals for BCI-based communication systems

BackgroundThe electroencephalogram (EEG) based brain-computer interface (BCI) system employing imagined speech serves as a mechanism for decoding EEG signals to facilitate control over external devices or communication with the external world at the moment the user desires. To effectively deploy such BCIs, it is imperative to accurately discern various brain states from continuous EEG signals when users initiate word imagination. New methodThis study involved the acquisition of EEG signals from 15 subjects engaged in four states: resting, listening, imagined speech, and actual speech, each involving a predefined set of 10 words. The EEG signals underwent preprocessing, segmentation, spatio-temporal and spectral analysis of each state, and functional connectivity analysis using the phase locking value (PLV) method. Subsequently, five features were extracted from the frequency and time-frequency domains. Classification tasks were performed using four machine learning algorithms in both pair-wise and multiclass scenarios, considering subject-dependent and subject-independent data. ResultsIn the subject-dependent scenario, the random forest (RF) classifier achieved a maximum accuracy of 94.60 % for pairwise classification, while the artificial neural network (ANN) classifier achieved a maximum accuracy of 66.92 % for multiclass classification. In the subject-independent scenario, the random forest (RF) classifier achieved maximum accuracies of 81.02 % for pairwise classification and 55.58 % for multiclass classification. Moreover, EEG signals were classified based on frequency bands and brain lobes, revealing that the theta (θ) and delta (δ) bands, as well as the frontal and temporal lobes, are sufficient for distinguishing between brain states. ConclusionThe findings promise to develop a system capable of automatically segmenting imagined speech segments from continuous EEG signals.

Multiclass classification of Autism Spectrum Disorder, attention deficit hyperactivity disorder, and typically developed individuals using fMRI functional connectivity analysis.

Neurodevelopmental conditions, such as Autism Spectrum Disorder (ASD) and Attention Deficit Hyperactivity Disorder (ADHD), present unique challenges due to overlapping symptoms, making an accurate diagnosis and targeted intervention difficult. Our study employs advanced machine learning techniques to analyze functional magnetic resonance imaging (fMRI) data from individuals with ASD, ADHD, and typically developed (TD) controls, totaling 120 subjects in the study. Leveraging multiclass classification (ML) algorithms, we achieve superior accuracy in distinguishing between ASD, ADHD, and TD groups, surpassing existing benchmarks with an area under the ROC curve near 98%. Our analysis reveals distinct neural signatures associated with ASD and ADHD: individuals with ADHD exhibit altered connectivity patterns of regions involved in attention and impulse control, whereas those with ASD show disruptions in brain regions critical for social and cognitive functions. The observed connectivity patterns, on which the ML classification rests, agree with established diagnostic approaches based on clinical symptoms. Furthermore, complex network analyses highlight differences in brain network integration and segregation among the three groups. Our findings pave the way for refined, ML-enhanced diagnostics in accordance with established practices, offering a promising avenue for developing trustworthy clinical decision-support systems.

Learning Robust Sequential Recommenders through Confident Soft Labels

Sequential recommenders that are trained on implicit feedback are usually learned as a multi-class classification task through softmax-based loss functions on one-hot class labels. However, one-hot training labels are sparse and may lead to biased training and sub-optimal performance. Dense, soft labels have been shown to help improve recommendation performance. However, how to generate high-quality and confident soft labels from noisy sequential interactions between users and items is still an open question. We propose a new learning framework for sequential recommenders, CSRec, which introduces c onfident s oft labels to provide robust guidance when learning from user-item interactions. CSRec contains a teacher module that generates high-quality and confident soft labels and a student module that acts as the target recommender and is trained on the combination of dense, soft labels and sparse, one-hot labels. We propose and compare three approaches to constructing the teacher module: (i) model-level, (ii) data-level, and (iii) training-level. To evaluate the effectiveness and generalization ability of CSRec, we conduct experiments using various state-of-the-art sequential recommendation models as the target student module on four benchmark datasets. Our experimental results demonstrate that CSRec is effective in training better-performing sequential recommenders.

Machine Learning based Suggestion Method for Land Suitability Assessment and Production Sustainability

The global population is projected to increase by an additional two billion by 2050, as per the assessment conducted by Food and Agriculture Management. However, the arable land is anticipated to expand by just 5%. Consequently, intelligent and effective agricultural practices are essential to enhancing farming production. Evaluating rural Land Suitability (LS) is a crucial instrument for agricultural growth. Numerous novel methods and concepts are being adopted in agriculture as alternatives for gathering and processing farm data. The swift advancement of wireless Sensor Networks (WSN) has prompted the creation of economical and compact sensor gadgets, with the Internet of Things (IoT) serving as a viable instrument for automation and decision-making in farmers. To evaluate agricultural LS, this study offers an expert system integrating networked sensors with Machine Learning (ML) technologies, including neural networks. The suggested approach would assist farmers in evaluating agricultural land for cultivating across four decision categories: very appropriate, suitable, somewhat suitable, and inappropriate. This evaluation is based on the data gathered from various sensor devices for system training. The findings achieved with the MLP with four concealed layers demonstrate efficacy for the multiclass categorization method compared to other current models. This trained system will assess future evaluations and categorize the land post-cultivation.

DDoS Attack Detection with Machine Learning

Nowadays, Distributed Denial of Service (DDoS) attacks are a major issue in internet security. These attacks target servers or network infrastructure. Similar to an unanticipated traffic jam on highway (lagging/crash) that prevent normal traffic reach to destination. DDoS may prevent users to access any system services. Researchers and scientists have developed numerous methods and algorithms to improve the performance of DDoS detection. In this paper, a DDoS detection method utilizing machine learning is proposed. There are three type of supervised machine learning classification methods which are K-Nearest Neighbor, Multilayer Perceptron and Random Forest, are applied in the proposed work to assess the accuracy of the model in training and testing processes. RF classification provides robustness and interpretability, MLP offers deep learning capabilities for complex patterns, and K-NN delivers simplicity and adaptability for instance-based learning. Together, these methods can contribute to a comprehensive DDoS attack detection system using machine learning. There are two types of classification setups: binary and multi-class classification. Binary classification involves identifying traffic as either a DDoS attack or normal using the NSL-KDD dataset. Multi-class classification, on the other hand, distinguishes between various types of DDoS attacks (such as DoS, Probe, U2R, and Sybil) and normal traffic using the NSL-KDD dataset. Feature engineering is also involved in this experiment to convert the categorical features into numerical values for detecting DDoS attack. Our model's performance was effective compared to other machine learning methods. RF achieved the highest accuracy rates: 99.35% in binary classification and 97.71% in multi-class classification. K-NN followed with 99.15% in binary and 97.35% in multi-class classification, while MLP achieved 90.63% in binary and 84.33% in multi-class classification.

Convolution neural network based multi-class classification of rehabilitation exercises for diastasis recti abdominis using wearable EMG-IMU sensors

PurposeGlobally, postnatal women endure a prominent issue caused by midline separation of abdominal recti muscles, characterized by a sagging and pouch-like appearance of the belly termed as Diastasis Recti Abdominis (DRA). The necessity of ensuring the efficacy of rehabilitative workouts for individuals with DRA cannot be overstated, as inaccurate exercises can exacerbate the condition and deteriorate the health of affected women. The purpose of these exercises is to specifically focus on the rectus abdominis muscles to facilitate the reapproximation of the linea alba. The primary aim of this research work is to assess the effectiveness of rehabilitation exercises for DRA women obtained from Inertial Measurement Unit (IMU) and Electromyography (EMG) sensors.Design/methodology/approachConvolutional neural networks (CNN) employs convolutional activation functions and pooling layers. Recently, 1D CNNs have emerged as a promising approach used in various applications, including personalized biomedical data classification and early diagnosis, structural health monitoring and anomaly detection. Yet another significant benefit is the feasibility of a real-time and cost-effective implementation of 1D CNN. The EMG and IMU signals serve as inputs for the 1D CNN. Features are then extracted from the fully connected layer of the CNN and fed into a boosting machine learning algorithm for classification.FindingsThe findings demonstrate that a combination of sensors provides more details about the exercises, thereby contributing to the classification accuracy.Practical implicationsIn real time, collecting data from postnatal women was incredibly challenging. The process of examining these women was time-consuming, and they were often preoccupied with their newborns, leading to a reluctance to focus on their own health. Additionally, postnatal women might not be fully aware of the implications of DRA and the importance of rehabilitation exercises. Many might not realize that neglecting DRA can lead to long-term issues such as back pain, pelvic floor dysfunction, and compromised core strength.Social implicationsDuring our data collection camps, there were educational sessions to raise awareness about the DRA problem and the benefits of rehabilitation exercises. This dual approach helped in building trust and encouraging participation. Moreover, the use of wearable sensors in this study provided a non-invasive and convenient way for new mothers to engage in rehabilitation exercises without needing frequent visits to a clinic, which is often impractical for them.Originality/valueThe utilization of discriminating features retrieved from the output layer of 1D CNN is a significant contribution to this work. The responses of this study indicate that 1D convolutional neural network (1D CNN) and Boosting algorithms used in a transfer learning strategy produce successful discrimination between accurate and inaccurate performance of exercises by achieving an accuracy of 96%.

All is attention for multi-label text classification

All is attention for multi-label text classification

DeepPhylo: Phylogeny-Aware Microbial Embeddings Enhanced Predictive Accuracy in Human Microbiome Data Analysis.

Microbial data analysis poses significant challenges due to its high dimensionality, sparsity, and compositionality. Recent advances have shown that integrating abundance and phylogenetic information is an effective strategy for uncovering robust patterns and enhancing the predictive performance in microbiome studies. However, existing methods primarily focus on the hierarchical structure of phylogenetic trees, overlooking the evolutionary distances embedded within them. This study introduces DeepPhylo, a novel method that employs phylogeny-aware amplicon embeddings to effectively integrate abundance and phylogenetic information. DeepPhylo improves both the unsupervised discriminatory power and supervised predictive accuracy of microbiome data analysis. Compared to the existing methods, DeepPhylo demonstrates superiority in informing biologically relevant insights across five real-world microbiome use cases, including clustering of skin microbiomes, prediction of host chronological age and gender, diagnosis of inflammatory bowel disease (IBD) across 15 studies, and multilabel disease classification.

Exploring online public survey lifestyle datasets with statistical analysis, machine learning and semantic ontology

Lifestyle diseases significantly contribute to the global health burden, with lifestyle factors playing a crucial role in the development of depression. The COVID-19 pandemic has intensified many determinants of depression. This study aimed to identify lifestyle and demographic factors associated with depression symptoms among Indians during the pandemic, focusing on a sample from Kolkata, India. An online public survey was conducted, gathering data from 1,834 participants (with 1,767 retained post-cleaning) over three months via social media and email. The survey consisted of 44 questions and was distributed anonymously to ensure privacy. Data were analyzed using statistical methods and machine learning, with principal component analysis (PCA) and analysis of variance (ANOVA) employed for feature selection. K-means clustering divided the pre-processed dataset into five clusters, and a support vector machine (SVM) with a linear kernel achieved 96% accuracy in a multi-class classification problem. The Local Interpretable Model-agnostic Explanations (LIME) algorithm provided local explanations for the SVM model predictions. Additionally, an OWL (web ontology language) ontology facilitated the semantic representation and reasoning of the survey data. The study highlighted a pipeline for collecting, analyzing, and representing data from online public surveys during the pandemic. The identified factors were correlated with depressive symptoms, illustrating the significant influence of lifestyle and demographic variables on mental health. The online survey method proved advantageous for data collection, visualization, and cost-effectiveness while maintaining anonymity and reducing bias. Challenges included reaching the target population, addressing language barriers, ensuring digital literacy, and mitigating dishonest responses and sampling errors. In conclusion, lifestyle and demographic factors significantly impact depression during the COVID-19 pandemic. The study’s methodology offers valuable insights into addressing mental health challenges through scalable online surveys, aiding in the understanding and mitigation of depression risk factors.