Label Vector Research Articles

A cosine similarity-based labeling technique for vulnerability type detection using source codes

Vulnerability detection is of great importance in providing reliability to software systems. Although existing methods achieve remarkable success in vulnerability detection, they have several disadvantages as follows: (1) The irrelevant information is removed from source codes, which have a high noise ratio, thereby utilizing deep learning methods and devising experiments featuring high accuracy. However, deep learning-based detection methods necessitate large-scale datasets. This results in computational hardship with respect to vulnerability detection in small-scale software systems. (2) The majority of the studies perform feature selection by processing vulnerability commits. Despite tremendous endeavors, there are few works detecting vulnerability with source codes. To solve these two problems, in this study, a novel labeling and vulnerability detection algorithm is proposed. The algorithm first exploits source codes with the help of a keyword vulnerability matrix. After that, an ultimate encoded matrix is generated by word2vec, thereby combining the labeling vector with the source code matrix to reveal a trainable dataset for a generalized linear model (GLM). Different from preceding studies, our method performs vulnerability detection without requiring vulnerability commits but using source codes. In addition to this, similar studies generally aim to bring sophisticated solutions for just one type of programming language. Conversely, our study develops vulnerability keywords for three programming languages including C#, Java, and C++, and creates the related labeling vectors by regarding the keyword matrix. The proposed method outperformed the baseline approaches for most of the experimental datasets with over 90% of the area under the curve (AUC). Further, there is a 7.7% margin between our method and the alternatives on average for Recall, Precision, and F1-score with respect to five types of vulnerabilities.

Feature selection for multi-labeled data based on label enhancement technique and mutual information

In multi-label data, the importance of each label within the logical label vector varies for each sample, and there exist inherent correlations among the labels. However, the logical label vector fails to capture these nuances. Consequently, relying solely on this vector for feature selection in multi-label data results in underutilization of supervisory information. To address this issue, this paper introduces a novel label enhancement algorithm. This algorithm leverages neighborhood information derived from features to transform the logical label vector into a label distribution that effectively reflects label differences and correlations. Subsequently, we propose a feature selection algorithm tailored for multi-label data, which incorporates both the transformed label distribution and mutual information. This algorithm not only accounts for the mutual information between features and label distributions but also captures the mutual information among features themselves. Finally, we evaluate our proposed feature selection algorithm against five state-of-the-art multi-label feature selection algorithms on ten publicly available datasets. The experimental results reveal that our algorithm outperforms its competitors in six distinct evaluation metrics, achieving an average performance improvement of approximately 9%. This substantial enhancement underscores the efficacy of our algorithm in handling complex multi-label data.

Distance mapping overlap complexity metric for class-imbalance problems

Data complexity for class-imbalance problems is a hot topic in the field of data mining. Classical data complexity measures use techniques such as kNN, which computes the nearest neighbors of each instance. However, the kNN-based method to obtain the nearest neighbors of all instances is an NP-hard problem, which is not conducive to large-scale data complexity calculations. In order to solve this problem, from the global and local perspectives, four distance map complexity metric methods are proposed, and the complexity values is called the distance map overlap index (DMOI). Firstly, we use Mahalanobis distance or standardized Euclidean distance to calculate the distance of the instance to the center point of the data set, and arrange by the distance value of each instance. Then, according to the sequential label vector, iteratively searches the number of map cut-points for each class. Finally, according to the number of map cut-points of different classes, the DMOI of the data set is calculated. Experiments on 50 class-imbalanced datasets show that the DMOI method outperforms state-of-the-art complexity measures for class-imbalanced problems. Although there is still a gap between DMOI and ONB for rule-based or tree-based classifiers, the use of Pearson correlation coefficient shows that DMOI can effectively approximate ONB, and there is a strong positive correlation between them.

A new ECT image reconstruction algorithm based on Vision transformer (ViT)

Aiming at the problem of low accuracy of ECT image reconstruction, this paper proposes an ECT image reconstruction algorithm based on Vision Transformer (ViT). ViT is a method that applies the Transformer to the field of image classification, which is characterized by strong long-distance dependency learning ability and strong multi-modal fusion ability compared to CNN. This paper fully utilizes the characteristics of ViT to transform the image reconstruction process of ECT into the classification process of ViT. Here, a method is proposed to use the object field distribution as the image classification label. This method converts a two-dimensional image of the object field distribution into a one-dimensional vector, which is the label vector of the image classification. These vectors and their corresponding reconstructed images obtained by the Landweber algorithm form the sample set of ViT. Extracting a large number of flow pattern samples with various shapes and distributions through COMSOL is used as a training set. After training ViT, this network model is used to infer the labels of the predicted flow patterns. After post-processing this label, the corresponding reconstructed image can be obtained. Finally, simulation experiments are conducted, and the experiments results show that the image errors and correlation coefficients of the reconstructed images obtained through the algorithm in this paper are better than those of Tikhonov algorithm, Landweber algorithm and Long Short-Term Memory Network(LSTM). And this algorithm has better resistance to noise interference than Tikhonov algorithm, Landweber algorithm, and LSTM. This also provides a new approach and means for ECT image reconstruction.

A generalized stochastic block model for overlapping community detection

Over the past two decades, community detection has been extensively explored. Yet, the problem of identifying overlapping communities has not been fully solved. In this paper, we introduce a novel approach, called the generalized stochastic block model, to address this issue by allowing nodes to belong to multiple communities. This approach extends the traditional representation of nodal community assignment from a single community label to a label vector, with each element indicating the membership of a node in a specific community. We develop a Markov chain Monte Carlo algorithm to tackle the model. Through numerical experiments conducted on synthetic and empirical networks, we demonstrate the efficacy of the proposed framework in accurately detecting overlapping communities.

Rethinking Regularization with Random Label Smoothing

Regularization helps to improve machine learning techniques by penalizing the models during training. Such approaches act in either the input, internal, or output layers. Regarding the latter, label smoothing is widely used to introduce noise in the label vector, making learning more challenging. This work proposes a new label regularization method, Random Label Smoothing, that attributes random values to the labels while preserving their semantics during training. The idea is to change the entire label into fixed arbitrary values. Results show improvements in image classification and super-resolution tasks, outperforming state-of-the-art techniques for such purposes.

Weighted cross-modal hashing with label enhancement

Hashing has been extensively studied for large-scale cross-modal retrieval. However, there are still some unexplored issues with existing methods. First, both the initially-provided similarity graph and one-hot label vector only logically describe the yes or no information, which cannot exactly reflect the correlation between instances. Second, all bits in a binary code contribute equally to Hamming distance, which limits its discriminability to preserve refined semantics. To address these limitations, this paper presents a new method, dubbed Weighted Cross-Modal Hashing (WCMH). It introduces the topological structure of categories to enhance the logical labels, i.e., to recover the real-valued distribution of categories. Moreover, we design a weighted Hamming distance that can enlarge the range of distance values by adaptively learning the weights of different hash bits. In light of this, more discriminative information is embedded into hash codes, and the enhanced semantics can be preserved as much as possible. Extensive experiments validate the superior performance of our WCMH, in comparison with some state-of-the-art cross-modal hashing approaches.

Absent Multiview Semisupervised Classification.

With the advent of vast data collection ways, data are often with multiple modalities or coming from multiple sources. Traditional multiview learning often assumes that each example of data appears in all views. However, this assumption is too strict in some real applications such as multisensor surveillance system, where every view suffers from some data absent. In this article, we focus on how to classify such incomplete multiview data in semisupervised scenario and a method called absent multiview semisupervised classification (AMSC) has been proposed. Specifically, partial graph matrices are constructed independently by anchor strategy to measure the relationships among between each pair of present samples on each view. And to obtain unambiguous classification results for all unlabeled data points, AMSC learns view-specific label matrices and a common label matrix simultaneously. AMSC measures the similarity between pair of view-specific label vectors on each view by partial graph matrices, and consider the similarity between view-specific label vectors and class indicator vectors based on the common label matrix. To characterize the contributions of different views, the p th root integration strategy is adopted to incorporate the losses of different views. By further analyzing the relation between the p th root integration strategy and exponential decay integration strategy, we develop an efficient algorithm with proved convergence to solve the proposed nonconvex problem. To validate the effectiveness of AMSC, comparisons are made with some benchmark methods on real-world datasets and in the document classification scenario as well. The experimental results demonstrate the advantages of our proposed approach.

Named entity recognition using transfer learning and small human‐ and meta‐pseudo‐labeled datasets

AbstractWe introduce a high‐performance named entity recognition (NER) model for written and spoken language. To overcome challenges related to labeled data scarcity and domain shifts, we use transfer learning to leverage our previously developed KorBERT as the base model. We also adopt a meta‐pseudo‐label method using a teacher/student framework with labeled and unlabeled data. Our model presents two modifications. First, the student model is updated with an average loss from both human‐ and pseudo‐labeled data. Second, the influence of noisy pseudo‐labeled data is mitigated by considering feedback scores and updating the teacher model only when below a threshold (0.0005). We achieve the target NER performance in the spoken language domain and improve that in the written language domain by proposing a straightforward rollback method that reverts to the best model based on scarce human‐labeled data. Further improvement is achieved by adjusting the label vector weights in the named entity dictionary.

Dynamic Correlation Learning and Regularization for Multi-Label Confidence Calibration.

Modern visual recognition models often display overconfidence due to their reliance on complex deep neural networks and one-hot target supervision, resulting in unreliable confidence scores that necessitate calibration. While current confidence calibration techniques primarily address single-label scenarios, there is a lack of focus on more practical and generalizable multi-label contexts. This paper introduces the Multi-Label Confidence Calibration (MLCC) task, aiming to provide well-calibrated confidence scores in multi-label scenarios. Unlike single-label images, multi-label images contain multiple objects, leading to semantic confusion and further unreliability in confidence scores. Existing single-label calibration methods, based on label smoothing, fail to account for category correlations, which are crucial for addressing semantic confusion, thereby yielding sub-optimal performance. To overcome these limitations, we propose the Dynamic Correlation Learning and Regularization (DCLR) algorithm, which leverages multi-grained semantic correlations to better model semantic confusion for adaptive regularization. DCLR learns dynamic instance-level and prototype-level similarities specific to each category, using these to measure semantic correlations across different categories. With this understanding, we construct adaptive label vectors that assign higher values to categories with strong correlations, thereby facilitating more effective regularization. We establish an evaluation benchmark, re-implementing several advanced confidence calibration algorithms and applying them to leading multi-label recognition (MLR) models for fair comparison. Through extensive experiments, we demonstrate the superior performance of DCLR over existing methods in providing reliable confidence scores in multi-label scenarios.

Deep Learning From Multiple Noisy Annotators as A Union.

Crowdsourcing is a popular solution for large-scale data annotations. So far, various end-to-end deep learning methods have been proposed to improve the practical performance of learning from crowds. Despite their practical effectiveness, most of them have two major limitations-they do not hold learning consistency and suffer from computational inefficiency. In this article, we propose a novel method named UnionNet, which is not only theoretically consistent but also experimentally effective and efficient. Specifically, unlike existing methods that either fit a given label from each annotator independently or fuse all the labels into a reliable one, we concatenate the one-hot encoded vectors of crowdsourced labels provided by all the annotators, which takes all the labeling information as a union and coordinates multiple annotators. In this way, we can directly train an end-to-end deep neural network by maximizing the likelihood of this union with only a parametric transition matrix. We theoretically prove the learning consistency and experimentally show the effectiveness and efficiency of our proposed method.

A partition-based problem transformation algorithm for classifying imbalanced multi-label data

Multi-label learning has garnered much research interest due to its wide range of real-world applications. Many multi-label learning methods have been proposed; however, few have addressed the class imbalance problem existing in multi-label data. Even though some studies have taken this issue into account, most of them have ignored the label correlations or only considered random correlations between them. In this study, we propose a novel partition-based imbalanced multi-label learning algorithm, named Multi-label Learning based on Hierarchical Clustering (MLHC), to tackle this problem. MLHC first carries out hierarchical clustering on the original label space to divide it into several disconnected subspaces, each of which contains several labels that are strongly correlated with each other. Then, for each label subspace, we use the problem transformation strategy to convert it into a multi-class problem by binary coding. Any multi-class imbalance learning algorithm can be applied to the transformed multi-class data. Finally, the classification results will be decoded to retrieve the corresponding label subspace, and all label subspace results are combined to show the predicted label vector in the original label space. We conducted experiments not only on thirteen benchmark multi-label datasets but also carried out them on XJTU-SY which is a multi-label engineering application dataset, and the results indicated that our proposed MLHC learning algorithm outperforms several state-of-the-art class imbalance multi-label learning algorithms, demonstrating the effectiveness and necessity of discovering label correlations and transforming the original imbalanced multi-label learning problem into multiple strongly correlated multi-class imbalanced learning problems.

FPLV: Enhancing recommender systems with fuzzy preference, vector similarity, and user community for rating prediction.

Rating prediction is crucial in recommender systems as it enables personalized recommendations based on different models and techniques, making it of significant theoretical importance and practical value. However, presenting these recommendations in the form of lists raises the challenge of improving the list's quality, making it a prominent research topic. This study focuses on enhancing the ranking quality of recommended items in user lists while ensuring interpretability. It introduces fuzzy membership functions to measure user attributes on a multi-dimensional item label vector and calculates user similarity based on these features for prediction and recommendation. Additionally, the user similarity network is modeled to extract community information, leading to the design of a set of corresponding recommendation algorithms. Experimental results on two commonly used datasets demonstrate the effectiveness of the proposed algorithm in enhancing list ranking quality, reducing prediction errors, and maintaining recommendation diversity and accurate user preference classification. This research highlights the potential of integrating heuristic methods with complex network theory and fuzzy techniques to enhance recommendation system performance with interpretability in mind.

2022 White Paper on Recent Issues in Bioanalysis: FDA Draft Guidance on Immunogenicity Information in Prescription Drug Labeling, LNP & Viral Vectors Therapeutics/Vaccines Immunogenicity, Prolongation Effect, ADA Affinity, Risk-based Approaches, NGS, qPCR, ddPCR Assays (Part 3 - Recommendations on Gene Therapy, Cell Therapy, Vaccines Immunogenicity & Technologies; Immunogenicity & Risk Assessment of Biotherapeutics and Novel Modalities; NAb Assays Integrated Approach).

The 2022 16th Workshop on Recent Issues in Bioanalysis (WRIB) took place in Atlanta, GA, USA on September 26-30, 2022. Over 1000 professionals representing pharma/biotech companies, CROs, and multiple regulatory agencies convened to actively discuss the most current topics of interest in bioanalysis. The 16th WRIB included 3 Main Workshops and 7 Specialized Workshops that together spanned 1week in order to allow exhaustive and thorough coverage of all major issues in bioanalysis, biomarkers, immunogenicity, gene therapy, cell therapy and vaccines. Moreover, in-depth workshops on ICH M10 BMV final guideline (focused on this guideline training, interpretation, adoption and transition); mass spectrometry innovation (focused on novel technologies, novel modalities, and novel challenges); and flow cytometry bioanalysis (rising of the 3rd most common/important technology in bioanalytical labs) were the special features of the 16th edition. As in previous years, WRIB continued to gather a wide diversity of international, industry opinion leaders and regulatory authority experts working on both small and large molecules as well as gene, cell therapies and vaccines to facilitate sharing and discussions focused on improving quality, increasing regulatory compliance, and achieving scientific excellence on bioanalytical issues. This 2022 White Paper encompasses recommendations emerging from the extensive discussions held during the workshop and is aimed to provide the bioanalytical community with key information and practical solutions on topics and issues addressed, in an effort to enable advances in scientific excellence, improved quality and better regulatory compliance. Due to its length, the 2022 edition of this comprehensive White Paper has been divided into three parts for editorial reasons. This publication (Part 3) covers the recommendations on Gene Therapy, Cell therapy, Vaccines and Biotherapeutics Immunogenicity. Part 1 (Mass Spectrometry and ICH M10) and Part 2 (LBA, Biomarkers/CDx and Cytometry) are published in volume 15 of Bioanalysis, issues 16 and 15 (2023), respectively.

Generative Label Enhancement with Gaussian Mixture and Partial Ranking

Label distribution learning (LDL) is an effective learning paradigm for dealing with label ambiguity. When applying LDL, the datasets annotated with label distributions (i.e., the real-valued vectors like the probability distribution) are typically required. Unfortunately, most existing datasets only contain the logical labels, and manual annotating with label distributions is costly. To address this problem, we treat the label distribution as a latent vector and infer its posterior by variational Bayes. Specifically, we propose a generative label enhancement model to encode the process of generating feature vectors and logical label vectors from label distributions in a principled way. In terms of features, we assume that the feature vector is generated by a Gaussian mixture dominated by the label distribution, which captures the one-to-many relationship from the label distribution to the feature vector and thus reduces the feature generation error. In terms of logical labels, we design a probability distribution to generate the logical label vector from a label distribution, which captures partial label ranking in the logical label vector and thus provides a more accurate guidance for inferring the label distribution. Besides, to approximate the posterior of the label distribution, we design a inference model, and derive the variational learning objective. Finally, extensive experiments on real-world datasets validate our proposal.

Cost-constrained feature selection in multilabel classification using an information-theoretic approach

Feature selection is one of the key steps in building a predictive model in multi-label classification. However, most of the existing methods do not take into account information about the costs associated with considered features, such as the costs of performing diagnostic medical tests. We consider a problem of cost-constrained multilabel feature selection, which aims to select a feature subset relevant to multiple labels while satisfying a user-specific maximal admissible budget. This approach allows for building a model with high predictive power, for which the cost of making a prediction for a single instance does not exceed the user-specified budget. In this problem, the balance between the feature subset relevance and its cost should be considered concurrently, which is nontrivial in practice because their optimal balance is unknown. In this paper, we propose a novel criterion for cost-constrained multilabel feature selection that combines the relevance and cost of the candidate feature. The relevance measure is derived using the lower bound of the mutual information between the feature subset and label vector. Moreover, we propose an effective method for determining the cost-factor value that controls the trade-off between relevancy and cost. The experimental results on multilabel datasets with various characteristics demonstrate the superiority of the proposed method over conventional methods.

Orthogonal semi-supervised regression with adaptive label dragging for cross-session EEG emotion recognition

Owning to its merits of great temporal resolution, portability and low cost, electroencephalogram (EEG) signals have received increasing attention in emotion recognition. All the time, least square regression (LSR) has been widely employed in classification tasks. However, there exist two issues based on the LSR method, which limits its performance. The first problem is that the LSR method usually cannot make EEG data retain more discriminative information. The second is that it is improper for multiple emotion classification to use hard discrete labels as regression objectives. To address these issues, we develop orthogonal semi-supervised regression with adaptive label dragging model (OSRLD) to recognize emotions. OSRLD can get a closed-form solution with less computational costs. Furthermore, experimental findings using the open SEED-IV dataset show that 1) Compared to seven methods in terms of classification accuracy, OSRLD achieves the best average accuracy of 77.96%, 80.20% and 81.45%, 2) A technology of label dragging which modifies the label vector of each sample can effectively enlarge the distance between classes, and 3) Based on learned transformation matrix, the primary EEG frequency bands and brain areas are automatically identified, which lays theoretical basis for simplifying the hardware design of EEG acquisition devices.

Deep learning based classification of multi-label chest X-ray images via dual-weighted metric loss

—Thoracic disease, like many other diseases, can lead to complications. Existing multi-label medical image learning problems typically include rich pathological information, such as images, attributes, and labels, which are crucial for supplementary clinical diagnosis. However, the majority of contemporary efforts exclusively focus on regression from input to binary labels, ignoring the relationship between visual features and semantic vectors of labels. In addition, there is an imbalance in data amount between diseases, which frequently causes intelligent diagnostic systems to make erroneous disease predictions. Therefore, we aim to improve the accuracy of the multi-label classification of chest X-ray images. Chest X-ray14 pictures were utilized as the multi-label dataset for the experiments in this study. By fine-tuning the ConvNeXt network, we got visual vectors, which we combined with semantic vectors encoded by BioBert to map the two different forms of features into a common metric space and made semantic vectors the prototype of each class in metric space. The metric relationship between images and labels is then considered from the image level and disease category level, respectively, and a new dual-weighted metric loss function is proposed. Finally, the average AUC score achieved in the experiment reached 0.826, and our model outperformed the comparison models.

Multiple color representation and fusion for diabetes mellitus diagnosis based on back tongue images

Tongue images have been proved to be effective in diabetes mellitus (DM) diagnosis. Without requirement of collecting blood sample, tongue image based diagnosis approach is non-invasive and convenient for the patients. Meanwhile, the colors of tongues play an important in aiding accurate diagnosis. However, the tongues’ colors fall on a small color gamut that makes it difficult for the existing color descripts to identify and distinguish the tiny difference of the tongues. To tackle this problem, we introduce a novel color descriptor by representing the colors with the clustering centers, namely color centroid points, of the color points sampled from tongue images. In order to boost the capacity of the descriptor, we extend it into three color spaces, i.e., RGB, HSV and LAB to mine a rich set of color information and exploit the complementary information among the three spaces. Since there exist correlation and complementarity among the features extracted from the three color spaces, we propose a novel multiple color features fusion method for DM diagnosis. Particularly, two projections are learned to project the multiple features to their corresponding shared and specific subspaces, in which their similarity and diversity are firstly measured by the Euclidean Distance and Hilbert Schmidt Independence Criterion (HSIC), respectively. To fully exploit the similar and complementary information, the two components are jointly transformed to their label vector, efficiently embedding the discriminant prior into the model, leading to significant improvement in the diagnosis outcomes. Experimental results on clinical tongue dataset substantiated the effectiveness of our proposed clustering-based color descriptor and the proposed multiple colors fusion approach. Overall, the proposed pipeline for the diagnosis of DM using back tongue images, achieved an average accuracy of up to 93.38%, indicating its potential toward realization of a clinical diagnostic tool for DM. Without loss generality, we also assessed the performance of the novel multiple features fusion method on two public datasets. The experiments prove the superiority of our multiple features learning model on general real-life application.

Few-shot Aspect Category Sentiment Analysis via Meta-learning

Existing aspect-based/category sentiment analysis methods have shown great success in detecting sentiment polarity toward a given aspect in a sentence with supervised learning, where the training and inference stages share the same pre-defined set of aspects. However, in practice, the aspect categories are changing rather than keeping fixed over time. Dealing with unseen aspect categories is under-explored in existing methods. In this article, we formulate a new few-shot aspect category sentiment analysis (FSACSA) task, which aims to effectively predict the sentiment polarity of previously unseen aspect categories. To this end, we propose a novel Aspect-Focused Meta-Learning (AFML) framework that constructs aspect-aware and aspect-contrastive representations from external knowledge to match the target aspect with aspects in the training set. Concretely, we first construct two auxiliary contrastive sentences for a given sentence with the incorporation of external knowledge, enabling the learning of sentence representations with a better generalization. Then, we devise an aspect-focused induction network to leverage the contextual sentiment toward a given aspect to refine the label vectors. Furthermore, we employ the episode-based meta-learning algorithm to train the whole network, so as to learn to generalize to novel aspects. Extensive experiments on multiple real-life datasets show that our proposed AFML framework achieves the state-of-the-art results for the FSACSA task.