Multi-label Linear Discriminant Analysis Research Articles

An Application of Multi-label Linear Discriminant Analysis and Binary Relevance K-Nearest Neighbor in Multi-label Classification of Annotated Images

Although Binary Relevance (BR) is an adaptive and conceptually simple multi-label learning technique, its inability to exploit label dependencies and other inherent problems in multi-label examples makes it difficult to generalize well in the classification of real-world multi-label examples like annotated images. Thus, to strengthen the generalization ability of Binary Relevance, this study used Multi-label Linear Discriminant Analysis (MLDA) as a preprocessing technique to take care of the label dependencies, the curse of dimensionality, and label over counting inherent in multi-labeled images. After that, Binary Relevance with K Nearest Neighbor as the base learner was fitted and its classification performance was evaluated on randomly selected 1000 images with a label cardinality of 2.149 of the five most frequent categories, namely; "person", "chair", "bottle", "dining table" and "cup" in the Microsoft Common Objects in Context 2017 (MS COCO 2017) dataset. Experimental results showed that micro averages of precision, recall, and f1-score of Multi-label Linear Discriminant Analysis followed by Binary Relevance K Nearest Neighbor (MLDA-BRKNN) achieved a more than 30% improvement in classification of the 1000 annotated images in the dataset when compared with the micro averages of precision, recall, and f1-score of Binary Relevance K Nearest Neighbor (BRKNN), which was used as the reference classifier method in this study.

MpsLDA-ProSVM: Predicting multi-label protein subcellular localization by wMLDAe dimensionality reduction and ProSVM classifier

Multi-label proteins play a significant role in life processes such as cell growth, development, and reproduction. Exploring protein subcellular localization (SCL) is a direct way to better understand the function of multi-label proteins in cells. This paper firstly presents a new prediction model named MpsLDA-ProSVM which predicts the SCL of multi-label proteins. Firstly, we utilize four coding algorithms including pseudo position-specific scoring matrix (PsePSSM), gene ontology (GO), conjoint triad (CT) and pseudo amino acid composition (PseAAC) to draw the feature information from protein sequences. Then, for the first time, we use a weighted multi-label linear discriminant analysis framework based on entropy weight form (wMLDAe) to refine and purify features. Finally, we input the optimal feature subset into the multi-label learning with label-specific features (LIFT) and multi-label k-nearest neighbor (ML-KNN) algorithms to obtain a synthetic ranking of relevant labels, and then use Prediction and Relevance Ordering based SVM (ProSVM) classifier to predict the SCLs. Tested by leave-one-out cross-validation (LOOCV), the overall actual accuracy on virus, plant, Gram-positive bacteria and Gram-negative bacteria datasets are 98.06%, 98.97%, 99.81% and 98.49%, which are 0.56%–9.16%, 1.07%–30.87%, 0.21%–6.91% and 3.99%–8.59% higher than other advanced methods respectively. By comparison, the model MpsLDA-ProSVM can effectively predict the specific location of multi-label proteins in cells.

DMLDA-LocLIFT: Identification of multi-label protein subcellular localization using DMLDA dimensionality reduction and LIFT classifier

Multi-label proteins occur in two or more subcellular locations, which play a vital role in cell development and metabolism. Prediction and analysis of multi-label subcellular localization (SCL) can present new perspective with drug target identification and new drug design. However, the prediction of multi-label protein SCL using biological experiments is expensive and labor-intensive. Therefore, predicting large-scale SCL with machine learning methods has turned into a popular study topic in bioinformatics. In this study, a novel multi-label learning methods for protein SCL prediction, called DMLDA-LocLIFT, is proposed. Firstly, the dipeptide composition (DC), encoding based on grouped weight (EBGW), pseudo amino acid composition (PseAAC), gene ontology (GO) and pseudo-position specific scoring matrix (PsePSSM) are employed to encode subcellular protein sequences. Then, using direct multi-label linear discriminant analysis (DMLDA) to get rid of noise information of the fused feature vector. Lastly, the first-best feature vectors are input into the multi-label learning with Label-specIfic FeaTures (LIFT) classifier to predict. The leave-one-out cross validation (LOOCV) shows that the overall actual accuracy on Gram-negative bacteria, Gram-positive bacteria, plant datasets, virus dataset and human dataset are 98.6%, 99.6%, 97.9%, 94.7% and 96.1% respectively, which are obviously better than other state-of-the-art prediction methods. The proposed model can effectively predict SCL of multi-label proteins and provide references for experimental identification of SCL. The source codes and datasets are available at https://github.com/QUST-AIBBDRC/DMLDA-LocLIFT/.

Learning shared subspace regularization with linear discriminant analysis for multi-label action recognition

Human action recognition under complex environment is a challenging work, while in deep learning and in these specific difficulty recognition tasks, the multi-label linear discriminant analysis (MLDA) is already utilized. As is known to all, MLDA is used for dimensionality feature reduction. Nevertheless, MLDA contains the eigendecomposition of dense matrices, which will cost a huge money on a high dimension computing. In this study, we demonstrate that the MLDA formula is able to equivalent to the least squares problem, which greatly reduces the computation and size of high-dimensional datasets. In addition, it is found that introducing attractive regularization technique into the classical least squares strategy can improve the robustness. The established equivalence relationship is proved by laboratory results on three action datasets. In addition, through comparing with some typical and recent algorithms, the superiority of our constructed model is also demonstrated.

Using Evolutionary Information and Multi-Label Linear Discriminant Analysis to Predict the Subcellular Location of Multi-Site Bacterial Proteins via Chou’s 5-Steps Rule

The function of a protein is closely tied to its subcellular location. Identifying the subcellular location of proteins is a crucial step to understand their functions. However, determining the subcellular location of proteins experimentally is time-consuming and costly. Therefore, developing effective computational methods to predict the subcellular positions of proteins is a hotspot in bioinformatics. Though many models have been proposed to improve the prediction accuracy of protein subcellular localization, there are still several shortcomings: (1) numerous methods ignore the multi-site proteins; (2) high dimensional features bring the burden to the construction of the prediction model. In this work, we proposed a method to predict the subcellular location of bacterial proteins with both single and multiple locations. Two features based on evolutionary information are extracted to solve the multi-site prediction problem, of which one is a 190-dimensional feature vector from absolute entropy correlation analysis (AECA-PSSM) and another is a 480-dimensional feature vector extracted using discrete wavelet transform (PSSM-DWT). After combining both proposed features, multi-label linear discriminant analysis (MLDA) is employed to transform the high-dimensional feature space into a lower-dimensional space. Multi-label k-nearest neighbors algorithm (ML-KNN) is utilized to predict the subcellular location of both single-site and multi-site proteins. Experimental results on Gram-positive dataset and Gram-negative dataset show the effectiveness of the proposed method.

Shared subspace least squares multi-label linear discriminant analysis

Multi-label linear discriminant analysis (MLDA) has been explored for multi-label dimension reduction. However, MLDA involves dense matrices eigen-decomposition which is known to be computationally expensive for large-scale problems. In this paper, we show that the formulation of MLDA can be equivalently casted as a least squares problem so as to significantly reduce the computation burden and scale to the data collections with higher dimension. Further, it is also found that appealing regularization techniques can be incorporated into the least-squares model to boost generalization accuracy. Experimental results on several popular multi-label benchmarks not only verify the established equivalence relationship, but also demonstrate the effectiveness and efficiency of our proposed algorithms.

Deep Robust Framework for Protein Function Prediction Using Variable-Length Protein Sequences.

The order of amino acids in a protein sequence enables the protein to acquire a conformation suitable for performing functions, thereby motivating the need to analyze these sequences for predicting functions. Although machine learning based approaches are fast compared to methods using BLAST, FASTA, etc., they fail to perform well for long protein sequences (with more than 300 amino acids). In this paper, we introduce a novel method for construction of two separate feature sets for protein using bi-directional long short-term memory network based on the analysis of fixed 1) single-sized segments and 2) multi-sized segments. The model trained on the proposed feature set based on multi-sized segments is combined with the model trained using state-of-the-art Multi-label Linear Discriminant Analysis (MLDA) features to further improve the accuracy. Extensive evaluations using separate datasets for biological processes and molecular functions demonstrate not only improved results for long sequences, but also significantly improve the overall accuracy over state-of-the-art method. The single-sized approach produces an improvement of +3.37 percent for biological processes and +5.48 percent for molecular functions over the MLDA based classifier. The corresponding numbers for multi-sized approach are +5.38 and +8.00 percent. Combining the two models, the accuracy further improves to +7.41 and +9.21 percent, respectively.

From Protein Sequence to Protein Function via Multi-Label Linear Discriminant Analysis.

Sequence describes the primary structure of a protein, which contains important structural, characteristic, and genetic information and thereby motivates many sequence-based computational approaches to infer protein function. Among them, feature-base approaches attract increased attention because they make prediction from a set of transformed and more biologically meaningful sequence features. However, original features extracted from sequence are usually of high dimensionality and often compromised by irrelevant patterns, therefore dimension reduction is necessary prior to classification for efficient and effective protein function prediction. A protein usually performs several different functions within an organism, which makes protein function prediction a multi-label classification problem. In machine learning, multi-label classification deals with problems where each object may belong to more than one class. As a well-known feature reduction method, linear discriminant analysis (LDA) has been successfully applied in many practical applications. It, however, by nature is designed for single-label classification, in which each object can belong to exactly one class. Because directly applying LDA in multi-label classification causes ambiguity when computing scatters matrices, we apply a new Multi-label Linear Discriminant Analysis (MLDA) approach to address this problem and meanwhile preserve powerful classification capability inherited from classical LDA. We further extend MLDA by l1-normalization to overcome the problem of over-counting data points with multiple labels. In addition, we incorporate biological network data using Laplacian embedding into our method, and assess the reliability of predicted putative functions. Extensive empirical evaluations demonstrate promising results of our methods.

A least squares formulation of multi-label linear discriminant analysis

The classical linear discriminant analysis has been recently extended to the multi-label dimensionality reduction. However, Multi-label Linear Discriminant Analysis (MLDA) involves dense matrices eigen-decomposition that is known to be computationally expensive for the large-scale problems. In this paper, we present that the formulation of MLDA can be equivalently casted as a new least-squares framework so as to significantly mitigate the computational overhead and scale to the data collections with higher dimension. Further, it is also found that appealing regularization techniques can be incorporated into the least-squares model to boost generalization accuracy. Experimental results on several popular multi-label benchmarks not only verify the established equivalence relationship, but also corroborate the effectiveness and efficiency of our proposed algorithms.

Multi-label dimensionality reduction and classification with extreme learning machines

In the need of some real applications, such as text categorization and image classification, the multi-label learning gradually becomes a hot research point in recent years. Much attention has been paid to the research of multi-label classification algorithms. Considering the fact that the high dimensionality of the multi-label datasets may cause the curse of dimensionality and will hamper the classification process, a dimensionality reduction algorithm, named multi-label kernel discriminant analysis (MLKDA), is proposed to reduce the dimensionality of multi-label datasets. MLKDA, with the kernel trick, processes the multi-label integrally and realizes the nonlinear dimensionality reduction with the idea similar with linear discriminant analysis (LDA). In the classification process of multi-label data, the extreme learning machine (ELM) is an efficient algorithm in the premise of good accuracy. MLKDA, combined with ELM, shows a good performance in multi-label learning experiments with several datasets. The experiments on both static data and data stream show that MLKDA outperforms multi-label dimensionality reduction via dependence maximization (MDDM) and multi-label linear discriminant analysis (MLDA) in cases of balanced datasets and stronger correlation between tags, and ELM is also a good choice for multi-label classification.