Twin Support Vector Machine Classifier Research Articles

Machine Failure Prediction Using Multilabel Classification Methods

Early detection of machine failure is crucial in every industrial setting as it may prevent unexpected process downtimes as well as system failures. However, machine learning (ML) models are increasingly being utilized to forecast system failures in industrial maintenance, and among them, multilabel classification techniques act as efficient methods. Therefore, this study analyzed machine failure data with five types of machine failures. Initially, a feature selection approach was also carried out in this study to determine the variables which directly cause machine failure. Furthermore, multilabel k-nearest neighbours (MLkNN), multilabel adaptive resonance associative map (MLARAM), and multilabel twin support vector machine classifier (MLTSVM) in adapted algorithms, Binary Relevance, ClassifierChain, and LabelPowerSet in problem transformation approaches, and Random Label Space Partitioning with Label Powerset (RakelD) in ensemble classifiers were employed. To train these models, both the original data set as well as data frame after the feature selection was used, and hamming loss, accuracy, macro, and micro averages were calculated for each of these classifiers. According to the results, MLkNN in adapted algorithms and LabelPowerset in problem transformation approaches performed better than other classifiers used in this study. Therefore, it can be concluded that MLkNN and LabelPowerset could be used to classify multilabel with positive results. KEYWORDS: adapted algorithms, ensemble classifiers, feature selection, machine failure, machine learning, multilabel classification, problem transformation.

Deep multi-view multiclass twin support vector machines

Multi-view learning (MVL) is a rapidly evolving direction in the field of machine learning. Despite the positive results, most algorithms that combine multi-view learning with twin support vector machines (TSVM) focus on the traditional machine learning domain. No method has been accomplished for combining MVL, TSVM, and deep learning. In this paper, we propose two novel multi-view deep models to solve the multiclass classification problem, namely deep multi-view twin support vector machines (DMvTSVM) based on deep neural network (DNN) and auto-encoder (AE) network. They find two non-parallel hyperplanes such that each hyperplane is as close to its own class as possible while being as far away from the other class as possible. Meanwhile, we apply similarity regularization to the output of the Deep TSVM classifier for each view to learn consensus information between views, and use this to refine the joint weights of the deep model and TSVM. Finally, the novel models employ the one−vs−rest strategy to allow the DMvTSVM classifier to solve the multiclass classification problems. In the experiments, the proposed methods are compared with existing state-of-the-art algorithms to prove their effectiveness.

Multiclass recognition of AD neurological diseases using a bag of deep reduced features coupled with gradient descent optimized twin support vector machine classifier for early diagnosis

SummaryAlzheimer's disease (AD) is an advanced neurodegenerative disease of the brain that affects the nerve system of brain. Previously, several feature extraction and classification methods were discussed, but that methods provide high over fitting problem, which leads to minimization of detection accuracy. To overcome these issues, the multi class classification of AD diseases using bag of deep feature reduction technique and twin support vector machine classifier (TSVM) optimized with gradient decent optimizer is proposed in this manuscript for classifying the AD disease as severe AD, mild cognitive impairment, healthy control. At first, the input EEG signals are pre‐processed. To decrease the execution time and processing time with feature size, a bag of deep features reduction technique is used. The reduced feature signals are classified by optimized TSVM. The simulation process is implemented in MATLAB environment. The proposed model achieves higher accuracy 33.84%, 28.93%, 33.03%, 27.93%, higher precision 22.87%, 16.97%, 16.97%, and 36.97%, compared with the existing methods, such as piecewise aggregate approximation support vector machine (MCC‐EEG‐PAA‐SVM), convolutional neural network (MCC‐EEG‐CNN), conformal kernel‐based fuzzy support vector machine (MCC‐EEG‐CKF‐SVM), Pearson correlation coefficient‐based feature selection strategy with linear discriminant analysis classifier (MCC‐EEG‐ PCC‐LDA).

Learning Pinball TWSVM efficiently using Privileged Information and its Applications

Expert knowledge plays a vital role in any learning framework. However, in the field of machine learning, an expert's knowledge is rarely utilised. Furthermore, most machine learning methods (support vector machine, SVM-based) use a hinge loss function that is sensitive to noise. Thus, in order to benefit from expert knowledge while reducing noise sensitivity, we propose in this paper a fast and novel Twin Support Vector Machine classifier based on privileged information with a pinball loss function, dubbed Pin-TWSVMPI, where expert knowledge is in the form of privileged information. The proposed Pin-TWSVMPI incorporates privileged information into two nonparallel decision hyperplanes by employing a correction function. Furthermore, we employ the Sequential Minimal Optimization (SMO) technique to get the classifier in order to make computations more efficient and faster, and we have demonstrated its applicability for pedestrian detection and handwritten digit recognition. Furthermore, for UCI datasets, we first construct a process that retrieves privileged information from the dataset's features, which is then used by Pin-TWSVMPI, resulting in improved classification accuracy with a reduced computing time.

Learning Pinball TWSVM efficiently using Privileged Information and their applications

In any learning framework, an expert knowledge always plays a crucial role. But, in the field of machine learning, the knowledge offered by an expert is rarely used. Moreover, machine learning algorithms (SVM based) generally use hinge loss function which is sensitive towards the noise. Thus, in order to get the advantage from an expert knowledge and to reduce the sensitivity towards the noise, in this paper, we propose a fast and novel Twin Support Vector Machine classifier based on privileged information with pinball loss function which has been termed as Pin-TWSVMPI where expert's knowledge is in the form of privileged information. The proposed Pin-TWSVMPI incorporates privileged information by using correcting function so as to obtain two nonparallel decision hyperplanes. Further, in order to make computations more efficient and fast, we use Sequential Minimal Optimization (SMO) technique for obtaining the classifier and have also shown its application for Pedestrian detection and Handwritten digit recognition. Further, for UCI datasets, we first implement a procedure which extracts privileged information from the features of the dataset which are then further utilized by Pin-TWSVMPI to which lead to enhancement in classification accuracy with comparatively lesser computational time.

Multimodal Fusion-Based Deep Learning Network for Effective Diagnosis of Alzheimer’s Disease

Alzheimer’s disease (AD) is a prevalent, irreversible, chronic, and degenerative disorder whose diagnosis at the prodromal stage is critical. Mostly, single modality data, such as magnetic resonance imaging (MRI) or positron emission tomography (PET), are used to make predictions in AD studies. However, the metabolic and structural data fusion can provide a holistic view of AD-staging analysis. To achieve this objective, a novel multimodal fusion-based method is proposed in this article. An optimal fusion of MRI and PET is achieved by harnessing demon algorithm and discrete wavelet transform. Finally, the fused image features are extracted using ResNet-50, and these features are classified using robust energy least square twin support vector machine classifier. Experiments on the AD neuroimaging initiative dataset show descent accuracy of 97%, 94%, and 97.5% for cognitive normal (CN) versus AD, CN versus mild cognitive impairment (MCI), and AD versus MCI, respectively. The proposed model will be beneficial for health professionals in accurately diagnosing AD at an early stage.

Large-Scale Least Squares Twin SVMs

In the last decade, twin support vector machine (TWSVM) classifiers have achieved considerable emphasis on pattern classification tasks. However, the TWSVM formulation still suffers from the following two shortcomings: (1) TWSVM deals with the inverse matrix calculation in the Wolfe-dual problems, which is intractable for large-scale datasets with numerous features and samples, and (2) TWSVM minimizes the empirical risk instead of the structural risk in its formulation. With the advent of huge amounts of data today, these disadvantages render TWSVM an ineffective choice for pattern classification tasks. In this article, we propose an efficient large-scale least squares twin support vector machine (LS-LSTSVM) for pattern classification that rectifies all the aforementioned shortcomings. The proposed LS-LSTSVM introduces different Lagrangian functions to eliminate the need for calculating inverse matrices. The proposed LS-LSTSVM also does not employ kernel-generated surfaces for the non-linear case, and thus uses the kernel trick directly. This ensures that the proposed LS-LSTSVM model is superior to the original TWSVM and LSTSVM. Lastly, the structural risk is minimized in LS-LSTSVM. This exhibits the essence of statistical learning theory, and consequently, classification accuracy on datasets can be improved due to this change. The proposed LS-LSTSVM is solved using the sequential minimal optimization (SMO) technique, making it more suitable for large-scale problems. We further proved the convergence of the proposed LS-LSTSVM. Exhaustive experiments on several real-world benchmarks and NDC-based large-scale datasets demonstrate that the proposed LS-LSTSVM is feasible for large datasets and, in most cases, performed better than existing algorithms.

A heuristic technique to detect phishing websites using TWSVM classifier

Phishing websites are on the rise and are hosted on compromised domains such that legitimate behavior is embedded into the designed phishing site to overcome the detection. The traditional heuristic techniques using HTTPS, search engine, Page Ranking and WHOIS information may fail in detecting phishing sites hosted on the compromised domain. Moreover, list-based techniques fail to detect phishing sites when the target website is not in the whitelisted data. In this paper, we propose a novel heuristic technique using TWSVM to detect malicious registered phishing sites and also sites which are hosted on compromised servers, to overcome the aforementioned limitations. Our technique detects the phishing websites hosted on compromised domains by comparing the log-in page and home page of the visiting website. The hyperlink and URL-based features are used to detect phishing sites which are maliciously registered. We have used different versions of support vector machines (SVMs) for the classification of phishing websites. We found that twin support vector machine classifier (TWSVM) outperformed the other versions with a significant accuracy of 98.05% and recall of 98.33%.

An Efficient IoT-Based Platform for Remote Real-Time Cardiac Activity Monitoring

In this article, a novel and an efficient methodology is presented for real-time monitoring of ECG signals. The method involves fast Fourier transform (FFT) based discrete wavelet transform (DWT) for extracting the features from the heartbeats which involves less computational complexity in terms of additions and multiplications operations for higher order filter lengths. These features extracted are recognized using particle swarm optimization (PSO) tuned twin support vector machines (TSVM) classifier. The TSVM classifier is four times faster than the standard SVM while the PSO technique is employed to gradually tune the classifier parameters to achieve more accuracy. The proposed methodology is implemented on IoT based microcontroller platform and validated on the benchmark Physionet data to classify 16 categories of ECG signals. Once an abnormality is detected, the platform generates a pop-up message as a warning and sends the information to a remote platform allowing hospitals to take preventive measures. The platform reported a higher overall accuracy of 95.68% than the existing studies. Further, such implementation can be utilized as a warning system in both homecare as well as tele-monitoring applications to continuously monitor the cardiac condition of a subject anywhere to the state-of-art heart disease diagnosis.

An enhanced KNN-based twin support vector machine with stable learning rules

Among the extensions of twin support vector machine (TSVM), some scholars have utilized K-nearest neighbor (KNN) graph to enhance TSVM's classification accuracy. However, these KNN-based TSVM classifiers have two major issues such as high computational cost and overfitting. In order to address these issues, this paper presents an enhanced regularized K-nearest neighbor based twin support vector machine (RKNN-TSVM). It has three additional advantages: (1) Weight is given to each sample by considering the distance from its nearest neighbors. This further reduces the effect of noise and outliers on the output model. (2) An extra stabilizer term was added to each objective function. As a result, the learning rules of the proposed method are stable. (3) To reduce the computational cost of finding KNNs for all the samples, location difference of multiple distances based k-nearest neighbors algorithm (LDMDBA) was embedded into the learning process of the proposed method. The extensive experimental results on several synthetic and benchmark datasets show the effectiveness of our proposed RKNN-TSVM in both classification accuracy and computational time. Moreover, the largest speedup in the proposed method reaches to 14 times.

Benchmarking tree-based least squares twin support vector machine classifiers

Benchmarking tree-based least squares twin support vector machine classifiers

Multiclass Twin Support Vector Machine for plant species identification

Automatic plant species identification is one of the recent and fascinating research area as plants are crucial element of ecosystem. Several plant species exist with significant importance but most of us are unaware of the diversity of plant species available on earth. Their utility to humans starts as oxygen provider, food source, and medicinal compounds essential for medicines that are difficult to develop in right proportions. Being the first living habitants of earth, they have roots far deeper in the ecosystem than any living being. Hence, it is utmost important to develop automatic plant species identification system in which the digital image of the plant is given as input and the label of the plant is determined by the system. In this paper, we have focused on three different aspects (i) Significance of threshold (ii) Feature descriptor that can best describe the leaf images and (iii) Proposed a novel classification method called Multi class Twin Support Vector Machine which in an extension of widely used Twin Support Vector Machine classifier. The performance of the proposed method is compared with SVM, Multi Birth SVM and Probabilistic Neural Network. It is observed that the proposed classifier outperforms all the aforementioned classifiers on publicly available datasets.

Detecting facial emotions using normalized minimal feature vectors and semi-supervised twin support vector machines classifier

In this paper, human facial emotions are detected through normalized minimal feature vectors using semi-supervised Twin Support Vector Machine (TWSVM) learning. In this study, face detection and tracking are carried out using the Constrained Local Model (CLM), which has 66 entire feature vectors. Based on Facial Animation Parameter’s (FAPs) definition, entire feature vectors are those things that visibly affect human emotion. This paper proposes the 13 minimal feature vectors that have high variance among the entire feature vectors are sufficient to identify the six basic emotions. Using the Max & Min and Z-normalization technique, two types of normalized minimal feature vectors are formed. The novelty of this study is methodological in that the normalized data of minimal feature vectors fed as input to the semi-supervised multi-class TWSVM classifier to classify the human emotions is a new contribution. The macro facial expression datasets are used by a standard database and several real-time datasets. 10-fold and hold out cross-validation is applied with the cross-database (combining standard and real-time). In the experimental result, using ‘One vs One’ and ‘One vs All’ multi-class techniques with 3 kernel functions produce a 36 trained model of each emotion and their validation parameters are calculated. The overall accuracy achieved for 10-fold cross-validation is 93.42 ± 3.25% and hold out cross-validation is 92.05 ± 3.79%. The overall performance (Precision, Recall, F1-score, Error rate and Computation Time) of the proposed model was also calculated. The performance of the proposed model and existing methods were compared and results indicate them to be more reliable than existing models.

ML-SLSTSVM: a new structural least square twin support vector machine for multi-label learning

Multi-label learning (MLL) is a special supervised learning task, where any single instance possibly belongs to several classes simultaneously. Nowadays, MLL methods are increasingly required by modern applications, such as protein function classification, speech recognition and textual data classification. In this paper, a structural least square twin support vector machine (SLSTSVM) classifier for multi-label learning is presented. This proposed ML-SLSTSVM focuses on the cluster-based structural information of the corresponding class in each optimization problem, which is vital for designing a good classifier in different real-world problems. This method is extended to a nonlinear version by the kernel trick. Experimental results demonstrate that proposed method is superior in generalization performance to other classifiers.

Blind modulation format identification using decision tree twin support vector machine in optical communication system

This paper proposes a method of blind modulation format identification using decision tree twin support vector machine classifier trained with the features extracted from the high-order cumulant and cyclic spectrum after compressed sensing in order to solve the problem of identification efficiency and computing speed and reduce the performance requirements of the sampling system in low optical signal to noise ratio. By reconstructing the feature parameters of fourth-order, eighth-order cumulants and cyclic spectrum under the theory of compressed sensing, and introducing the decision tree twin support vector machine classifier to achieve high-precision classification, the different modulation formats of amplitude shift keying, multiple phase-shift keying and multiple quadrature amplitude modulation are effectively identified. Simulation analysis of the influence of identification accuracy and identification time improves the identification performance and achieves the purpose of identifying more signals with fewer feature parameters. The results indicate that the average identification accuracy of the optical modulation format signals can be achieved over 94% when the OSNR is -5dB, and the identification time is 4.3 times higher than the standard SVM. Owing to its excellent performance, this method can be employed in the next generation optical transport network for auto-adaption real-time modulation format identification.

Efficient and robust TWSVM classification via a minimum L1-norm distance metric criterion

A twin support vector machine (TWSVM) is a classic distance metric learning method for classification problems. The TWSVM criterion is formulated based on the squared L2-norm distance, making it prone to being influenced by the presence of outliers. In this paper, to develop a robust distance metric learning method, we propose a new objective function, called L1-TWSVM, for the TWSVM classifier using the robust L1-norm distance metric. The optimization strategy is to maximize the ratio of the inter-class distance dispersion to the intra-class distance dispersion by using the robust L1-norm distance rather than the traditional L2-norm distance. The resulting objective function is much more challenging to optimize because it involves a non-smooth L1-norm term. As an important contribution of this paper, we design a simple but valid iterative algorithm for solving L1-norm optimal problems. This algorithm is easy to implement, and its convergence to an optimum is theoretically guaranteed. The efficiency and robustness of L1-TWSVM have been validated by extensive experiments on both UCI datasets as well as synthetic datasets. The promising experimental results indicate that our proposal approaches outperform relevant state-of-the-art methods in all kinds of experimental settings.

KNN-based multi-label twin support vector machine with priority of labels

Abstract A common approach to solve multi-label classification problems is the transformation method, in which a multi-label problem is converted into multiple single-label representations. With an efficient implementation of single-label algorithms, and considering dependency between labels and the fact that similar samples often share the same labels, we can expect a highly effective classification in multi-label datasets. In this paper, to tackle multi-label classification problem, first using an improved twin support vector machine classifier, the hyperplanes containing structural information of samples and local information of each class label, are found. Then the prior probability of each hyperplane and the sample points that are located in the margins of the hyperplanes are extracted. For the prediction phase, several facts are applied to help finding the sets of relevant labels of a sample: (1) samples with similar labels share the same information, (2) local information has a great impact on the performance and efficiency of a multi-label algorithm, and (3) the samples that are most important in classifications are located in the margin of hyperplanes. To obtain the sets of relevant labels for a test sample, first its k nearest samples in the margin space of the hyperplanes are found. Then, relevant labels are extracted using statistical and membership counting methods. The nonlinear version of the algorithm is also developed through kernel trick. The experimental results obtained from different datasets and different measures indicated good performances of the proposed algorithm, compared to several relevant methods.

Clustering by twin support vector machine and least square twin support vector classifier with uniform output coding

The recently proposed twin support vector clustering (TWSVC) is a powerful clustering method. However, TWSVC may encounter the singularity problem and it is time consuming in the learning stage. In this paper, we introduce some efficient techniques into TWSVC, and propose two clustering models, called twin bounded support vector clustering (TBSVC) and least square twin bounded support vector clustering (LSTBSVC), respectively. TBSVC introduces a maximum margin regularization term into TWSVC, which not only avoids its singularity but also significantly improves the performance. LSTBSVC introduces the least square formation into TBSVC to greatly accelerate its learning speed. Moreover, a uniform output coding for LSTBSVC is introduced to cope with the non-uniformed problem in the learning procedures. In addition, nonlinear clustering is also extended to the above clustering methods by using the kernel trick. Experimental results show the effectiveness and efficiency of our methods.

Benchmarking Tree based Least Squares Twin Support Vector Machine Classifiers

Least square twin support vector machine is an emerging learning method applied in classification problem. This paper present a tree-based least square twin support vector machine (T-LSTWSVM) for classification. Classification procedure depends on the correlation of input feature as well as output feature. UCI benchmark data sets are used to evaluate the test set performance of tree-based least square twin support vector machine (T-LSTWSVM) classifiers with multiple kernel functions such as linear, polynomial and radial basis function (RBF) kernels. This method applies on two main types of classification problems such as binary class problem as well as multi-class problem. The evaluation and accuracy is calculated in terms of distance metric. It was observed that multi-class classification problem performed excellently by tree-based method.

A self-adaptive frequency selection common spatial pattern and least squares twin support vector machine for motor imagery electroencephalography recognition

Motor imagery brain-computer interface (BCI) systems require accurate and fast recognition of brain activity patterns for reliable communication and interaction. Achieving this accuracy is still a challenge because of the low signal-to-noise ratio in electroencephalography signals and high variability of sensorimotor rhythms. To address this need, we proposed a novel scheme that combined a frequency band selection common spatial pattern algorithm and a particle swarm optimization least squares twin support vector machine classifier for recognition of motor imagery patterns. We used self-adaptive artifact removal and the common spatial pattern method to obtain the most distinguishable features. To improve the classification results, we investigated linear, polynomial, sigmoid, wavelet and gaussian kernel functions for the nonlinear least squares twin support vector machine classifier. Particle swarm optimization, chaotic particle swarm optimization, a genetic algorithm and a quantum genetic algorithm were compared and used to tune the hyper-parameters for the classifier. To evaluate our proposed method, we used BCI Competition IV data sets 2A and 2B. Experimental results showed that for our method, the average recognition accuracy of data set 2A is increased by 6.10%, 6.71%, 3.87%, 4.01%, 2.55% and 4.86% compared with the results obtained by regularization projection twin support vector machine, twin support vector machine, support vector machine, linear discriminant analysis, back propagation and probabilistic neural network, respectively. Using data set 2B, the average recognition accuracy achieved by our method was greater by 4.73%, 5.46%, 4.45%, 4.10%, 8.62% and 4.27%, respectively. The standard deviation of the accuracy values of the two data sets decreased. Furthermore, compared with the traditional support vector machines, our proposed method achieved a faster central processing unit running time for training classifiers.