SV Machine Research Articles

Peak load forecasting based on multi‐source data and day‐to‐day topological network

Electric load variation results from a variety of factors. This study discusses the inner mechanism of load variation and presents a load forecasting method based on multi-source data and day-to-day topological network. Multi-source data including weather data, seasonal attributes, calendar attributes, holiday attributes and load growth rate are considered to be the dominant factors that cause the load variation. Initially, a day-to-day topological network is generated that reflects the similarity between any 2 days. Additionally, the random walk with restart (RWR) algorithm is applied to the topological network to construct the training set. The support vector regression model is then adopted. With case studies using the data of Guangzhou, the authors obtain a decrease of the mean absolute percentage errors of 19.5 and 25.7% over the SV machine method and a neural network ensemble model. Their research reveals three points: (i) weather elements, especially temperature, air pressure and water vapour pressure, have a dominant influence on load variation besides historical load; (ii) multi-source data improves the accuracy, meanwhile the applying of RWR algorithm maximises its effect; and (iii) since some of the factors that cause a constant error remains unknown, they can use the feedforward correction to decrease the affects.

Tennessee Eastman Process Diagnosis Based on Dynamic Classification With SVDD

The support vector domain description (SVDD) is an efficient kernel method inspired from the SV machine (SVM) by Vapnik. It is commonly used for one-classification problems or novelty detection. The training algorithm solves a constrained convex quadratic programming (QP) problem. This assumes prior dense sampling (offline training) and it requires large memory and enormous amounts of training time. In this paper, we propose a fast SVDD dedicated for multiclassification problems. The proposed classifier deals with stationary as well as nonstationary (NS) data. The principle is based on the dynamic removal/insertion of informations according to adequate rules. To ensure the rapidity of convergence, the algorithm considers in each run a limited frame of samples for the training process. These samples are selected according to some approximations based on Karush–Kuhn–Tucker (KKT) conditions. An additional merge mechanism is proposed to avoid local optima drawbacks and improve performances. The developed method is assessed on some synthetic data to prove its effectiveness. Afterward, it is employed to solve a diagnosis problem and faults detection. We considered for this purpose a real industrial plant consisting in Tennessee Eastman process (TEP).

Fast pruning superfluous support vectors in SVMs

Support vector machines (SV machines, SVMs) often contain many SVs, which reduce runtime speeds of decision functions. To simplify the decision functions and improve SVM succinctness, the efforts to remove SVs in trained SVMs have been made. By meticulously designing some pruning coefficients and solving for the rest, this paper presents a simple method for fast removing superfluous SVs. The method empowers users to remove those SVs in a single iteration, thereby significantly enhancing the pruning speed of currently used methods, which remove the SVs one by one. The existence and uniqueness of the fast pruning coefficients are shown. The nexus of primal and dual optimizations is illustrated geometrically. The fast pruning method can also be applied to other kernel-based machines without any modifications. The computational complexity is discussed. Examples are given first and experiments on larger data sets demonstrate the effectiveness of the fast simplification method.

Feature space versus empirical kernel map and row kernel space in SVMs

In machine-learning technologies, the support vector machine (SV machine, SVM) is a brilliant invention with many merits, such as freedom from local minima, the widest possible margins separating different clusters, and a solid theoretical foundation. In this paper, we first explore the linear separability relationships between the high-dimensional feature space H and the empirical kernel map U as well as between H and the space of kernel outputs K. Second, we investigate the relations of the distances between separating hyperplanes and SVs in H and U, and derive an upper bound for the margin width in K. Third, as an application, we show experimentally that the separating hyperplane in H can be slightly adjusted through U. The experiments reveal that existing SVM training can linearly separate the data in H with considerable success. The results in this paper allow us to visualize the geometry of H by studying U and K.

Pruning Support Vector Machines Without Altering Performances

Support vector machines (SV machines, SVMs) have many merits that distinguish themselves from many other machine-learning algorithms, such as the nonexistence of local minima, the possession of the largest distance from the separating hyperplane to the SVs, and a solid theoretical foundation. However, SVM training algorithms such as the efficient sequential minimal optimization (SMO) often produce many SVs. Some scholars have found that the kernel outputs are frequently of similar levels, which insinuate the redundancy of SVs. By analyzing the overlapped information of kernel outputs, a succinct separating-hyperplane-securing method for pruning the dispensable SVs based on crosswise propagation (CP) is systematically developed. The method also circumvents the problem of explicitly discerning SVs in feature space as the SVM formulation does. Experiments with the famous SMO-based software LibSVM reveal that all typical kernels with different parameters on the data sets contribute the dispensable SVs. Some 1% ~ 9% (in some scenarios, more than 50%) dispensable SVs are found. Furthermore, the experimental results also verify that the pruning method does not alter the SVMs' performances at all. As a corollary, this paper further contributes in theory a new lower upper bound on the number of SVs in the high-dimensional feature space.

Multi-Classification by Using Tri-Class SVM

The standard form for dealing with multi-class classification problems when bi-classifiers are used is to consider a two-phase (decomposition, reconstruction) training scheme. The most popular decomposition procedures are pairwise coupling (one versus one, 1-v-1), which considers a learning machine for each Pair of classes, and the one-versus-all scheme (one versus all, 1-v-r), which takes into consideration each class versus the remaining classes. In this article a 1-v-1 tri-class Support Vector Machine (SVM) is presented. The expansion of the architecture of this machine into three categories specifically addresses the decomposition problem of how to prevent the loss of information which occurs in the usual 1-v-1 training procedure. The proposed machine, by means of a third class, allows all the information to be incorporated into the remaining training patterns when a multi-class problem is considered in the form of a 1-v-1 decomposition. Three general structures are presented where each improves some features from the precedent structure. In order to deal with multi-classification problems, it is demonstrated that the final machine proposed allows ordinal regression as a form of decomposition procedure. Examples and experimental results are presented which illustrate the performance of the new tri-class SV machine.

K-SVCR. A support vector machine for multi-class classification

The problem of multi-class classification is usually solved by a decomposing and reconstruction procedure when two-class decision machines are implied. During the decomposing phase, training data are partitioned into two classes in several manners and two-class learning machines are trained. To assign the class for a new entry, machines’ outputs are evaluated in a specific pulling scheme. This article introduces the “Support Vector Classification-Regression” machine for K-class classification purposes ( K-SVCR), a new training algorithm with ternary outputs {−1,0,+1} based on Vapnik's Support Vector theory. This new machine evaluates all the training data into a 1-versus-1-versus-rest structure during the decomposing phase by using a mixed classification and regression SV Machine (SVM) formulation. For the reconstruction, a specific pulling scheme considering positive and negative votes has been designed, making the overall learning architecture more fault-tolerant as it will be demonstrated.