Structural Risk Minimization Principle Research Articles

Efficient Optimization of a Ka-Band Branch Waveguide Power Divider

This paper presents an efficient optimization technique, Support Vector Regression (SVR) approach, for the designing of a Ka-band branch waveguide power divider. This SVR approach comes from the Support Vector Machine (SVM) learning theory, which is based on the structural risk minimization (SRM) principle and leads good generalization ability. With this method, a Ka-band branch waveguide power divider is proposed. Unlike the traditional Chebyschev-type branch-line coupler, this branch waveguide power divider has the uniform main guide and all the branches in same size to minimize the difficulty and sensitivity in the mechanical fabrication. A validation of the proposed design is also presented by means of full-wave simulation and measurement.

Reliability forecasting by recurrent Support Vector Regression

Support Vector Regression (SVR), which employs the structural risk minimisation principle to minimise an upper bound of the generalisation errors instead of minimising the training errors used by ANNs, has been successfully applied to solve nonlinear forecasting and times series problems. However, the application of SVR to reliability forecasting has still not been extensively explored. In general, Recurrent Neural Networks (RNNs) are trained by back-propagation algorithms. In the study, the learning algorithms of SVR are applied to RNNs for forecasting system reliability, and the Immune Algorithm (IA) is applied to the parameter determining the SVR model. A numerical example in the existing literature is employed to demonstrate the prediction performance of the proposed model. Empirical results illustrate that the proposed model outperforms other approaches in the existing literature.

Mim Capacitor Modeling by Support Vector Regression

The support vector regression (SVR) method is introduced to model the MIM capacitor in this paper. SVM is a type of learning machine based on the statistical learning theory, which implements the structural risk minimization principle to obtain a good generalization from limited size data sets. The SVR model of the MIM capacitor is trained and tested by using the data generated from EM simulation. Once the model is constructed, it can provides results approaching the accuracy of the EM simulated results without increasing the analysis time significantly, which proves the validity of the method.

Cross-correlation aided support vector machine classifier for classification of EEG signals

Over the last few decades pattern classification has been one of the most challenging area of research. In the present-age pattern classification problems, the support vector machines (SVMs) have been extensively adopted as machine learning tools. SVM achieves higher generalization performance, as it utilizes an induction principle called structural risk minimization (SRM) principle. The SRM principle seeks to minimize the upper bound of the generalization error consisting of the sum of the training error and a confidence interval. SVMs are basically designed for binary classification problems and employs supervised learning to find the optimal separating hyperplane between the two classes of data. The main objective of this paper is to introduce a most promising pattern recognition technique called cross-correlation aided SVM based classifier. The idea of using cross-correlation for feature extraction is relatively new in the domain of pattern recognition. In this paper, the proposed technique has been utilized for binary classification of EEG signals. The binary classifiers employ suitable features extracted from crosscorrelograms of EEG signals. These cross-correlation aided SVM classifiers have been employed for some benchmark EEG signals and the proposed method could achieve classification accuracy as high as 95.96% compared to a recently proposed method where the reported accuracy was 94.5%.

USING LEAST SQUARE SVM FOR NONLINEAR SYSTEMS MODELING AND CONTROL

ABSTRACTSupport vector machine is a learning technique based on the structural risk minimization principle, and it is also a class of regression method with good generalization ability. The paper firstly introduces the mathematical model of regression least squares support vector machine (LSSVM), and designs incremental learning algorithms by the calculation formula of block matrix, then uses LSSVM to model nonlinear system, based on which to control nonlinear systems by model predictive method. Simulation experiments indicate that the proposed method provides satisfactory performance, and it achieves superior modeling performance to the conventional method based on neural networks, moreover it achieves well control performance.

Hybrid Support Vector Machines-Based Multi-fault Classification

Abstract Support Vector Machines (SVM) is a new general machine-learning tool based on structural risk minimization principle. This characteristic is very signific ant for the fault diagnostics when the number of fault samples is limited. Considering that SVM theory is originally designed for a two-class classification, a hybrid SVM scheme is proposed for multi-fault classification of rotating machinery in our paper. Two SVM strategies, 1-v-1 (one versus one) and 1-v-r (one versus rest), are respectively adopted at different classification levels. At the parallel classification level, using 1-v-1 strategy, the fault features extracted by various signal analysis methods are transferred into the multiple parallel SVM and the local classification results are obtained. At the serial classification level, these local results values are fused by one serial SVM based on 1-v-r strategy. The hybrid SVM scheme introduced in our paper not only generalizes the performance of signal binary SVMs but improves the precision and reliability of the fault classification results. The actually testing results show the availability suitability of this new method.

Face recognition using an NNSRM classifier in LDA subspace

NNSRM is an implementation of the structural risk minimization (SRM) principle using the nearest neighbor (NN) rule, and linear discriminant analysis (LDA) is a dimension-reducing method, which is usually used in classifications. This paper combines the two methods for face recognition. We first project the face images into a PCA subspace, then project the results into a much lower-dimensional LDA subspace, and then use an NNSRM classifier to recognize them in the LDA subspace. Experimental results demonstrate that the combined method can achieve a better performance than NN by selecting different distances and a comparable performance with SVM but costing less computational time.

Support Vector Regression Model for Millimeter Wave Transitions

In this paper, we introduce a new method, support vector regression (SVR) method, to model millimeter wave transitions. SVR is based on the structural risk minimization (SRM) principle, which leads to good generalization ability for regression problem. The SVR model can be electromagnetically developed with a set of training data and testing data which produced by the electromagnetic simulation. Two Ka-band millimeter wave transitions, i.e., waveguide to microstrip transition and coaxial to waveguide adapter, are used as examples to validate the method. Experimental results show that the developed SVR models have a good predictive ability, and they are useful for interactive CAD of millimeter wave transitions.

Support Vector Echo-State Machine for Chaotic Time-Series Prediction

A novel chaotic time-series prediction method based on support vector machines (SVMs) and echo-state mechanisms is proposed. The basic idea is replacing "kernel trick" with "reservoir trick" in dealing with nonlinearity, that is, performing linear support vector regression (SVR) in the high-dimension "reservoir" state space, and the solution benefits from the advantages from structural risk minimization principle, and we call it support vector echo-state machines (SVESMs). SVESMs belong to a special kind of recurrent neural networks (RNNs) with convex objective function, and their solution is global, optimal, and unique. SVESMs are especially efficient in dealing with real life nonlinear time series, and its generalization ability and robustness are obtained by regularization operator and robust loss function. The method is tested on the benchmark prediction problem of Mackey-Glass time series and applied to some real life time series such as monthly sunspots time series and runoff time series of the Yellow River, and the prediction results are promising.

KScore: a novel machine learning approach that is not dependent on the data structure of the training set

Currently machine learning approaches used in Quantitative Structure Activity Relationship (QSAR) model generation impose restrictions and/or make assumptions on how the training set descriptors correlate with a target activity. kScore has been developed as the first machine learning approach that does not require the training data to conform to a defined kernel, accommodates uneven data point distributions in the descriptor space, and optimizes the weight of each dimension in the descriptor space in order to identify the descriptors most relevant to the target property. The ability of kScore to adapt to virtually any correlation makes it essential that generalization terms be included to inhibit overtraining. The Structural Risk Minimization principle and the linear epsilon-insensitive loss terms have been added to the kScore optimization function. The resulting kScore algorithm has proven to be quite universal across several datasets and either produces results similar to or outperforms the most predictive machine learning algorithms tested, such as SVM, kNN, Recursive Partitioning, Neural Networks, Gaussian Process, and the Bayesian Classifier.

DESIGN OF DECISION TREE VIA KERNELIZED HIERARCHICAL CLUSTERING FOR MULTICLASS SUPPORT VECTOR MACHINES

As a very effective method for universal purpose pattern recognition, support vector machine (SVM) was proposed for dichotomic classification problem, which exhibits a remarkable resistance to overfitting, a feature explained by the fact that it directly implements the principle of structural risk minimization. However, in real world, most of classification problems consist of multiple categories. In an attempt to extend the binary SVM classifier for multiclass classification, decision-tree-based multiclass SVM was proposed recently, in which the structure of decision tree plays an important role in minimizing the classification error. The present study aims at developing a systematic way for the design of decision tree for multiclass SVM. Kernel-induced distance function between datasets was discussed and then kernelized hierarchical clustering was developed and used in determining the structure of decision tree. Further, simulation results on satellite image interpretation show the superiority of the proposed classification strategy over the conventional multiclass SVM algorithms.

Hybridizing mixtures of experts with support vector machines: Investigation into nonlinear dynamic systems identification

Mixture of experts (ME) models comprise a family of modular neural network architectures aiming at distilling complex problems into simple subtasks. This is done by deploying a separate gating module for softly dividing the input space into overlapping regions to be each assigned to one or more expert networks. Conversely, support vector machines (SVMs) refer to kernel-based methods, neural-network-alike models that constitute an approximate implementation of the structural risk minimization principle. Such learning machines follow the simple, but powerful idea of nonlinearly mapping input data into high-dimensional feature spaces wherein a linear decision surface discriminating different regions is properly designed. In this work, we formally characterize and empirically evaluate a novel approach, named as Mixture of Support Vector Machine Experts (MSVME), whose main purpose is to combine the complementary properties of both SVM and ME models. In the formal characterization, an algorithm based on a maximum likelihood criterion is considered for the MSVME training, and we demonstrate that it is possible to train each expert based on an SVM perspective. Regarding the empirical evaluation, simulation results involving nonlinear dynamic system identification problems are reported, contrasting the performance shown by the MSVME approach with that exhibited by conventional SVM and ME models.

Multi-fault diagnosis for turbo-pump based on mesh support vector machines

Support vector machine(SVM) is a new general machine-learning tool based on structural risk minimization principle that exhibits good generalization even when fault samples are few. Fault diagnosis based on support vector ma- chines is discussed. Since basic support vector machines is originally designed for two-class classification, while most of fault diagnosis problems are multi-class cases, a new multi-class classification algorithm named mesh support vector machines is presented to solve the multi-class recognition prob- lems. It is a mesh classifier in which every class constructs two-class SVM classifiers with less than 4 other classes. It is simple and extensible, and has little repeated training amount, so the rate of training and recognition is expedited. The effec- tiveness of the method is verified by the application to the multi-fault diagnosis for turbo pump test bed.

A Model for Prediction of Activity of Insecticidal Crystal Proteins from Bacillus thuringiensis Based on Support Vector Machine

A quantitative structure-property relationship (QSPR) model in terms of amino acid composition and the activity of Bacillus thuringiensis insecticidal crystal proteins was established. Support vector machine (SVM) is a novel general machine-learning tool based on the structural risk minimization principle that exhibits good generalization when fault samples are few; it is especially suitable for classification, forecasting, and estimation in cases where small amounts of samples are involved such as fault diagnosis; however, some parameters of SVM are selected based on the experience of the operator, which has led to decreased efficiency of SVM in practical application. The uniform design (UD) method was applied to optimize the running parameters of SVM. It was found that the average accuracy rate approached 73 % when the penalty factor was 0.01, the epsilon 0.2, the gamma 0.05, and the range 0.5. The results indicated that UD might be used as an effective method to optimize the parameters of SVM and SVM and could be used as an alternative powerful modeling tool for QSPR studies of the activity of Bacillus thuringiensis (Bt) insecticidal crystal proteins. Therefore, a novel method for predicting the insecticidal activity of Bt insecticidal crystal proteins was proposed by the authors of this study.

Structural Risk Minimisation based gene expression profiling analysis

For microarray based cancer classification, feature selection is a common method for improving classifier generalisation. Most wrapper methods use cross validation methods to evaluate feature sets. For small sample problems like microarray, however, cross validation methods may overfit the data. In this paper, we propose a Structural Risk Minimisation (SRM) based method for gene selection in cancer classification. SRM principle allows for reducing the probable bound on generalisation error and thus avoids overfitting problems. The experimental results show that the proposed method produces significantly better performance than general wrapper methods that use cross validations.

LTCC INTERCONNECT MODELING BY SUPPORT VECTOR REGRESSION

In this paper, we introduce a new method: support vector regression (SVR) method to modeling low temperature co-fired ceramic (LTCC) multilayer interconnect. SVR bases on structural risk minimization (SRM) principle, which leads to good generalization ability. A LTCC based stripline-to-stripline interconnect used as example to verify the proposed method. Experiment results show that the developed SVR model perform a good predictive ability in analyzing the electrical performance.

APPLICATION OF SUPPORT VECTOR MACHINES TO GLOBAL PREDICTION OF NUCLEAR PROPERTIES

Advances in statistical learning theory present the opportunity to develop statistical models of quantum many-body systems exhibiting remarkable predictive power. The potential of such "theory-thin" approaches is illustrated with the application of Support Vector Machines (SVMs) to global prediction of nuclear properties as functions of proton and neutron numbers Z and N across the nuclidic chart. Based on the principle of structural-risk minimization, SVMs learn from examples in the existing database of a given property Y, automatically and optimally identify a set of "support vectors" corresponding to representative nuclei in the training set, and approximate the mapping (Z, N) → Y in terms of these nuclei. Results are reported for nuclear masses, beta-decay lifetimes, and spins/parities of nuclear ground states. These results indicate that SVM models can match or even surpass the predictive performance of the best conventional "theory-thick" global models based on nuclear phenomenology.

Modeling of 3-D Vertical Interconnect Using Support Vector Machine Regression

In this letter, the support vector machine (SVM) regression approach is introduced to model the three-dimensional (3-D) high density microwave packaging structure. The SVM is based on the structural risk minimization principle, which leads to a good generalization ability. With a 3-D vertical interconnect used as an example, the SVM regression model is electromagnetically developed with a set of training data and testing data, which is produced by the electromagnetic simulation. Experimental results suggest that the developed model performs with a good predictive ability in analyzing the electrical performance

Forecasting: Adopting the Methodology of Support Vector Machines to Nursing Research

In nursing studies, linear statistical methods are commonly used to analyze data on subjective perceptions. Most of the collected data include participants’ subjective perceptions and are usually not collected in a controlled environment. For linear regression analyses, there is a common understanding that the distributions of some studied variables might not be close to linear in nature. However, when using the support vector machine (SVM), there is no violation concern for the regression assumption of linearity. SVM, as an artificial intelligence learning method, is based on the structural risk minimization principle that relies on the computational learning theory. It is a powerful technique for nonlinear regression or pattern recognition, which can learn complex patterns or trends from input data (independent variables) and create outputs (dependent variables). A trained learning machine can then generate output for unseen data. The quality of SVM was determined by the machine’s training error rate and generation capability. Choosing the approaches of nonlinear regression or pattern recognition is based on the characteristics of the dependent variable (Cristianini & ShaweTaylor 2000; Figure 1). How does SVM differ from the statistical methods? How could nursing researchers adopt SVM to address nursing issues? In this commentary, these two questions are answered. The process of developing SVM and three example studies using SVM are introduced in the following.

A robust deterministic annealing algorithm for data clustering

In this paper, a novel robust deterministic annealing (RDA) algorithm is developed for data clustering. This method takes advantage of conventional noise clustering (NC) and deterministic annealing (DA) algorithms in terms of the independence of data initialization, the ability to avoid poor local optima, the better performance for unbalanced data, and the robustness against noise and outliers. In addition, a cluster validity criterion, i.e., Vapnik–Chervonenkis (VC)-bound induced index, which is estimated based on the structural risk minimization (SRM) principle, is specifically extended for RDA to determine the optimal cluster number for a given data set. The superiority of the proposed RDA clustering algorithm is supported by experimental results.