Nonlinear Support Vector Research Articles

Data-driven prediction models of multi-dimensional energy consumed in public buildings

Due to the intense pressure from energy shortage and environmental protection, an accurate prediction of building energy consumption is crucial for different energy conservation applications and policies. Besides simulation models and traditional statistical approaches, a data-driven modelling based on energy records provides new opportunities for predicting the building energy demand. This research is conducted based on the whole procedure of data mining with limited datasets, by making use of machine learning techniques and mathematical statistics. Especially, regarding the temporal and the architectural scales, models can be categorized into the short-term prediction, medium-term prediction and long-term prediction of classified energy consumptions, which also represent different modelling characteristics derived from mass data, limited data and poor data respectively. During the modelling process, the fuzzy C-means clustering and the interdisciplinary Lorenz curve were utilized to recognize different energy patterns. Afterwards, models of the nonlinear Support Vector Regression, the Grey model and the traditional polynomial regression were utilized respectively to output the predicted sequence. In summary, with datasets in current energy platforms, this paper presents a study of data-driven models based on energy records considering the nonlinear and uncertain features of different multi-dimensional models.

Neutron-gamma discrimination based on support vector machine combined to nonnegative matrix factorization and continuous wavelet transform

Recent developments of digital signal processing have played an effective role to achieve a fast and accurate neutron-gamma discrimination. Thus, we present in this research work, a novel method which combines supervised and unsupervised machine learning to perform the neutron-gamma discrimination task at the output of a stilbene organic scintillation detector. We propose a three steps procedure that highly qualifies the discrimination. First, the detector’s output signals are processed as mixtures of several unknown sources, through nonnegative matrix factorization algorithms. Second, the continuous wavelet transform is performed to characterize the recovered original sources. The resulting time-scale representation is considered as an image which is segmented in order to extract main features of neutron signals versus the gamma ones. The features are then used as input of a nonlinear support vector machine classifier to finally achieve the neutron-gamma discrimination in mixed radiation field. Furthermore, the proposed method provides the classification precision for each radiation.

Robustness of Support Vector Machines in Algorithmic Trading on Cryptocurrency Market

Abstract This study investigates the profitability of an algorithmic trading strategy based on training SVM model to identify cryptocurrencies with high or low predicted returns. A tail set is defined to be a group of coins whose volatility-adjusted returns are in the highest or the lowest quintile. Each cryptocurrency is represented by a set of six technical features. SVM is trained on historical tail sets and tested on the current data. The classifier is chosen to be a nonlinear support vector machine. The portfolio is formed by ranking coins using the SVM output. The highest ranked coins are used for long positions to be included in the portfolio for one reallocation period. The following metrics were used to estimate the portfolio profitability: %ARC (the annualized rate of change), %ASD (the annualized standard deviation of daily returns), MDD (the maximum drawdown coefficient), IR1, IR2 (the information ratio coefficients). The performance of the SVM portfolio is compared to the performance of the four benchmark strategies based on the values of the information ratio coefficient IR1, which quantifies the risk-weighted gain. The question of how sensitive the portfolio performance is to the parameters set in the SVM model is also addressed in this study.

The Use of Artificial Intelligence as a Tool Supporting Sustainable Development Local Policy

This paper addresses the problem of noise in spa protection areas. Its aim is to determine the delimitation of the areas that exceed a permissible noise level around the sanatorium on the example of a health resort in Inowrocław. The determination of the exceedance of permissible noise levels allows us to develop directly effective local policy tools to be included in planning documents. In order to reduce noise infiltration, it is important to define environmental priorities. Taking into account their impact on the health of users in the protection area, environmental priorities enable us to introduce additional elements to street architecture. In order to properly manage space, in accordance with the idea of sustainable development, zones of environmental sensitivity—and their socio-environmental vulnerability—have been designated for assessing damage (exceeding permissible noise in health facilities) and defining methods of building resilience (proper management). This has provided the basis for a natural balance optimized for the people living in these areas. To achieve the goal above, non-linear support vector machine (SVM) networks were used. This technique allows us to classify the linearly inseparable data and to determine the optimal separation margin. The boundaries of the areas which exceeded permissible noise levels (separation margin) were estimated on the basis of noise pollution maps, created by means of the SVM technique. Thus, the study results in establishing buffer zones where it is possible to use varied land utilization in terms of form and function, as described in the planning documents. Such an activity would limit the spread of noise.

Robust face anti‐spoofing using CNN with LBP and WLD

Automatically recognising people by their biometric characteristics is a well-established research area. Biometric systems are vulnerable to many different types of presentation attacks made by persons showing photo, video, or mask to spoof the real identity. This study introduces a novel approach to detect face-spoofing, by extracting the local features local binary pattern (LBP) and simplified weber local descriptor (SWLD) encoded convolutional neural network (CNN) models, WLD and LBP features are combined together to ensure the preservation of the local intensity information and the orientations of the edges. These two components are complementary to each other. Specifically, differential excitation preserves the local intensity information but omits the orientations of edges. On the contrary, LBP describes the orientations of the edges but ignore the intensity information, the proposed approach presents a very low degree of complexity which makes it suitable for real-time applications, Finally, a non-linear support vector machine (SVM) classifier with kernel function was used for determining whether the input image corresponds to a live face or not. Authors’ experimental analysis on two publicly available databases REPLAY-ATTACK and CASIA face anti-spoofing showed that their approach performs better than state-of-the-art techniques following the provided evaluation protocols of each database.

Optimization of soluble solids content prediction models in ‘Hami’ melons by means of Vis-NIR spectroscopy and chemometric tools

Nondestructive assessment of internal quality in fruit with thick rind and large size was always a challenge in terms of the fruit quality automatic grading application. This study compared different spectral measurement positions and prediction models for determining soluble solids content (SSC) of ‘Hami’ melon. Visible and short wavelength near-infrared with the spectral range of 550–950 nm was used and three local positions (calyx, equator and stem regions) in each sample were selected for spectral collection, respectively. The partial least squares (PLS) was employed to establish three local calibration models (calyx-model, equator-model and stem-model) and calyx-model was proved to be optimal one for SSC prediction of ‘Hami’ melon. In order to optimize the calyx-model, four algorithms including successive projections algorithm (SPA), Monte Carlo-uninformative variable elimination (MC-UVE), competitive adaptive reweighted sampling (CARS) and combination algorithm MC-UVE-SPA were applied to select effective variables from full spectra, respectively. And, the non-linear least square support vector machine (LSSVM), linear multiple linear regression (MLR) and PLS models were also developed based on those selected effective variables, respectively. The results indicated that MC-UVE-SPA was the optimal method for selecting the effective variable. Based on 18 variables selected by MC-UVE-SPA, all established models including MC-UVE-SPA-PLS, MC-UVE-SPA-LSSVM and MC-UVE-SPA-MLR can successfully predict the SSC of all ‘Hami’ melon fruit samples with the prediction accuracy of RMSEP = 0.95–0.99 °Brix.

An Ensemble Approach for Classification of Breast Histopathology Images

This paper presents an approach to classify breast histopathology images based on the mitotic count. An ensemble of handcrafted features, a bag of features (BoF), and convolutional neural network (CNN) are done for mitosis detection. Initially, mitotic regions are segmented followed by extraction of handcrafted features, BoF, and CNN-learned features from the candidate region. A non-linear support vector machine with Gaussian radial basis function kernel is used for the classification of each feature set independently. The proposed work utilizes the main tissue components of the images in an ensemble style to correctly classify the input histopathology image into benign or malignant. It also enables to maximize the performance by leveraging the disconnected feature set. Publicly available ICPR 2012 and 2014 mitosis dataset of is used for evaluation. The proposed ensemble method yielded a much improved F1 score. Better accuracy and fewer computing resources of our approach make it feasible for clinical use.

Breast Cancer Treatment Response Monitoring Using Quantitative Ultrasound and Texture Analysis: Comparative Analysis of Analytical Models

PURPOSE: The purpose of this study was to develop computational algorithms to best determine tumor responses early after the start of neoadjuvant chemotherapy, based on quantitative ultrasound (QUS) and textural analysis in patients with locally advanced breast cancer (LABC). METHODS: A total of 100 LABC patients treated with neoadjuvant chemotherapy were included in this study. Breast tumors were scanned with a clinical ultrasound system prior to treatment, during the first, fourth and eighth weeks of treatment, and prior to surgery. QUS parameters were calculated from ultrasound radio frequency data within tumor regions. Texture features were extracted from each QUS parametric map. Patients were classified into two groups based on identified clinical/pathological response: responders and non-responders. In order to differentiate treatment responders, three multi-feature response classification algorithms, namely a linear discriminant, a k-nearest-neighbor and a nonlinear support vector machine classifier were compared. RESULTS: All algorithms distinguished responders and non-responders with accuracies ranging between 68% and 92%. In particular, support vector machine performed the best in differentiating responders from non-responders with accuracies of 78%, 90% and 92% at weeks 1, 4 and 8 after the start of treatment, respectively. The most relevant features in separating the two response groups at early stages (weeks 1and 4) were texture features and at a later stage (week 8) were mean QUS parameters, particularly ultrasound backscatter intensity-based parameters. CONCLUSION: An early stage treatment response prediction model developed by quantitative ultrasound and texture analysis combined with modern computational methods permits offering effective alternatives to standard treatment for refractory patients.

Explaining and predicting the impact of authors within a community: an assessment of the bibliometric literature and application of machine learning

AbstractFollowing widespread availability of computerized databases, much research has correlated bibliometric measures from papers or patents to subsequent success, typically measured as the number of publications or citations. Building on this large body of work, we ask the following questions: given available bibliometric information in one year, along with the combined theories on sources of creative breakthroughs from the literatures on creativity and innovation, how accurately can we explain the impact of authors in a given research community in the following year? In particular, who is most likely to publish, publish highly cited work, and even publish a highly cited outlier? And, how accurately can these existing theories predict breakthroughs using only contemporaneous data? After reviewing and synthesizing (often competing) theories from the literatures, we simultaneously model the collective hypotheses based on available data in the year before RNA interference was discovered. We operationalize author impact using publication count, forward citations, and the more stringent definition of being in the top decile of the citation distribution. Explanatory power of current theories altogether ranges from less than 9% for being top cited to 24% for productivity. Machine learning (ML) methods yield similar findings as the explanatory linear models, and tangible improvement only for non-linear Support Vector Machine models. We also perform predictions using only existing data until 1997, and find lower predictability than using explanatory models. We conclude with an agenda for future progress in the bibliometric study of creativity and look forward to ML research that can explain its models.

Assessment of lipid peroxidation and artificial neural network models in early Alzheimer Disease diagnosis

Objective: Lipid peroxidation constitutes a molecular mechanism involved in early Alzheimer Disease (AD) stages, and artificial neural network (ANN) analysis is a promising non-linear regression model, characterized by its high flexibility and utility in clinical diagnosis. ANN simulates neuron learning procedures and it could provide good diagnostic performances in this complex and heterogeneous disease compared with linear regression analysis. Design and Methods: In our study, a new set of lipid peroxidation compounds were determined in urine and plasma samples from patients diagnosed with early Alzheimer Disease (n = 70) and healthy controls (n = 26) by means of ultra-performance liquid chromatography coupled with tandem mass-spectrometry. Then, a model based on ANN was developed to classify groups of participants. Results: The diagnostic performances obtained using an ANN model for each biological matrix were compared with the corresponding linear regression model based on partial least squares (PLS), and with the non-linear (radial and polynomial) support vector machine (SVM) models. Better accuracy, in terms of receiver operating characteristic-area under curve (ROC-AUC), was obtained for the ANN models (ROC-AUC 0.882 in plasma and 0.839 in urine) than for PLS and SVM models. Conclusion: Lipid peroxidation and ANN constitute a useful approach to establish a reliable diagnosis when the prognosis is complex, multidimensional and non-linear.

Bias-corrected support vector machine with Gaussian kernel in high-dimension, low-sample-size settings

In this paper, we study asymptotic properties of nonlinear support vector machines (SVM) in high-dimension, low-sample-size settings. We propose a bias-corrected SVM (BC-SVM) which is robust against imbalanced data in a general framework. In particular, we investigate asymptotic properties of the BC-SVM having the Gaussian kernel and compare them with the ones having the linear kernel. We show that the performance of the BC-SVM is influenced by the scale parameter involved in the Gaussian kernel. We discuss a choice of the scale parameter yielding a high performance and examine the validity of the choice by numerical simulations and actual data analyses.

Indoor Scene recognition for Micro Aerial Vehicles Navigation using Enhanced SIFT-ScSPM Descriptors

In this paper, a new scene recognition visual descriptor called Enhanced Scale Invariant Feature Transform-based Sparse coding Spatial Pyramid Matching (Enhanced SIFT-ScSPM) descriptor is proposed by combining a Bag of Words (BOW)-based visual descriptor (SIFT-ScSPM) and Gist-based descriptors (Enhanced Gist-Enhanced multichannel Gist (Enhanced mGist)). Indoor scene classification is carried out by multi-class linear and non-linear Support Vector Machine (SVM) classifiers. Feature extraction methodology and critical review of several visual descriptors used for indoor scene recognition in terms of experimental perspectives have been discussed in this paper. An empirical study is conducted on the Massachusetts Institute of Technology (MIT) 67 indoor scene classification data set and assessed the classification accuracy of state-of-the-art visual descriptors and the proposed Enhanced mGist, Speeded Up Robust Features-Spatial Pyramid Matching (SURF-SPM) and Enhanced SIFT-ScSPM visual descriptors. Experimental results show that the proposed Enhanced SIFT-ScSPM visual descriptor performs better with higher classification rate, precision, recall and area under the Receiver Operating Characteristic (ROC) curve values with respect to the state-of-the-art and the proposed Enhanced mGist and SURF-SPM visual descriptors.

Development of a health monitoring and diagnosis framework for fused deposition modeling process based on a machine learning algorithm

In this article, a data-driven approach is applied to develop a health monitoring and diagnosis framework for a fused deposition modeling process based on a machine learning algorithm. For the data-driven approach, three accelerometers, an acoustic emission sensor, and three thermocouples are installed, and associated data are collected from those sensors. The collected data are processed to obtain root mean square values, and they are used for constructing health monitoring and diagnosis models for the fused deposition modeling process based on a support vector machine algorithm, which is one of machine learning algorithms. Among various root mean square values, those of acceleration data from the frame were most effective for diagnosing health states of the fused deposition modeling process with the non-linear support vector machine–based model.

A systematic analysis of genomics-based modeling approaches for prediction of drug response to cytotoxic chemotherapies

BackgroundThe availability and generation of large amounts of genomic data has led to the development of a new paradigm in cancer treatment emphasizing a precision approach at the molecular and genomic level. Statistical modeling techniques aimed at leveraging broad scale in vitro, in vivo, and clinical data for precision drug treatment has become an active area of research. As a rapidly developing discipline at the crossroads of medicine, computer science, and mathematics, techniques ranging from accepted to those on the cutting edge of artificial intelligence have been utilized. Given the diversity and complexity of these techniques a systematic understanding of fundamental modeling principles is essential to contextualize influential factors to better understand results and develop new approaches.MethodsUsing data available from the Genomics of Drug Sensitivity in Cancer (GDSC) and the NCI60 we explore principle components regression, linear and non-linear support vector regression, and artificial neural networks in combination with different implementations of correlation based feature selection (CBF) on the prediction of drug response for several cytotoxic chemotherapeutic agents.ResultsOur results indicate that the regression method and features used have marginal effects on Spearman correlation between the predicted and measured values as well as prediction error. Detailed analysis of these results reveal that the bulk relationship between tissue of origin and drug response is a major driving factor in model performance.ConclusionThese results display one of the challenges in building predictive models for drug response in pan-cancer models. Mainly, that bulk genotypic traits where the signal to noise ratio is high is the dominant behavior captured in these models. This suggests that improved techniques of feature selection that can discriminate individual cell response from histotype response will yield more successful pan-cancer models.

Machine-Learning-Based Prediction of Treatment Outcomes Using MR Imaging-Derived Quantitative Tumor Information in Patients with Sinonasal Squamous Cell Carcinomas: A Preliminary Study

The purpose of this study was to determine the predictive power for treatment outcome of a machine-learning algorithm combining magnetic resonance imaging (MRI)-derived data in patients with sinonasal squamous cell carcinomas (SCCs). Thirty-six primary lesions in 36 patients were evaluated. Quantitative morphological parameters and intratumoral characteristics from T2-weighted images, tumor perfusion parameters from arterial spin labeling (ASL) and tumor diffusion parameters of five diffusion models from multi-b-value diffusion-weighted imaging (DWI) were obtained. Machine learning by a non-linear support vector machine (SVM) was used to construct the best diagnostic algorithm for the prediction of local control and failure. The diagnostic accuracy was evaluated using a 9-fold cross-validation scheme, dividing patients into training and validation sets. Classification criteria for the division of local control and failure in nine training sets could be constructed with a mean sensitivity of 0.98, specificity of 0.91, positive predictive value (PPV) of 0.94, negative predictive value (NPV) of 0.97, and accuracy of 0.96. The nine validation data sets showed a mean sensitivity of 1.0, specificity of 0.82, PPV of 0.86, NPV of 1.0, and accuracy of 0.92. In conclusion, a machine-learning algorithm using various MR imaging-derived data can be helpful for the prediction of treatment outcomes in patients with sinonasal SCCs.

Early Detection of Human Epileptic Seizures Based on Intracortical Microelectrode Array Signals.

We examine, for the first time, the use of intracortical microelectrode array (MEA) signals for early detection of human epileptic seizures. 4×4 mm2 96-channel-MEA recordings were obtained during neuro-monitoring preceding resective surgery in five participants. The participant-specific seizure-detection framework consisted of: first, feature extraction from local field potentials (LFPs) and multiunit activity (MUA); second, nonlinear cost-sensitive support vector machine (SVM) classification of ictal and interictal states based on LFP, MUA, and combined LFP-MUA (a SVM was trained for each participant separately); and third, Kalman filter postprocessing of SVM scoring functions. Performance was assessed on data including 17 seizures and 39.0h interictal and preictal recordings. The use of combined LFP-MUA features resulted in 100% sensitivity with short detection latency (average: 2.7 s; median: 2.5 s) and five false alarms (0.13/h). The average detection performance based on the area under the receiver operating characteristic corresponded to 0.97. Importantly, technically false alarms were related to epileptiform activity, subclinical seizures, and recording artifacts. Extreme gradient boosting classifiers ranked features based on LFP spectral coherence or MUA count among the top features for seizures characterized by spike-wave complexes, whereas features related to LFP power spectra were ranked higher for seizures characterized by sustained gamma LFP oscillations. The combination of intracortical LFP and MUA signals may allow reliable detection of human epileptic seizures by improving latency and false alarm rate. Intracortical MEAs provide promising signals for closed-loop seizure-control systems based on seizure early-detection in people with pharmacologically resistant epilepsies.

Multi-parameter safe sample elimination rule for accelerating nonlinear multi-class support vector machines

K-SVCR and Twin-KSVC are two novel algorithms to deal with multi-class problems. They have achieved good performance since they evaluate all training samples into a “1-versus-1-versus-rest” structure. But they are extremely time consuming, so it remains challenging to apply them into large-scale problems directly. Inspired by the sparse solution of SVMs, in this paper, we propose a safe sample elimination rule (SSE) for multi-class classifiers K-SVCR and Twin-KSVC, termed as SSE-K-SVCR and SSE-T-KSVC, to reduce computation time. With our rule, many redundant samples of all classes can be identified and deleted before actually solving the problem, so the scale of dual problems can be reduced a lot. And our methods are safe, i.e., they can derive identical optimal solutions as K-SVCR and Twin-KSVC, respectively. So the testing accuracy keeps unchanged. Besides, the methods can be embedded into grid search method to accelerate the whole training process, and they are effective both for penalty parameter and kernel parameter. Finally, a fast algorithm clipDCD is introduced to reduce the computation time for larger datatset. Experimental results on one artificial dataset and seventeen benchmark datasets demonstrate the effectiveness and safety of our proposed methods.

Performance evaluation of DWT based sigmoid entropy in time and frequency domains for automated detection of epileptic seizures using SVM classifier

The electroencephalogram (EEG) signal contains useful information on physiological states of the brain and has proven to be a potential biomarker to realize the complex dynamic behavior of the brain. Epilepsy is a brain disorder described by recurrent and unpredictable interruption of healthy brain function. Diagnosis of patients with epilepsy requires monitoring and visual inspection of long-term EEG by the neurologist, which is found to be a time-consuming procedure. Therefore, this study proposes an automated seizure detection model using a novel computationally efficient feature named sigmoid entropy derived from discrete wavelet transforms. The sigmoid entropy was estimated from the wavelet coefficients in each sub-band and classified using a non-linear support vector machine classifier with leave-one-subject-out cross-validation. The performance of the proposed method was tested with the Ramaiah Medical College and Hospital (RMCH) database, which consists of the 58 Hours of EEG from 115 subjects, the University of Bonn (UBonn), and CHB-MIT databases. Results showed that sigmoid entropy exhibits lower values for epileptic EEG in contrary to other existing entropy methods. We observe a seizure detection rate of 96.34%, a false detection rate of 0.5/h and a mean detection delay of 1.2 s for the RMCH database. The highest sensitivity of 100% and 94.21% were achieved for UBonn and CHB-MIT databases respectively. The performance comparison confirms that sigmoid entropy was found to be better and computationally efficient as compared to other entropy methods. It can be concluded that the proposed sigmoid entropy could be used as a potential biomarker for recognition and detection of epileptic seizures.

Non-Intrusive Detection of Drowsy Driving Based on Eye Tracking Data

Drowsy driving is one of the leading causes of motor vehicle accidents in North America. This paper presents the use of eye tracking data as a non-intrusive measure of driver behavior for detection of drowsiness. Eye tracking data were acquired from 53 subjects in a simulated driving experiment, whereas the simultaneously recorded multichannel electroencephalogram (EEG) signals were used as the baseline. A random forest (RF) and a non-linear support vector machine (SVM) were employed for binary classification of the state of vigilance. Different lengths of eye tracking epoch were selected for feature extraction, and the performance of each classifier was investigated for every epoch length. Results revealed a high accuracy for the RF classifier in the range of 88.37% to 91.18% across all epoch lengths, outperforming the SVM with 77.12% to 82.62% accuracy. A feature analysis approach was presented and top eye tracking features for drowsiness detection were identified. Altogether, this study showed a high correspondence between the extracted eye tracking features and EEG as a physiological measure of vigilance and verified the potential of these features along with a proper classification technique, such as the RF, for non-intrusive long-term assessment of drowsiness in drivers. This research would ultimately lead to development of technologies for real-time assessment of the state of vigilance, providing early warning of fatigue and drowsiness in drivers.

A machine-learning–based prediction model of fistula formation after interstitial brachytherapy for locally advanced gynecological malignancies

PurposeExternal beam radiotherapy combined with interstitial brachytherapy is commonly used to treat patients with bulky, advanced gynecologic cancer. However, the high radiation dose needed to control the tumor may result in fistula development. There is a clinical need to identify patients at high risk for fistula formation such that treatment may be managed to prevent this toxic side effect. This work aims to develop a fistula prediction model framework using machine learning based on patient, tumor, and treatment features. Methods and MaterialsThis retrospective study included 35 patients treated at our institution using interstitial brachytherapy for various gynecological malignancies. Five patients developed rectovaginal fistula and two developed both rectovaginal and vesicovaginal fistula. For each patient, 31 clinical features of multiple data types were collected to develop a fistula prediction framework. A nonlinear support vector machine was used to build the prediction model. Sequential backward feature selection and sequential floating backward feature selection methods were used to determine optimal feature sets. To overcome data imbalance issues, the synthetic minority oversampling technique was used to generate synthetic fistula cases for model training. ResultsSeven mixed data features were selected by both sequential backward selection and sequential floating backward selection methods. Our prediction model using these features achieved a high prediction accuracy, that is, 0.904 area under the curve, 97.1% sensitivity, and 88.5% specificity. ConclusionsA machine-learning–based prediction model of fistula formation has been developed for patients with advanced gynecological malignancies treated using interstitial brachytherapy. This model may be clinically impactful pending refinement and validation in a larger series.