Linear Decision Boundaries Research Articles

How well do rudimentary plasticity rules predict adult visual object learning?

A core problem in visual object learning is using a finite number of images of a new object to accurately identify that object in future, novel images. One longstanding, conceptual hypothesis asserts that this core problem is solved by adult brains through two connected mechanisms: 1) the re-representation of incoming retinal images as points in a fixed, multidimensional neural space, and 2) the optimization of linear decision boundaries in that space, via simple plasticity rules applied to a single downstream layer. Though this scheme is biologically plausible, the extent to which it explains learning behavior in humans has been unclear-in part because of a historical lack of image-computable models of the putative neural space, and in part because of a lack of measurements of human learning behaviors in difficult, naturalistic settings. Here, we addressed these gaps by 1) drawing from contemporary, image-computable models of the primate ventral visual stream to create a large set of testable learning models (n = 2,408 models), and 2) using online psychophysics to measure human learning trajectories over a varied set of tasks involving novel 3D objects (n = 371,000 trials), which we then used to develop (and publicly release) empirical benchmarks for comparing learning models to humans. We evaluated each learning model on these benchmarks, and found those based on deep, high-level representations from neural networks were surprisingly aligned with human behavior. While no tested model explained the entirety of replicable human behavior, these results establish that rudimentary plasticity rules, when combined with appropriate visual representations, have high explanatory power in predicting human behavior with respect to this core object learning problem.

Cluster-based data relabelling for classification

Linear classifiers are generally simpler and more explainable than their nonlinear variants. They can achieve satisfactory classification performance on linearly separable data, but not on nonlinear data. So, linear classifiers need extending, typically by modification of their algorithms, resulting in their nonlinear variants. In this paper we present one general method, cluster-based data relabelling (CBDR), that allows linear classifiers to work effectively on nonlinear data. CBDR partitions the data set into several non-overlapping class-specific clusters and relabels data by the clusters. A linear classifier can then be applied to the relabelled data to seek cluster-based linear decision boundaries instead of class-based decision boundaries. Extensive experimentation has demonstrated that CBDR can significantly enhance the classification performance of linear classifiers, and even outperform their nonlinear variants. Further experimentation has demonstrated that CBDR can also improve the classification performance of nonlinear classifiers. Most significant outperformance was observed on imbalanced data in both cases.

Comparison of linear and non-linear decision boundaries to detect feedlot bloat using intensive data collection systems on Angus × Hereford steers

Ruminal tympany (bloat) has long been an issue for large and small livestock operations. Though improvements in feedlot management practices have reduced its occurrence, it is still highly prevalent and is known to detrimentally affect animal performance, welfare, and in many instances, lead to animal death. Current decision support systems and diet formulation software omit the inclusion of bloat prediction based on animal performance. Here, we aim to predict bloat incidence in implanted and non-implanted feedlot steers from performance data comparing linear (LDB) and non-linear decision boundaries. Eighteen crossbred Angus × Hereford steers: BW (491.13 ± 25.78 kg) and age (12 ± 1 mo) were randomly distributed into implanted and non-implanted treatments. All animals were randomly assigned to one of two pens fit with automated monitoring systems for BW, freshwater intake, and water intake behavior: water intake event visit, no water intake event visit (NWIE), and time spent drinking. DM intake (DMI) was individually recorded from all animals through the Calan Gate system for 135 d (30 d adaptation, 105 d experimental diet). Incidences of bloat were recorded as bloat instances regardless of severity to ensure that early onset detection of bloat was recorded and properly identified in predictive models. Logistic regression with a binomial distribution and a logit link function was utilized to predict the incidences of bloat through LDB. Feature selection and penalization of coefficients were explored through L1 (sum of absolute values) and L2 (sum of squares) penalization to avoid overfitting of models. Additional NLDB and a non-parametric LDB are examined for prediction. Accuracy, specificity, and sensitivity were high for the models reported. No significant differences were observed between LDB and NLDB, with the highest specificity (predicting bloat) value of 0.820 for stepwise feature selection algorithms, and a value of 0.832 for the artificial neural network. Highest accuracy was 0.829 for ridge regression, and 0.847 for the random forest with hyperparameter tuning. DM intake, BW, and NWIE were the three most important variables for the prediction of feedlot bloat showing clear drops in DMI and BW and increases in NWIE when animals bloated. The lack of difference in predictive performance between LDB and NLDB highlights the often-overlooked concept that machine learning algorithms are not always the only/best modeling technique. Additionally, the models reported herein carry acceptable predictive performance for inclusion into management decisions that reduce bloat incidences in feedlot cattle.

One class SVM model based on neural tangent kernel for anomaly detection task on small-scale data

Recent studies have shown that the evolution of infinitely wide neural networks satisfying certain conditions can be described by a kernel function called neural tangent kernel (NTK). We introduce NTK into a one-class support vector machine model and select data from different domains in UCI for a small-sample outlier detection task, demonstrate that NTK-OCSVM generally outperforms a variety of commonly used classification models, with more than 20% improvement in accuracy for similar models. When the kernel function parameters are varied, the experiments show that the model has strong robustness within a certain parameter range. Finally, we experimentally compare the time complexity of different models and the decision boundaries, and demonstrate that NTK-OCSVM improves accuracy at the expense of operational efficiency and has linear decision boundaries.

An efficient approach for discriminant analysis based on adaptive feature augmentation

Effective discriminant analysis is of great practical importance, as demonstrated by economic and genetic applications. Feature augmentation via nonparametric and selection (FANS) is an efficient approach that has been widely used in classification. However, FANS may impair efficiency when a linear decision boundary separates data reasonably well. The available remedy is to use both the transformed features and original ones, which may increase computational cost and model complexity. Motivated by these concerns, this paper proposes an efficient nonparametric approach for binary discriminant analysis, called adaptive FANS, integrating augmentation and nonparametric tests. In this procedure, the original features or transformed ones are used selectively to keep the number of features constant. Thus, this procedure avoids the ergodic transformation and reduces error caused by nonparametric estimation and computational complexity. Simulation and real data analysis demonstrate its competitiveness and significant adaptability. Moreover, our approach can be easily extended to other linear frameworks.

Mushroom data creation, curation, and simulation to support classification tasks

Predicting if a set of mushrooms is edible or not corresponds to the task of classifying them into two groups—edible or poisonous—on the basis of a classification rule. To support this binary task, we have collected the largest and most comprehensive attribute based data available. In this work, we detail the creation, curation and simulation of a data set for binary classification. Thanks to natural language processing, the primary data are based on a text book for mushroom identification and contain 173 species from 23 families. While the secondary data comprise simulated or hypothetical entries that are structurally comparable to the 1987 data, it serves as pilot data for classification tasks. We evaluated different machine learning algorithms, namely, naive Bayes, logistic regression, and linear discriminant analysis (LDA), and random forests (RF). We found that the RF provided the best results with a five-fold Cross-Validation accuracy and F2-score of 1.0 (mu =1, sigma =0), respectively. The results of our pilot are conclusive and indicate that our data were not linearly separable. Unlike the 1987 data which showed good results using a linear decision boundary with the LDA. Our data set contains 23 families and is the largest available. We further provide a fully reproducible workflow and provide the data under the FAIR principles.

A Novel Linear Classifier for Class Imbalance Data Arising in Failure-Prone Air Pressure Systems

An Air Pressure System (APS) is one of the crucial components of an automobile. Its failure leads to financial loses, and it may lead to loss of lives. Thus predicting such failure is a critical problem that requires a rigorous solution. Recently, many researchers have presented machine learning techniques to deal with APS failure detection. One of the major challenges in dealing with APS failure data is the presence of high class imbalance. Conventional classification criteria may not be able to efficiently handle such data. In this paper, a new machine learning method for APS failure detection is proposed. It is designed to specifically deal with the class imbalance. The method uses a linear decision boundary by maximizing Area Under the Curve (maxAUC) criterion. The proposed method was experimentally validated on an industrial dataset of APS failure. The results of the proposed method are thoroughly compared with existing linear as well as non-linear classifiers.

Multi-marker quantitative radiomics for mass characterization in dedicated breast CT imaging.

PurposeTo develop and evaluate the diagnostic performance of an algorithm for multi‐marker radiomic‐based classification of breast masses in dedicated breast computed tomography (bCT) images.MethodsOver 1000 radiomic descriptors aimed at quantifying mass and border heterogeneity, morphology, and margin sharpness were developed and implemented. These included well‐established texture and shape feature descriptors, which were supplemented with additional approaches for contour irregularity quantification, spicule and lobe detection, characterization of degree of infiltration, and differences in peritumoral compartments. All descriptors were extracted from a training set of 202 bCT masses (133 benign and 69 malignant), and their individual diagnostic performance was investigated in terms of area under the receiver operating characteristics (ROC) curve (AUC) of single‐feature‐based linear discriminant analysis (LDA) classifiers. Subsequently, the most relevant descriptors were selected through a multiple‐step feature selection process (including stability analysis, statistical significance, evaluation of feature interaction, and dimensionality reduction), and used to develop a final LDA radiomic model for classification of benign and malignant masses, which was then tested on an independent test set of 82 cases (45 benign and 37 malignant).ResultsThe majority of the individual radiomic descriptors showed, on the training set, an AUC value deriving from a linear decision boundary higher than 0.65, with the lower limit of the associated 95% confidence interval (C.I.) not overlapping with random chance (AUC = 0.5). The final LDA radiomic model resulted in a test set AUC of 0.90 (95% C.I. 0.80–0.96).ConclusionsThe proposed multi‐marker radiomic approach achieved high diagnostic accuracy in bCT mass classification, using a radiomic signature based on different feature types. While future studies with larger datasets are needed to further validate these results, quantitative radiomics applied to bCT shows potential to improve the breast cancer diagnosis pipeline.

Machine Learning Algorithm for Gait Analysis and Classification on Early Detection of Parkinson

Gait analysis using kinetic data such as pressure distribution underneath the foot has been a topic of interest for assessing falls in elderly and certain pathology such as Parkinson's disease. The disease which affects the central nervous system cannot be ultimately diagnosed by a test. In this letter, we describe a detection algorithm able to classify subjects into Parkinson or normal subjects based on load distribution during gait. This will allow those with the disease to benefit from early detection and thus early treatment. We perform spatial and time signal analyses over vertical ground reaction forces to categorize gaits as balanced or unbalanced, where unbalanced gaits correspond to subjects with Parkinson's disease and balanced gaits could be relevant to both normal and diseased subjects. Then simple features like correlation are used to further differentiate between balanced-normal subjects and balanced-diseased subjects. A 95% overall classification accuracy has been achieved using a linear decision boundary. This letter can be employed to form the basis of designing a portable device for early Parkinson's disease detection on a real-time basis. Moreover, it can be used for evaluation purposes of a rehabilitation program.

A Machine Learning-Based Data Fusion Approach for Improved Corrosion Testing

This work presents machine learning-inspired data fusion approaches to improve the nondestructive testing of reinforced concrete. The principal effects that are used for data fusion are shown theoretically. Their effectiveness is tested in case studies carried out on large-scale concrete specimens with built-in chloride-induced rebar corrosion. The dataset consists of half-cell potential mapping, Wenner resistivity, microwave moisture and ground penetrating radar measurements. Data fusion is based on the logistic regression algorithm. It learns an optimal linear decision boundary from multivariate labeled training data, to separate intact and defect areas. The training data are generated in an experiment that simulates the entire life cycle of chloride-exposed concrete building parts. The unique possibility to monitor the deterioration, and targeted corrosion initiation, allows data labeling. The results exhibit an improved sensitivity of the data fusion with logistic regression compared to the best individual method half-cell potential.

(Invited) Single Nanoarray Sensor for Multi-Analyte Detection

Chemical sensors usually adopted a single resistance-metric mode, which has a drawback of significant cross-sensitivities in mixed analytes scenario, thus showing a poor selectivity towards target analytes. By integration of multiple sensors in a so-called sensor array platform, multi-analyte detection and enhanced selectivity can be achieved. However such a strategy requires significant device integration work involving sophisticated electrical interconnects with increasing circuit complexity. In the past few years, we has been focused on establishing a multi-mode sensing platform based on a single nanostructure array (nanoarray) device. Such a single device platform can be operated at variable temperatures with different modes such as photocurrent, resistance, and impedance. These nanoarray sensors have been fabricated with good scalability using a variety of synthetic methods ranging from vapor to solution phase depositions. The array composition could be rationally designed from pristine homogeneous to dissimilar heterogeneous across metals, ceramics, and polymers. Depending on the comprised material composition, decoration of a nanolayer of perovskite, metal oxide, or noble metal may enable formation of electronic and catalytic sensitizers, as well as hetero-contacts that can drastically boost sensor performance under various gaseous analyte atmospheres. Toward mixed multi-analyte conditions, such a new type of multi-mode nanoarray sensors allows selective and sensitive detection of multiple chemical species in a single-device configuration. By relating the multiple sensing modes using a single nanoarray sensor, efficient and selective detection of multiple gas analytes has been accomplished. A database was established based on multiple sets of data, namely the sensitivities for different analytes obtained from the outputs of different sensing modes. The signals of multiple gas analytes could be differentiated through the principal component analysis process. The linear and non-linear decision boundaries could be readily derived for pattern recognition, and the multi-analyte quantification and detection.

Neurons With Paraboloid Decision Boundaries for Improved Neural Network Classification Performance.

In mathematical terms, an artificial neuron computes the inner product of a d -dimensional input vector x with its weight vector w , compares it with a bias value w0 and fires based on the result of this comparison. Therefore, its decision boundary is given by the equation wTx+w0=0 . In this paper, we propose replacing the linear hyperplane decision boundary of a neuron with a curved, paraboloid decision boundary. Thus, the decision boundary of the proposed paraboloid neuron is given by the equation (hTx+h0)2-||x-p||22=0 , where h and h0 denote the parameters of the directrix and p denotes the coordinates of the focus. Such paraboloid neural networks are proven to have superior recognition accuracy in a number of applications.

Forecasting bank failures and stress testing: A machine learning approach

This paper presents a forecasting model of bank failures based on machine-learning. The proposed methodology defines a linear decision boundary that separates the solvent banks from those that failed. This setup generates a novel alternative stress-testing tool. Our sample of 1443 U.S. banks includes all 481 banks that failed during the period 2007–2013. The set of explanatory variables is selected using a two-step feature selection procedure. The selected variables were then fed to a support vector machines forecasting model, through a training–testing learning process. The model exhibits a 99.22% overall forecasting accuracy and outperforms the well-established Ohlson’s score.

Improved selection criteria for H ii regions, based on IRAS sources

We present new criteria for selecting HII regions from the Infrared Astronomical Satellite (IRAS) Point Source catalogue (PSC), based on an HII region catalogue derived manually from the all-sky Wide-field Infrared Survey Explorer (WISE). The criteria are used to augment the number of HII region candidates in the Milky Way. The criteria are defined by the linear decision boundary of two samples: IRAS point sources associated with known HII regions, which serve as the HII region sample, and IRAS point sources at high Galactic latitudes, which serve as the non-HII region sample. A machine learning classifier, specifically a support vector machine (SVM), is used to determine the decision boundary. We investigate all combinations of four IRAS bands and suggest that the optimal criterion is log(F$_{\rm 60}$/F$_{\rm 12}$)$\ge$(-0.19$\times$log(F$_{\rm 100}$/F$_{\rm 25}$)+ 1.52), with detections at 60 and 100 micron. This selects 3041 HII region candidates from the IRAS PSC. We find that IRAS HII region candidates show evidence of evolution on the two-colour diagram. Merging the WISE HII catalogue with IRAS HII region candidates, we estimate a lower limit of approximately 10200 for the number of HII regions in the Milky Way.

Performance improvement of 64-QAM coherent optical communication system by optimizing symbol decision boundary based on support vector machine

High-order modulation signals are suited for high-capacity communication systems because of their high spectral efficiency, but they are more vulnerable to various impairments. For the signals that experience degradation, when symbol points overlap on the constellation diagram, the original linear decision boundary cannot be used to distinguish the classification of symbol. Therefore, it is advantageous to create an optimum symbol decision boundary for the degraded signals. In this work, we experimentally demonstrated the 64-quadrature-amplitude modulation (64-QAM) coherent optical communication system using support-vector machine (SVM) decision boundary algorithm to create the optimum symbol decision boundary for improving the system performance. We investigated the influence of various impairments on the 64-QAM coherent optical communication systems, such as the impairments caused by modulator nonlinearity, phase skew between in-phase (I) arm and quadrature-phase (Q) arm of the modulator, fiber Kerr nonlinearity and amplified spontaneous emission (ASE) noise. We measured the bit-error-ratio (BER) performance of 75-Gb/s 64-QAM signals in the back-to-back and 50-km transmission. By using SVM to optimize symbol decision boundary, the impairments caused by I/Q phase skew of the modulator, fiber Kerr nonlinearity and ASE noise are greatly mitigated.

Forecasting Bank Failures and Stress Testing: A Machine Learning Approach

In this paper, we present a forecasting model of bank failures based on machine-learning. The proposed methodology defines a linear decision boundary separating the solvent from the failed banks. This setup generates a novel alternative stress testing tool. Our sample of 1443 U.S. banks includes all 481 failed banks during the 2007-2013 period. The set of explanatory variables is selected using a two-step feature selection procedure. The selected variables were then fed to a to Support Vector Machines forecasting model, through a training-testing learning process. The model exhibits 99.22% overall forecasting accuracy and outperforms the well-established Ohlson’s score.

Multi-Directional Space Tessellation to Improve the Decision Boundary in Indirect Mixed-Signal Testing

One of the most challenging aspects in nowadays microelectronics industry is production test and verification of mixed-signal circuits. In order to cope with some of the drawbacks encountered in this scenario, researchers have found alternate test as a promising solution in achieving such endeavor. This work prospects the possibilities of using space tessellations along multiple directions in order to improve the test decision boundary definition in the alternate measure space. The proposed method is able to reduce false positive test outcomes, i.e. test escapes, with acceptable penalty in test yield loss metric. The key idea presented in this work is to use an ensemble of octrees, each of them tessellating the plane along different directions. Such tessellations create a refinement in the non linear test decision boundaries without the need of including extra circuit samples. The tree ensemble, together with a strict test decision criterion, serve as a classifier during the production testing phase. The proposed multi-directional ensemble tessellation strategy has been applied to test a band-pass Biquad filter affected by parametric variations. The proposed method has reported promising simulation results in lowering the test escapes metric as compared to a single octree classifier. The computational overhead of evaluating several octrees is insignificant since 2n-tree data structures are traversed efficiently. The octree ensemble technique has also been compared against a classic specification guard-banding technique reporting better test yield loss metrics for the same test escape target.

Improving Classification Accuracy based on Random Forest Model through Weighted Sampling for Noisy Data with Linear Decision Boundary

Background: Random forest algorithms tend to use a simple random sampling of observations in building their decision trees. The random selection has the chance for noisy, outlier and non informative data to take place during the construction of trees. This leads to inappropriate and poor ensemble classification decision. This paper aims to optimize, the sample selection through probability proportional to size sampling (weighted sampling) in which the noisy, outlier and non informative data points are down weighted to improve the classification accuracy of the model. Methods: The weights of each data pointis determined in two aspects,finding each data pointinfluence on the modelthrough Leave-One-Out method using a single classification tree and measuring the deviance residual of each data point using logistic regression model, these are combined as the final weight. Results: The proposed Finest Random Forest (FRF) performs consistently better than the conventional Random Forest (RF) in terms of classification accuracy. Conclusion: The classification accuracy is improved when random forest is composed with probability proportional to size sampling (weighted sampling) for noisy data with linear decision boundary.Keywords: Classification Accuracy, Decision Trees, Noisy Data, Outlier, Random Forest, Weighted Sampling

Risk-based adaptive metric learning for nearest neighbour classification

The performance of k-nearest neighbour classification highly depends on the appropriateness of distance metric designation. Optimal performance can be obtained when the distance metric is matched to the characteristics of data. Existing works on distance-metric learning typically learn a global linear transform from training samples, and the effectiveness is limited to data, which are well-separated by linear decision boundaries. To address this problem, we propose a locally adaptive weighted distance-metric learning method to deal with the non-linearity of the data. The metric are learned based on local leave-one-out cross-validation (LOOCV) risks in each dimension, so that the local variations in feature component discriminability are taken into account. Experiments on both public datasets and hyper-spectral imagery classification demonstrate that the classification accuracy of the proposed method shows about 2–10% improvements over other competitive methods.

A robust dissimilarity representation for writer‐independent signature modelling

In this study, the authors present a novel dissimilarity-based signature modelling framework for writer-independent off-line signature verification. The proposed framework utilises a discrete Radon transform and a dynamic time warping algorithm for writer-independent signature representation in dissimilarity space, and a writer-specific strategy for dissimilarity normalisation. A discriminative classifier, either a discriminant function or a support vector machine, is utilised for verification purposes. Both linear and non-linear decision boundaries are considered. The authors show that the novel techniques presented in this study provide an improved platform for writer-independent signature modelling. When evaluated on Dolfing's data set, a signature database that contains 1530 genuine signatures and 3000 amateur skilled forgeries, the systems presented in this study outperform all previous systems also evaluated on this data set.