Average Misclassification Rate Research Articles

Research on feature vector construction and classification recognition of typical mine microseismic waveforms

In the realm of mining seismology, the identification of microseismic waveforms is a critical step for enhancing the precision of microseismic event localization. Addressing the complexities and diverse sources of noise in subterranean mining environments, this study has developed a novel methodology for the automated categorization and recognition of characteristic microseismic waveforms. Initially, a comprehensive database for mining-induced microseismic waveforms was established, catering to their non-stationary and nonlinear attributes. Based on the distinct features of different microseismic waveforms, this study introduces an approach for classification and identification that relies on waveform characteristics. A feature selection model, KNN-LOO (K-Nearest Neighbor - Leave-One-Out), was employed to optimize the feature vectors, in addition to the utilization of Principal Component Analysis (PCA) for dimensionality reduction of the feature vectors. Ultimately, an enhanced BT-SVM (Binary Tree-Support Vector Machine) model was developed to accomplish the automated classification and identification of mining-induced microseismic waveforms. Experimental results indicate that, compared to conventional SVM classification methods, the proposed methodology not only exhibits theoretical and mechanistic advancements but also significantly boosts performance in practical applications. Specifically, the average misclassification rate for each category of waveforms is reduced to less than 5%, and the overall accuracy has increased by 20%. This considerably elevates both the accuracy and speed of microseismic waveform classification and recognition, thereby providing valuable insights for monitoring and analyzing microseismic activities within mining operations as well as enhancing the safety protocols for mineral extraction.

Assessing Cognitive Workload Using Cardiovascular Measures and Voice.

Monitoring cognitive workload has the potential to improve both the performance and fidelity of human decision making. However, previous efforts towards discriminating further than binary levels (e.g., low/high or neutral/high) in cognitive workload classification have not been successful. This lack of sensitivity in cognitive workload measurements might be due to individual differences as well as inadequate methodology used to analyse the measured signal. In this paper, a method that combines the speech signal with cardiovascular measurements for screen and heartbeat classification is introduced. For validation, speech and cardiovascular signals from 97 university participants and 20 airline pilot participants were collected while cognitive stimuli of varying difficulty level were induced with the Stroop colour/word test. For the trinary classification scheme (low, medium, high cognitive workload) the prominent result using classifiers trained on each participant achieved 15.17 ± 0.79% and 17.38 ± 1.85% average misclassification rates indicating good discrimination at three levels of cognitive workload. Combining cardiovascular and speech measures synchronized to each heartbeat and consolidated with short-term dynamic measures might therefore provide enhanced sensitivity in cognitive workload monitoring. The results show that the influence of individual differences is a limiting factor for a generic classification and highlights the need for research to focus on methods that incorporate individual differences to achieve even better results. This method can potentially be used to measure and monitor workload in real time in operational environments.

Predictive Data Mining Approaches for Diabetes Mellitus Type II Disease

Diabetes is among the major public health problem especially in developing countries which cause by abnormal insulin secretion in human body. It is a common disease that can led to several health complications and mortality. In Malaysia, most of the cases are categorized as Diabetes Mellitus (DM) Type II. Patients with diabetes increases from year to year due to unhealthy lifestyles e.g. smoking, overweight and hypertension. Therefore, this study meant to identify the influential factors that may contribute to DM Type II by comparing the performance of different data mining approaches. Between April 2017 and November 2018, 684 patients from a public clinic participated in this retrospective cross-sectional study. Four predictive models involved in the study are Logistic Regression, Decision Tree, Naïve Bayes, and Artificial Neural Network (ANN). The error measures (Average Squared Error and Misclassification Rate) with ROC Index are used to evaluate the performance of the models. Results show that the performance of Logistic Regression-Stepwise outperformed to other predictive models with classification accurateness of 73% and able to predict positive outcome (Y=1) correctly by 90%. The significant inputs that affect DM Type II prediction (Y=1) are Hypertension and Glycated Hemoglobin (HbA1c) given the Root Mean Squared Error (RMSE) of model is 0.424. The importance of study may be able to contribute in improving the strategies and planning on diabetes diseases in Malaysia.

Evaluation of Deep Learning-Based Automated Detection of Primary Spine Tumors on MRI Using the Turing Test.

BackgroundRecently, the Turing test has been used to investigate whether machines have intelligence similar to humans. Our study aimed to assess the ability of an artificial intelligence (AI) system for spine tumor detection using the Turing test.MethodsOur retrospective study data included 12179 images from 321 patients for developing AI detection systems and 6635 images from 187 patients for the Turing test. We utilized a deep learning-based tumor detection system with Faster R-CNN architecture, which generates region proposals by Region Proposal Network in the first stage and corrects the position and the size of the bounding box of the lesion area in the second stage. Each choice question featured four bounding boxes enclosing an identical tumor. Three were detected by the proposed deep learning model, whereas the other was annotated by a doctor; the results were shown to six doctors as respondents. If the respondent did not correctly identify the image annotated by a human, his answer was considered a misclassification. If all misclassification rates were >30%, the respondents were considered unable to distinguish the AI-detected tumor from the human-annotated one, which indicated that the AI system passed the Turing test.ResultsThe average misclassification rates in the Turing test were 51.2% (95% CI: 45.7%–57.5%) in the axial view (maximum of 62%, minimum of 44%) and 44.5% (95% CI: 38.2%–51.8%) in the sagittal view (maximum of 59%, minimum of 36%). The misclassification rates of all six respondents were >30%; therefore, our AI system passed the Turing test.ConclusionOur proposed intelligent spine tumor detection system has a similar detection ability to annotation doctors and may be an efficient tool to assist radiologists or orthopedists in primary spine tumor detection.

A depth-based modification of the k-nearest neighbour method

We propose a new nonparametric procedure to solve the problem of classifying objects represented by $d$-dimensional vectors into $K \geq 2$ groups. The newly proposed classifier was inspired by the $k$ nearest neighbour (kNN) method. It is based on the idea of a depth-based distributional neighbourhood and is called $k$ nearest depth neighbours (kNDN) classifier. The kNDN classifier has several desirable properties: in contrast to the classical kNN, it can utilize global properties of the considered distributions (symmetry). In contrast to the maximal depth classifier and related classifiers, it does not have problems with classification when the considered distributions differ in dispersion or have unequal priors. The kNDN classifier is compared to several depth-based classifiers as well as the classical kNN method in a simulation study. According to the average misclassification rates, it is comparable to the best current depth-based classifiers.

Fingerprint liveness detection through fusion of pores perspiration and texture features

Spoofing attacks on the fingerprint scanners become major and serious concern with the growing development and use of biometric technologies. Level 1 and Level 2 features; which are said to be unique and most commonly used features in fingerprint verification systems, are easily get spoofed. Moreover, single feature based specifically designed spoof detection methods are not performed well on different fingerprint scanners and spoofing materials. This paper proposed the fusion of pores perspiration and texture features in static software based approach to identify live and fake fingerprints. The pores perspiration activity is quantified by computing the ridge signal energy and gray level distributions around the detected pores. These pore characteristics are statically determined instead of dynamic measurement. Autoencoder neural network is used to reduce the high dimensional feature vector and learn its low dimensional hidden representation unsupervisedly. The binary classification in two classes: live and spoof is performed by the supervisedly trained softmax classifier. The performance of the classifier is evaluated in terms of Average Classification Error (ACE) and misclassification rates: FerrLive and FerrFake. The experimental results carried out on LivDet 2013 and LivDet 2015 databases show the improvement of the classifier performance in comparison to the state-of-the-art methods.

Performance Analysis of Predictive Models using Generic Datasets

Today over 2.5 quintillion bytes of data is being created every single day where 753 crore people on this planet are creating 1.7mb of data each second. Most often than not, Researchers only scratch the surface when it comes to analyzing which algorithm will be best suited with their dataset and which one will give the highest efficiency. Sometimes, this analysis takes more computational time than the actual execution itself. Aim of this paper is to understand and solve this dilemma by applying different predictions models like Neural Networks, Regression and Decision Tree algorithms to different datasets where their performance was measured using ROC Index, Average Square Error and Misclassification Rate. A comparative analysis is done to show their best performance in different scopes and conditions. All data sets and results were compared and analyzed using SAS tool.

Improving the Use of Mortality Data in Public Health: A Comparison of Garbage Code Redistribution Models.

Objectives. To describe and compare 3 garbage code (GC) redistribution models: naïve Bayes classifier (NB), coarsened exact matching (CEM), and multinomial logistic regression (MLR).Methods. We analyzed Taiwan Vital Registration data (2008-2016) using a 2-step approach. First, we used non-GC death records to evaluate 3 different prediction models (NB, CEM, and MLR), incorporating individual-level information on multiple causes of death (MCDs) and demographic characteristics. Second, we applied the best-performing model to GC death records to predict the underlying causes of death. We conducted additional simulation analyses for evaluating the predictive performance of models.Results. When we did not account for MCDs, all 3 models presented high average misclassification rates in GC assignment (NB, 81%; CEM, 86%; MLR, 81%). In the presence of MCD information, NB and MLR exhibited significant improvement in assignment accuracy (19% and 17% misclassification rate, respectively). Furthermore, CEM without a variable selection procedure resulted in a substantially higher misclassification rate (40%).Conclusions. Comparing potential GC redistribution approaches provides guidance for obtaining better estimates of cause-of-death distribution and highlights the significance of MCD information for vital registration system reform.

Multiple-Antenna Emitters Identification Based on a Memoryless Power Amplifier Model.

Power amplifier (PA) nonlinearity is typically unique at the radio frequency (RF) front-end for particular emitters. It can play a crucial role in the application of specific emitter identification (SEI). In this paper, under the Multi-Input Multi-Output (MIMO) multipath communication scenario, two data-aided approaches are proposed to identify multi-antenna emitters using PA nonlinearity. Built upon a memoryless polynomial model, the first approach formulates a linear least square (LLS) problem and presents the closed-form solution of nonlinear coefficients in a MIMO system by means of singular value decomposition (SVD) operation. Another alternative approach estimates nonlinear coefficients of each individual PA through nonlinear least square (NLS) solved by the regularized Gauss–Newton iterative scheme. Moreover, there are some practical discussions of our proposed approaches about the mismatch of the order of PA model and the rank-deficient condition. Finally, the average misclassification rate is derived based on the minimum error probability (MEP) criterion, and the proposed approaches are validated to be effective through extensively numerical simulations.

Brain wave classification using long short-term memory network based OPTICAL predictor

Brain-computer interface (BCI) systems having the ability to classify brain waves with greater accuracy are highly desirable. To this end, a number of techniques have been proposed aiming to be able to classify brain waves with high accuracy. However, the ability to classify brain waves and its implementation in real-time is still limited. In this study, we introduce a novel scheme for classifying motor imagery (MI) tasks using electroencephalography (EEG) signal that can be implemented in real-time having high classification accuracy between different MI tasks. We propose a new predictor, OPTICAL, that uses a combination of common spatial pattern (CSP) and long short-term memory (LSTM) network for obtaining improved MI EEG signal classification. A sliding window approach is proposed to obtain the time-series input from the spatially filtered data, which becomes input to the LSTM network. Moreover, instead of using LSTM directly for classification, we use regression based output of the LSTM network as one of the features for classification. On the other hand, linear discriminant analysis (LDA) is used to reduce the dimensionality of the CSP variance based features. The features in the reduced dimensional plane after performing LDA are used as input to the support vector machine (SVM) classifier together with the regression based feature obtained from the LSTM network. The regression based feature further boosts the performance of the proposed OPTICAL predictor. OPTICAL showed significant improvement in the ability to accurately classify left and right-hand MI tasks on two publically available datasets. The improvements in the average misclassification rates are 3.09% and 2.07% for BCI Competition IV Dataset I and GigaDB dataset, respectively. The Matlab code is available at https://github.com/ShiuKumar/OPTICAL.

An Efficient Adaptive Reliability Analysis Method Based on Kriging and Weighted Average Misclassification Rate Improvement

In the past few years, the Kriging model based on the adaptive design of experiments (DoE) has attracted extensive attention in analyzing the reliability of structures involving time-consuming simulations or complicated implicit performance function. Although varieties of sampling strategies have been proposed, they update DoE mainly by selecting the sample at which its corresponding learning function value is the maximum or minimum. However, there are usually two drawbacks. First, the training samples in DoE are easily clustered or overlapped. Second, the existing strategies usually only consider the improvement of prediction accuracy at the new training sample rather than the accuracy improvement of the region near the new training sample. Unfortunately, these two drawbacks can cause some unnecessary performance function evaluations. Therefore, an efficient adaptive sampling strategy and reliability analysis algorithm based on Kriging model, weighted average misclassification rate, sampling uniformity and gradient of prediction uncertainty are proposed. Furthermore, an improved stopping criterion based on the relative error estimation of failure probability is also developed to further reduce iteration. Subsequently, two explicit examples from the literature are analyzed to verify the effectiveness and superiority of the proposed method. Finally, a truss structure subjected to six external loads is investigated to illustrate the feasibility of the proposed reliability analysis method in engineering applications.

Depth-weighted Bayes classification

Two procedures for supervised classification are proposed. These are based on data depth and focus on the centre of each class. The classifiers add either a depth or a depth rank term to the objective function of the Bayes classifier. The cost of misclassifying a point depends not only on a class where it belongs, but also on its centrality with respect to this class. The classification of points that are more central is enforced while outliers are downweighted. The proposed objective function can also be used to evaluate the performance of other classifiers instead of the usual average misclassification rate. Use of the depth function increases robustness of the new procedures against the large inclusion of contaminated data that often impede the Bayes classifier. Properties of the new methods are investigated and compared with those of the Bayes classifier. Theoretical results are derived for elliptically symmetric distributions, while comparison for non-symmetric distributions is conducted by means of a simulation study. Comparisons are conducted for both theoretical classifiers and their empirical counterparts. The performance of the newly proposed classifiers is also compared to the performance of several standard methods in some real life situations.

Correlation-based linear discriminant classification for gene expression data.

Microarray gene expression technology provides a systematic approach to patient classification. However, microarray data pose a great computational challenge owing to their large dimensionality, small sample sizes, and potential correlations among genes. A recent study has shown that gene-gene correlations have a positive effect on the accuracy of classification models, in contrast to some previous results. In this study, a recently developed correlation-based classifier, the ensemble of random subspace (RS) Fisher linear discriminants (FLDs), was utilized. The impact of gene-gene correlations on the performance of this classifier and other classifiers was studied using simulated datasets and real datasets. A cross-validation framework was used to evaluate the performance of each classifier using the simulated datasets or real datasets, and misclassification rates (MRs) were computed. Using the simulated data, the average MRs of the correlation-based classifiers decreased as the correlations increased when there were more correlated genes. Using real data, the correlation-based classifiers outperformed the non-correlation-based classifiers, especially when the gene-gene correlations were high. The ensemble RS-FLD classifier is a potential state-of-the-art computational method. The correlation-based ensemble RS-FLD classifier was effective and benefited from gene-gene correlations, particularly when the correlations were high.

Comparing classification methods for diffuse reflectance spectra to improve tissue specific laser surgery.

BackgroundIn the field of oral and maxillofacial surgery, newly developed laser scalpels have multiple advantages over traditional metal scalpels. However, they lack haptic feedback. This is dangerous near e.g. nerve tissue, which has to be preserved during surgery. One solution to this problem is to train an algorithm that analyzes the reflected light spectra during surgery and can classify these spectra into different tissue types, in order to ultimately send a warning or temporarily switch off the laser when critical tissue is about to be ablated. Various machine learning algorithms are available for this task, but a detailed analysis is needed to assess the most appropriate algorithm.MethodsIn this study, a small data set is used to simulate many larger data sets according to a multivariate Gaussian distribution. Various machine learning algorithms are then trained and evaluated on these data sets. The algorithms’ performance is subsequently evaluated and compared by averaged confusion matrices and ultimately by boxplots of misclassification rates. The results are validated on the smaller, experimental data set.ResultsMost classifiers have a median misclassification rate below 0.25 in the simulated data. The most notable performance was observed for the Penalized Discriminant Analysis, with a misclassifiaction rate of 0.00 in the simulated data, and an average misclassification rate of 0.02 in a 10-fold cross validation on the original data.ConclusionThe results suggest a Penalized Discriminant Analysis is the most promising approach, most probably because it considers the functional, correlated nature of the reflectance spectra.The results of this study improve the accuracy of real-time tissue discrimination and are an essential step towards improving the safety of oral laser surgery.

Predicting Fusarium Head Blight Epidemics with Boosted Regression Trees

Predicting major Fusarium head blight (FHB) epidemics allows for the judicious use of fungicides in suppressing disease development. Our objectives were to investigate the utility of boosted regression trees (BRTs) for predictive modeling of FHB epidemics in the United States, and to compare the predictive performances of the BRT models with those of logistic regression models we had developed previously. The data included 527 FHB observations from 15 states over 26 years. BRTs were fit to a training data set of 369 FHB observations, in which FHB epidemics were classified as either major (severity ≥ 10%) or non-major (severity < 10%), linked to a predictor matrix consisting of 350 weather-based variables and categorical variables for wheat type (spring or winter), presence or absence of corn residue, and cultivar resistance. Predictive performance was estimated on a test (holdout) data set consisting of the remaining 158 observations. BRTs had a misclassification rate of 0.23 on the test data, which was 31% lower than the average misclassification rate over 15 logistic regression models we had presented earlier. The strongest predictors were generally one of mean daily relative humidity, mean daily temperature, and the number of hours in which the temperature was between 9 and 30°C and relative humidity ≥ 90% simultaneously. Moreover, the predicted risk of major epidemics increased substantially when mean daily relative humidity rose above 70%, which is a lower threshold than previously modeled for most plant pathosystems. BRTs led to novel insights into the weather-epidemic relationship.

Single-trial decoding of imagined grip force parameters involving the right or left hand based on movement-related cortical potentials

Time–domain feature representation for imagined grip force movement-related cortical potentials (MRCP) of the right or left hand and the decoding of imagined grip force parameters based on electroencephalogram (EEG) activity recorded during a single trial were here investigated. EEG signals were acquired from eleven healthy subjects during four different imagined tasks performed with the right or left hand. Subjects were instructed to execute imagined grip movement at two different levels of force. Each task was executed 60 times in random order. The imagined grip force MRCP of the right or left hand was analyzed by superposition and averaging technology, a single-trial extraction method, analysis of variance (ANOVA), and multiple comparisons. Significantly different features were observed among different imagined grip force tasks. These differences were used to decode imagined grip force parameters using Fisher linear discrimination analysis based on kernel function (k-FLDA) and support vector machine (SVM). Under the proposed experimental paradigm, the study showed that MRCP may characterize the dynamic processing that takes place in the brain during the planning, execution, and precision of a given imagined grip force task. This means that features related to MRCP can be used to decode imagined grip force parameters based on EEG. ANOVA and multiple comparisons of time–domain features for MRCP showed that movement-monitoring potentials (MMP) and specific interval (0–150 ms) average potentials to be significantly different among 4 different imagined grip force tasks. The minimum peak negativity differed significantly between high and low amplitude grip force. Identification of the 4 different imagined grip force tasks based on MMP was performed using k-FLDA and SVM, and the average misclassification rates of 27 % ± 5 % and 24 % ± 4 % across 11 subjects were achieved respectively. The minimum misclassification rate was 15 %, and the average minimum misclassification rate across 11 subjects was 24 % ± 4.5 %. This investigation indicates that imagined grip force MRCP may encode imagined grip force parameters. Single-trial decoding of imagined grip force parameters based on MRCP may be feasible. The study may provide some additional and fine control instructions for brain–computer interfaces.

Vehicle classification using GPS data

Vehicle classification information is crucial to transportation planning, facility design, and operations. Traditional vehicle classification methods are either too expensive to be deployed for large areas or subject to errors under specific situations. In this paper, we propose methods to classify vehicles using GPS data extracted from mobile traffic sensors, which is considered to be low-cost especially for large areas of urban arterials. It is found that features related to the variations of accelerations and decelerations (e.g., the proportions of accelerations and decelerations larger than 1meter per square second, and the standard deviations of accelerations and decelerations) are the most effective in terms of vehicle classification using GPS data. By classifying general trucks from passenger cars, the average misclassification rate is about 1.6% for the training data, and 4.2% for the testing data.

Predicting Fusarium Head Blight Epidemics With Weather-Driven Pre- and Post-Anthesis Logistic Regression Models

Our objective was to identify weather-based variables in pre- and post-anthesis time windows for predicting major Fusarium head blight (FHB) epidemics (defined as FHB severity ≥ 10%) in the United States. A binary indicator of major epidemics for 527 unique observations (31% of which were major epidemics) was linked to 380 predictor variables summarizing temperature, relative humidity, and rainfall in 5-, 7-, 10-, 14-, or 15-day-long windows either pre- or post-anthesis. Logistic regression models were built with a training data set (70% of the 527 observations) using the leaps-and-bounds algorithm, coupled with bootstrap variable and model selection methods. Misclassification rates were estimated on the training and remaining (test) data. The predictive performance of models with indicator variables for cultivar resistance, wheat type (spring or winter), and corn residue presence was improved by adding up to four weather-based predictors. Because weather variables were intercorrelated, no single model or subset of predictor variables was best based on accuracy, model fit, and complexity. Weather-based predictors in the 15 final empirical models selected were all derivatives of relative humidity or temperature, except for one rainfall-based predictor, suggesting that relative humidity was better at characterizing moisture effects on FHB than other variables. The average test misclassification rate of the final models was 19% lower than that of models currently used in a national FHB prediction system.

Assessing a Binary Measurement System With Varying Misclassification Rates When a Gold Standard Is Available

In manufacturing, we often use a binary measurement system (BMS) for 100% inspection to protect customers from receiving nonconforming product. We can assess the performance of a BMS by estimating the consumer's and producer's risks, the two misclassification rates. Here, we consider assessment plans and their analysis when a gold standard system (GSS) is available for the assessment study but is too expensive for everyday use. We propose a random-effects model to allow for variation in the misclassification rates within the populations of conforming and nonconforming parts. One possibility, here denoted the standard plan, is to randomly sample n parts and measure them once with the GSS and r times with the inspection system. We provide a simple analysis and planning advice for standard plans. In practice, the misclassification rates are often low and the underlying process has high capability. This combination of conditions makes the assessment of the BMS challenging. We show that we need a very large number of measurements with the standard plan in order to get precise estimators of the average misclassification rates and the true process performance. We consider an alternate design, here denoted the conditional assessment plan, where we select random samples from the sets of previously passed and failed parts. The sampled parts are measured once with the GSS and r times with the inspection system. When we augment the data from the conditional plans with available baseline information on the overall pass rate, we show that we can precisely estimate the parameters of interest with many fewer measurements. In the online supplementary materials, we provide R code to find maximum likelihood estimates and corresponding approximate standard errors, and to find the asymptotic standard deviation of the estimators with a selected plan size and assumed parameter values for both the standard and the conditional sampling plans.

Grip Kinetic Profile Variability in Adult Signature Writing

Previous studies of handwriting grip kinetics have demonstrated the ability to classify writers based on the topography of grip forces associated with signature writing. However, the topographic representation requires a large array of individual sensors in practice. The possibility of differentiating participants on the basis of a summative, temporal force profile is yet unknown. In this study, we investigated the variability of features derived from a time-evolving total grip force profile. Using an instrumented writing utensil, twenty adult participants provided 600 samples of a well-practised bogus signature over a period of 10 days. Deploying a combination of temporal, spectral and information-theoretic features, a linear discriminant analysis classifier outperformed nonparametric and nonlinear classifier alternatives and discriminated among participants with an average misclassification rate of 5.8% as estimated by cross-validation. These results suggest the existence of a unique kinetic profile for each writer even when generating the same written product. Our findings highlight the potential of using grip kinetics as a biometric measure.