Tested Machine Learning Algorithms Research Articles

Boosting for Mineral Prospectivity Modeling: A New GIS Toolbox

With an increasing demand for raw materials, predictive models that support successful mineral exploration targeting are of great importance. We evaluated different machine learning techniques with an emphasis on boosting algorithms and implemented them in an ArcGIS toolbox. Performance was tested on an exploration dataset from the Iberian Pyrite Belt (IPB) with respect to accuracy, performance, stability, and robustness. Boosting algorithms are ensemble methods used in supervised learning for regression and classification. They combine weak classifiers, i.e., classifiers that perform slightly better than random guessing to obtain robust classifiers. Each time a weak learner is added; the learning set is reweighted to give more importance to misclassified samples. Our test area, the IPB, is one of the oldest mining districts in the world and hosts giant volcanic-hosted massive sulfide (VMS) deposits. The spatial density of ore deposits, as well as the size and tonnage, makes the area unique, and due to the high data availability and number of known deposits, well-suited for testing machine learning algorithms. We combined several geophysical datasets, as well as layers derived from geological maps as predictors of the presence or absence of VMS deposits. Boosting algorithms such as BrownBoost and Adaboost were tested and compared to Logistic Regression (LR), Random Forests (RF) and Support Vector machines (SVM) in several experiments. We found performance results relatively similar, especially to BrownBoost, which slightly outperformed LR and SVM with respective accuracies of 0.96 compared to 0.89 and 0.93. Data augmentation by perturbing deposit location led to a 7% improvement in results. Variations in the split ratio of training and test data led to a reduction in the accuracy of the prediction result with relative stability occurring at a critical point at around 26 training samples out of 130 total samples. When lower numbers of training data were introduced accuracy dropped significantly. In comparison with other machine learning methods, Adaboost is user-friendly due to relatively short training and prediction times, the low likelihood of overfitting and the reduced number of hyperparameters for optimization. Boosting algorithms gave high predictive accuracies, making them a potential data-driven alternative for regional scale and/or brownfields mineral exploration.

Abstract P5-02-03: Testing machine learning algorithms' performance for non-invasively identifying breast cancer molecular subtypes using BI-RADS evaluations

Abstract Purpose: Inter- and intra-observer variability adversely affects interpretation of highly advantageous MRI data in breast cancer diagnosis, prognosis and therapeutic decisions. Machine learning (ML) might potentially provide support to the physicians for objective decision strategies. In this work, ML methods using BI-RADS features were tested for non-invasively determining molecular subtypes of breast cancer. Methods: In this IRB approved study, Out of 126 consecutive patients' retrospective data with written consent, 69 patients (mean±std age 48.24±11.62 and range [27-82]) with full histopathological and MRI data were selected. In surgical histopathological data ER+PR+HER2- was classified as luminal HER2(-), ER+PR+HER+ was classified as luminal HER2(+), ER-PR-HER2+ was classified as HER2(enriched) and ER-PR-HER2- was classified as triple negative. The cohort revealed 51 luminal HER2(-), 11 luminal HER2(+), 7 triple negative cases without any HER2(enriched) occurrence. DCE, DW, T2W MRI data were obtained on a 1.5T Magnetom Espree (Siemens, Erlangen, Germany) scanner which were subsequently interpreted in consensus by 2 radiologists of 15 and 10 years of experience. Age, mass and non-mass properties, non-enhancing BI-RADS findings, ADC values obtained from radiologist drawn ROIS and kinetic curve properties were fed to 22 standard ML algorithms provided in Matlab® (Mathworks, Natick, MA, USA) as predictors for 3 categories. The algorithms were run by cross validating on 50 folds whereby reported classifier accuracy was obtained from each of the observations when in held-out fold. Results: Out of 22 ML algorithms tested Support Vector Machine's Fine Gaussian variant, Ensemble Boosted Trees and Fine K-Nearest Neighbor resulted in 76.8%,71.0% and 53.6% accuracy with (100%, 18%, 0%), (92%, 9%, 14%) and (67%, 18%, 14%) true positive rates for predicting each histopathological category respectively. Furthermore, PCA based dimension reduction worsened the outcomes indicating high sensitivity to the feature set. Conclusion: ML algorithms demonstrated potential as a decision support system increasing assessment objectivity with an added non-invasiveness advantage. However, overall algorithm performance indicates further studies with larger cohorts and broader feature are necessary for refinements and improving ML methodology in breast cancer imaging. Citation Format: Ozcan A, Aksoy Ozcan U, Ekemen S, Ulus S, Oyan B. Testing machine learning algorithms' performance for non-invasively identifying breast cancer molecular subtypes using BI-RADS evaluations [abstract]. In: Proceedings of the 2018 San Antonio Breast Cancer Symposium; 2018 Dec 4-8; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2019;79(4 Suppl):Abstract nr P5-02-03.

Abstract WP382: Using Natural Language Processing Algorithms to Identify Stroke Cases and Stroke Subtypes From Neuroimaging Reports

Background/Objective: The long-term goal of our research is to develop automated, accurate methods for conducting stroke outcome surveillance across large populations. To facilitate this, we developed natural language processing (NLP) based machine learning algorithms to classify stroke cases and sub-type strokes using neuroimaging reports. We report on the performance of our algorithms. Methods: Our population of interest included patients with stroke symptoms presenting to the emergency room of our large academic healthcare system. We randomly sampled 332 probable stroke cases. A trained neurologist validated stroke diagnoses using neuroimaging reports. Data preprocessing included cleaning and normalizing the reports into a standardized format. We trained and tested machine learning algorithms using the formatted reports. The NLP-based algorithms predicted a stroke diagnosis (binary classification) and a stroke type diagnosis (multiclass classification) using n-grams of length 1 (i.e., ‘stroke’, ‘hemorrhage’) through 3 (i.e., ‘no mass effect’), term-frequency weighting, and feature dimensionality reduction via truncated singular value decomposition (SVD). We report algorithm performance using the area under the receiver operating characteristic curve (AUC-ROC). Classification methods we tested included Multinomial Naïve Bayes, Logistic Regression, Random Forest Classifier, and Support Vector Machine (SVM). Results: The highest performing algorithm for both stroke and stroke sub-type classification contained 1 to 2 n-grams, no term-frequency, and 200 SVD components. For the stroke case detection, SVM achieved the best AUC-ROC of 95.6%. For stroke sub-type detection, SVM also yielded the highest AUC-ROC of 93.5% for no stroke, 92.3% for ischemic stroke, 91.9% for intraparenchymal hemorrhage, and 94.8% for subarachnoid hemorrhage. Conclusions: We report very promising results in our pilot study using machine learning algorithms to classify stroke cases from neuroimaging reports. Feature selection on this pilot data revealed a subset of words that are highly effective in categorizing stroke. Future work will focus on algorithm improvement, finer grained stroke sub-type stratification, and multi-label phenotyping.

An Experimental Approach of Applying Boruta and Elastic Net for Variable Selection in Classifying Breast Cancer Datasets

Feature selection identifies the key aspects involved in predicting the outcome. In this study, we propose boruta and elastic net (Enet) feature selection for classifying breast cancer datasets. A comparative study of boruta, Enet along with genetic algorithms (GA) and consistency-based subset feature selections are done, where Enet selected best features for Wisconsin diagnostic breast cancer (WDBC) and breast cancer datasets. To prove the stability of Enet feature selection, variable importance of machine learning algorithms like naive Bayes (NB), multilayer perceptron (MLP) and random forest (RF) is evaluated and compared. It is proved that the features obtained by Enet contain all the common variables selected by tested machine learning algorithms. The proposed Enet feature selection along with MLP for classification yields a better receiver operating characteristic (ROC): 0.990, 0.687 and a reduced root mean squared error (RMSE): 0.159, 0.429 for WDBC and breast cancer datasets, when compared with naive Bayes and RF.

Machine Learning: An Approach in Identifying Risk Factors for Coercion Compared to Binary Logistic Regression.

Introduction: Although knowledge about negative effects of coercive measures in psychiatry exists, its prevalence is still high in clinical routine. This study aimed at define risk factors and test machine learning algorithms for their accuracy in the prediction of the risk to being subjected to coercive measures.Methods: In a sample of involuntarily hospitalized patients (n = 393) at the University Hospital of Psychiatry Zurich, we analyzed risk factors for the experience of coercion (n = 170 patients) using chi-square tests and Mann Whitney U tests. We trained machine learning algorithms [logistic regression, Supported Vector Machine (SVM), and decision trees] with these risk factors and tested obtained models for their accuracy via five-fold cross validation. To verify the results we compared them to binary logistic regression.Results: In a model with 8 risk-factors which were available at admission, the SVM algorithm identified 102 out of 170 patients, which had experienced coercion and 174 out of 223 patients without coercion (69% accuracy with 60% sensitivity and 78% specificity, AUC 0.74). In a model with 18 risk-factors, available after discharge, the logistic regression algorithm identified 121 out of 170 with and 176 out of 223 without coercion (75% accuracy, 71% sensitivity, and 79% specificity, AUC 0.82).Discussion: Incorporating both clinical and demographic variables can help to estimate the risk of experiencing coercion for psychiatric patients. This study could show that trained machine learning algorithms are comparable to binary logistic regression and can reach a good or even excellent area under the curve (AUC) in the prediction of the outcome coercion/no coercion when cross validation is used. Due to the better generalizability machine learning is a promising approach for further studies, especially when more variables are analyzed. More detailed knowledge about individual risk factors may help to prevent the occurrence of situations involving coercion.

Structuring Data for Intelligent Predictive Maintenance in Asset Management

Predictive maintenance (PdM) within asset management improves savings in operational cost, productivity, and safety management capabilities. While PdM can be administered using various methods, growing interest in Artificial Intelligence (AI) has lead to current state of the art PdM relying on machine learning (ML) technology. Like other tools used in PdM for asset management, standards for applying ML technology for PdM are required. This work introduces a standard of practice in regards to usage of asset data to develop ML analytic tools for PdM. It provides a standard method for ensuring asset data is in a form conducive to ML algorithms, and ensuring retention of asset information necessary for optimum PdM during the data transform. In the ML domain, it has been proven through research initiatives that the data structure used to train and test ML algorithms has a great impact on their performance and accuracy. Using poorly trained models for estimation due to improper data usage, can leave some AI-based PdM tools vulnerable to high rates of inaccurate estimations. Thus, leading to value loss during an asset’s life cycle.

Comparison of Machine Learning Techniques and Variables for Groundwater Dissolved Organic Nitrogen Prediction in an Urban Area

Dissolved inorganic nitrogen (DIN) are typically the main focus of nutrient management strategies; however, some studies have found that dissolved organic nitrogen (DON) can be the dominant form of total nitrogen (TN) in several Australian estuaries and catchments. To better understand nitrogen cycling and explore the relationships between measured groundwater DON and environmental factors, thirteen machine learning (ML) techniques were compared in this study. DON was simulated under two scenarios using a range of input variables: 1) detailed nutrient data with landscape and sampling factors, and 2) limited nutrient data with landscape and sampling factors. Most of the tested ML algorithms more accurately predicted DON than when it was estimated from the difference between TN and DIN. Some models show greater adaptability to different modelling conditions, with only a few approaches able to predict with high accuracy using limited input variables (scenario 2). From the models tested, bagged mars, cubist and random forest were selected as optimal. Sample depth, sampling date and specific surface water area were the important non-nutrient input variables for DON prediction, which reveals the significant effect of surface environmental factors and seasonality on groundwater DON.

Data Collection, Statistical Analysis and Machine Learning Studies of Cancer Dataset from North Costal Districts of AP, India

Abstract The data collection and understanding provides better understanding of cancer patients in the regions from north coastal districts of AP, India. The present studies were shown more number of breast cancer patients compares to other types of cancer. Most of the questionnaire provided good understanding about cancer patients. There are more number of females (64.3%) compared to males (35.7%). Most of the patients are using mobile phones and mosquito repellents. KStar was shown 100% accuracy with the dataset compared to other tested machine learning algorithms.

Natural control capabilities of robotic hands by hand amputated subjects.

People with transradial hand amputations who own a myoelectric prosthesis currently have some control capabilities via sEMG. However, the control systems are still limited and not natural. The Ninapro project is aiming at helping the scientific community to overcome these limits through the creation of publicly available electromyography data sources to develop and test machine learning algorithms. In this paper we describe the movement classification results gained from three subjects with an homogeneous level of amputation, and we compare them with the results of 40 intact subjects. The number of considered subjects can seem small at first sight, but it is not considering the literature of the field (which has to face the difficulty of recruiting trans-radial hand amputated subjects). The classification is performed with four different classifiers and the obtained balanced classification rates are up to 58.6% on 50 movements, which is an excellent result compared to the current literature. Successively, for each subject we find a subset of up to 9 highly independent movements, (defined as movements that can be distinguished with more than 90% accuracy), which is a deeply innovative step in literature. The natural control of a robotic hand in so many movements could lead to an immediate progress in robotic hand prosthetics and it could deeply change the quality of life of amputated subjects.

Classification of hand movements in amputated subjects by sEMG and accelerometers.

Numerous recent studies have aimed to improve myoelectric control of prostheses. However, the majority of these studies is characterized by two problems that could be easily fulfilled with recent resources supplied by the scientific literature. First, the majority of these studies use only intact subjects, with the unproved assumption that the results apply equally to amputees. Second, usually only electromyography data are used, despite other sensors (e.g., accelerometers) being easy to include into a real life prosthesis control system. In this paper we analyze the mentioned problems by the classification of 40 hand movements in 5 amputated and 40 intact subjects, using both sEMG and accelerometry data and applying several different state of the art methods. The datasets come from the NinaPro database, which supplies publicly available sEMG data to develop and test machine learning algorithms for prosthetics. The number of subjects can seem small at first sight, but it is not considering the literature of the field (which has to face the difficulty of recruiting trans-radial hand amputated subjects). Our results indicate that the maximum average classification accuracy for amputated subjects is 61.14%, which is just 15.86% less than intact subjects, and they show that intact subjects results can be used as proxy measure for amputated subjects. Finally, our comparison shows that accelerometry as a modality is less affected by amputation than electromyography, suggesting that real life prosthetics performance may easily be improved by inclusion of accelerometers.

Feature-Cost Sensitive Learning with Submodular Trees of Classifiers

During the past decade, machine learning algorithms have become commonplace in large-scale real-world industrial applications. In these settings, the computation time to train and test machine learning algorithms is a key consideration. At training-time the algorithms must scale to very large data set sizes.At testing-time, the cost of feature extraction can dominate the CPU runtime. Recently, a promising method was proposed to account for the feature extraction cost at testing time, called Cost-sensitive Tree of Classifiers (CSTC). Although the CSTC problem is NP-hard, the authors suggest an approximation through a mixed-norm relaxation across many classifiers. This relaxation is slow to train and requires involved optimization hyperparameter tuning. We propose a different relaxation using approximate submodularity, called Approximately Submodular Tree of Classifiers (ASTC). ASTC is much simpler to implement, yields equivalent results but requires no optimization hyperparameter tuning and is up to two orders of magnitude faster to train.