Simple Machine Learning Methods Research Articles

Artificial neural network prediction of particle size of agglomerated polybutadiene latex

AbstractPolymer agglomeration is a useful way to enlarge the particle size of polybutadiene latex (PBL). However, the agglomeration process is very complex and is affected by many factors. Therefore, it is difficult to predict the mean particle size and distribution of agglomerated PBL. This study proposed a novel simulation method based on the artificial neural network (ANN) model. Firstly, the influence of agglomeration conditions on the particle size of agglomerated PBL were investigated, including the particle size of pre‐agglomerate PBL, the amount, the α‐methacrylic acid (α‐MAA) content, the solid content and the particle size of agglomerating agent. It is found that these factors have a close nonlinear relationship with the mean particle size and polydispersion index (PDI) of agglomerated PBL. Then, a model containing 236 data points was built using ANN including these properties as input with mean particle size and PDI of agglomerated PBL as output. The coefficient of determination (R2) values predicted by ANN for training, validation, test and total data are 0.9879, 0.9964, 0.9930, and 0.9902, respectively. It indicates that there is a good agreement between experimental data and predicted results by ANN. This study demonstrates that a simple machine learning method has good performance in predicting the particle size of agglomerated PBL, which further improves the development efficiency of PBL polymer agglomeration.

Natural language processing of clinical notes enables early inborn error of immunity risk ascertainment

There are now approximately 450 discrete inborn errors of immunity (IEI) described; however, diagnostic rates remain suboptimal. Use of structured health record data has proven useful for patient detection but may be augmented by natural language processing (NLP). Here we present a machine learning model that can distinguish patients from controls significantly in advance of ultimate diagnosis date. We sought to create an NLP machine learning algorithm that could identify IEI patients early during the disease course and shorten the diagnostic odyssey. Our approach involved extracting a large corpus of IEI patient clinical-note text from a major referral center's electronic health record (EHR) system and a matched control corpus for comparison. We built text classifiers with simple machine learning methods and trained them on progressively longer time epochs before date of diagnosis. The top performing NLP algorithm effectively distinguished cases from controls robustly 36 months before ultimate clinical diagnosis (area under precision recall curve> 0.95). Corpus analysis demonstrated that statistically enriched, IEI-relevant terms were evident 24+ months before diagnosis, validating that clinical notes can provide a signal for early prediction of IEI. Mining EHR notes with NLP holds promise for improving early IEI patient detection.

Predicting Cardiovascular Disease Using Simple Machine Learning Techniques

Cardiovascular diseases (CVD) become a major health concern, which needs improved prediction models for intervention. In this study, this research explores the possibilities of using simple machine learning methods, including Logistic Regression, Decision Tree, and Random Forest, for predicting cardiovascular diseases effectively. Using a large dataset (308854 instances) containing demographic, binary, and numerical information, the research applied the above machine learning algorithms to develop predictive models. This study involved data preprocessing, including feature selection and handling duplicate values. This study then divides the dataset into two subsets: training set and testing set by 7:3 and 8:2. The Logistic Regression algorithm demonstrated good predictive performance, with an accuracy rate of 92%. Decision Tree exhibited a similar accuracy of 92%, while Random Forest underperformed both slightly with an accuracy of 91%.

A Machine Learning Augmented Data Assimilation Method for High‐Resolution Observations

AbstractThe accuracy of initial conditions is an important driver of the forecast skill of numerical weather prediction models. Increases in the quantity of available measurements, particularly high‐resolution remote sensing observational data products from satellites, are valuable inputs for improving those initial condition estimates. However, the traditional data assimilation methods for integrating observations into forecast models are computationally expensive. This makes incorporating dense observations into operational forecast systems challenging, and it is often prohibitively time‐consuming. Additionally, high‐resolution observations often have correlated observation errors which are difficult to estimate and create problems for assimilation systems. As a result, large quantities of data are discarded and not used for state initialization. Using the Lorenz‐96 system for testing, we demonstrate that a simple machine learning method can be trained to assimilate high‐resolution data. Using it to do so improves both initial conditions and forecast accuracy. Compared to using the Ensemble Kalman Filter with high‐resolution observations ignored, our augmented method has an average root‐mean‐squared error reduced by 37%. Ensemble forecasts using initial conditions generated by the augmented method are more accurate and reliable at up to 10 days of forecast lead time.

A hybrid method of combination probability and machine learning for Chinese geological text segmentation

To address the issues surrounding incomplete coverage of core dictionaries, limited training corpora, and low precision in Chinese geological text segmentation, a knowledge- and data-driven word segmentation method by combining combination probability and machine learning was proposed in this paper. We extracted mathematical feature information from terms in Chinese geological text to construct a Term Combination Probability Model (TCPM) for Chinese word combinations by integrating the combination features of geological terms and the Chinese writing styles under zero-sample conditions. The TCPM was used to extract geological terms with high combination characteristics as a user-defined dictionary, and then a geological corpus was constructed by using a general domain word segmentation method based on this dictionary. After a small amount of manual review and optimization, the geological corpus was trained with a BiLSTM-CRF model to segment Chinese geological text. The proposed method in this paper was tested using a regional geological survey report set in Henan Province, and the precision, recall, and F1-score of the method are 92.65%, 92.53%, and 92.59%, respectively. The experimental results demonstrated that this method, combined with the inherent knowledge features of geological text and the machine learning method, can assist in expanding the core dictionary for Chinese geological text segmentation based on zero-sample, and can improve the segmentation precision of Chinese geological text compared to simple general word segmentation methods or machine learning methods.

Estimation of photosynthetic parameters from hyperspectral images using optimal deep learning architecture

The maximum carboxylation rate (Vcmax) and maximum electron transport rate (Jmax) of leaves are crucial for comprehending carbon cycling in farmland. Nevertheless, estimating these photosynthetic parameters precisely and rapidly faces a considerable challenge. This study designed an optimal deep learning architecture that accurately extracts photosynthetic parameters from hyperspectral images and evaluated its stability across different crop species. Photosynthetic parameter models jointly driven by hyperspectral images and deep learning were compared with models jointly driven by one-dimensional reflectance and simple machine learning methods, as well as chlorophyll driven models. The proposed optimal deep learning architecture incorporated spatial attention and prior knowledge of spectral indices calculations. Our results demonstrated that the hyperspectral images and deep learning jointly driven models outperformed traditional models. Notably, incorporating the module of spatial attention and spectral indices calculation networks achieved the best estimations for Vcmax25 (R2: 0.86, RMSE: 10.18 μmol m−2 s−1) and Jmax25 (R2: 0.83, RMSE: 24.27 μmol m−2 s−1). In contrast, the performance of the one-dimension reflectance driven models deteriorated (R2: 0.43–0.58, RMSE: 19.71–39.99 μmol m−2 s−1). Moreover, the best architecture was interpreted. Weight analysis revealed that the hyperspectral information on the middle part of the leaf contributed most to photosynthetic parameters. Feature map analysis indicated that the spectral indices calculation module utilized the information of the visible light spectrum. Even though the migration of models across various crop species may lead to a slight degradation, the model performance remained satisfactory after fine-tuning. The proposed deep learning models, which used 3D hyperspectral images for estimation of photosynthetic parameters, outperformed the models jointly driven by 1D reflectance data and conventional machine learning algorithms. The results highlighted the significance of spatial information from hyperspectral images and prior knowledge through spectral indices calculations. Moreover, the stability and effectiveness of the proposed architecture remained excellent across different species. This study presents advanced and highly effective deep learning techniques for evaluating the photosynthetic capacity of crop leaves and modeling the carbon cycle.

Using Lower Limb Wearable Sensors to Identify Gait Modalities: A Machine-Learning-Based Approach.

Real-world gait analysis can aid in clinical assessments and influence related interventions, free from the restrictions of a laboratory setting. Using individual accelerometers, we aimed to use a simple machine learning method to quantify the performance of the discrimination between three self-selected cyclical locomotion types using accelerometers placed at frequently referenced attachment locations. Thirty-five participants walked along a 10 m walkway at three different speeds. Triaxial accelerometers were attached to the sacrum, thighs and shanks. Slabs of magnitude, three-second-long accelerometer data were transformed into two-dimensional Fourier spectra. Principal component analysis was undertaken for data reduction and feature selection, followed by discriminant function analysis for classification. Accuracy was quantified by calculating scalar accounting for the distances between the three centroids and the scatter of each category's cloud. The algorithm could successfully discriminate between gait modalities with 91% accuracy at the sacrum, 90% at the shanks and 87% at the thighs. Modalities were discriminated with high accuracy in all three sensor locations, where the most accurate location was the sacrum. Future research will focus on optimising the data processing of information from sensor locations that are advantageous for practical reasons, e.g., shank for prosthetic and orthotic devices.

Beyond tsunami fragility functions: experimental assessment for building damage estimation

Tsunami fragility functions (TFF) are statistical models that relate a tsunami intensity measure to a given building damage state, expressed as cumulative probability. Advances in computational and data retrieval speeds, coupled with novel deep learning applications to disaster science, have shifted research focus away from statistical estimators. TFFs offer a “disaster signature” with comparative value, though these models are seldom applied to generate damage estimates. With applicability in mind, we challenge this notion and investigate a portion of TFF literature, selecting three TFFs and two application methodologies to generate a building damage estimation baseline. Further, we propose a simple machine learning method, trained on physical parameters inspired by, but expanded beyond, TFF intensity measures. We test these three methods on the 2011 Ishinomaki dataset after the Great East Japan Earthquake and Tsunami in both binary and multi-class cases. We explore: (1) the quality of building damage estimation using TFF application methods; (2) whether TFF can generalize to out-of-domain building damage datasets; (3) a novel machine learning approach to perform the same task. Our findings suggest that: both TFF methods and our model have the potential to achieve good binary results; TFF methods struggle with multiple classes and out-of-domain tasks, while our proposed method appears to generalize better.

Simple machine learning methods work surprisingly well for Ramanomics

AbstractRecently, a deep convolutional neural network was employed to detect liver cancer by Ramanomics. Results on a demo dataset claimed an accuracy of about 90% to be achieved after around an hour of training in a modern desktop. However, my experience with another Ramanomics dataset taught me that simple methods could potentially outperform deep learning. Here, I tested the simple and interpretable method of logistic regression. It achieved an accuracy of around 90.4% in under a minute. Employing a random decision forest yields an accuracy of 92.6% in under 10 s. Thus, although deep learning is promising, it is yet to provide a quantum‐leap in performance for Ramanomics. A biophysics aware machine learning method would be more welcome!

A case study of microarray breast cancer classification using machine learning algorithms with grid search cross validation

Breast cancer is one of the leading causes of death and most frequently diagnosed cancer amongst women. Annually, almost half a million women do not survive the disease and die from breast cancer. Machine learning is a subfield of artificial intelligence (AI) and computer science that uses data and algorithms to mimic how humans learn, and gradually improving its accuracy. In this work, simple machine learning methods are used to classify breast cancer microarray data to normal and relapse. The data is from the gene expression omnibus (GEO) website namely GSE45255 and GSE15852. These two datasets are integrated and combined to form a single dataset. The study involved three machine learning algorithms, random forest (RF), extra tree (ET), and support vector machine (SVM). Grid search cross validation (CV) is applied for hyperparameter tuning of the algorithms. The result shows that the tuned SVM is best among the tested algorithms with accuracy of 97.78%. In the future it is recommended to include feature selection method to get the optimal features and better classification accuracies.

Machine learning methods for predicting failures of US commercial bank

ABSTRACT In this paper, we attempt to study the effectiveness of various simple machine learning methods in the prediction of bank failures. From a raw dataset of 10,938 US banks during the period of 2000–2020, we find that machine learning approaches do not really outperform the benchmark of conventional statistical method, logistic regression. However, using PCA to retain relevant variance in variables significantly improve the performance of machine learning methods and raise the out-of-sample accuracy of those method to over 70% to over 80%. Of all the machine learning methods used in this paper, the simple KNN seems to be the best model in forecasting bank failure in the United States.

Machine Learning for Single‐Station Detection of Transient Deformation in GPS Time Series With a Case Study of Cascadia Slow Slip

AbstractInspired by recent studies using various machine learning methods on different types of time series data (e.g., seismic, sea floor pressure), this study proposes a simple machine learning method, based on the recurrent neural network approach, for transient deformation detection in Global Positioning System time series. Unlike most previous studies using a sliding window technique, our model uses a single data point of the entire time series as sequential input and directly outputs the transient probability for all points in the time series. As a case study, we apply our method to detect slow‐slip events in Cascadia between 2005 and 2016. The specific model is first trained and validated using synthetic data, and then used to detect slow‐slip events on real Cascadia data. Based on our detection results, the spatial extent, duration, and migration of the major events are consistent with previous studies. As a benchmark, we compared our results in detail with those based on the Relative Strength Index (RSI). In general, our ML model detects more stations likely to be associated with nearby slow‐slip events than the RSI model, especially if there are data gaps in the timeseries.

Online Machine Learning for 1-Day-Ahead Prediction of Indoor Photovoltaic Energy

We explore the potential for predicting indoor photovoltaic energy on a forecasting horizon of up to 24 hours. The objective is to enable energy management approaches that exploit harvesting opportunities more strategically, for which they require more accurate energy intake predictions. Our study is based on a data set covering over 3 years, for which we simulate online machine learning algorithms with different amounts of training data and input features. Our results show that relatively simple machine learning methods can outperform a persistent predictor considerably, and we observed a reduction of errors of up to 56%. When devices obtain a significant amount of sunlight, adding the weather forecast improves the prediction accuracy. We discuss prediction features, the amount of training data and analyze the sources of errors to understand the potential of indoor photovoltaic energy harvesting predictions.

Simple, efficient, and generalized ECG signal quality assessment method for telemedicine applications

Electrocardiogram (ECG) signals recorded by paramedics in an unsupervised environment are prone to errors and noise due to factors such as electrode misplacements and daily activities during prolonged monitoring. Signal Quality Assessment (SQA) systems can prevent false diagnoses by automated arrhythmia detection systems and reduce the workload of cardiologists. However, current SQA methods are quite complex and/or exhibit poor performances in presence of certain arrhythmias. This study proposed a novel SQA method consisting of three main steps: (1) feasibility conditions check, (2) average beat correlation algorithm, and (3) beat clustering algorithm. The system uses empirical probability functions to automatically adapt the quality score according to the presence of pathological beats and/or noises. We evaluated the proposed method on three datasets and considered both three-quality classes ('Bad', 'HR' and 'Diagnostic') and two-quality classes ('Acceptable' and 'Unacceptable'). For the three-class classification, we obtained an average accuracy of 81.42 % while for the two-class, our algorithm achieved an average sensitivity of 94.59 %, a specificity of 98.38 %, and an accuracy of 97.10 %. The results show that our method achieves high accuracy, specificity and generalizability, outperforming the simple heuristic rule, machine learning, and deep learning methods. Additionally, the algorithm requires less computational time, making it suitable for telemedicine applications.

Energy performance estimation for large building portfolios with machine learning-based techniques

Building operation is responsible for 28% of the world’s carbon emissions. In this context, establishing priorities in refurbishment strategies at the scale of a city or a group of buildings is important. Such procedures are usually led by experts in energy performance and, therefore, they are rarely carried out due to their long and costly nature.This research aims at the estimation of building energy performance to pave the way towards finding near-optimal refurbishment strategies. Thanks to the identification of easily-accessible building characteristics, the method applies machine learning models to scan a building portfolio based on a low level of details. The results show good potential to identify low-performer buildings with simple machine learning methods. It also opens the door for further improvements through the inclusion of supplementary building features at the input of the predictive system.This work includes (a) the integration of a knowledge database thanks to the Swiss CECB energy performance certificates, referencing more than 70 000 buildings, (b) the preparation of a training data set through the selection of relevant physical characteristics of buildings (input) and the corresponding energy consumption labels (output), (c) the development of predictive models used in a supervised way, (d) their evaluation on an independent test set.

Research on the Application of Time Convolution Series in Futures Price Forecasting

Aiming at the problem of anomaly detection of time series data with unbalanced distribution among classes, a detection method based on depth convolution neural network is proposed. With the increasing trading scale of the futures market, it covers more and more economic and financial fields, and the volatility of the futures market is more and more intense, which constantly presents many complex phenomena that cannot be explained by other classical financial theories. Investors predict stock prices based on statistical analysis or simple machine learning methods, but because the stock market is a complex nonlinear dynamic system, these methods have huge limitations. By using the existing data and the chart obtained by the game master software for prediction analysis, the advantages of several technical indicators have been effectively complemented, and the accuracy of prediction has been improved. On the basis of the concept of attribute set and attribute measure, aiming at the problem of pattern recognition, this paper proposes a recognition method, establishes an attribute mathematical model, and combines it with other methods, which has been gradually applied to artificial intelligence, neural network and other fields. It has been successfully applied in the prediction of futures price trend. Building a futures price forecasting model to reveal the inherent law implied in futures market price index and show its evolution mechanism can improve the control ability to deal with economic and financial risks, and provide an objective and rigorous basis for regulatory authorities to formulate relevant policies, which is also the significance of this study.

Towards Explainable Machine Learning for Bank Churn Prediction Using Data Balancing and Ensemble-Based Methods

The diversity of data collected on both social networks and digital interfaces is extremely increased, raising the problem of heterogeneous variables that are not often favourable to classification algorithms. Despite the significant improvement in machine learning (ML) and predictive analysis efficiency for classification in customer relationship management systems (CRM), their performance remains very limited by heterogeneous data processing, class imbalance, and feature scales. This impact turned out to be more important for simple ML methods which in addition often suffer from over-fitting. This paper proposes a succinct and detailed ML model building process including cross-validation of the combination of SMOTE to balance data and ensemble methods for modelling. From the conducted experiments, the random forest (RF) model yielded the best performance of 0.86 in terms of accuracy and f1-scoreusing balanced data. It confirms the literature summary about this topic which shows that RF was among the most effective algorithms for customer predictive classification issues. The constructed and optimized models were interpreted by Shapley values and feature importance analysis which shows that the “age” feature was the most significant while “HasCrCard” was the less one. This process has proven effective in bridging previously reported research gaps and the resulting model should be used for supporting bank customer loyalty decision-making.

Integrating climate and satellite remote sensing data for predicting county-level wheat yield in China using machine learning methods

• We assessed the potential of multi-source data of wheat yield prediction in China. • Large spatial divergences for yield prediction in three wheat planting zones. • Water-related variables outperformed other climatic predictors for yield prediction. • SIF had better predictability for yield than traditional vegetation indices did. • Climate data provided unique predicted information beyond satellite data. Early and reliable crop yield prediction on a large scale is imperative for making in-season crop management decisions as well as for ensuring global food security. The integrated use of climate and remote sensing data for predicting yield at regional and national scales has been previously investigated in various parts of the world. However, such attempts for national scale yield prediction, particularly in different planting zones in China have been rarely reported. For this purpose, this study explored the potential of nine climate variables, three remote sensing-derived metrics, and three machine learning methods (random forest, support vector machine, and least absolute shrinkage and selection operator) for predicting wheat yield based on data acquired during 2002–2010 from 1582 counties across China’s three wheat planting zones. Our results illustrated large spatial divergences for yield prediction. The best performance (R 2 = 0.79 and R 2 = 0.66) was achieved for the northern winter wheat and northern spring wheat planting zones, respectively. Water-related climatic variables outperformed temperature-related variables, with the best individual predictive performance (R 2 = 0.67). Solar-induced chlorophyll fluorescence had better performance (R 2 = 0.60) for predicting the crop yield than NDVI and EVI. Climate data across the whole growing season has provided additional information for yield prediction as compared to remote sensing data. The additional contribution for yield prediction in winter wheat planting zones benefiting from climate data decreased from sowing to maturity, which was the opposite in remote sensing data. Typically, the support vector machine outperformed other models and the prediction in winter wheat planting zones performed better than the spring wheat planting zone. Our study demonstrates the effectiveness of integrating climate and remote sensing data for accurate county-level yield prediction in China. These kinds of simple and scalable machine learning methods could be targeted for further work by agricultural researchers and advisors.

Machine learning decision tree algorithm role for predicting mortality in critically ill adult COVID-19 patients admitted to the ICU

BackgroundCoronavirus disease-19 (COVID-19) is caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and is currently a major cause of intensive care unit (ICU) admissions globally. The role of machine learning in the ICU is evolving but currently limited to diagnostic and prognostic values. A decision tree (DT) algorithm is a simple and intuitive machine learning method that provides sequential nonlinear analysis of variables. It is simple and might be a valuable tool for bedside physicians during COVID-19 to predict ICU outcomes and help in critical decision-making like end-of-life decisions and bed allocation in the event of limited ICU bed capacities. Herein, we utilized a machine learning DT algorithm to describe the association of a predefined set of variables and 28-day ICU outcome in adult COVID-19 patients admitted to the ICU. We highlight the value of utilizing a machine learning DT algorithm in the ICU at the time of a COVID-19 pandemic. MethodsThis was a prospective and multicenter cohort study involving 14 hospitals in Saudi Arabia. We included critically ill COVID-19 patients admitted to the ICU between March 1, 2020, and October 31, 2020. The predictors of 28-day ICU mortality were identified using two predictive models: conventional logistic regression and DT analyses. ResultsThere were 1468 critically ill COVID-19 patients included in the study. The 28-day ICU mortality was 540 (36.8 %), and the 90-day mortality was 600 (40.9 %). The DT algorithm identified five variables that were integrated into the algorithm to predict 28-day ICU outcomes: need for intubation, need for vasopressors, age, gender, and PaO2/FiO2 ratio. ConclusionDT is a simple tool that might be utilized in the ICU to identify critically ill COVID-19 patients who are at high risk of 28-day ICU mortality. However, further studies and external validation are still required.

Искусственный интеллект в диагностике и лечении мочекаменной болезни.

Complex health-care data will be growing exponentially as we enter the era of “big data”. Interpretation and analysis of a large amount of medical data for rapid and personalized decision-making require artificial intelligence-driven technologies and data mining techniques. Over the past decade, a rapid transition to the analysis, treatment and monitoring of kidney stone disease using the artificial intelligence (AI) with machine learning algorithms and artificial neural networks ensures the development of precise support systems. Machine learning algorithms able to predict the location of stones using computed tomography scans or ultrasound images, determine their chemical composition, recurrence rate based on demographic and genetic variables, spontaneous ureteral calculous passage, and treatment outcomes. Recent studies use rather simple machine learning methods and small training sets that reduce their accuracy, sensitivity, and specificity and limit their routine application. Therefore, the development of complex architectures, using multilayer artificial neural networks and convolutional neural networks, and integration other heterogeneous variables will improve the resultant predictive accuracy ensuring their introduction in real clinical practice.