Statistical Machine Learning Methods Research Articles

Prediction of atherosclerosis using machine learning based on operations research

Atherosclerosis is one of the major reasons for cardiovascular disease including coronary heart disease, cerebral infarction and peripheral vascular disease. Atherosclerosis has no obvious symptoms in its early stages, so the key to the treatment of atherosclerosis is early intervention of risk factors. Machine learning methods have been used to predict atherosclerosis, but the presence of strong causal relationships between features can lead to extremely high levels of information redundancy, which can affect the effectiveness of prediction systems. We aim to combine statistical analysis and machine learning methods to reduce information redundancy and further improve the accuracy of disease diagnosis. We cleaned and collated the relevant data obtained from the retrospective study at Affiliated Hospital of Nanjing University of Chinese Medicine through data analysis. First, some features that with too many missing values are filtered out of the 34 features, leaving 25 features. 49% of the samples were categorized as the atherosclerosis risk group while the rest 51% as the control group without atherosclerosis risk under the guidance of relevant experts. We compared the prediction results of a single indicator that had been medically proven to be highly correlated with atherosclerosis with the prediction results of multiple features to fully demonstrate the effect of feature information redundancy on the prediction results. Then the features that could distinguish whether have atherosclerosis risk or not were retained by statistical tests, leaving 20 features. To reduce the information redundancy between features, after drawing inspiration from graph theory, machine learning combined with optimal correlation distances was then used to screen out 15 significant features, and the prediction models were evaluated under the 15 features. Finally, the information of the 5 screened-out non-significant features was fully utilized by ensemble learning to improve the prediction superiority for atherosclerosis. Area Under the Receiver Operating Characteristic (ROC) Curve (AUC), which is used to measure the predictive performance of the model, was 0.84035 and Kolmogorov-Smirnov (KS) value was 0.646. After feature selection model based on optimal correlation distance, the AUC value was 0.88268 and the KS value was 0.688, both of which were improved by about 0.04. Finally, after ensemble learning, the AUC value of the model was further improved by 0.01369 to 0.89637. The optimal distance feature screening model proposed in this paper improves the performance of atherosclerosis prediction models in terms of both prediction accuracy and AUC metrics. Code and models are available at https://github.com/Cesartwothousands/Prediction-of-Atherosclerosis.

Research on Time Series Anomaly Detection: Based on Deep Learning Methods

Time series anomaly detection has always been an important research direction. The early time series anomaly detection methods are mainly statistical methods and machine learning methods. With the powerful functions of deep neural network being continuously mined by researchers, the effect of deep neural network in anomaly detection task has been significantly better than the traditional methods. In view of the continuous development and application of deep neural networks such as transformer and graph neural network (GNN) in time series anomaly detection in recent years, the body of research lacks a comparative evaluation of deep learning methods in recent years. This paper studies various deep neural networks suitable for time series, which are divided into three categories according to anomaly detection methods. The evaluation is conducted on public datasets. By analyzing the evaluation criteria, this paper discusses the performance of each model, as well as the problems and development direction in the field of time series anomaly detection in the future. This study found that in the time series anomaly detection task, transformer is suitable for dealing with long-time series prediction, and studying the graph structure of time series may be the best way to deal with time series anomaly detection in the future

From demand forecasting to inventory ordering decisions for red blood cells through integrating machine learning, statistical modeling, and inventory optimization.

The demand and supply of blood are highly variable over time. Blood inventory management that relies heavily on experience-based decisions may not be adaptive to real demand, leading to high operational costs, wastage, and shortages. We combined statistical modeling, machine learning, and optimization methods to develop a data-driven demand forecasting and inventory management strategy for red blood cells (RBCs). We then used the strategy to inform daily blood orders. A secondary semi-weekly (twice per week) ordering strategy was developed to handle the last-mile split delivery problem for blood suppliers, characterized by multi-deliveries to the same location multiple times during a short period of time. Both strategies were evaluated using the TRUST database including all patient data across four hospitals in Hamilton, Ontario. We identified 227,944 RBC transfusions for 40,787 patients in Hamilton, Ontario from 2012 to 2018. The predicted daily demand from the hybrid demand forecasting model was not significantly different from the actual daily demand (paired t-test p-value=0.163); however, the proposed daily ordering quantity from the model was significantly lower than the actual ordering quantity (p-value <0.001). The proposed daily ordering strategy reduced inventory levels by 38.4% without risk of shortages, leading to an overall cost reduction of 43.0% (95% confidence interval [CI]: 42.3%, 43.7%) compared with the actual cost. The semi-weekly ordering strategy reduced ordering frequency by 62.6% (95% CI: 61.5%, 63.7%). The proposed data-driven ordering strategy combining demand forecasting and inventory optimization can achieve significant cost savings for healthcare systems and blood suppliers.

GANBOT: a GAN-based framework for social bot detection.

Nowadays, a massive number of people are involved in various social media. This fact enables organizations and institutions to more easily access their audiences across the globe. Some of them use social bots as an automatic entity to gain intangible access and influence on their users by faster content propagation. Thereby, malicious social bots are populating more and more to fool humans with their unrealistic behavior and content. Hence, that’s necessary to distinguish these fake social accounts from real ones. Multiple approaches have been investigated in the literature to answer this problem. Statistical machine learning methods are one of them focusing on handcrafted features to represent characteristics of social bots. Although they reached successful results in some cases, they relied on the bot’s behavior and failed in the behavioral change patterns of bots. On the other hands, more advanced deep neural network-based methods aim to overcome this limitation. Generative adversarial network (GAN) as new technology from this domain is a semi-supervised method that demonstrates to extract the behavioral pattern of the data. In this work, we use GAN to leak more information of bot samples for state-of-the-art textual bot detection method (Contextual LSTM). Although GAN augments low labeled data, original textual GAN (Sequence Generative Adversarial Net (SeqGAN)) has the known limitation of convergence. In this paper, we invested this limitation and customized the GAN idea in a new framework called GANBOT, in which the generator and classifier connect by an LSTM layer as a shared channel between them. Our experimental results on a bench-marked dataset of Twitter social bot show our proposed framework outperforms the existing contextual LSTM method by increasing bot detection probabilities.

Domain-Aware Word Segmentation for Chinese Language: A Document-Level Context-Aware Model

Word segmentation is an essential and challenging task in natural language processing, especially for the Chinese language due to its high linguistic complexity. Existing methods for Chinese word segmentation, including statistical machine learning methods and neural network methods, usually have good performance in specific knowledge domains. Given the increasing importance of interdisciplinary and cross-domain studies, one of the challenges in cross-domain word segmentation is to handle the out-of-vocabulary (OOV) words. Existing methods show unsatisfactory performance to meet the practical standard. To this end, we propose a document-level context-aware model that can automatically perceive and identify OOV words from different domains. Our method jointly implements a word-based and a character-based model and then processes the results with a newly proposed reconstruction model. We evaluate the new method by designing and conducting comprehensive experiments on two real-world datasets (e.g., news from different domains). The results demonstrate the superiority of our method over the state-of-the-art models in handling texts from different domains. Importantly, when doing the word segmentation under the cross-domain scenario, our proposed method can improve the performance of OOV words recognition.

Neonatal Jaundice Diagnosis Using a Smartphone Camera Based on Eye, Skin, and Fused Features with Transfer Learning.

Neonatal jaundice is a common condition worldwide. Failure of timely diagnosis and treatment can lead to death or brain injury. Current diagnostic approaches include a painful and time-consuming invasive blood test and non-invasive tests using costly transcutaneous bilirubinometers. Since periodic monitoring is crucial, multiple efforts have been made to develop non-invasive diagnostic tools using a smartphone camera. However, existing works rely either on skin or eye images using statistical or traditional machine learning methods. In this paper, we adopt a deep transfer learning approach based on eye, skin, and fused images. We also trained well-known traditional machine learning models, including multi-layer perceptron (MLP), support vector machine (SVM), decision tree (DT), and random forest (RF), and compared their performance with that of the transfer learning model. We collected our dataset using a smartphone camera. Moreover, unlike most of the existing contributions, we report accuracy, precision, recall, f-score, and area under the curve (AUC) for all the experiments and analyzed their significance statistically. Our results indicate that the transfer learning model performed the best with skin images, while traditional models achieved the best performance with eyes and fused features. Further, we found that the transfer learning model with skin features performed comparably to the MLP model with eye features.

A representation and deep learning model for annotating ubiquitylation sentences stating E3 ligase\xa0- substrate interaction

BackgroundUbiquitylation is an important post-translational modification of proteins that not only plays a central role in cellular coding, but is also closely associated with the development of a variety of diseases. The specific selection of substrate by ligase E3 is the key in ubiquitylation. As various high-throughput analytical techniques continue to be applied to the study of ubiquitylation, a large amount of ubiquitylation site data, and records of E3-substrate interactions continue to be generated. Biomedical literature is an important vehicle for information on E3-substrate interactions in ubiquitylation and related new discoveries, as well as an important channel for researchers to obtain such up to date data. The continuous explosion of ubiquitylation related literature poses a great challenge to researchers in acquiring and analyzing the information. Therefore, automatic annotation of these E3-substrate interaction sentences from the available literature is urgently needed.ResultsIn this research, we proposed a model based on representation and attention mechanism based deep learning methods, to automatic annotate E3-substrate interaction sentences in biomedical literature. Focusing on the sentences with E3 protein inside, we applied several natural language processing methods and a Long Short-Term Memory (LSTM)-based deep learning classifier to train the model. Experimental results had proved the effectiveness of our proposed model. And also, the proposed attention mechanism deep learning method outperforms other statistical machine learning methods. We also created a manual corpus of E3-substrate interaction sentences, in which the E3 proteins and substrate proteins are also labeled, in order to construct our model. The corpus and model proposed by our research are definitely able to be very useful and valuable resource for advancement of ubiquitylation-related research.ConclusionHaving the entire manual corpus of E3-substrate interaction sentences readily available in electronic form will greatly facilitate subsequent text mining and machine learning analyses. Automatic annotating ubiquitylation sentences stating E3 ligase-substrate interaction is significantly benefited from semantic representation and deep learning. The model enables rapid information accessing and can assist in further screening of key ubiquitylation ligase substrates for in-depth studies.

GRU-based deep learning approach for network intrusion alert prediction

The exponential growth in the number of cyber attacks in the recent past has necessitated active research on network intrusion detection, prediction and mitigation systems. While there are numerous solutions available for intrusion detection, the prediction of future network intrusions still remains an open research problem. Existing approaches employ statistical and/or shallow machine learning methods for the task, and therefore suffer from the need for feature selection and engineering. This paper presents a deep learning based approach for prediction of network intrusion alerts. A Gated Recurrent Unit (GRU) based deep learning model is proposed which is shown to be capable of learning dependencies in security alert sequences, and to output likely future alerts given a past history of alerts from an attacking source. The performance of the model is evaluated on intrusion alert sequences obtained from the Warden alert sharing platform.

Probit Regressive Tversky Indexed Rocchio Convolutive Deep Neural Learning for Legal Document Data Analytics

Legal documents data analytics is a very significant process in the field of computational law.Semantically analyzing the documents is more challenging since it’s often more complicated than open domain documents. Efficient document analysis is crucial to current legal applications, such as case-based reasoning, legal citations, and so on. Due to the extensive growth of documents of data, several statistical machine learning methods have been developed for Legal documents data analytics. However, documents are large and highly complex, so the traditional machine learning-based classification models are inefficient for accurate data analytics with minimum time. In order to improve the accurate legal documents data analytics with minimum time, an efficient technique called Probit Regressive Tversky Indexed Rocchio Convolutive Deep Neural Learning (PRTIRCDNL) is introduced. The PRTIRCDNL technique uses the Convolutive Deep neural learning concept to learn the given input with help of many layers and provides accurate classification results. Convolutive Deep Neural Learning uses two different processing steps such as keyword extraction and classification in the different layers such as input, two hidden layers and output layer. Initially, large numbers of legal documents are collected from the dataset. Then the collected legal documents are sent to the input layer of the convolutive deep neural learning. The input legal documents are transferred into the first hidden layer where the keyword extraction process is carried out by applying the Target projective probit Regression. Then the regression function extracts the keywordsbased on frequent occurrence score.Then the extracted keywords are transferred into the second hidden layer where the document classification is performed using the Tversky similarity indexive Rocchio classifier. Likewise, all the legal documents are classified into different classes. The experimental evaluation is carried out using different performancemetrics such as accuracy, precision, recall,F-measure and computational time with respect to the number of legaldocuments collected from the dataset.The observed results confirmed that the presented PRTIRCDNL techniqueprovides the better performance interms of achieving higher accuracy, precision, recall and F-measure with minimum computation time.

Using statistical machine learning methods to optimize well operation

Using statistical machine learning methods to optimize well operation

Variational auto-encoders based on the shift correction for imputation of specific missing in multivariate time series

Data missing is a ubiquitous phenomenon in multivariate time series analysis because of failure measurements, improper installation or other unfavorable factors. Most imputation methods, such as statistical methods and machine learning methods, are mainly for dealing with random or continuous missing, but do not take into consideration the non-random specific missing situation. This paper proposes an imputation model based on the variational auto-encoders (VAE) and shift correction for specific missing values, which is also extended to the β-VAE model. The shift correction is used to correct the original probability distribution deviation caused by specific values concentrated missing. Then taking the meteorological and air quality data sets in Beijing as the example, this paper mainly explores two common specific missing situations, including large values missing and small values missing. The experimental results show that the proposed model can effectively impute specific missing values, which has high imputation accuracy and robustness.

Research on Intelligent Structural Health Monitoring System

The basic concepts of structural health monitoring are explained, the research status and its application in engineering are outlined, and the key activities of the health monitoring system are introduced. Finally, the application of statistical methods and machine learning methods in the process of damage identification is summarized.

Applying Statistical Machine Learning Methods to Analysis Differences in the Severity Level of COVID-19 among Countries

The COVID-19 pandemic has caused a significant negative impact on countries around the world, and there appears to be an observable difference in severity among nations. This study aims to provide an insight into the roles many social and economic factors played in contributing to this variation. By investigating potential patterns through exploratory data analysis, followed by constructing models using several popular machine learning techniques, we examine the validity of the underlying assumptions and identifying any potential limitations. Total deaths per million population is used as dependent variable with log transformation to remove outliers. A set of factors such as life expectancy, unemployment rate and population are available in the dataset. After removing and transforming outliers, various machine learning methods with cross validation are implemented and the optimal model is determined by predefined metrics such as root-mean-squared-error (RMSE) and mean-squared-error (MAE). The results show that the Gradient Boost Machine (GBM) technique achieves the most optimal results in terms of minimum RMSE and MAE. The RMSE and MAE values indicate no over fitting issues and the GBM algorithm captures the most influential factors such as life expectancy, healthcare expense per Gross Domestic Product (GDP) and GDP per capita, which are clearly critical explanatory variables for predicting total deaths per million population.

Early Prediction of Fire Accident in Petroleum Industries by Statistical Machine Learning Algorithm

Due to unpredictability of climatic conditions across the world, early fire forecasting has become more challenging and critical for many oil and gas sectors. It is extremely hard for anyone to predict fires with any degree of certainty, especially in the gas or oil sectors. Until now, the models in use have not been adequate. However, this is critical in order to maintain workers and property safe. As a result, this research work investigates the different approaches available for fire hazard assessment and prediction in order to deal with fire dangers. Also, this research work presents the statistical machine learning methods to detect fire accidents in petroleum industries based on risk index models and risk assessment parameters by performing a statistical process. Moreover, this research work develops a statistical machine learning method to enhance the accuracy in predicting the fire occurrence. Finally, the proposed algorithm is measured by utilizing the performance metrics such as accuracy, proposed risk index, and sensitivity.

Machine learning approach for predicting crude oil stability based on NMR spectroscopy

Crude oils are extremely complex organic mixtures, composed of various constituents ranging in size, shape and polarity. Obtaining a detailed insight into the petroleum composition is of highest priority for quality evaluation of crude oils and crude oil product performances. The stability of crude oils and their components represents one of the major challenges in petroleum industry, since there is no existing single method to determine the stability of all fractions. In this study, statistical multi-way analysis (MWA) and machine learning (ML) methods were coupled with diffusion-ordered NMR spectroscopy (DOSY) and compared to different crude oil stability affecting parameters in order to explore possibilities to predict crude oil stability. The potential of this approach was explored to identify and classify the crude oils of different origin according to their composition, stability, density and diffusion properties. With the application of MWA using the TUCKER3 decomposition model for a set of DOSY NMR spectra, the principal components were determined for the model (5,5,5), which described 99.89 % of the total variance. The reduced space of the first 3 principal components was used for the sample classification. Similar samples were identified, and reduced space was further utilized for the regression of measured stabilities. Extensive ML multivariate linear regression was carried out for modeling crude oil stability in relation to DOSY NMR spectra and other measured properties, such as aromaticity, API gravity, percentage of aliphatic chains, asphaltene content and relative diffusivities. In both MWA and ML cases the best predictive models were determined. For such complex mixtures as crude oils are, exceptionally good correlations were obtained, proving that this new and robust model can accurately predict crude oil stability and other important parameters relevant for petroleum industry thus showing a great potential for practical applications.

Ventricular and atrial function and deformation is largely preserved after arterial switch operation

ObjectiveTo test the hypothesis that ventricular and atrial function are different between patients with transposition of the great arteries (TGA) after arterial switch operation (ASO) and healthy controls.Methods103 consecutive patients...

Product sales probabilistic forecasting: An empirical evaluation using the M5 competition data

Supply chain management depends heavily on uncertain point forecasts of product sales. In order to deal with such uncertainty and optimize safety stock levels, methods that can estimate the right part of the sales distribution are required. Given the limited work that has been done in the field of probabilistic product sales forecasting, we propose and test some novel methods to estimate uncertainty, utilizing empirical computations and simulations to determine quantiles. To do so, we use the M5 competition data to empirically evaluate the forecasting and inventory performance of these methods by making comparisons both with established statistical approaches and advanced machine learning methods. Our results indicate that different methods should be employed based on the quantile of interest and the characteristics of the series being forecast, concluding that methods that employ relatively simple and faster to compute empirical estimations result in better inventory performance than more sophisticated and computer intensive approaches.

An adaptive adversarial domain adaptation approach for corn yield prediction

Recently, statistical machine learning and deep learning methods have been widely explored for corn yield prediction. Though successful, machine learning models generated within a specific spatial domain often lose their validity when directly applied to new regions. To address this issue, we designed an unsupervised adaptive domain adversarial neural network (ADANN). Specifically, through domain adversarial training, the ADANN model reduced the impact of domain shift by projecting data from different domains into the same subspace. Also, the ADANN model was designed to be trained in an adaptive way, which guaranteed the model can learn the domain-invariant features and perform accurate yield prediction simultaneously. Informative variables including time-series vegetation indices and sequential weather observations were first collected from multiple data sources and aggregated to the county level. Then, we trained the ADANN model with the extracted features and corresponding reported county-level corn yield from the U.S. Department of Agriculture (USDA). Finally, the trained model was evaluated in four testing years 2016–2019. The U.S. corn belt was used as the study area and counties under study were grouped into two diverse ecological regions. The experimental results showed that the developed ADANN model had better performance than three other state-of-the-art machine learning models in both local experiments (train and test in the same region) and transfer experiments (train and test in different regions). As the first study using adversarial learning for crop yield prediction, this research demonstrates a novel solution for improving model transferability on crop yield prediction.

Artificial Neuro Probit Regression Based Associative Frequent Pattern Mining for Heart Disease Prediction

Data mining is a technique used to explore the big data from the different sources to obtain the hidden patterns through statistical technique and machine learning methods. Due to implication in the development of a variety of applications, data mining algorithms are applied to a healthcare domain for the prediction of disease at an earlier stage. The various data mining applications have been useful for medical experts to diagnose disease in an accurate method is a challenging one. To improve the accuracy in heart disease prediction using Artificial Neuro Probit Regressed Associative Frequent Pattern Mining (ANPRFPM) model is introduced. Initially, the number of attributes and the data are collected from dataset in the ANPRFPM model (i.e. patterns). The ANPRFPM model trained by the input layer, hidden layer, and the output layer using large set of attributes and patterns. Input layer contains set of attributes and patterns are given as input to the network. In hidden layer of the proposed ANPRFPM model contains probit regressed associative rule mining concept to identify the frequent or non-frequent itemsets based on the support value. Continued by, the confidence is measured between the training and testing patterns of disease and conveyed into the output layer. Finally, soft sign activation function at the output layer for analyzing the confidence value and providing the two classification outcomes as normal or abnormal with higher accuracy in ANPRFPM model. To revolve, the error in the disease prediction gets minimized at the output layer. Using a heart disease dataset with different performance metrics an experimentation is carried out such as prediction accuracy, false-positive rate and prediction time with respect to a number of patterns. In an ANPRFPM model achieves higher prediction accuracy with minimum time as well as the false positive rate than the state-of-the-art methods proves an observed result.

Complex environmental contaminant mixtures and their associations with thyroid hormones using supervised and unsupervised machine learning techniques

Evaluating complex mixtures and their associated health effects poses a challenge in human populations. Herein, we assess the association between 17 organic and metal contaminants in blood with thyroid hormones in a remote Indigenous (First Nations) region from Quebec, Canada (n=526). Using principal component analysis (PCA) to reduce the number of variables, we generated varimax rotated principal component (PC) loadings of contaminants on these uncorrelated synthetic axes. Associations with levels of thyroid hormones (TSH, free T4, and total T3) were conducted using multivariable linear regression methods with the participant PC loadings and adjusting for covariates. Additionally, Bayesian kernel machine regression (BKMR) analysis was used to evaluate the univariate contaminant exposure effect as well as the contaminant mixture effects on levels of thyroid hormones. Significant and positive associations were found between total T3 and PC-2 (high positive nickel and cadmium loadings), total T3 and PC-3 (negative association with negative loading for nickel and positive loading for cadmium) and TSH and PC-1 (high positive loadings for organic contaminants). No significant observations were observed for free T4. BKMR provided additional insight into the PCA results and suggested that nickel, and not cadmium, was responsible for driving the observed effects with this effect remaining when evaluating the entire mixture. BKMR analysis did not support the association of TSH with organic contaminants that were found in the PCA regression. Our findings reinforced other studies which showed that metals such as nickel may alter thyroid hormone levels and highlighted how complex environmental mixtures interact with each other. These observations represent an important step to determining how complex mixtures of contaminants can be assessed in human populations, especially those living a subsistence lifestyle who may have high body burdens of contaminants, and to help understand the resultant net effect of exposures on endogenous thyroid hormones utilizing novel machine learning statistical methods.