High-performance Machine Learning Model Research Articles

Designing clinically translatable artificial intelligence systems for high-dimensional medical imaging

The National Institutes of Health in 2018 identified key focus areas for the future of artificial intelligence in medical imaging, creating a foundational roadmap for research in image acquisition, algorithms, data standardization and translatable clinical decision support systems. Among the key issues raised in the report, data availability, the need for novel computing architectures and explainable artificial intelligence algorithms are still relevant, despite the tremendous progress made over the past few years alone. Furthermore, translational goals of data sharing, validation of performance for regulatory approval, generalizability and mitigation of unintended bias must be accounted for early in the development process. In this Perspective, we explore challenges unique to high-dimensional clinical imaging data, in addition to highlighting some of the technical and ethical considerations involved in developing machine learning systems that better represent the high-dimensional nature of many imaging modalities. Furthermore, we argue that methods that attempt to address explainability, uncertainty and bias should be treated as core components of any clinical machine learning system. Substantial advances have been made in the past decade in developing high-performance machine learning models for medical applications, but translating them into practical clinical decision-making processes remains challenging. This Perspective provides insights into a range of challenges specific to high-dimensional, multimodal medical imaging.

A novel integrated approach of ELM and modified equilibrium optimizer for predicting soil compression index of subgrade layer of Dedicated Freight Corridor

This study proposes a high-performance machine learning model to sidestep the time of conducting actual laboratory tests of soil compression index (Cc), one of the important criteria for determining the settlement of subgrade layers of roadways, railways, and airport runways. The suggested method combines the modified equilibrium optimizer (MEO) and the extreme learning machine (ELM) in a novel way. In this study, Gaussian mutation with an exploratory search mechanism was incorporated to construct the MEO and used to enhance the performance of conventional ELM by optimizing its learning parameters. PCA (Principal component analysis)-based results exhibit that the developed ELM-MEO attained the most precise prediction with R2 = 0.9746, MAE = 0.0184, and RMSE = 0.0284 in training, and R2 = 0.9599, MAE = 0.0232, and RMSE = 0.0357 in the testing phase. The results showed that the proposed ELM-MEO model outperformed the other developed models, confirming the ELM-MEO model's superiority over the other models, such as random forest, gradient boosting machine, genetic programming, including the ELM and artificial neural network (ANN)-based models optimized with equilibrium optimizer, particle swarm optimization, Harris hawks optimization, slime mould algorithm, and marine predators algorithm. Based on the experimental results, the proposed ELM-MEO can be used as a promising alternative to predict soil Cc in civil engineering projects, including rail and road projects.

Development of a high-performance machine learning model to predict ground ozone pollution in typical cities of China

High ozone concentrations have adverse effects on human health and ecosystems. In recent years, the ambient ozone concentration in China has shown an upward trend, and high-quality prediction of ozone concentrations has become critical to support effective policymaking. In this study, a novel hybrid model combining wavelet decomposition (WD), a gated recurrent unit (GRU) neural network and a support vector regression (SVR) model was developed to predict the daily maximum 8 h ozone. We used the ground ozone observation data in six representative megacities across China from Jan. 1, 2015 to Jun. 15, 2020 for model training, and we used data from Jun. 15 to Dec. 31, 2020 for model testing. The results show that the developed model performs very well for megacities; against observations, the model obtains an average cross-validated R2 (coefficient of determination) ranging from 0.90 for Shanghai to 0.97 for Chengdu in the one-step predictions, thereby indicating that the model outperformed any single algorithm or other hybrid algorithms reported. The developed model can also capture high ozone pollution episodes with an average accuracy of 92% for the next five days in inland cities. This study will be useful for the environmental health community to prevent high ozone exposure more efficiently in megacities in China and shows great potential for accurate ozone prediction using machine learning approaches.

Rule-based explanations based on ensemble machine learning for detecting sink mark defects in the injection moulding process

Manufacturing quality control (QC) in plastic injection moulding is of the upmost importance since almost one third of plastic products are manufactured via the injection moulding process. Moreover, smart manufacturing technologies are enabling the generation of huge amounts of data in production lines. This data can be used for predicting the quality of manufactured plastic products using machine learning methods, allowing companies to save costs and improve their production efficiency. However, high-performance machine learning models are usually too complicated to be understood by human intuition. Therefore, we have introduced a rule-based explanations (RBE) framework that combines several machine learning interpretation methods to help to understand the decision mechanisms of accurate and complex predictive models – specifically tree ensemble models. These generated rules can be used to visually and easily understand the main factors that affect the quality in the manufacturing process. To demonstrate the applicability of RBE, we present two experiments with real industrial data gathered from a plastic injection moulding machine in a Singapore model factory. The collected datasets contain condition data for several manufacturing processes as well as the QC results for sink mark defects in the production of small plastic products. The experiments revealed that it is possible to extract meaningful explanations in the form of simple decision rules that are enhanced with partial dependence plots and feature importance rankings for a better understanding of the underlying mechanisms and data relationships of accurate tree ensembles.

Distinct composition and metabolic functions of human gut microbiota are associated with cachexia in lung cancer patients

Cachexia is associated with decreased survival in cancer patients and has a prevalence of up to 80%. The etiology of cachexia is poorly understood, and limited treatment options exist. Here, we investigated the role of the human gut microbiome in cachexia by integrating shotgun metagenomics and plasma metabolomics of 31 lung cancer patients. The cachexia group showed significant differences in the gut microbial composition, functional pathways of the metagenome, and the related plasma metabolites compared to non-cachectic patients. Branched-chain amino acids (BCAAs), methylhistamine, and vitamins were significantly depleted in the plasma of cachexia patients, which was also reflected in the depletion of relevant gut microbiota functional pathways. The enrichment of BCAAs and 3-oxocholic acid in non-cachectic patients were positively correlated with gut microbial species Prevotella copri and Lactobacillus gasseri, respectively. Furthermore, the gut microbiota capacity for lipopolysaccharides biosynthesis was significantly enriched in cachectic patients. The involvement of the gut microbiome in cachexia was further observed in a high-performance machine learning model using solely gut microbial features. Our study demonstrates the links between cachectic host metabolism and specific gut microbial species and functions in a clinical setting, suggesting that the gut microbiota could have an influence on cachexia with possible therapeutic applications.

Prenatal exposure to ambient fine particulate matter and early childhood neurodevelopment: A population-based birth cohort study

Although previous studies have reported the adverse effect of air pollution exposure during pregnancy on neurodevelopment in children, epidemiological evidence is limited, and the results are inconsistent. This study aimed to explore the association between prenatal ambient fine particulate matter (PM2.5) exposure and early childhood neurodevelopment in a large birth cohort study of 4009 maternal-child pairs. Prenatal daily PM2.5 exposure concentrations at 1 km spatial revolution were estimated using high-performance machine-learning models. Neurodevelopmental outcomes of children at ages 2, 6, 12, and 24 months were assessed using the Ages and Stages Questionnaire (ASQ). Distributed lag nonlinear models were used to identify critical windows of prenatal PM2.5 exposure. General linear mixed models with binomially distributed errors were used to estimate the effect of prenatal PM2.5 exposure on suspected developmental delay (SDD) in five developmental domains based on the longitudinal design. Prenatal PM2.5 exposure was significantly associated with decreased scores for all neurodevelopmental domains of children at ages 2, 6, and 24 months. Each 10-μg/m3 increase in PM2.5 exposure was significantly associated with increased risk of SDD for all subjects (RR: 1.52 95% CI: 1.19, 2.03), specifically, in problem-solving domain for girls (RR: 2.23, 95% CI: 1.22, 4.35). Prenatal PM2.5 exposure in weeks 18 to 34 was significantly associated with both ASQ scores and SDDs. Our study proposed that prenatal PM2.5 exposure affected early childhood neurodevelopment evaluated with the ASQ scale. PM2.5 exposure might increase the risk of SDD for boys and girls, specifically in the problem-solving domain for girls.

Feature selection using cloud-based parallel genetic algorithm for intrusion detection data classification

With the exponential growth of the amount of data being generated, stored and processed on a daily basis in the machine learning, data analytics and decision-making systems, the data preprocessing established itself as the key factor for building reliable high-performance machine learning models. One of the roles in preprocessing is variable reduction using feature selection methods; however, the processing time needed for these methods is a major drawback. This study aims at mitigating this problem by migrating the algorithm to a MapReduce implementation suitable for parallelization on a high number of commodity hardware units. The genetic algorithm-based methods were put in the focus of this study. Hadoop, an open-source MapReduce library, was used as a framework for implementing parallel genetic algorithms within our research. The representative machine learning methods, SVM (support vector machine), ANN (artificial neural network), RT (random tree), logistic regression and Naive Bayes, were embedded into implementation for feature selection. The feature selection methods were applied to four NSL-KDD data sets, and the number of features is reduced from cca 40 to cca 10 data sets with the accuracy of 90.45%. These results have both significant practical and theoretical impact. On the one hand, the genetic algorithm has been parallelized in the MapReduce manner, which has been considered unachievable in a strict sense. Furthermore, the genetic algorithm allows randomness-enhanced feature selection and its parallelization reduces overall data preprocessing and allows larger population count which in turn leads to better feature selection. On the practical side, it has been shown that this implementation outperforms the existing feature selection methods.

High Performance Machine Learning Models of Large Scale Air Pollution Data in Urban Area

Abstract Preserving the air quality in urban areas is crucial for the health of the population as well as for the environment. The availability of large volumes of measurement data on the concentrations of air pollutants enables their analysis and modelling to establish trends and dependencies in order to forecast and prevent future pollution. This study proposes a new approach for modelling air pollutants data using the powerful machine learning method Random Forest (RF) and Auto-Regressive Integrated Moving Average (ARIMA) methodology. Initially, a RF model of the pollutant is built and analysed in relation to the meteorological variables. This model is then corrected through subsequent modelling of its residuals using the univariate ARIMA. The approach is demonstrated for hourly data on seven air pollutants (O3, NOx, NO, NO2, CO, SO2, PM10) in the town of Dimitrovgrad, Bulgaria over 9 years and 3 months. Six meteorological and three time variables are used as predictors. High-performance models are obtained explaining the data with R2 = 90%-98%.

The 17-y spatiotemporal trend of PM2.5 and its mortality burden in China

Investigations on the chronic health effects of fine particulate matter (PM2.5) exposure in China are limited due to the lack of long-term exposure data. Using satellite-driven models to generate spatiotemporally resolved PM2.5 levels, we aimed to estimate high-resolution, long-term PM2.5 and associated mortality burden in China. The multiangle implementation of atmospheric correction (MAIAC) aerosol optical depth (AOD) at 1-km resolution was employed as a primary predictor to estimate PM2.5 concentrations. Imputation techniques were adopted to fill in the missing AOD retrievals and provide accurate long-term AOD aggregations. Monthly PM2.5 concentrations in China from 2000 to 2016 were estimated using machine-learning approaches and used to analyze spatiotemporal trends of adult mortality attributable to PM2.5 exposure. Mean coverage of AOD increased from 56 to 100% over the 17-y period, with the accuracy of long-term averages enhanced after gap filling. Machine-learning models performed well with a random cross-validation R2 of 0.93 at the monthly level. For the time period outside the model training window, prediction R2 values were estimated to be 0.67 and 0.80 at the monthly and annual levels. Across the adult population in China, long-term PM2.5 exposures accounted for a total number of 30.8 (95% confidence interval [CI]: 28.6, 33.2) million premature deaths over the 17-y period, with an annual burden ranging from 1.5 (95% CI: 1.3, 1.6) to 2.2 (95% CI: 2.1, 2.4) million. Our satellite-based techniques provide reliable long-term PM2.5 estimates at a high spatial resolution, enhancing the assessment of adverse health effects and disease burden in China.

A Grey-Box Ensemble Model Exploiting Black-Box Accuracy and White-Box Intrinsic Interpretability

Machine learning has emerged as a key factor in many technological and scientific advances and applications. Much research has been devoted to developing high performance machine learning models, which are able to make very accurate predictions and decisions on a wide range of applications. Nevertheless, we still seek to understand and explain how these models work and make decisions. Explainability and interpretability in machine learning is a significant issue, since in most of real-world problems it is considered essential to understand and explain the model’s prediction mechanism in order to trust it and make decisions on critical issues. In this study, we developed a Grey-Box model based on semi-supervised methodology utilizing a self-training framework. The main objective of this work is the development of a both interpretable and accurate machine learning model, although this is a complex and challenging task. The proposed model was evaluated on a variety of real world datasets from the crucial application domains of education, finance and medicine. Our results demonstrate the efficiency of the proposed model performing comparable to a Black-Box and considerably outperforming single White-Box models, while at the same time remains as interpretable as a White-Box model.

Machine Learning Applications in Endocrinology and Metabolism Research: An Overview.

Machine learning (ML) applications have received extensive attention in endocrinology research during the last decade. This review summarizes the basic concepts of ML and certain research topics in endocrinology and metabolism where ML principles have been actively deployed. Relevant studies are discussed to provide an overview of the methodology, main findings, and limitations of ML, with the goal of stimulating insights into future research directions. Clear, testable study hypotheses stem from unmet clinical needs, and the management of data quality (beyond a focus on quantity alone), open collaboration between clinical experts and ML engineers, the development of interpretable high-performance ML models beyond the black-box nature of some algorithms, and a creative environment are the core prerequisites for the foreseeable changes expected to be brought about by ML and artificial intelligence in the field of endocrinology and metabolism, with actual improvements in clinical practice beyond hype. Of note, endocrinologists will continue to play a central role in these developments as domain experts who can properly generate, refine, analyze, and interpret data with a combination of clinical expertise and scientific rigor.

Can we justify producing univariate machine-learning forecasts with satellite-derived solar irradiance?

Ground-based radiometry is the most reliable method for obtaining data for solar energy applications. However, due to potential calibration and data quality-control issues, as well as the high cost associated with instrumentation, reliable and long-term ground-based measurements are scarce. For this reason, irradiance data from alternative sources, such as modeled or remote-sensed data, are of interest. Nonetheless, data coming from these sources are known to have systematic errors that may deteriorate the performance of subsequent applications. This study aims to evaluate the quality of solar forecasts generated using two types of irradiance data, namely, ground-based and bias-corrected satellite-derived global horizontal irradiance using error decomposition frameworks. Hourly forecasts are generated using 5 high-performance machine-learning models, with ground-based data from 15 research-grade monitoring stations in Europe, South America, and Africa, and satellite-derived data at the collocated pixels, separately. On top of these component forecasts, ensemble forecasts are also evaluated. Instead of limiting the forecast verification to a set of accuracy measures (e.g., mean bias error or root mean square error), the Murphy–Winkler forecast verification framework is considered, in which the joint, conditional, and marginal distributions of observations and forecasts are examined. The framework describes the qualities of forecast, such as calibration, conditional bias, or discrimination, with quantitative measures. Hence, it provides additional insights to the traditional measure-oriented approach. The main results show that the forecasts generated using bias-corrected satellite-derived data have comparable quality as compared to those generated using ground-based data.

Predicting monthly high-resolution PM2.5 concentrations with random forest model in the North China Plain

Exposure to fine particulate matter (PM2.5) remains a worldwide public health issue. However, epidemiological studies on the chronic health impacts of PM2.5 in the developing countries are hindered by the lack of monitoring data. Despite the recent development of using satellite remote sensing to predict ground-level PM2.5 concentrations in China, methods for generating reliable historical PM2.5 exposure, especially prior to the construction of PM2.5 monitoring network in 2013, are still very rare. In this study, a high-performance machine-learning model was developed directly at monthly level to estimate PM2.5 levels in North China Plain. We developed a random forest model using the latest Multi-angle implementation of atmospheric correction (MAIAC) aerosol optical depth (AOD), meteorological parameters, land cover and ground PM2.5 measurements from 2013 to 2015. A multiple imputation method was applied to fill the missing values of AOD. We used 10-fold cross-validation (CV) to evaluate model performance and a separate time period, January 2016 to December 2016, was used to validate our model's capability of predicting historical PM2.5 concentrations. The overall model CV R2 and relative prediction error (RPE) were 0.88 and 18.7%, respectively. Validation results beyond the modeling period (2013–2015) shown that this model can accurately predict historical PM2.5 concentrations at the monthly (R2 = 0.74, RPE = 27.6%), seasonal (R2 = 0.78, RPE = 21.2%) and annual (R2 = 0.76, RPE = 16.9%) level. The annual mean predicted PM2.5 concentration from 2013 to 2016 in our study domain was 67.7 μg/m3 and Southern Hebei, Western Shandong and Northern Henan were the most polluted areas. Using this computationally efficient, monthly and high-resolution model, we can provide reliable historical PM2.5 concentrations for epidemiological studies on PM2.5 health effects in China.

Editorial of the special issue: adaptive and intelligent systems (AIS) for learning, control and optimization in dynamic environments

An increasing number of real world applications, such as network traffic, network monitoring, power systems etc., generates non-stationary behaviors. Therefore, the model built in order to predict their dynamics, optimize their performance or control their behavior must be adapted over time. During the last decade, the modeling based on learning in non-stationary environments has received more and more attention in the machine learning and data mining communities. This is because the requirements from practical (industrial) side induced a permanent paradigm shift from classical batch off-line learning procedures (as conducted throughout the 80ties and 90ties to build up accurate and high-performance machine learning models) to incremental, adaptive algorithms which are able to evolve the model structures, architectures and parameters fully autonomously and on the fly, typical in a single-pass manner. It is especially the case within the scope of dynamic data streaming context, changing environmental conditions or as part of large-scale problems, e.g. web mining, multi-sensor networks, sequential video analysis, Big Data or predictive maintenance in factories for the future (FoF) as part of the Industry 4.0 activities (even an important standpoint of the Horizon 2020 objective programme see https://ec.europa.eu/programmes/horizon2020/en/h2020-sections-projects). The necessity of such algorithms is also underlined by the increasing size of the databases and storages, which induces that conventional batch learning systems cannot be applied within a reasonable time frame and sufficient accuracy. Thus, a block-wise loading of the data has to be carried out, which again requires the usage of incremental and evolving learning algorithms. Thereby the learning algorithms are either guided by a permanent update of model parameters and structures, leading to a so-called anytime up-to-date experience, or by occasional update cycles