High-performance Machine Learning Algorithms Research Articles

A general framework of high-performance machine learning algorithms: application in structural mechanics

Data-driven models utilizing powerful artificial intelligence (AI) algorithms have been implemented over the past two decades in different fields of simulation-based engineering science. Most numerical procedures involve processing data sets developed from physical or numerical experiments to create closed-form formulae to predict the corresponding systems’ mechanical response. Efficient AI methodologies that will allow the development and use of accurate predictive models for solving computational intensive engineering problems remain an open issue. In this research work, high-performance machine learning (ML) algorithms are proposed for modeling structural mechanics-related problems, which are implemented in parallel and distributed computing environments to address extremely computationally demanding problems. Four machine learning algorithms are proposed in this work and their performance is investigated in three different structural engineering problems. According to the parametric investigation of the prediction accuracy, the extreme gradient boosting with extended hyper-parameter optimization (XGBoost-HYT-CV) was found to be more efficient regarding the generalization errors deriving a 4.54% residual error for all test cases considered. Furthermore, a comprehensive statistical analysis of the residual errors and a sensitivity analysis of the predictors concerning the target variable are reported. Overall, the proposed models were found to outperform the existing ML methods, where in one case the residual error was decreased by 3-fold. Furthermore, the proposed algorithms demonstrated the generic characteristic of the proposed ML framework for structural mechanics problems.

Interpretable by Design: Learning Predictors by Composing Interpretable Queries.

There is a growing concern about typically opaque decision-making with high-performance machine learning algorithms. Providing an explanation of the reasoning process in domain-specific terms can be crucial for adoption in risk-sensitive domains such as healthcare. We argue that machine learning algorithms should be interpretable by design and that the language in which these interpretations are expressed should be domain- and task-dependent. Consequently, we base our model's prediction on a family of user-defined and task-specific binary functions of the data, each having a clear interpretation to the end-user. We then minimize the expected number of queries needed for accurate prediction on any given input. As the solution is generally intractable, following prior work, we choose the queries sequentially based on information gain. However, in contrast to previous work, we need not assume the queries are conditionally independent. Instead, we leverage a stochastic generative model (VAE) and an MCMC algorithm (Unadjusted Langevin) to select the most informative query about the input based on previous query-answers. This enables the online determination of a query chain of whatever depth is required to resolve prediction ambiguities. Finally, experiments on vision and NLP tasks demonstrate the efficacy of our approach and its superiority over post-hoc explanations.

Applications of machine learning in neuroscience and inspiration of reinforcement learning for computational neuroscience

High-performance machine learning algorithms have always been one of the concerns of many researchers. Since its birth, machine learning has been a product of multidisciplinary integration. Especially in the field of neuroscience, models from related fields continue to inspire the development of neural networks and deepen people's understanding of neural networks. The mathematical and quantitative modeling approach to research brought about by machine learning is also feeding into the development of neuroscience. One of the emerging products of this is computational neuroscience. Computational neuroscience has been pushing the boundaries of models of brain function in recent years, and just as early studies of visual hierarchy influenced neural networks, computational neuroscience has great potential to lead to higher performance machine learning algorithms, particularly in the development of deep learning algorithms with strong links to neuroscience. In this paper, it first reviews the help and achievements of machine learning for neuroscience in recent years specially in fMRI image recognition and look at the possibilities for the future development of neural networks due to the recent development of the computational neuroscience in psychiatry of the temporal difference model for dopamine and serotonin.

Development and validation of a machine learning-based detection system to improve precision screening for medication errors in the neonatal intensive care unit.

Aim: To develop models that predict the presence of medication errors (MEs) (prescription, preparation, administration, and monitoring) using machine learning in NICU patients. Design: Prospective, observational cohort study randomized with machine learning (ML) algorithms. Setting: A 22-bed capacity NICU in Ankara, Turkey, between February 2020 and July 2021. Results: A total of 11,908 medication orders (28.9 orders/patient) for 412 NICU patients (5.53 drugs/patient/day) who received 2,280 prescriptions over 32,925 patient days were analyzed. At least one physician-related ME and nurse-related ME were found in 174 (42.2%) and 235 (57.0%) of the patients, respectively. The parameters that had the highest correlation with ME occurrence and subsequently included in the model were: total number of drugs, anti-infective drugs, nervous system drugs, 5-min APGAR score, postnatal age, alimentary tract and metabolism drugs, and respiratory system drugs as patient-related parameters, and weekly working hours of nurses, weekly working hours of physicians, and number of nurses' monthly shifts as care provider-related parameters. The obtained model showed high performance to predict ME (AUC: 0.920; 95% CI: 0.876-0.970) presence and is accessible online (http://softmed.hacettepe.edu.tr/NEO-DEER_Medication_Error/). Conclusion: This is the first developed and validated model to predict the presence of ME using work environment and pharmacotherapy parameters with high-performance ML algorithms in NICU patients. This approach and the current model hold the promise of implementation of targeted/precision screening to prevent MEs in neonates. Clinical Trial Registration: ClinicalTrials.gov, identifier NCT04899960.

Data-driven state-of-health estimation for lithium-ion battery based on aging features

Reliable state-of-health (SOH) estimation is crucial to the safe operation of lithium-ion battery. Data-driven SOH estimation becomes a hot research topic with the booming of high-performance machine learning algorithms. The effectiveness of a data-driven approach will be enhanced significantly if the input features are extracted properly. In order to improve the SOH estimation accuracy, the features highly associated with battery degradation should be utilized in the data-driven model. In this paper, an aging feature extraction method based on electrochemical model (EM) is proposed to account for the battery degradation mechanisms. The aging features of EM such as charge transfer resistance, solid phase diffusion coefficient and electrode volume fraction are defined as internal health features (IHFs) for SOH estimation. Moreover, external health features (EHFs) are directly extracted from the voltage and temperature curves that split into multiple stages. Then, IHFs and multi-stage EHFs are selected appropriately for SOH estimation in offline and online application scenarios. Finally, two well-known machine learning algorithms are employed to construct data-driven SOH estimation model using IHFs and EHFs. Experimental data are used to prove that the proposed method can effectively improve the accuracy of SOH estimation under different application scenarios and battery charge–discharge modes.

An Intelligent Two-Stage Energy Dispatch Management System for Hybrid Power Plants: Impact of Machine Learning Deployment

The utilization of renewable energy sources such as PV and wind power has become imperative due to the increase of carbon dioxide emissions, which leads to the increase in global temperature and the negative consequences of climate change. As a result, renewable energy sources are constantly gaining popularity to be integrated in power systems to create hybrid power plants (HPPs). However, HPPs come with great complications due to the uncertainty in renewable energy output, which has given rise to the need for a reliable and effective energy dispatch management system for HPPs. In this paper, a two-stage machine learning (ML) based energy dispatch management system for HPPs is designed to control renewable energy sources (PV and wind power), reserve energy sources (energy storage systems) and backup energy sources (diesel, fuel cells, auxiliary loads, etc.). The system aims to minimize the power variance in the HPPs to achieve peak shaving and valley filling. The first stage aims to forecast the power output of renewable energy sources, as well as the load demand. The second stage aims to coordinate the energy output of the reserve and backup sources to achieve the required objective. Different ML techniques were compared to find the highest performing ML algorithm to achieve the required objective of the system, where long short-term memory (LSTM) provided the highest results with an average mean squared error of 0.005 and an average explained variance score of 0.9. The results of the management system verify the effectiveness of the system for the management of the energy dispatch in HPPs, through the successful flattening of the load curve of the HPP, which increases the reliability of the power system with the integration of renewable energy sources. Also, the system was shown to be robust against the uncertainty of the PV and wind power output, and the load demand.

Extraction of Photovoltaic Plants Using Machine Learning Methods: A Case Study of the Pilot Energy City of Golmud, China

Solar energy is an abundant, clean, and renewable source that can mitigate global climate change, environmental pollution, and energy shortage. However, comprehensive datasets and efficient identification models for the spatial distribution of photovoltaic (PV) plants locally and globally over time remain limited. In the present study, a model that combines original spectral features, PV extraction indexes, and terrain features for the identification of PV plants is established based on the pilot energy city Golmud in China, which covers 71,298.7 km2 and has the highest density of PV plants in the world. High-performance machine learning algorithms were integrated with PV plant extraction models, and performances of the XGBoost, random forest (RF), and support vector machine (SVM) algorithms were compared. According to results from the investigations, the XGBoost produced the highest accuracy (OA = 99.65%, F1score = 0.9631) using Landsat 8 OLI imagery. The total area occupied by PV plants in Golmud City in 2020 was 10,715.85 ha based on the optimum model. The model also revealed that the area covered by the PV plant park in the east of Golmud City increased by approximately 10% from 2018 (5344.2 ha) to 2020 (5879.34 ha). The proposed approach in this study is one of the first attempts to identify time-series large-scale PV plants based on a pixel-based machine learning algorithm with medium-resolution free images in an efficient way. The study also confirmed the effectiveness of combining original spectral features, PV extraction indexes, and terrain features for the identification of PV plants. It will shed light on larger- and longer-scale identification of PV plants around the world and the evaluation of the associated dynamics of PV plants.

Boosting phase-contrast MRI performance in idiopathic normal pressure hydrocephalus diagnostics by means of machine learning approach.

Phase-contrast MRI allows detailed measurements of various parameters of CSF motion. This examination is technically demanding and machine dependent. The literature on this topic is ambiguous. Machine learning (ML) approaches have already been successfully utilized in medical research, but none have yet been applied to enhance the results of CSF flowmetry. The aim of this study was to evaluate the possible contribution of ML algorithms in enhancing the utilization and results of MRI flowmetry in idiopathic normal pressure hydrocephalus (iNPH) diagnostics. The study cohort consisted of 30 iNPH patients and 15 healthy controls examined on one MRI machine. All major phase-contrast parameters were inspected: peak positive, peak negative, and average velocities; peak amplitude; positive, negative, and average flow rates; and aqueductal area. The authors applied ML algorithms to 85 complex features calculated from a phase-contrast study. The most distinctive parameters with p < 0.005 were the peak negative velocity, peak amplitude, and negative flow. From the ML algorithms, the Adaptive Boosting classifier showed the highest specificity and best discrimination potential overall, with 80.4% ± 2.9% accuracy, 72.0% ± 5.6% sensitivity, 84.7% ± 3.8% specificity, and 0.812 ± 0.047 area under the receiver operating characteristic curve (AUC). The highest sensitivity was 85.7% ± 5.6%, reached by the Gaussian Naive Bayes model, and the best AUC was 0.854 ± 0.028 by the Extra Trees classifier. Feature extraction algorithms combined with ML approaches simplify the utilization of phase-contrast MRI. The highest-performing ML algorithm was Adaptive Boosting, which showed good calibration and discrimination on the testing data, with 80.4% accuracy, 72.0% sensitivity, 84.7% specificity, and 0.812 AUC. Phase-contrast MRI boosted by the ML approach can help to determine shunt-responsive iNPH patients.

Prediction of Shunt Responsiveness in Suspected Patients With Normal Pressure Hydrocephalus Using the Lumbar Infusion Test: A Machine Learning Approach

Machine learning (ML) approaches can significantly improve the classical Rout-based evaluation of the lumbar infusion test (LIT) and the clinical management of the normal pressure hydrocephalus. To develop a ML model that accurately identifies patients as candidates for permanent cerebral spinal fluid shunt implantation using only intracranial pressure and electrocardiogram signals recorded throughout LIT. This was a single-center cohort study of prospectively collected data of 96 patients who underwent LIT and 5-day external lumbar cerebral spinal fluid drainage (external lumbar drainage) as a reference diagnostic method. A set of selected 48 intracranial pressure/electrocardiogram complex signal waveform features describing nonlinear behavior, wavelet transform spectral signatures, or recurrent map patterns were calculated for each patient. After applying a leave-one-out cross-validation training-testing split of the data set, we trained and evaluated the performance of various state-of-the-art ML algorithms. The highest performing ML algorithm was the eXtreme Gradient Boosting. This model showed a good calibration and discrimination on the testing data, with an area under the receiver operating characteristic curve of 0.891 (accuracy: 82.3%, sensitivity: 86.1%, and specificity: 73.9%) obtained for 8 selected features. Our ML model clearly outperforms the classical Rout-based manual classification commonly used in clinical practice with an accuracy of 62.5%. This study successfully used the ML approach to predict the outcome of a 5-day external lumbar drainage and hence which patients are likely to benefit from permanent shunt implantation. Our automated ML model thus enhances the diagnostic utility of LIT in management.

Low-Power On-Chip Implementation of Enhanced SVM Algorithm for Sensors Fusion-Based Activity Classification in Lightweighted Edge Devices

Smart homes assist users by providing convenient services from activity classification with the help of machine learning (ML) technology. However, most of the conventional high-performance ML algorithms require relatively high power consumption and memory usage due to their complex structure. Moreover, previous studies on lightweight ML/DL models for human activity classification still require relatively high resources for extremely resource-limited embedded systems; thus, they are inapplicable for smart homes’ embedded system environments. Therefore, in this study, we propose a low-power, memory-efficient, high-speed ML algorithm for smart home activity data classification suitable for an extremely resource-constrained environment. We propose a method for comprehending smart home activity data as image data, hence using the MNIST dataset as a substitute for real-world activity data. The proposed ML algorithm consists of three parts: data preprocessing, training, and classification. In data preprocessing, training data of the same label are grouped into further detailed clusters. The training process generates hyperplanes by accumulating and thresholding from each cluster of preprocessed data. Finally, the classification process classifies input data by calculating the similarity between the input data and each hyperplane using the bitwise-operation-based error function. We verified our algorithm on ‘Raspberry Pi 3’ and ‘STM32 Discovery board’ embedded systems by loading trained hyperplanes and performing classification on 1000 training data. Compared to a linear support vector machine implemented from Tensorflow Lite, the proposed algorithm improved memory usage to 15.41%, power consumption to 41.7%, performance up to 50.4%, and power per accuracy to 39.2%. Moreover, compared to a convolutional neural network model, the proposed model improved memory usage to 15.41%, power consumption to 61.17%, performance to 57.6%, and power per accuracy to 55.4%.

A new auditory algorithm in stock market prediction on oil and gas sector in Nigerian stock exchange

Stock market prediction is the process of forecasting future prices of stocks. Stock market prediction is a challenging process as a result of uncertainties that influence the market change of price. This paper proposes a nature-inspired algorithm, called Auditory Algorithm (AA), which follows the pathway of the auditory system like that of the human ear. The performance of AA is compared with that of high performance machine learning algorithms and continuous-time stochastic process. The machine learning algorithms used in this paper are Logistic Regression (LR), Support Vector Machine (SVM), Feed forward neural network (FFN) and Recurrent Neural Network (RNN) while continuous-time models such as Stochastic Differential Equation (SDE) and Geometric Brownian Motion (GBM) are also used. The results show that the overall performance of AA is superior to that of other algorithms compared in this paper, as it drastically reduced the forecast error to the barest minimum.

Robust video content analysis schemes for human action recognition.

Action recognition is a challenging time series classification task that has received much attention in the recent past due to its importance in critical applications, such as surveillance, visual behavior study, topic discovery, security, and content retrieval. The main objective of the research is to develop a robust and high-performance human action recognition techniques. A combination of local and holistic feature extraction methods used through analyzing the most effective features to extract to reach the objective, followed by using simple and high-performance machine learning algorithms. This paper presents three robust action recognition techniques based on a series of image analysis methods to detect activities in different scenes. The general scheme architecture consists of shot boundary detection, shot frame rate re-sampling, and compact feature vector extraction. This process is achieved by emphasizing variations and extracting strong patterns in feature vectors before classification. The proposed schemes are tested on datasets with cluttered backgrounds, low- or high-resolution videos, different viewpoints, and different camera motion conditions, namely, the Hollywood-2, KTH, UCF11 (YouTube actions), and Weizmann datasets. The proposed schemes resulted in highly accurate video analysis results compared to those of other works based on four widely used datasets. The First, Second, and Third Schemes provides recognition accuracies of 57.8%, 73.6%, and 52.0% on Hollywood2, 94.5%, 97.0%, and 59.3% on KTH, 94.5%, 95.6%, and 94.2% on UCF11, and 98.9%, 97.8% and 100% on Weizmann. Each of the proposed schemes provides high recognition accuracy compared to other state-of-art methods. Especially, the Second Scheme as it gives excellent comparable results to other benchmarked approaches.

Prediction of Worse Functional Status After Surgery for Degenerative Cervical Myelopathy: A Machine Learning Approach.

Surgical decompression for degenerative cervical myelopathy (DCM) is one of the mainstays of treatment, with generally positive outcomes. However, some patients who undergo surgery for DCM continue to show functional decline. To use machine learning (ML) algorithms to determine predictors of worsening functional status after surgical intervention for DCM. This is a retrospective analysis of prospectively collected data. A total of 757 patients enrolled in 2 prospective AO Spine clinical studies, who underwent surgical decompression for DCM, were analyzed. The modified Japanese Orthopedic Association (mJOA) score, a marker of functional status, was obtained before and 1 yr postsurgery. The primary outcome measure was the dichotomized change in mJOA at 1 yr according to whether it was negative (worse functional status) or non-negative. After applying an 80:20 training-testing split of the dataset, we trained, optimized, and tested multiple ML algorithms to evaluate algorithm performance and determine predictors of worse mJOA at 1 yr. The highest-performing ML algorithm was a polynomial support vector machine. This model showed good calibration and discrimination on the testing data, with an area under the receiver operating characteristic curve of 0.834 (accuracy: 74.3%, sensitivity: 88.2%, specificity: 72.4%). Important predictors of functional decline at 1 yr included initial mJOA, male gender, duration of myelopathy, and the presence of comorbidities. The reasons for worse mJOA are frequently multifactorial (eg, adjacent segment degeneration, tandem lumbar stenosis, ongoing neuroinflammatory processes in the cord). This study successfully used ML to predict worse functional status after surgery for DCM and to determine associated predictors.

The Detection of Opioid Misuse and Heroin Use From Paramedic Response Documentation: Machine Learning for Improved Surveillance.

BackgroundTimely, precise, and localized surveillance of nonfatal events is needed to improve response and prevention of opioid-related problems in an evolving opioid crisis in the United States. Records of naloxone administration found in prehospital emergency medical services (EMS) data have helped estimate opioid overdose incidence, including nonhospital, field-treated cases. However, as naloxone is often used by EMS personnel in unconsciousness of unknown cause, attributing naloxone administration to opioid misuse and heroin use (OM) may misclassify events. Better methods are needed to identify OM.ObjectiveThis study aimed to develop and test a natural language processing method that would improve identification of potential OM from paramedic documentation.MethodsFirst, we searched Denver Health paramedic trip reports from August 2017 to April 2018 for keywords naloxone, heroin, and both combined, and we reviewed narratives of identified reports to determine whether they constituted true cases of OM. Then, we used this human classification as reference standard and trained 4 machine learning models (random forest, k-nearest neighbors, support vector machines, and L1-regularized logistic regression). We selected the algorithm that produced the highest area under the receiver operating curve (AUC) for model assessment. Finally, we compared positive predictive value (PPV) of the highest performing machine learning algorithm with PPV of searches of keywords naloxone, heroin, and combination of both in the binary classification of OM in unseen September 2018 data.ResultsIn total, 54,359 trip reports were filed from August 2017 to April 2018. Approximately 1.09% (594/54,359) indicated naloxone administration. Among trip reports with reviewer agreement regarding OM in the narrative, 57.6% (292/516) were considered to include information revealing OM. Approximately 1.63% (884/54,359) of all trip reports mentioned heroin in the narrative. Among trip reports with reviewer agreement, 95.5% (784/821) were considered to include information revealing OM. Combined results accounted for 2.39% (1298/54,359) of trip reports. Among trip reports with reviewer agreement, 77.79% (907/1166) were considered to include information consistent with OM. The reference standard used to train and test machine learning models included details of 1166 trip reports. L1-regularized logistic regression was the highest performing algorithm (AUC=0.94; 95% CI 0.91-0.97) in identifying OM. Tested on 5983 unseen reports from September 2018, the keyword naloxone inaccurately identified and underestimated probable OM trip report cases (63 cases; PPV=0.68). The keyword heroin yielded more cases with improved performance (129 cases; PPV=0.99). Combined keyword and L1-regularized logistic regression classifier further improved performance (146 cases; PPV=0.99).ConclusionsA machine learning application enhanced the effectiveness of finding OM among documented paramedic field responses. This approach to refining OM surveillance may lead to improved first-responder and public health responses toward prevention of overdoses and other opioid-related problems in US communities.

Commercial quality “awards” are not a strong indicator of quality surgical care

BackgroundThis study aimed to determine whether publicized hospital rankings can be used to predict surgical outcomes. MethodsPatients undergoing one of nine surgical procedures were identified, using the Healthcare Cost and Utilization Project State Inpatient Database for Florida and New York 2011–2013 and merged with hospital data from the American Hospital Association Annual Survey. Nine quality designations were analyzed as possible predictors of inpatient mortality and postoperative complications, using logistic regression, decision trees, and support vector machines. ResultsWe identified 229,657 patients within 177 hospitals. Decision trees were the highest performing machine learning algorithm for predicting inpatient mortality and postoperative complications (accuracy 0.83, P<.001). The top 3 variables associated with low surgical mortality (relative impact) were Hospital Compare (42), total procedure volume (16) and, Joint Commission (12). When analyzed separately for each individual procedure, hospital quality awards were not predictors of postoperative complications for 7 of the 9 studied procedures. However, when grouping together procedures with a volume-outcome relationship, hospital ranking becomes a significant predictor of postoperative complications. ConclusionHospital quality rankings are not a reliable indicator of quality for all surgical procedures. Hospital and provider quality must be evaluated with an emphasis on creating consistent, reliable, and accurate measures of quality that translate to improved patient outcomes.

AZOrange - High performance open source machine learning for QSAR modeling in a graphical programming environment

BackgroundMachine learning has a vast range of applications. In particular, advanced machine learning methods are routinely and increasingly used in quantitative structure activity relationship (QSAR) modeling. QSAR data sets often encompass tens of thousands of compounds and the size of proprietary, as well as public data sets, is rapidly growing. Hence, there is a demand for computationally efficient machine learning algorithms, easily available to researchers without extensive machine learning knowledge. In granting the scientific principles of transparency and reproducibility, Open Source solutions are increasingly acknowledged by regulatory authorities. Thus, an Open Source state-of-the-art high performance machine learning platform, interfacing multiple, customized machine learning algorithms for both graphical programming and scripting, to be used for large scale development of QSAR models of regulatory quality, is of great value to the QSAR community.ResultsThis paper describes the implementation of the Open Source machine learning package AZOrange. AZOrange is specially developed to support batch generation of QSAR models in providing the full work flow of QSAR modeling, from descriptor calculation to automated model building, validation and selection. The automated work flow relies upon the customization of the machine learning algorithms and a generalized, automated model hyper-parameter selection process. Several high performance machine learning algorithms are interfaced for efficient data set specific selection of the statistical method, promoting model accuracy. Using the high performance machine learning algorithms of AZOrange does not require programming knowledge as flexible applications can be created, not only at a scripting level, but also in a graphical programming environment.ConclusionsAZOrange is a step towards meeting the needs for an Open Source high performance machine learning platform, supporting the efficient development of highly accurate QSAR models fulfilling regulatory requirements.