Complex Machine Learning Algorithms Research Articles

Explainable discovery of disease biomarkers: The case of ovarian cancer to illustrate the best practice in machine learning and Shapley analysis

Objective:Ovarian cancer is a significant health issue with lasting impacts on the community. Despite recent advances in surgical, chemotherapeutic and radiotherapeutic interventions, they have had only marginal impacts due to an inability to identify biomarkers at an early stage. Biomarker discovery is challenging, yet essential for improving drug discovery and clinical care. Machine learning (ML) techniques are invaluable for recognising complex patterns in biomarkers compared to conventional methods, yet they can lack physical insights into diagnosis. eXplainable Artificial Intelligence (XAI) is capable of providing deeper insights into the decision-making of complex ML algorithms increasing their applicability. We aim to introduce best practice for combining ML and XAI techniques for biomarker validation tasks. Methods:We focused on classification tasks and a game theoretic approach based on Shapley values to build and evaluate models and visualise results. We described the workflow and apply the pipeline in a case study using the CDAS PLCO Ovarian Biomarkers dataset to demonstrate the potential for accuracy and utility. Results:The case study results demonstrate the efficacy of the ML pipeline, its consistency, and advantages compared to conventional statistical approaches. Conclusion:The resulting guidelines provide a general framework for practical application of XAI in medical research that can inform clinicians and validate and explain cancer biomarkers.

Constrained IoT-Based Machine Learning for Accurate Glycemia Forecasting in Type 1 Diabetes Patients.

Individuals with diabetes mellitus type 1 (DM1) tend to check their blood sugar levels multiple times daily and utilize this information to predict their future glycemic levels. Based on these predictions, patients decide on the best approach to regulate their glucose levels with considerations such as insulin dosage and other related factors. Nevertheless, modern developments in Internet of Things (IoT) technology and innovative biomedical sensors have enabled the constant gathering of glucose level data using continuous glucose monitoring (CGM) in addition to other biomedical signals. With the use of machine learning (ML) algorithms, glycemic level patterns can be modeled, enabling accurate forecasting of this variable. Constrained devices have limited computational power, making it challenging to run complex machine learning algorithms directly on these devices. However, by leveraging edge computing, using lightweight machine learning algorithms, and performing preprocessing and feature extraction, it is possible to run machine learning algorithms on constrained devices despite these limitations. In this paper we test the burdens of some constrained IoT devices, probing that it is feasible to locally predict glycemia using a smartphone, up to 45 min in advance and with acceptable accuracy using random forest.

Modelling forest biomass dynamics in relation to climate change in Romania using complex data and machine learning algorithms

Modelling forest biomass dynamics in relation to climate change in Romania using complex data and machine learning algorithms

Logistic regression technique is comparable to complex machine learning algorithms in predicting cognitive impairment related to post intensive care syndrome

To evaluate the performance of machine learning (ML) models and to compare it with logistic regression (LR) technique in predicting cognitive impairment related to post intensive care syndrome (PICS-CI). We conducted a prospective observational study of ICU patients at two tertiary hospitals. A cohort of 2079 patients was screened, and finally 481 patients were included. Seven different ML models were considered, decision tree (DT), random forest (RF), XGBoost, neural network (NN), naïve bayes (NB), and support vector machine (SVM), and compared with logistic regression (LR). Discriminative ability was evaluated by area under the receiver operating characteristic curve (AUC), calibration belt plots, and Hosmer–Lemeshow test was used to assess calibration. Decision curve analysis was performed to quantify clinical utility. Duration of delirium, poor Richards–Campbell sleep questionnaire (RCSQ) score, advanced age, and sepsis were the most frequent and important candidates risk factors for PICS-CI. All ML models showed good performance (AUC range: 0.822–0.906). NN model had the highest AUC (0.906 [95% CI 0.857–0.955]), which was slightly higher than, but not significantly different from that of LR (0.898 [95% CI 0.847–0.949]) (P > 0.05, Delong test). Given the overfitting and complexity of some ML models, the LR model was then used to develop a web-based risk calculator to aid decision-making (https://model871010.shinyapps.io/dynnomapp/). In a low dimensional data, LR may yield as good performance as other complex ML models to predict cognitive impairment after ICU hospitalization.

Systematic Evaluation of Local and Global Machine Learning Models for the Prediction of ADME Properties.

Machine learning (ML) has become an indispensable tool to predict absorption, distribution, metabolism, and excretion (ADME) properties in pharmaceutical research. ML algorithms are trained on molecular structures and corresponding ADME assay data to develop quantitative structure-property relationship (QSPR) models. Traditional QSPR models were trained on compound sets of limited size. With the advent of more complex ML algorithms and data availability, training sets have become larger and more diverse. Most common training approaches consist in either training a model with a small set of similar compounds, namely, compounds designed for the same drug discovery project or chemical series (local model approach) or with a larger set of diverse compounds (global model approach). Global models are built with all experimental data available for an assay, combining compound data from different projects and disease areas. Despite the ML progress made so far, the choice of the appropriate data composition for building ML models is still unclear. Herein, a systematic evaluation of local and global ML models was performed for 10 different experimental assays and 112 drug discovery projects. Results show a consistent superior performance of global models for ADME property predictions. Diagnostic analyses were also carried out to investigate the influence of training set size, structural diversity, and data shift in the relative performance of local and global ML models. Training set and structural diversity did not have an impact in the relative performance on the methods. Instead, data shift helped to identify the projects with larger performance differences between local and global models. Results presented in this work can be leveraged to improve ML-based ADME properties predictions and thus decision-making in drug discovery projects.

Detecting Data Anomalies from Their Formal Specifications: A Case Study in IoT Systems

We present in this paper a new method in detecting anomalies in datasets representing systems behaviour, which is based on comparing a dataset to the data blueprint of the system representing its normal behaviour. This method removes some of the need for applying complex machine learning algorithms that aim at detecting abnormalities in such datasets and gives a more assured outcome of the presence of abnormalities. Our method first models a system using the formal language of the π-calculus, and then applies an abstract interpretation that ultimately generates an abstract multiset representing the messages exchanged in the system model. We term this multiset as the data blueprint of the system, and it represents the normal behaviour expected. We apply this method to the case of a recent study in literature, which attempts to analyse normal and abnormal behaviour in datasets representing runs of the MQTT protocol, both under attack and no attack conditions. We show that our method is able to detect these conditions in an easier and more straightforward manner than the original case study attempts to.

The Impact of Renewable Electricity Output on Sustainability in the Context of Circular Economy: A Global Perspective

In this article, we investigate the impact of “Renewable Electricity Output” on the green economy in the context of the circular economy for 193 countries in the period 2011–2020. We use data from the World Bank ESG framework. We perform Panel Data with Fixed Effects, Panel Data with Random Effects, Weighted Last Squares-WLS, and Pooled Ordinary Least Squares-OLS. Our results show that Renewable Electricity Output is positively associated, among others, with “Adjusted Savings-Net Forest Depletion” and “Renewable Energy Consumption” and negatively associated, among others, with “CO2 Emission” and “Cooling Degree Days”. Furthermore, we perform a cluster analysis implementing the k-Means algorithm optimized with the Elbow Method and we find the presence of four clusters. In adjunct, we confront seven different machine learning algorithms to predict the future level of “Renewable Electricity Output”. Our results show that Linear Regression is the best algorithm and that the future value of renewable electricity output is predicted to growth on average at a rate of 0.83% for the selected countries. Furthermore, we improve the machine learning analysis with a Deep Learning approach using Convolutional Neural Network-CNN but the algorithm is not appropriate for the analyzed dataset. Less complex machine learning algorithms show better statistical results.

Emerging AI Technologies Inspiring the Next Generation of E-Textiles

The smart textile and wearables sector is looking towards advancing technologies to meet both industry, consumer and new emerging innovative textile application demands, within a fast paced textile industry. In parallel, inspiration based on the biological neural workings of the human brain is driving the next generation of Artificial Intelligence (AI). AI inspired hardware (neuromorphic computing) and software modules mimicking the processing capabilities and properties of neural networks and the human nervous system are taking shape. The textile sector needs to actively look at such emerging and new technologies, taking inspiration from their workings and processing methods in order to stimulate new and innovative embedded intelligence advancements in the e-textile world. This emerging next generation of AI is rapidly gaining interest across varying industries (textile, medical, automotive, aerospace, military). It brings the promise of new innovative applications enabled by low size, weight and processing power technologies. Such properties meet the need for enhanced performing integrated circuits (IC’s) and complex machine learning algorithms. How such properties can inspire and drive advancements within the e-textiles sector needs to be considered. This paper will provide an insight into AI advancements in the e-textiles domain, before focusing specifically on the future vision and direction around the potential application of neuromorphic computing and spiking neural network inspired AI technologies within the textile sector. We investigate the core architectural elements of artificial neural networks, neuromorphic computing (2D and 3D structures) and how such neuroscience inspired technologies could impact and inspire change and new research developments within the e-textile sector.

Interpretation of Explainable AI Methods as Identification of Local Linearized Models

Artificial intelligence (AI) models are increasingly ubiquitous in daily life, their unparalleled predictive and decision-making capabilities are being utilized for applications of all magnitudes, ranging from minor decisions to those with significant impacts on individuals and society. However, many of these models feature a multitude of redundant parameters, which render them incomprehensible to human understanding. The lack of transparency raises concerns about the reliability and fairness of the decisions made by AI models motivating a new field of research called eXplainable AI (XAI), which aims to elucidate complex AI model outcomes and develop tools to enable human understanding. Given the increasing impact of machine learning in the fields of data-driven estimation and control, it becomes crucial to integrate XAI tools with control theory to better comprehend the decisions made by AI models in estimation and control. In this article, we propose the use of an XAI method called Local Interpretable Model-Agnostic Explanations (LIME) to explain the mechanisms behind a black-box estimation algorithm processing time-series data. Moreover, we demonstrate that LIME can be used to identify a local linearized model that approximates the complex machine learning algorithm. We show that the identified local linearized model can shed light on the dynamic of the model that generated the training data.

A Comparitive Study on Brain Tumor Detection Methods using MRI Scans

A brain tumour is formed due to the overgrowth of abnormal cells inside the brain. There are several types of tumours that are broadly classified based on their cancerous properties as malignant tumours or cancerous tumours and benign tumours or non-cancerous tumours. They can either begin in the brain and be known as primary tumours or spread to the brain from another body part and be called metastatic. These tumour developments can be life-threatening if not detected and treated at an early stage. MRI or Magnetic Resonance Imaging Scans serve as the preliminary test to diagnose a tumour. Then, further analysis is done by the neurosurgeons for prescribed medical care and treatment in the future. About 40-50 thousand patients suffer from a brain tumour growth every year in India. Thus, it is crucial to enhance the efficiency of the health sector by automating tasks that require preliminary diagnosis to reduce the burden on doctors and provide critical patients with timely care and treatment. Hence, this project proposes to develop an algorithm to detect brain tumours from MRI Scans on MATLAB and also provides a comparison between the different models that can be implemented to perform this task. The proposed algorithms in this study reflect the importance of creating a system that directly detects tumours without the requirement of complex machine learning algorithms that require the use of training and testing data sets.

Interpretable Machine Learning Techniques in ECG-Based Heart Disease Classification: A Systematic Review.

Heart disease is one of the leading causes of mortality throughout the world. Among the different heart diagnosis techniques, an electrocardiogram (ECG) is the least expensive non-invasive procedure. However, the following are challenges: the scarcity of medical experts, the complexity of ECG interpretations, the manifestation similarities of heart disease in ECG signals, and heart disease comorbidity. Machine learning algorithms are viable alternatives to the traditional diagnoses of heart disease from ECG signals. However, the black box nature of complex machine learning algorithms and the difficulty in explaining a model's outcomes are obstacles for medical practitioners in having confidence in machine learning models. This observation paves the way for interpretable machine learning (IML) models as diagnostic tools that can build a physician's trust and provide evidence-based diagnoses. Therefore, in this systematic literature review, we studied and analyzed the research landscape in interpretable machine learning techniques by focusing on heart disease diagnosis from an ECG signal. In this regard, the contribution of our work is manifold; first, we present an elaborate discussion on interpretable machine learning techniques. In addition, we identify and characterize ECG signal recording datasets that are readily available for machine learning-based tasks. Furthermore, we identify the progress that has been achieved in ECG signal interpretation using IML techniques. Finally, we discuss the limitations and challenges of IML techniques in interpreting ECG signals.

Detection of Schizophrenia from EEG Signals using Dual Tree Complex Wavelet Transform and Machine Learning Algorithms

This research was conducted with the aim to detect schizophrenia automatically from EEG signals using machine learning algorithms. The 16 electrode EEG data were collected from the online repository where 43 schizophrenic and 39 healthy persons’ dataset is available. By applying Low Pass Filter and Total Variation Denoising method, raw EEG signals were denoised and were decomposed into beta, alpha, theta and delta waves by using Dual Tree Complex Wavelet Transform. To apply machine learning algorithms, five features: mean, median, standard deviation, energy and kurtosis were considered for all the four wave bands. With Linear Support Vector Machine and Random Forest classifier machine learning algorithms, 12 out of 16 channels were classified with test accuracy above 95% and F1 score above 90%. Among them, 7 channels were predicted with 100% test accuracy. This research thus has the potential to detect schizophrenia unsupervised and within a noticeably short period of time giving the opportunity to real time monitoring of patients. Hence, people living in remote areas or deprived of adequate healthcare professionals can be benefitted through the outcome of this research.
 Bangladesh Journal of Medical Physics Vol.15 No.1 2022 P 8-27

Determining and Validating Population Differences in Magnetic Resonance Angiography Using Sparse Representation.

Identifying population differences can serve as an insightful tool for diagnostic radiology. To do so, a reliable preprocessing framework and data representation are vital. We build a machine learning model to visualize gender differences in the circle of Willis (CoW), an integral part of the brain's vasculature. We start with a dataset of 570 individuals and process them for analysis using 389 for the final analysis. We find statistical differences between male and female patients in one image plane and visualize where they are. We can see differences between the right and left-hand sides of the brain confirmed using Support Vector Machines (SVM). This process can be applied to detect population variations in the vasculature automatically. It can guide debugging and inferring complex machine learning algorithms such as SVM and deep learning models.

Electric energy consumption predictions for residential buildings: Impact of data-driven model and temporal resolution on prediction accuracy

This study aims to investigate the measurement parameters for predicting the electric energy consumption of residential buildings using a data-driven model. Herein, the temporal resolution of data and algorithms that can improve prediction accuracy are comparatively investigated. For the investigation, the time units of the data collected from the monitoring system of an actual residential building are set as 10 min and 1 h. Further, algorithms such as multiple linear regression (MLR), multilayer perceptron (MLP), support vector machine (SVM), and random forest (RF) are employed to predict the electric energy consumption of the building. The parameters of the data collection include electric energy consumption based on the usage type, occupancy information, and indoor environmental information. The model is validated using a K-fold technique, and the prediction accuracy is compared using R 2 and the t -value. Analyses using seven input variables reveal that the prediction accuracy for the 1 h interval data is better than that for the 10 min interval data, based on the temporal resolution of the data. In addition, the results of the algorithms reveal that the prediction accuracy is the highest when the MLR algorithm is used, followed by those when using the RF, MLP, and SVM algorithms. A relatively simple statistical method and low-resolution data rather than a complex machine learning algorithm or high-resolution data achieved the best prediction accuracy. These results are expected to facilitate high-accuracy predictions of the electric energy consumption of residential buildings. • Predictions of residential energy consumption were studied using a data-driven model. • Energy consumption information was measured and collected in a residential building. • The collected data were usage type, occupancy, and indoor environmental information. • The prediction accuracy is compared based on temporal resolutions and algorithms. • These results will be useful for the energy management of residential buildings.

Accelerating machine learning at the edge with approximate computing on FPGAs

Performing inference of complex machine learning (ML) algorithms at the edge is becoming important to unlink the system functionality from the cloud. However, the ML models increase complexity faster than the available hardware resources. This research aims to accelerate machine learning by offloading the computation to low-end FPGAs and using approximate computing techniques to optimise resource usage, taking advantage of the inaccurate nature of machine learning models. In this paper, we propose a generic matrix multiply-add processing element design, parameterised in datatype, matrix size, and data width. We evaluate the resource consumption and error behaviour while varying the matrix size and the data width given a fixed-point data type. We determine that the error scales with the matrix size, but it can be compensated by increasing the data width, posing a trade-off between data width and matrix size with respect to the error.

Proximity Detection During Epidemics: Direct UWB TOA Versus Machine Learning Based RSSI.

In this paper, we compare the direct TOA-based UWB technology with the RSSI-based BLE technology using machine learning algorithms for proximity detection during epidemics in terms of complexity of implementation, availability in existing smart phones, and precision of the results. We establish the theoretical limits on the precision and confidence of proximity estimation for both technologies using the Cramer Rao Lower Bound (CRLB) and validate the theoretical foundations using empirical data gathered in diverse practical operating scenarios. We perform our empirical experiments at eight distances in three flat environments and one non-flat environment encompassing both Line of Sight (LOS) and Obstructed-LOS (OLOS) situations. We also analyze the effects of various postures (eight angles) of the person carrying the sensor, and four on-body locations of the sensor. To estimate the range with BLE RSSI, we use 14 features for training the Gradient Boosted Machines (GBM) learning algorithm and we compare the precision of results with those obtained from memoryless UWB TOA ranging algorithm. We show that the memoryless UWB TOA algorithm achieves 93.60% confidence, slightly outperforming the 92.85% confidence of the BLE RSSI with more complex GBM machine learning (ML) algorithm and the need for substantial training. The training process for the RSSI-based BLE social distance measurements involved 3000 measurements to create a training dataset for each scenario and post-processing of data to extract 14 features of RSSI, and the ML classification algorithm consumed 200 s of computational time. The memoryless UWB ranging algorithm achieves more robust results without any need for training in less than 0.5 s of computation time.Graphical

Multifunctional 2D MoS2 Optoelectronic Artificial Synapse with Integrated Arithmetic and Reconfigurable Logic Operations for In‐Memory Neuromorphic Computing Applications

AbstractThe hardware implementation of advanced artificial intelligence (AI) technology based on complex deep learning and machine learning algorithms is constricted by the limitation of conventional Von–Neuman architecture. Emerging neuromorphic computing architecture based on the human brain with in‐memory computing capability could instigate unprecedented breakthroughs in AI technology. In this pursuit, 2D MoS2 optoelectronic artificial synapse imitating complex biological neuromorphic behavior such as short/long‐term memory, paired‐pulse facilitation, and long‐term depression‐potentiation is proposed and demonstrated. Furthermore, the broadband sensitivity of the device can be utilized to emulate Pavlov's classical conditioning for associative learning of the biological brain. More importantly, reconfigurable Boolean AND and OR logic gate operation is demonstrated within the same device by synergistically modulating the device conductance via the persistent photoconductivity and electrical gate stress. The linear response of the photocurrent to the optical stimulus can perform arithmetic operations such as counting, addition, and subtraction within a single device. This novel integration of memory, synaptic behavior, and processing within a single monolayer MoS2 device is believed to put forth a new horizon for the Non‐Von–Neuman type in‐memory computing architecture for highly advanced AI applications based on 2D materials.

Machine learning-enabled nanosafety assessment of multi-metallic alloy nanoparticles modified TiO2 system

Establishing toxicological predictive modeling frameworks for heterogeneous nanomaterials is crucial for rapid environmental and health risk assessment. However, existing structure-toxicity correlation models for such nanomaterials are only based on simple linear regression algorithms that are prone to underfitting the training data. These models rely heavily on experimental and expensive computational quantum mechanical descriptors, which significantly limit their practical use. Herein, we present the application of empirical descriptors and complex machine learning algorithms to the development of high-performance quantitative structure-toxicity relationship (QSTR) models of TiO2 hybridized with multi-metallic (Ag, Au, Pt) alloy nanoparticles (multi-metallic NPs/TiO2). To confirm the viability of empirical descriptors as model input, we selected five distinct machine learning algorithms for predicting the toxicity of multi-metallic alloy NPs/TiO2 system in Chinese hamster ovary cell line. Notably, an empirical descriptor-based QSTR model (kernel ridge regression) revealed a predictive performance that is on par with density functional theory (DFT) descriptor-based counterparts. More specifically, the results indicated that model selection is influenced by descriptor choice, such that complex DFT descriptors worked best with a complex algorithm (random forest regression; RMSET = 0.0954, MAET = 0.0811, RT2 = 0.9411), whereas more straightforward empirical descriptors were most suitable with a simpler algorithm (kernel ridge regression; RMSET = 0.1244, MAET = 0.1106, RT2 = 0.8999). Moreover, our model outperforms existing QSAR models built on the same data set. This study offers a new perspective on using empirical features to develop accurate predictive computational models for the rapid discovery and profiling of safe-by-design nanomaterials.

A CRITICAL EXAMINATION OF SHIELDING THE CYBERSPACE: A REVIEW ON THE ROLE OF AI IN CYBER SECURITY

This review focuses on the critical analysis of AI’s role in cybersecurity to investigate the issues in the cybersecurity space. By utilizing synchronous and scattered sources on the topic in question, the opinion paper focuses capacity on AI-embedded technologies to enhance cybersecurity and discourage cyber-attacks making critical infrastructure impenetrable by malicious persons [1]. The paper is concerned with applying AI solutions paired with research documenting theoretical outcomes obtained from academic sources, field reports, and practical case examples. Suggestions, pros, and cons of those solutions, as well as the implications, are presented. More importantly, the paper does not only touch on the ethical and policy aspects of AI application in cyber security, but there is also a need for a detailed approach to cyber security, an approach that unites technological innovation to privacy, transparency, and accountability [1]. In contrast, the article brings attention to future developments and innovations in AI-based cybersecurity, which is a crucial need of the hour. AI evolution is seen in different aspects, such as designing complex Machine Learning algorithms till the inclusion of AI in Security Operations Centers (SOCs) and Threat Intelligence Platforms, which have prepared reliable weapons for cyber defense systems.

Auto Labeling Methods Developed Through Semi-Weakly Supervised Learning in Prognostics and Health Management Applications for Rolling Ball Bearing

In recent years, complex machine learning and deep learning algorithms have been used in prognostics and health management applications to monitor the state of industrial systems in real-time and predict failures in advance. However, the performance of these algorithms depends on the size of the collected labeled datasets. This study proposes a semi-weakly supervised learning method that creates label functions using a small amount of data and, consequently, combines weak supervision and semi-supervision methods to expand the labeled dataset for training. The proposed model training method creates labels automatically in the life prediction dataset of rolling element bearings, which were manually labeled by expert groups. The performance of the deep neural network is continuously improved through experiments, and a performance improvement similar to that of manually labeled training data is presented. Future research applications include meta-learner experiments (e.g., use of pre-trained word embedding or other model architectures) and more appropriate selection methods and definitions for the basic learner.