Standard Machine Learning Approaches Research Articles

When is it acceptable to break the rules? Knowledge representation of moral judgements based on empirical data

Constraining the actions of AI systems is one promising way to ensure that these systems behave in a way that is morally acceptable to humans. But constraints alone come with drawbacks as in many AI systems, they are not flexible. If these constraints are too rigid, they can preclude actions that are actually acceptable in certain, contextual situations. Humans, on the other hand, can often decide when a simple and seemingly inflexible rule should actually be overridden based on the context. In this paper, we empirically investigate the way humans make these contextual moral judgements, with the goal of building AI systems that understand when to follow and when to override constraints. We propose a novel and general preference-based graphical model that captures a modification of standard dual process theories of moral judgment. We then detail the design, implementation, and results of a study of human participants who judge whether it is acceptable to break a well-established rule: no cutting in line. We then develop an instance of our model and compare its performance to that of standard machine learning approaches on the task of predicting the behavior of human participants in the study, showing that our preference-based approach more accurately captures the judgments of human decision-makers. It also provides a flexible method to model the relationship between variables for moral decision-making tasks that can be generalized to other settings.

Impact of functional electrical stimulation on nerve-damaged muscles by quantifying fat infiltration using deep learning.

Quantitative imaging in life sciences has evolved into a powerful approach combining advanced microscopy acquisition and automated analysis of image data. The focus of the present study is on the imaging-based evaluation of the posterior cricoarytenoid muscle (PCA) influenced by long-term functional electrical stimulation (FES), which may assist the inspiration of patients with bilateral vocal fold paresis. To this end, muscle cross-sections of the PCA of sheep were examined by quantitative image analysis. Previous investigations of the muscle fibers and the collagen amount have not revealed signs of atrophy and fibrosis due to FES by a laryngeal pacemaker. It was therefore hypothesized that regardless of the stimulation parameters the fat in the muscle cross-sections would not be significantly altered. We here extending our previous investigations using quantitative imaging of intramuscular fat in cross-sections. In order to perform this analysis both reliably and faster than a qualitative evaluation and time-consuming manual annotation, the selection of the automated method was of crucial importance. To this end, our recently established deep neural network IMFSegNet, which provides more accurate results compared to standard machine learning approaches, was applied to more than 300 H&E stained muscle cross-sections from 22 sheep. It was found that there were no significant differences in the amount of intramuscular fat between the PCA with and without long-term FES, nor were any significant differences found between the low and high duty cycle stimulated groups. This study on a human-like animal model not only confirms the hypothesis that FES with the selected parameters has no negative impact on the PCA, but also demonstrates that objective and automated deep learning-based quantitative imaging is a powerful tool for such a challenging analysis.

Structure-exploiting interior-point solver for high-dimensional entropy-sparsified regression learning

The solution of high-dimensional nonlinear regression problems through standard machine learning approaches often relies on first-order information, due to the numerical and memory challenges arising from the computation of the Hessian matrix and of the higher-order derivatives. While this scenario seems not favorable to second-order methods, here we show that an efficient and modular structure-exploiting interior-point solver can be successfully applied to the recently introduced class of entropy-based methods for regression learning. Specifically, by exploiting the favorable structure of the problem and of the Hessian matrix, we suggest a robust solution strategy based on explicit low-rank updates combined with an iterative Symmetric Quasi-Minimal Residual (SQMR) algorithm to solve the underlying system of linear equations. The results show that the proposed structure-exploiting solver – which relies on the hybrid parallelism and distributed-memory computing paradigm – allows a significant solution time speed-up with respect to a naive solution strategy. Furthermore, through an adequate use of the Message Passing Interface (MPI) and of Open Multi-Processing (OpenMP), the proposed solver enables the solution of large-scale problems on high-performance computing architectures consisting of thousands of compute nodes. The accompanying detailed convergence and performance analyses demonstrate both numerical robustness and high-performance capabilities for increasingly high-dimensional problems.

A Siamese Network with Node Convolution for Individualized Predictions Based on Connectivity Maps Extracted from Resting-State fMRI Data.

Deep learning has demonstrated great potential for objective diagnosis of neuropsychiatric disorders based on neuroimaging data, which includes the promising resting-state functional magnetic resonance imaging (RS-fMRI). However, the insufficient sample size has long been a bottleneck for deep model training for the purpose. In this study, we proposed a Siamese network with node convolution (SNNC) for individualized predictions based on RS-fMRI data. With the involvement of Siamese network, which uses sample pair (rather than a single sample) as input, the problem of insufficient sample size can largely be alleviated. To adapt to connectivity maps extracted from RS-fMRI data, we applied node convolution to each of the two branches of the Siamese network. For regression purposes, we replaced the contrastive loss in classic Siamese network with the mean square error loss and thus enabled Siamese network to quantitatively predict label differences. The label of a test sample can be predicted based on any of the training samples, by adding the label of the training sample to the predicted label difference between them. The final prediction for a test sample in this study was made by averaging the predictions based on each of the training samples. The performance of the proposed SNNC was evaluated with age and IQ predictions based on a public dataset (Cam-CAN). The results indicated that SNNC can make effective predictions even with a sample size of as small as 40, and SNNC achieved state-of-the-art accuracy among a variety of deep models and standard machine learning approaches.

Identifying treatment non-responders based on pre-treatment gait characteristics - A machine learning approach

BackgroundPaediatric movement disorders such as cerebral palsy often negatively impact walking behaviour. Although clinical gait analysis is usually performed to guide therapy decisions, not all respond positively to their assigned treatment. Identifying these individuals based on their pre-treatment characteristics could guide clinicians towards more appropriate and personalized interventions. Using routinely collected pre-treatment gait and anthropometric features, we aimed to assess whether standard machine learning approaches can be effective in identifying patients at risk of negative treatment outcomes. MethodsObservational data of 119 patients with movement disorders were retrospectively extracted from a local clinical database, comprising sagittal joint angles and spatiotemporal parameters, derived from motion capture data pre- and post-treatment (physiotherapy, orthosis, botulin toxin injections, or surgery). Participants were labelled based on their change in gait profile score (GPS, non-responders with a decline in GPS of <1.6° vs. responders). Their pre-treatment features (sagittal joint angles, spatiotemporal parameters, anthropometrics) were used to train a support vector machine classifier with 5-fold cross-validation and Bayesian optimization within a MATLAB-based Classification Learner App. ResultsAn average accuracy of 88.2 ± 0.5 % was achieved for identifying participants whose gait will not respond to treatment, with 64 % true negative rate and an area under the curve of 88 %. ConclusionOverall, a classical machine learning model was able to identify patients at risk of not responding to treatment, based on gait features and anthropometrics collected prior to treatment. The output of such a model could function as a warning signal, notifying clinicians that a certain individual might not respond well to the standard of care and that a more personalized intervention might be needed.

A reinforcing transfer learning approach to predict buildings energy performance

PurposeThe purpose of this paper is to propose a novel data-driven approach for predicting energy performance of buildings that can address the scarcity of quality data, and consider the dynamic nature of building systems.Design/methodology/approachThis paper proposes a reinforcing machine learning (ML) approach based on transfer learning (TL) to address these challenges. The proposed approach dynamically incorporates the data captured by the building management systems into the model to improve its accuracy.FindingsIt was shown that the proposed approach could improve the accuracy of the energy performance prediction compared to the conventional TL (non-reinforcing) approach by 19 percentage points in mean absolute percentage error.Research limitations/implicationsThe case study results confirm the practicality of the proposed approach and show that it outperforms the standard ML approach (with no transferred knowledge) when little data is available.Originality/valueThis approach contributes to the body of knowledge by addressing the limited data availability in the building sector using TL; and accounting for the dynamics of buildings’ energy performance by the reinforcing architecture. The proposed approach is implemented in a case study project based in London, UK.

Detecting Underwater Concrete Cracks with Machine Learning: A Clear Vision of a Murky Problem

This paper presents the development of an underwater crack detection system for structural integrity assessment of submerged structures, such as offshore oil and gas installations, underwater pipelines, underwater foundations for bridges, dams, etc. Our focus is on the use of machine-learning-based approaches. First, a detailed literature review of the state of the current methods for underwater surface crack detection is presented, highlighting challenges and opportunities. An overview of the image augmentation approach for the creation of underwater optical effects is also presented. Experimental results using a standard network-based machine learning approach, which is used for surface crack detection in onshore environments, are presented. A series of test cases is presented in which existing networks’ performance is improved using augmented images for underwater conditions. The effectiveness and accuracy of the proposed approach in detecting cracks in underwater concrete structures are demonstrated. The proposed approach has the potential to improve the safety and reliability of underwater structures and prevent catastrophic failures.

Estimating individual minimum calibration for deep-learning with predictive performance recovery: An example case of gait surface classification from wearable sensor gait data

Clinical datasets often comprise multiple data points or trials sampled from a single participant. When these datasets are used to train machine learning models, the method used to extract train and test sets must be carefully chosen. Using the standard machine learning approach (random-wise split), different trials from the same participant may appear in both training and test sets. This has led to schemes capable of segregating data points from a same participant into a single set (subject-wise split). Past investigations have demonstrated that models trained in this manner underperform compared to those trained using random-split schemes. Additional training of models via a small subset of trials, known as calibration, bridges the gap in performance across split schemes; however, the amount of calibration trials required to achieve strong model performance is unclear. Thus, this study aims to investigate the relationship between calibration training set size and prediction accuracy on the calibration test set. A database of 30 young, healthy adults performing multiple walking trials across nine different surfaces while fit with inertial measurement unit sensors on the lower limbs was used to develop a deep-learning classifier. For subject-wise trained models, calibration on a single gait cycle per surface yielded a 70% increase in F1-score, the harmonic mean of precision and recall, while 10 gait cycles per surface were sufficient to match the performance of a random-wise trained model. Code to generate calibration curves may be found at (https://github.com/GuillaumeLam/PaCalC).

TRANSFORMATION AND CLASSIFICATION OF ORDINAL SURVEY DATA

Currently, Machine Learning is being significantly used in almost all of the research domains. However, its applicability in survey research is still in its infancy. We in this paper, attempt to highlight the applicability of Machine Learning in survey research while working on two different aspects in parallel. First, we introduce a pattern-based transformation method for ordinal survey data. Our purpose behind developing such a transformation method is twofold. Our transformation facilitates easy interpretation of ordinal survey data and provides convenience while applying standard Machine Learning approaches. Second, we demonstrate the application of various classification techniques over real and transformed ordinal survey data and interpret their results in terms of their suitability in survey research. Our experimental results suggest that Machine Learning coupled with the Pattern Recognition paradigm has a tremendous scope in survey research.

Fundamental limits to learning closed-form mathematical models from data

Given a finite and noisy dataset generated with a closed-form mathematical model, when is it possible to learn the true generating model from the data alone? This is the question we investigate here. We show that this model-learning problem displays a transition from a low-noise phase in which the true model can be learned, to a phase in which the observation noise is too high for the true model to be learned by any method. Both in the low-noise phase and in the high-noise phase, probabilistic model selection leads to optimal generalization to unseen data. This is in contrast to standard machine learning approaches, including artificial neural networks, which in this particular problem are limited, in the low-noise phase, by their ability to interpolate. In the transition region between the learnable and unlearnable phases, generalization is hard for all approaches including probabilistic model selection.

Divide and conquer: A granular concept-cognitive computing system for dynamic classification decision making

Dynamic classification decision making is a crucial issue in management decision making and data mining, which is widely applied in different areas such as human-machine collaborative decision making, network intrusion detection, and traffic data stream mining. However, the existing strategies of static classification decision making are always unable to achieve desired outcomes in ill-structured domains, as the standard machine learning approaches mainly focus on static learning, which is not suitable to mine evolving dynamic data to support decision making. In addition, the main factors regarding incorrect classification predictions are also important for knowledge management and decision making, which is often ignored in many standard learning systems. Therefore, inspired by the idea of divide and conquer, we in this article propose a novel dynamic concept learning framework, namely granular concept-cognitive computing system (gC3S), for dynamic classification decision making by transforming instances into concepts. More specifically, to better characterize the process of dynamic classification decision making, we give the objective function of gC3S via mathematical programming theory. For management decision making, gC3S emphasizes tracing the corresponding approximate concepts via the incorrect classification predictions. Finally, we also apply gC3S to traffic data stream mining, and the experimental results on the different real-world situations further demonstrate that our approach is very effective for dynamic classification decision making.

Towards out of distribution generalization for problems in mechanics

There has been a massive increase in research interest towards applying data driven methods to problems in mechanics, with a particular emphasis on using data driven methods for predictive modeling and design of materials with novel functionality. While traditional machine learning (ML) methods have enabled many breakthroughs, they rely on the assumption that the training (observed) data and testing (unseen) data are independent and identically distributed (i.i.d). However, when these standard ML approaches are applied to real world mechanics problems with unknown test environments, they can be very sensitive to data distribution shifts, and can break down when evaluated on test datasets that violate the i.i.d. assumption. In contrast, out-of-distribution (OOD) generalization approaches assume that the data contained in test environments are allowed to shift (i.e., violate the i.i.d. assumption). To date, multiple methods have been proposed to improve the OOD generalization of ML methods. However, most of these OOD generalization methods have been focused on classification problems, driven in part by the lack of benchmark datasets available for OOD regression problems. Thus, the efficiency of these OOD generalization methods on regression problems, which are typically more relevant to mechanics research than classification problems, is unknown. To address this, we perform a fundamental study of OOD generalization methods for regression problems in mechanics. Specifically, we identify three OOD generalization problems: covariate shift, mechanism shift, and sampling bias. For each problem, we create two benchmark examples that extend the Mechanical MNIST dataset collection, and we investigate the performance of popular OOD generalization methods on these mechanics-specific regression problems. Our numerical experiments show that in most cases, while the OOD algorithms perform better compared to traditional ML methods on these OOD generalization problems, there is a compelling need to develop more robust OOD methods that can generalize the notion of invariance across multiple OOD scenarios. Overall, we expect that this study, as well as the associated open access benchmark datasets, will enable further development of OOD methods for mechanics specific regression problems.

A machine learning model for predicting, diagnosing, and mitigating health disparities in hospital readmission

The management of hyperglycemia in hospitalized patients has a significant impact on both morbidity and mortality. Therefore, it is important to predict the need for diabetic patients to be hospitalized. However, using standard machine learning approaches to make these predictions may result in health disparities caused by biases in the data related to social determinants (such as race, age, and gender). These biases must be removed early in the data collection process, before they enter the system and are reinforced by model predictions, resulting in biases in the model’s decisions. In this paper, we propose a machine learning pipeline capable of making predictions as well as detecting and mitigating biases in the data and model predictions. This pipeline analyses the clinical data and determines whether biases exist in the data, if so, it removes those biases before making predictions. We evaluate the performance of the proposed method on a clinical dataset using accuracy and fairness measures. The findings of the results show that when we mitigate biases early during the data ingestion, we get fairer predictions.

Hyperspectral Image Classification Model Using Squeeze and Excitation Network with Deep Learning.

In the domain of remote sensing, the classification of hyperspectral image (HSI) has become a popular topic. In general, the complicated features of hyperspectral data cause the precise classification difficult for standard machine learning approaches. Deep learning-based HSI classification has lately received a lot of interest in the field of remote sensing and has shown promising results. As opposed to conventional hand-crafted feature-based classification approaches, deep learning can automatically learn complicated features of HSIs with a greater number of hierarchical layers. Because HSI's data structure is complicated, applying deep learning to it is difficult. The primary objective of this research is to propose a deep feature extraction model for HSI classification. Deep networks can extricate features of spatial and spectral from HSI data simultaneously, which is advantageous for increasing the performances of the proposed system. The squeeze and excitation (SE) network is combined with convolutional neural networks (SE-CNN) in this work to increase its performance in extracting features and classifying HSI. The squeeze and excitation block is designed to improve the representation quality of a CNN. Three benchmark datasets are utilized in the experiment to evaluate the proposed model: Pavia Centre, Pavia University, and Salinas. The proposed model's performance is validated by a performance comparison with current deep transfer learning approaches such as VGG-16, Inception-v3, and ResNet-50. In terms of accuracy on each class of datasets and overall accuracy, the proposed SE-CNN model outperforms the compared models. The proposed model achieved an overall accuracy of 96.05% for Pavia University, 98.94% for Pavia Centre dataset, and 96.33% for Salinas dataset.

Graph neural networks for parameter estimation in micro-electro-mechanical system testing

Micro-electro-mechanical systems (MEMS) are of great importance in a broad range of applications including vehicle safety and consumer electronics. During the testing of these devices, large heterogeneous data sets containing a variety of parameters are recorded. Aiming to substitute costly measurements as well as to gain insight into the relations among the measured parameters, graph neural networks (GNNs) are investigated. Thus, the questions are addressed whether for inference of MEMS final module level test parameters, working on graph structures leads to an improvement of the predictive performance compared to the analysis via standard machine learning approaches on tabular data and how the graph structure and learning algorithm contribute to the overall performance. To evaluate this, in an empirical study different graph representations of the acquired test data were set up. On these, four different state-of-the-art GNN architectures were trained and compared on the task of raw sensitivity prediction for a MEMS gyroscope. Whereas the GNNs performed on par with a light gradient boosting machine, neural network and multivariate adaptive regression splines model used as baseline on the complete data set, in the presence of sparse data, the GNNs outperformed the baseline methods in terms of the overall root-mean-square error (RMSE) and achieved distinct improvement in the maximum error when trained on data with similar sparsity rates as observed during the validation. • Proposal and demonstration of GNNs for parameter estimation in MEMS testing. • Application to sensitivity estimation at final module testing of MEMS gyroscopes. • Evaluation of different graph construction variants and four GNN architectures. • Distinct performance improvement in the case of sparse measurement data.

Image Segmentation with Priority Based Apposite Feature Extraction Model for Detection of Multiple Sclerosis in MR Images Using Deep Learning Technique

Multiple Sclerosis (MS) is a degenerative neurological disease caused by damage to the central nervous system's axons and myelin sheaths. MS lesions alter shape, position, and size over time in patients, therefore radiologists must be vigilant in detecting and evaluating MS lesions appropriately. Magnetic resonance imaging (MRI) has gotten a lot of attention from doctors for diagnosing MS, but there are other ways as well. The information provided by MRI modalities to doctors about the brain's anatomy and function is critical for making an MS diagnosis quickly. For automatic extraction of MS lesions from three-dimensional (3D) MR images, this work introduces a new feature selection approach. The approach described here can be used to treat a variety of MS lesions. MS MRI diagnosis takes a long time, is difficult, and is prone to human mistake. The design of a Computer-Aided Diagnosis System (CADS) based on Artificial Intelligence (AI) to diagnose MS incorporates standard machine learning and deep learning approaches. Traditional machine learning uses trial and error to extract, select, and classify features. Deep learning, on the other hand, uses deep layers with values that are automatically learned. In this research work, a Priority based Apposite Feature Extraction Model with Image Segmentation (PAFEM-IS) is proposed for segmentation and feature extraction. With proposed method, a large number of image attributes can be learned with little effort and bias on the part of the user. Apart from that, by using unlabelled data for feature learning, the classifier training can benefit from the significantly larger amount of generally available unlabelled data. The proposed model is compared with the traditional models and the proposed model exhibits better performance levels.

Effective Intrusion Detection System Using Classifier Ensembles

The problem of network intrusion detection poses innumerable challenges to the research community, industry, and commercial sectors. Moreover, the persistent attacks occurring on the cyber-threat landscape compel researchers to devise robust approaches in order to address the recurring problem. Given the presence of huge web traffic, standard machine learning approaches are rather inefficient if adapted in network intrusion detection areas. Instead, a hybrid multiple classifier model when attempted enhances the performance henceforth leading to valid predictions. Thus, novel ensemble approaches are presented in this research work that involves bagged homogeneous classifier ensembles and arcing of heterogeneous ensembles. Then the classification performances of classifier models are assessed using accuracy. Here, classifier ensemble is built using base classifiers such as RBF and SVM. The feasibleness and the advantages of the proposed approaches are illustrated with the help of existing intrusion detection dataset. pre-processing phase, classification phase and combining phase are the three major phases of this proposed method. A broad series of analogous experiments are done for standard dataset of intrusion detection. Furthermore, comparisons with previous work on standard dataset of intrusion detection are also exhibited. The experimental outcomes demonstrate that this proposed ensemble approaches are competitive.

Spatio-temporal classification and prediction of land use and land cover change for the Vembanad Lake system, Kerala: a machine learning approach.

Land use and land cover (LULC) change has become a critical issue for decision planners and conservationists due to inappropriate growth and its effect on natural ecosystems. As a result, the goal of this study is to identify the LULC for the Vembanad Lake system (VLS), Kerala, in the short term, i.e., within a decade, utilizing three standard machine learning approaches, random forest (RF), classification and regression trees (CART), and support vector machines (SVM), on the Google Earth Engine (GEE) platform. When comparing the three techniques, SVM performed poor at an average accuracy of around 82.5%, CART being the next at accuracy of 87.5%, and the RF model being good at the average of 89.5%. The RF outperformed the SVM and CART in almost identical spectral classes such as barren land and built-up areas. As a result, RF-classified LULC is considered to predict the spatio-temporal distribution of LULC transition analysis for 2035 and 2050. The study was conducted in Idrisi TerrSet software using the cellular automata (CA)-Markov chain analysis. The model's efficiency is evaluated by comparing the projected 2019 image to the actual 2019 classified image. The efficiency was good with more than 94.5% accuracy for the classes except for barren land, which might have resulted from the recent natural calamities and the accelerated anthropogenic activity in the area.

Conditional generative models for sampling and phase transition indication in spin systems

In this work, we study generative adversarial networks (GANs) as a tool to learn the distribution of spin configurations and to generate samples, conditioned on external tuning parameters or other quantities associated with individual configurations. For concreteness, we focus on two examples of conditional variables---the temperature of the system and the energy of the samples. We show that temperature-conditioned models can not only be used to generate samples across thermal phase transitions, but also be employed as unsupervised indicators of transitions. To this end, we introduce a GAN-fidelity measure that captures the model’s susceptibility to external changes of parameters. The proposed energy-conditioned models are integrated with Monte Carlo simulations to perform over-relaxation steps, which break the Markov chain and reduce auto-correlations. We propose ways of efficiently representing the physical states in our network architectures, e.g., by exploiting symmetries, and to minimize the correlations between generated samples. A detailed evaluation, using the two-dimensional XY model as an example, shows that these incorporations bring in considerable improvements over standard machine-learning approaches. We further study the performance of our architectures when no training data is provided near the critical region.

Causal Machine Learning and Business Decision Making

Causal knowledge is critical for strategic and organizational decision making. By contrast, standard machine learning approaches remain purely pattern and prediction-based, rendering them unsuitable for being applied to a wide variety of managerial decision problems. Taking a mixed-methods approach, which relies on multiple sources, including semi-structured interviews with data scientists and decision makers, as well as quantitative survey data, this study makes a first attempt at delineating causality as a critical boundary condition for the application of machine learning in business. It highlights the crucial role of theory in causal inference and offers a new perspective on human-machine interaction for data-augmented decision making.