Machine Learning Inference Research Articles

Fast Inference for Machine Learning in ROOT/TMVA

ROOT provides, through TMVA, machine learning tools for data analysis at HEP experiments and beyond. However, with the rapidly evolving ecosystem for machine learning, the focus of TMVA is shifting. We present the new developments and strategy of TMVA, which will allow the analyst to integrate seamlessly, and effectively, different workflows in the diversified machine-learning landscape. Focus is put on a fast machine learning inference system, which will enable analysts to deploy their machine learning models rapidly on large scale datasets. We present the technical details of a fast inference system for decision tree algorithms, included in the next ROOT release (6.20). We further present development status and proposal for a fast inference interface and code generator for ONNX-based Deep Learning models.

Machine Learning with ROOT/TMVA

ROOT provides, through TMVA, machine learning tools for data analysis at HEP experiments and beyond. We present recently included features in TMVA and the strategy for future developments in the diversified machine learning landscape. Focus is put on fast machine learning inference, which enables analysts to deploy their machine learning models rapidly on large scale datasets. The new developments are paired with newly designed C++ and Python interfaces supporting modern C++ paradigms and full interoperability in the Python ecosystem. We present as well a new deep learning implementation for convolutional neural network using the cuDNN library for GPU. We show benchmarking results in term of training time and inference time, when comparing with other machine learning libraries such as Keras/Tensorflow.

Design of task-specific optical systems using broadband diffractive neural networks

Deep learning has been transformative in many fields, motivating the emergence of various optical computing architectures. Diffractive optical network is a recently introduced optical computing framework that merges wave optics with deep-learning methods to design optical neural networks. Diffraction-based all-optical object recognition systems, designed through this framework and fabricated by 3D printing, have been reported to recognize hand-written digits and fashion products, demonstrating all-optical inference and generalization to sub-classes of data. These previous diffractive approaches employed monochromatic coherent light as the illumination source. Here, we report a broadband diffractive optical neural network design that simultaneously processes a continuum of wavelengths generated by a temporally incoherent broadband source to all-optically perform a specific task learned using deep learning. We experimentally validated the success of this broadband diffractive neural network architecture by designing, fabricating and testing seven different multi-layer, diffractive optical systems that transform the optical wavefront generated by a broadband THz pulse to realize (1) a series of tuneable, single-passband and dual-passband spectral filters and (2) spatially controlled wavelength de-multiplexing. Merging the native or engineered dispersion of various material systems with a deep-learning-based design strategy, broadband diffractive neural networks help us engineer the light–matter interaction in 3D, diverging from intuitive and analytical design methods to create task-specific optical components that can all-optically perform deterministic tasks or statistical inference for optical machine learning.

FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing

New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that potentially requires minimal modification to the current computing model. As examples, we retrain the ResNet-50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC and apply a ResNet-50 model with transfer learning for neutrino event classification. Using Project Brainwave by Microsoft to accelerate the ResNet-50 image classification model, we achieve average inference times of 60 (10) milliseconds with our experimental physics software framework using Brainwave as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600--700 inferences per second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.

P5706Finding predictors and causes of cardiac surgery ICU readmission using machine learning and causal inference

Abstract Background “Bounce-back” to the intensive care unit (ICU) occurs when patients return to the ICU for critical changes in clinical status within the same hospital admission. Bounce-backs post-cardiac surgery increase resource utilisation, total cost of care, are associated with higher mortality and morbidity. However, prediction of bounce-back has proved to be challenging. Previous work addressed the feasibility of predicting bounce-back, but these models required significant physician input to design and calibrate the predictive variables. Purpose We aimed to develop an automated machine learning model that would identify patients at risk of bounce-back by selecting the most relevant variables from those available before onset of bounce-back. Additionally, we highlight the differences between predictive and causal inference, to demonstrate that purely associative methods of prediction can mislead clinical decision-making. Methods Clinical records of adult cardiac surgery patients between 2011 to 2016 were collected from our institutional Society for Thoracic Surgeons (STS) database and our institutional electronic health record (EHR) system. For bounce-back prediction, an L1 regularised logistic regression model was applied, which also automatically determined important variables with highest prediction effect from the initial 151 variables. For causal inference, the g-computation algorithm was used to compare the differences between causal and predictive regression effects. We quantified the performance of our system on clinically relevant metrics such as specificity, sensitivity, and area under the ROC curve (AUC). Results Of the 6189 patients, 357 (5.7%) bounced back to the ICU. The prediction model achieved an AUC score of 0.75 (0.03) and 22% specificity at 95% sensitivity, Further analysis showed 79% of the false positive patients had faced other severe postoperative complications but none of the false negative patients had downstream complications. Subsequent causal analysis revealed that the actual causal effects of treatments differed from the predictive model estimates, e.g. administration of intra-operative tranexamic acid increased the probability of bounce-back by 13% but its causal effect on bounce-back after removing confounders was negligible (an increase of only 0.5%). Conclusions Our predictive machine-learning model can successfully predict patients at risk of ICU bounce-backs, using linked STS registry data with the comprehensive electronic health record. The prediction model automatically detects important subset of variables. In addition, we note that causal and predictive model estimates of the same parameters differed, indicating that reliance on predictive models for interventional clinical decision-making may not be appropriate. Acknowledgement/Funding National Institutes of Health, Office of Naval Research, Defense Advanced Research Projects Agency

Semantic Information G Theory and Logical Bayesian Inference for Machine Learning

An important problem in machine learning is that, when using more than two labels, it is very difficult to construct and optimize a group of learning functions that are still useful when the prior distribution of instances is changed. To resolve this problem, semantic information G theory, Logical Bayesian Inference (LBI), and a group of Channel Matching (CM) algorithms are combined to form a systematic solution. A semantic channel in G theory consists of a group of truth functions or membership functions. In comparison with the likelihood functions, Bayesian posteriors, and Logistic functions that are typically used in popular methods, membership functions are more convenient to use, providing learning functions that do not suffer the above problem. In Logical Bayesian Inference (LBI), every label is independently learned. For multilabel learning, we can directly obtain a group of optimized membership functions from a large enough sample with labels, without preparing different samples for different labels. Furthermore, a group of Channel Matching (CM) algorithms are developed for machine learning. For the Maximum Mutual Information (MMI) classification of three classes with Gaussian distributions in a two-dimensional feature space,only 2–3 iterations are required for the mutual information between three classes and three labels to surpass 99% of the MMI for most initial partitions For mixture models, the Expectation-Maximization (EM) algorithm is improved to form the CM-EM algorithm, which can outperform the EM algorithm when the mixture ratios are imbalanced, or when local convergence exists. The CM iteration algorithm needs to combine with neural networks for MMI classification in high-dimensional feature spaces. LBI needs further investigation for the unification of statistics and logic.

Matrix Mapping on Crossbar Memory Arrays with Resistive Interconnects and Its Use in In-Memory Compression of Biosignals

Recent advances in nanoscale resistive memory devices offer promising opportunities for in-memory computing with their capability of simultaneous information storage and processing. The relationship between current and memory conductance can be utilized to perform matrix-vector multiplication for data-intensive tasks, such as training and inference in machine learning and analysis of continuous data stream. This work implements a mapping algorithm of memory conductance for matrix-vector multiplication using a realistic crossbar model with finite cell-to-cell resistance. An iterative simulation calculates the matrix-specific local junction voltages at each crosspoint, and systematically compensates the voltage drop by multiplying the memory conductance with the ratio between the applied and real junction potential. The calibration factors depend both on the location of the crosspoints and the matrix structure. This modification enabled the compression of Electrocardiographic signals, which was not possible with uncalibrated conductance. The results suggest potential utilities of the calibration scheme in the processing of data generated from mobile sensing or communication devices that requires energy/areal efficiencies.

Smart, Secure, Yet Energy-Efficient, Internet-of-Things Sensors

The proliferation of Internet-of-Things (IoT) has led to the generation of zettabytes of sensitive data each year. The generated data are usually raw, requiring cloud resources for processing and decision-making operations to extract valuable information (i.e., distill smartness). Use of cloud resources raises serious design issues: limited bandwidth, insufficient energy, and security concerns. Edge-side computing and cryptographic techniques have been proposed to get around these problems. However, as a result of increased computational load and energy consumption, it is difficult to simultaneously achieve smartness, security, and energy efficiency. We propose a novel way out of this predicament by employing signal compression and machine learning inference on the IoT sensor node. An important sensor operation scenario is for the sensor to transmit data to the base station immediately when an event of interest occurs, e.g., arrhythmia is detected by a smart electrocardiogram sensor or seizure is detected by a smart electroencephalogram sensor, and transmit data on a less urgent basis otherwise. Since on-sensor compression and inference drastically reduce the amount of data that need to be transmitted, we actually end up with a dramatic energy bonus relative to the traditional sense-and-transmit IoT sensor. We use a part of this energy bonus to carry out encryption and hashing to ensure data confidentiality and integrity. We analyze the effectiveness of this approach on six different IoT applications with two data transmission scenarios: alert notification and continuous notification. The experimental results indicate that relative to the traditional sense-and-transmit sensor, IoT sensor energy is reduced by $57.1\times$ for electrocardiogram (ECG) sensor based arrhythmia detection, $379.8\times$ for freezing of gait detection in the context of Parkinson's disease, $139.7\times$ for electroencephalogram (EEG) sensor based seizure detection, $216.6\times$ for human activity classification, $162.8\times$ for neural prosthesis spike sorting, and $912.6\times$ for chemical gas classification. Our approach not only enables the IoT system to push signal processing and decision-making to the extreme of the edge-side (i.e., the sensor node), but also solves data security and energy efficiency problems simultaneously.

Edge Machine Learning: Enabling Smart Internet of Things Applications

Machine learning has traditionally been solely performed on servers and high-performance machines. However, advances in chip technology have given us miniature libraries that fit in our pockets and mobile processors have vastly increased in capability narrowing the vast gap between the simple processors embedded in such things and their more complex cousins in personal computers. Thus, with the current advancement in these devices, in terms of processing power, energy storage and memory capacity, the opportunity has arisen to extract great value in having on-device machine learning for Internet of Things (IoT) devices. Implementing machine learning inference on edge devices has huge potential and is still in its early stages. However, it is already more powerful than most realise. In this paper, a step forward has been taken to understand the feasibility of running machine learning algorithms, both training and inference, on a Raspberry Pi, an embedded version of the Android operating system designed for IoT device development. Three different algorithms: Random Forests, Support Vector Machine (SVM) and Multi-Layer Perceptron, respectively, have been tested using ten diverse data sets on the Raspberry Pi to profile their performance in terms of speed (training and inference), accuracy, and power consumption. As a result of the conducted tests, the SVM algorithm proved to be slightly faster in inference and more efficient in power consumption, but the Random Forest algorithm exhibited the highest accuracy. In addition to the performance results, we will discuss their usability scenarios and the idea of implementing more complex and taxing algorithms such as Deep Learning on these small devices in more details.

Computational Power and the Social Impact of Artificial Intelligence

Machine learning is a computational process. To that end, it is inextricably tied to computational power - the tangible material of chips and semiconductors that the algorithms of machine intelligence operate on. Most obviously, computational power and computing architectures shape the speed of training and inference in machine learning, and therefore influence the rate of progress in the technology. But, these relationships are more nuanced than that: hardware shapes the methods used by researchers and engineers in the design and development of machine learning models. Characteristics such as the power consumption of chips also define where and how machine learning can be used in the real world. Despite this, many analyses of the social impact of the current wave of progress in AI have not substantively brought the dimension of hardware into their accounts. While a common trope in both the popular press and scholarly literature is to highlight the massive increase in computational power that has enabled the recent breakthroughs in machine learning, the analysis frequently goes no further than this observation around magnitude. This paper aims to dig more deeply into the relationship between computational power and the development of machine learning. Specifically, it examines how changes in computing architectures, machine learning methodologies, and supply chains might influence the future of AI. In doing so, it seeks to trace a set of specific relationships between this underlying hardware layer and the broader social impacts and risks around AI. On one hand, this examination shines a spotlight on how hardware works to exacerbate a range of concerns around ubiquitous surveillance, technological unemployment, and geopolitical conflict. On the other, it also highlights the potentially significant role that shaping the development of computing power might play in addressing these concerns.

Factor Analysis From Quadratic Sampling

Factor analysis decomposition, i.e., decomposition of a covariance matrix as a sum of a low-rank positive semidefinite matrix and a diagonal matrix is an important problem in a variety of areas, such as signal processing, machine learning, system identification, and statistical inference. In this letter, the focus is on computing the factor analysis decomposition from a set of quadratic (or symmetric rank-one) measurements of a covariance matrix. Commonly used minimum trace factor analysis heuristic can be adapted to solve this problem when all the measurements are available. However, the resulting convex program is not suitable for processing large-scale or streaming data. Therefore, this letter presents a low-complexity iterative algorithm, which recovers the unknowns through a series of rank-one updates. The iterative algorithm performs better than the convex program when only a finite number of data snapshots are available.

Improved proximal ADMM with partially parallel splitting for multi-block separable convex programming

For a type of multi-block separable convex programming raised in machine learning and statistical inference, we propose a proximal alternating direction method of multiplier with partially parallel splitting, which has the following nice properties: (1) to alleviate the weight of the proximal terms, the restrictions imposed on the proximal parameters are relaxed substantively; (2) to maintain the inherent structure of the primal variables $$x_i(i=1,2,\ldots ,m)$$ , the relaxation parameter $$\gamma $$ is only attached to the update formula of the dual variable $$\lambda $$ . For the resulted method, we establish its global convergence and worst-case $$\mathcal {O}(1/t)$$ convergence rate in an ergodic sense, where t is the iteration counter. Finally, three numerical examples are given to illustrate the theoretical results obtained.

A chaotic dynamical system that paints and samples

Can a dynamical system paint masterpieces such as Da Vinci’s Mona Lisa or Monet’s Water Lilies? Moreover, can this dynamical system be chaotic in the sense that although the trajectories are sensitive to initial conditions, the same painting is created every time? Setting aside the creative aspect of painting a picture, in this work, we develop a novel algorithm to reproduce paintings and photographs. Combining ideas from ergodic theory and control theory, we construct a chaotic dynamical system with predetermined statistical properties. If one makes the spatial distribution of colors in the picture the target distribution, akin to a human, the algorithm first captures large scale features and then goes on to refine small scale features. Beyond reproducing paintings, this approach is expected to have a wide variety of applications such as uncertainty quantification, sampling for efficient inference in scalable machine learning for big data, and developing effective strategies for search and rescue. In particular, our preliminary studies demonstrate that this algorithm provides significant acceleration and higher accuracy than competing methods for Monte Carlo, Quasi Monte Carlo, Markov Chain Monte Carlo (MCMC).

Formalizing Neurath's ship: Approximate algorithms for online causal learning.

Higher-level cognition depends on the ability to learn models of the world. We can characterize this at the computational level as a structure-learning problem with the goal of best identifying the prevailing causal relationships among a set of relata. However, the computational cost of performing exact Bayesian inference over causal models grows rapidly as the number of relata increases. This implies that the cognitive processes underlying causal learning must be substantially approximate. A powerful class of approximations that focuses on the sequential absorption of successive inputs is captured by the Neurath's ship metaphor in philosophy of science, where theory change is cast as a stochastic and gradual process shaped as much by people's limited willingness to abandon their current theory when considering alternatives as by the ground truth they hope to approach. Inspired by this metaphor and by algorithms for approximating Bayesian inference in machine learning, we propose an algorithmic-level model of causal structure learning under which learners represent only a single global hypothesis that they update locally as they gather evidence. We propose a related scheme for understanding how, under these limitations, learners choose informative interventions that manipulate the causal system to help elucidate its workings. We find support for our approach in the analysis of 3 experiments. (PsycINFO Database Record

Bootstrap bias corrections for ensemble methods

This paper examines the use of a residual bootstrap for bias correction in machine learning regression methods. Accounting for bias is an important obstacle in recent efforts to develop statistical inference for machine learning. We demonstrate empirically that the proposed bootstrap bias correction can lead to substantial improvements in both bias and predictive accuracy. In the context of ensembles of trees, we show that this correction can be approximated at only double the cost of training the original ensemble. Our method is shown to improve test set accuracy over random forests by up to 70% on example problems from the UCI repository.

The Mortality and Medical Costs of Air Pollution: Evidence from Changes in Wind Direction.

We estimate the causal effects of acute fine particulate matter exposure on mortality, health care use, and medical costs among the US elderly using Medicare data. We instrument for air pollution using changes in local wind direction and develop a new approach that uses machine learning to estimate the life-years lost due to pollution exposure. Finally, we characterize treatment effect heterogeneity using both life expectancy and generic machine learning inference. Both approaches find that mortality effects are concentrated in about 25 percent of the elderly population.

Orthogonal parallel MCMC methods for sampling and optimization

Monte Carlo (MC) methods are widely used for Bayesian inference and optimization in statistics, signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In order to foster better exploration of the state space, specially in high-dimensional applications, several schemes employing multiple parallel MCMC chains have been recently introduced. In this work, we describe a novel parallel interacting MCMC scheme, called orthogonal MCMC (O-MCMC), where a set of “vertical” parallel MCMC chains share information using some “horizontal” MCMC techniques working on the entire population of current states. More specifically, the vertical chains are led by random-walk proposals, whereas the horizontal MCMC techniques employ independent proposals, thus allowing an efficient combination of global exploration and local approximation. The interaction is contained in these horizontal iterations. Within the analysis of different implementations of O-MCMC, novel schemes in order to reduce the overall computational cost of parallel Multiple Try Metropolis (MTM) chains are also presented. Furthermore, a modified version of O-MCMC for optimization is provided by considering parallel Simulated Annealing (SA) algorithms. Numerical results show the advantages of the proposed sampling scheme in terms of efficiency in the estimation, as well as robustness in terms of independence with respect to initial values and the choice of the parameters.

Tracing retinal vessel trees by transductive inference

BackgroundStructural study of retinal blood vessels provides an early indication of diseases such as diabetic retinopathy, glaucoma, and hypertensive retinopathy. These studies require accurate tracing of retinal vessel tree structure from fundus images in an automated manner. However, the existing work encounters great difficulties when dealing with the crossover issue commonly-seen in vessel networks.ResultsIn this paper, we consider a novel graph-based approach to address this tracing with crossover problem: After initial steps of segmentation and skeleton extraction, its graph representation can be established, where each segment in the skeleton map becomes a node, and a direct contact between two adjacent segments is translated to an undirected edge of the two corresponding nodes. The segments in the skeleton map touching the optical disk area are considered as root nodes. This determines the number of trees to-be-found in the vessel network, which is always equal to the number of root nodes. Based on this undirected graph representation, the tracing problem is further connected to the well-studied transductive inference in machine learning, where the goal becomes that of properly propagating the tree labels from those known root nodes to the rest of the graph, such that the graph is partitioned into disjoint sub-graphs, or equivalently, each of the trees is traced and separated from the rest of the vessel network. This connection enables us to address the tracing problem by exploiting established development in transductive inference. Empirical experiments on public available fundus image datasets demonstrate the applicability of our approach.ConclusionsWe provide a novel and systematic approach to trace retinal vessel trees with the present of crossovers by solving a transductive learning problem on induced undirected graphs.

Dirichlet process mixture model, extensions and applications

Nonparametric Bayesian inference is a research topic in the fields of computational statistics and machine learning. The review conducts a comprehensive survey on one of the representative stochastic processes-Dirichlet process (DP), including theoretical fundamentals, representation & construction method, extensions, statistical inference and applications in machine learning & bioinformatics. By analyzing the relations between Dirichlet process and hierarchical Dirichlet process, nested Dirichlet process, dependent Dirichlet process, matrix stick-breaking process and kernel stick-breaking process, the review reveals how to extend Dirichlet process to specific application scenario. To conclude, the review analyses DP merits and demerits and discusses DP future research focus.

Characterizing and Extracting Multiplex Patterns in Complex Networks

Complex network theory provides a means for modeling and analyzing complex systems that consist of multiple and interdependent components. Among the studies on complex networks, structural analysis is of fundamental importance as it presents a natural route to understanding the dynamics, as well as to synthesizing or optimizing the functions, of networks. A wide spectrum of structural patterns of networks has been reported in the past decade, such as communities, multipartites, bipartite, hubs, authorities, outliers, and bow ties, among others. In this paper, we are interested in tackling the challenging task of characterizing and extracting multiplex patterns (multiple patterns as mentioned previously coexisting in the same networks in a complicated manner), which so far has not been explicitly and adequately addressed in the literature. Our work shows that such multiplex patterns can be well characterized as well as effectively extracted by means of a granular stochastic blockmodel, together with a set of related algorithms proposed here based on some machine learning and statistical inference ideas. These models and algorithms enable us to further explore complex networks from a novel perspective.