Enable Machine Learning Research Articles

Network-Aware Optimization of Distributed Learning for Fog Computing

Fog computing promises to enable machine learning tasks to scale to large amounts of data by distributing processing across connected devices. Two key challenges to achieving this goal are (i) heterogeneity in devices’ compute resources and (ii) topology constraints on which devices communicate with each other. We address these challenges by developing a novel network-aware distributed learning methodology where devices optimally share local data processing and send their learnt parameters to a server for periodic aggregation. Unlike traditional federated learning, our method enables devices to offload their data processing tasks to each other, with these decisions optimized to trade off costs associated with data processing, offloading, and discarding. We analytically characterize the optimal data transfer solution under different assumptions on the fog network scenario, showing for example that the value of offloading is approximately linear in the range of computing costs in the network when the cost of discarding is modeled as decreasing linearly in the amount of data processed at each node. Our experiments on real-world data traces from our testbed confirm that our algorithms improve network resource utilization substantially without sacrificing the accuracy of the learned model, for varying distributions of data across devices. We also investigate the effect of network dynamics on model learning and resource costs.

Enabling Machine Learning with Service Function Chaining for Security Enhancement at 5G Edges

With massive sorts of terminals, devices, and machines connecting to 5G, a tremendous surge of data makes cyber-security a pressing issue, and conventional countermeasures are facing unprecedented challenges. Recently, with the rise of ML (Machine Learning) and SDN/NFV-based (Software-Defined Networks/Network Functions Virtualization) SFC (Service Function Chaining) techniques, how to leverage them for security enhancement in MEC (Multi-access/Mobile Edge Computing) has received much attention. Hence, in this article, we first propose an elastic framework to integrate ML with virtualized SFC, aiming at smart and efficient provision of different services at MEC. Then, we propose an ML-based anomaly detection algorithm used as a kind of service policy for SFC classifiers, which guides the latter for quick traffic classification and subsequent redirections of attack flows. Finally, we build a corresponding prototype system and evaluate the performance of the proposed algorithm through extensive experiments. Related results have confirmed the feasibility and advantages of the proposed framework and algorithm.

Towards Development of an ISFET-Based Smart pH Sensor: Enabling Machine Learning for Drift Compensation in IoT Applications

Monitoring of pH is crucial for several chemical and biochemical processes. ISFET (Ion-Sensitive Field-Effect Transistors)-based pH sensors are promising candidates for pH monitoring applications. However, ISFET devices are prone to temporal and temperature drifts, which severely affects the precision of pH measurements. In this work, we collect experimental data of temporal and temperature drifts in an ISFET sensor to formulate an accurate SPICE macro model, incorporating both temporal and temperature non-idealities. The developed macro model is utilized for generating training data for state-of-the-art machine learning models for drift compensation, with a primary focus on the temporal characteristics. We utilize recurrent neural networks (RNNs) to model the temporal characteristics of ISFET, and thus, compensate the non-ideality. The sensor data is collected in various pH buffer solutions and a data set of sequences containing time-dependent voltage readings are generated by the device and the RNNs are trained to learn the crucial features from the data and map them to the precise pH of the solution. We compare two variants of RNNs, i.e. LSTM (long short-term memory) and GRU (gated recurrent unit), and their bidirectional low computational cost variants - biLSTM and biGRU. Each model is tested in a memory-constrained environment with the availability of a 32-bit and 64-bit floating-point number. Empirically, we find biLSTMs to perform best, where the achieved root mean square error (RMSE) between the model predicted pH and the true pH of the test solution is less than 0.212 pH, with an average RMSE of 0.126 pH. For temperature drift compensation, we collect data for four different temperatures and adapt well-established MLPs (Multi-layer Perceptrons) to compensate the intrinsic temperature drift in the sensor. We observe an average RMSE of the model predicted pH to the true pH to be less than 0.286 pH. The developed RNN models were implemented on Xilinx ZCU104 FPGA development kit using PYNQ framework, which demonstrates low power consumption. The proposed framework establishes the efficacy of Machine Learning (ML) techniques for drift compensation in ISFET-based pH sensors for deployment in IoT applications.

Bag-of-Words Technique in Natural Language Processing: A Primer for Radiologists.

Natural language processing (NLP) is a methodology designed to extract concepts and meaning from human-generated unstructured (free-form) text. It is intended to be implemented by using computer algorithms so that it can be run on a corpus of documents quickly and reliably. To enable machine learning (ML) techniques in NLP, free-form text must be converted to a numerical representation. After several stages of preprocessing including tokenization, removal of stop words, token normalization, and creation of a master dictionary, the bag-of-words (BOW) technique can be used to represent each remaining word as a feature of the document. The preprocessing steps simplify the documents but also potentially degrade meaning. The values of the features in BOW can be modified by using techniques such as term count, term frequency, and term frequency-inverse document frequency. Experience and experimentation will guide decisions on which specific techniques will optimize ML performance. These and other NLP techniques are being applied in radiology. Radiologists' understanding of the strengths and limitations of these techniques will help in communication with data scientists and in implementation for specific tasks. Online supplemental material is available for this article. ©RSNA, 2021.

Binary classification of single qubits using quantum machine learning method

Machine learning can make predictions on unseen data by extracting information and constructing algorithms and statistical models from the training data, which is highly desirable for speech recognition and computer vision. Quantum computing harnesses the phenomena of quantum mechanics realizing parallel computing, which could solve certain computational problems, such as RSA encryption significantly faster than classical computers. Quantum machine learning is reasonable to enable machine learning faster than that of classical computers by utilizing quantum operations. Binary classification is a typical quantum machine learning task applied to single qubits. Here we present a binary classifier for regions. Firstly, we sample two sets of quantum data points located in the X-Z plane of the Bloch sphere randomly. Then we utilize a single parameterized rotation gate and measure the qubits along the Z axis. We train the hybrid model combined by quantum mdoel and classical Neutral network and regard the corss entropy the predictions of the classical Netural network and labels as the loss function. Based on the above, we present the results of the trained hybrid model to classify new quantum qubits.

High Performance DAQ Infrastructure to Enable Machine Learning for the Advanced Photon Source Upgrade

High Performance DAQ Infrastructure to Enable Machine Learning for the Advanced Photon Source Upgrade

On the distributed software architecture of a data analysis workflow: A case study

AbstractHybrid distributed computing software architectures gain great importance in data analysis workflows as the number of available underlying machine learning libraries and data storage systems increase. We argue that there is a need for novel approaches for software architecture designs that can enable machine learning data analysis workflows to run on top of different subsystem libraries. To address this need, we propose a hybrid distributed software architecture in this manuscript. The proposed architecture manages machine learning models for both supervised and unsupervised machine learning data analysis workflows. To show the usability of the proposed architecture, we implement a prototype for the banking sector as a case study. The prototype application includes two data analysis workflows: a workflow for predicting the loan usage tendency of customers, and a workflow for clustering the customers based on the usage patterns of banking loans. The prototype is tested on a large scale banking dataset. Performance tests were carried out to investigate the performance in terms of both responsiveness and scalability of the system. The results obtained reveal the usability of the proposed architecture.

Abstract 2177: Tumor neoantigen profiling with validated patient-specific TCR characterization to improve neoepitope prediction

Abstract PACT Pharma has developed an ultra-sensitive approach to validate predicted neoantigens and the cognate T cell receptors (neoTCRs) from tumor specific somatic mutations by capturing neoantigen-specific T cells from peripheral blood. This process is used in clinical trials (NCT03970382) of personalized neoTCR-T therapy for persons with solid cancers. Using the state of the art prediction pipeline and screening process, more than one hundred twenty neoantigens were predicted from 5 type of solid cancers that were validated by characterizing cognate T cells and their close to 200 unique TCRs captured from the blood of the same individual. These validated neoepitopes of 8 to 11 amino acids represent broad HLA class I coverage with >30 alleles to date. Our analysis revealed that mutations can occur in all positions within the epitopes. Epitope immunogenicity is potentially affected by different mechanisms including mutation position, agretopicity, as well as HLA interacting positions and/or by interactions between mutated residues and its cognate TCRs. It was observed that these validated neoepitopes comprise broad ranges of predicted HLA binding affinities, stability, and neoantigen expression levels. The analysis presented here offers insights to enable machine learning to advance rules for epitope selection and prioritization that may be important for immunological approaches to address a broad range of diseases, including cancer. Citation Format: Zheng Pan, Olivier Dalmas, Songming Peng, Kyle Jacoby, Barbara Sennino, Yan Ma, Chad Smith, Amin Momin, Allison Xu, Katharine Heeringa, Jonathan Johnston, Duo An, Boi Quach, William Lu, Diana Nguyen, Andrew Conroy, Bhamini Purandare, Eva Huang, Eric Stawiski, Alex Franzusoff, Stefanie Mandl. Tumor neoantigen profiling with validated patient-specific TCR characterization to improve neoepitope prediction [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 2177.

SDN-Enabled FiWi-IoT Smart Environment Network Traffic Classification Using Supervised ML Models

Due to the rapid growth of the Internet of Things (IoT), applications such as the Augmented Reality (AR)/Virtual Reality (VR), higher resolution media stream, automatic vehicle driving, the smart environment and intelligent e-health applications, increasing demands for high data rates, high bandwidth, low latency, and the quality of services are increasing every day (QoS). The management of network resources for IoT service provisioning is a major issue in modern communication. A possible solution to this issue is the use of the integrated fiber-wireless (FiWi) access network. In addition, dynamic and efficient network configurations can be achieved through software-defined networking (SDN), an innovative and programmable networking architecture enabling machine learning (ML) to automate networks. This paper, we propose a machine learning supervised network traffic classification scheduling model in SDN enhanced-FiWi-IoT that can intelligently learn and guarantee traffic based on its QoS requirements (QoS-Mapping). We capture the different IoT and non-IoT device network traffic trace files based on the traffic flow and analyze the traffic traces to extract statistical attributes (port source and destination, IP address, etc.). We develop a robust IoT device classification process module framework, using these network-level attributes to classify IoT and non-IoT devices. We tested the proposed classification process module in 21 IoT/Non-IoT devices with different ML algorithms and the results showed that classification can achieve a Random Forest classifier with 99% accuracy as compared to other techniques.

Using Knowledge Graphs to Plausibly Infer Missing Associations in EMR Data.

Electronic Medical Records (EMRs) are increasingly being deployed at primary points of care and clinics for digital record keeping, increasing productivity and improving communication. In practice, however, there still exists an often incomplete picture of patient profiles, not only because of disconnected EMR systems but also due to incomplete EMR data entry - often caused by clinician time constraints and lack of data entry restrictions. To complete a patient's partial EMR data, we plausibly infer missing causal associations between medical EMR concepts, such as diagnoses and treatments, for situations that lack sufficient raw data to enable machine learning methods. We follow a knowledge-based approach, where we leverage open medical knowledge sources such as SNOMED-CT and ICD, combined with knowledge-based reasoning with explainable inferences, to infer clinical encounter information from incomplete medical records. To bootstrap this process, we apply a semantic Extract-Transform-Load process to convert an EMR database into an enriched domain-specific Knowledge Graph.

Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learning

One of the key factors of enabling machine learning models to comprehend and solve real-world tasks is to leverage multimodal data. Unfortunately, annotation of multimodal data is challenging and expensive. Recently, self-supervised multimodal methods that combine vision and language were proposed to learn multimodal representations without annotation. However, these methods often choose to ignore the presence of high levels of noise and thus yield sub-optimal results. In this work, we show that the problem of noise estimation for multimodal data can be reduced to a multimodal density estimation task. Using multimodal density estimation, we propose a noise estimation building block for multimodal representation learning that is based strictly on the inherent correlation between different modalities. We demonstrate how our noise estimation can be broadly integrated and achieves comparable results to state-of-the-art performance on five different benchmark datasets for two challenging multimodal tasks: Video Question Answering and Text-To-Video Retrieval. Furthermore, we provide a theoretical probabilistic error bound substantiating our empirical results and analyze failure cases. Code: https://github.com/elad-amrani/ssml.

AIEMLA: artificial intelligence enabled machine learning approach for routing attacks on internet of things

The Internet of things (IoT) is emerging as a prime area of research in the modern era. The significance of IoT in the daily life is increasing due to the increase in objects or things connected to the internet. In this paper, routing protocol for low power and lossy networks (RPL) is examined on the Contiki operating system. This paper used RPL attack framework to simulate three RPL attacks, namely hello-flood, decreased-rank and increased-version. These attacks are simulated in a separate and simultaneous manner. The focus remained on the detection of these attacks through artificial neural network (ANN)-based supervised machine learning approach. The accurate detection of the malicious nodes prevents the network from the severe effects of the attack. The accuracy of the proposed model is computed with hold-out approach and tenfold cross-validation technique. The hyperparameters have been optimized through parameter tuning. The model presented in this paper detected the aforesaid attacks simultaneously as well as individually with 100% accuracy. This work also investigated other performance measures like precision, recall, F1-score and Mathews correlation coefficient (MCC).

A Highly Scalable, Modular Test Bench Architecture for Large-Scale DC Power Cycling of SiC MOSFETs: Towards Data Enabled Reliability

Silicon carbide (SiC) power MOSFETs have superior conduction, switching and thermal properties compared to silicon (Si) MOSFETs and IGBTs [1]. However, unlike Si devices, whose reliability is well understood from decades of research and field data, SiC device technology is relatively nascent and has only recently started witnessing wide scale deployment. This reliability challenge can possibly be addressed through two complementary solutions as shown in Figure 1: 1) developing a large accelerated aging dataset of SiC devices under various conditions to understand their long term reliability and guide future device development, 2) using on-board, in-system prognostics and device health monitoring techniques to predict imminent device failures well ahead of time, thus ensuring reliable system operation [2]. In both cases, the large-scale reliability testing of SiC MOSFETs is needed. Generally, remaining useful lifetime (RUL) estimation methods for power devices use an empirical model obtained from fitting accelerated aging data [3]-[7], [9], [10]. Consequently, the accuracy of RUL models is directly related to the size and variety of available accelerated aging dataset. Moreover, large datasets are crucial in enabling machine learning (ML) and artificial intelligence (AI) based reliability models.

A Machine Learning Strategy for Drug Discovery Identifies Anti-Schistosomal Small Molecules.

Schistosomiasis is a chronic and painful disease of poverty caused by the flatworm parasite Schistosoma. Drug discovery for antischistosomal compounds predominantly employs in vitro whole organism (phenotypic) screens against two developmental stages of Schistosoma mansoni, post-infective larvae (somules) and adults. We generated two rule books and associated scoring systems to normalize 3898 phenotypic data points to enable machine learning. The data were used to generate eight Bayesian machine learning models with the Assay Central software according to parasite’s developmental stage and experimental time point (≤24, 48, 72, and >72 h). The models helped predict 56 active and nonactive compounds from commercial compound libraries for testing. When these were screened against S. mansoni in vitro, the prediction accuracy for active and inactives was 61% and 56% for somules and adults, respectively; also, hit rates were 48% and 34%, respectively, far exceeding the typical 1–2% hit rate for traditional high throughput screens.

In-Network Flow Classification With Knowledge Distillation

Recent research has incorporated machine learning with software-defined networking to support intelligent traffic engineering. However, most frameworks only enable machine learning in remote controllers, which introduce significant signaling overhead and data forwarding costs. In this work, we present a new architecture called in-network inference (INI) to realize local learning in Neural Compute Stick (NCS), a portable device that can be connected to a programmable switch via a USB port. While NCS can flexibly extend the computing power of a switch, its limited capacity however cannot afford real-time inference for enormous traffic demands. To develop a practical local learning architecture, we design a two-phase learning framework that combines local learning with knowledge distillation and remote learning to achieve lightweight but accurate traffic classification. We further design an inference model deployment and adaptation algorithm to utilize multiple NCS devices equipped with different switches to share the inference workload of a network. Our testbed experiments show that the two-phase learning framework reduces the inference rejection rate by 46.5% and maintains the inference accuracy of 98.10%. The trace-driven simulations verify that the proposed adaptive model placement scheme considers load balancing and, hence, better utilizes the computing resources of NCS to serve dynamic inference requests.

An Artificial Intelligence-based Intrusion Detection System

Intrusion detection systems have been used in many systems to avoid malicious attacks. Traditionally, these intrusion detection systems use signature-based classification to detect predefined attacks and monitor the network's overall traffic. These intrusion detection systems often fail when an unseen attack occurs, which does not match with predefined attack signatures, leaving the system hopeless and vulnerable. In addition, as new attacks emerge, we need to update the database of attack signatures, which contains the attack information. This raises concerns because it is almost impossible to define every attack in the database and make the process costly also. Recently, research in conjunction with artificial intelligence and network security has evolved. As a result, it created many possibilities to enable machine learning approaches to detect the new attacks in network traffic. Machine learning has already shown successful results in the domain of recommendation systems, speech recognition, and medical systems. So, in this paper, we utilize machine learning approaches to detect attacks and classify them. This paper uses the CSE-CIC-IDS dataset, which contains normal and malicious attacks samples. Multiple steps are performed to train the network traffic classifier. Finally, the model is deployed for testing on sample data.

An embedded system for the automated generation of labeled plant images to enable machine learning applications in agriculture.

A lack of sufficient training data, both in terms of variety and quantity, is often the bottleneck in the development of machine learning (ML) applications in any domain. For agricultural applications, ML-based models designed to perform tasks such as autonomous plant classification will typically be coupled to just one or perhaps a few plant species. As a consequence, each crop-specific task is very likely to require its own specialized training data, and the question of how to serve this need for data now often overshadows the more routine exercise of actually training such models. To tackle this problem, we have developed an embedded robotic system to automatically generate and label large datasets of plant images for ML applications in agriculture. The system can image plants from virtually any angle, thereby ensuring a wide variety of data; and with an imaging rate of up to one image per second, it can produce lableled datasets on the scale of thousands to tens of thousands of images per day. As such, this system offers an important alternative to time- and cost-intensive methods of manual generation and labeling. Furthermore, the use of a uniform background made of blue keying fabric enables additional image processing techniques such as background replacement and image segementation. It also helps in the training process, essentially forcing the model to focus on the plant features and eliminating random correlations. To demonstrate the capabilities of our system, we generated a dataset of over 34,000 labeled images, with which we trained an ML-model to distinguish grasses from non-grasses in test data from a variety of sources. We now plan to generate much larger datasets of Canadian crop plants and weeds that will be made publicly available in the hope of further enabling ML applications in the agriculture sector.

Portuguese word embeddings for the oil and gas industry: Development and evaluation

Over the last decades, oil and gas companies have been facing a continuous increase of data collected in unstructured textual format. New disruptive technologies, such as natural language processing and machine learning, present an unprecedented opportunity to extract a wealth of valuable information within these documents. Word embedding models are one of the most fundamental units of natural language processing, enabling machine learning algorithms to achieve great generalization capabilities by providing meaningful representations of words, being able to capture syntactic and semantic features based on their context. However, the oil and gas domain-specific vocabulary represents a challenge to those algorithms, in which words may assume a completely different meaning from a common understanding. The Brazilian pre-salt is an important exploratory frontier for the oil and gas industry, with increasing attractiveness for international investments in exploration and production projects, and most of its documentation is in Portuguese. Moreover, Portuguese is one of the largest languages in terms of number of native speakers. Nonetheless, despite the importance of the petroleum sector of Portuguese speaking countries, specialized public corpora in this domain are scarce. This work proposes PetroVec, a representative set of word embedding models for the specific domain of oil and gas in Portuguese. We gathered an extensive collection of domain-related documents from leading institutions to build a large specialized oil and gas corpus in Portuguese, comprising more than 85 million tokens. To provide an intrinsic evaluation, assessing how well the models can encode domain semantics from the text, we created a semantic relatedness test set, comprising 1,500 word pairs labeled by selected experts in geoscience and petroleum engineering from both academia and industry. In addition, we performed an extrinsic quantitative evaluation on a downstream task of named entity recognition in geoscience, plus a set of qualitative analyses, and conducted a comparative evaluation against a public general-domain embedding model. The obtained results suggest that our domain-specific models outperformed the general model on their ability to represent specialized terminology. To the best of our knowledge, this is the first attempt to generate and evaluate word embedding models for the oil and gas domain in Portuguese. Finally, all the resources developed by this work are made available for public use, including the pre-trained specialized models, corpora, and validation datasets.

Neural Network Vessel Lumen Regression for Automated Lumen Cross-Section Segmentation in Cardiovascular Image-Based Modeling.

We accelerate a pathline-based cardiovascular model building method by training machine learning models to directly predict vessel lumen surface points from computed tomography (CT) and magnetic resonance (MR) medical image data. We formulate vessel lumen detection as a regression task using a polar coordiantes representation. Neural networks trained with our regression formulation allow predictions to be made with significantly higher accuracy than existing methods that identify the vessel lumen through binary pixel classification. The regression formulation enables machine learning models to be trained end-to-end for vessel lumen detection without post-processing steps that reduce accuracy. By employing our models in a pathline-based cardiovascular model building pipeline we substantially reduce the manual segmentation effort required to build accurate cardiovascular models, and reduce the overall time required to perform patient-specific cardiovascular simulations. While our method is applied here for cardiovascular model building it is generally applicable to segmentation of tree-like and tubular structures from image data.

Robust Coreset Construction for Distributed Machine Learning

Coreset, which is a summary of the original dataset in the form of a small weighted set in the same sample space, provides a promising approach to enable machine learning over distributed data. Although viewed as a proxy of the original dataset, each coreset is only designed to approximate the cost function of a specific machine learning problem, and thus different coresets are often required to solve different machine learning problems, increasing the communication overhead. We resolve this dilemma by developing robust coreset construction algorithms that can support a variety of machine learning problems. Motivated by empirical evidence that suitably-weighted k -clustering centers provide a robust coreset, we harden the observation by establishing theoretical conditions under which the coreset provides a guaranteed approximation for a broad range of machine learning problems, and developing both centralized and distributed algorithms to generate coresets satisfying the conditions. The robustness of the proposed algorithms is verified through extensive experiments on diverse datasets with respect to both supervised and unsupervised learning problems.