Runtime Improvements Research Articles

DNN Is Not All You Need: Parallelizing Non-neural ML Algorithms on Ultra-low-power IoT Processors

Machine Learning (ML) functions are becoming ubiquitous in latency- and privacy-sensitive IoT applications, prompting a shift toward near-sensor processing at the extreme edge and the consequent increasing adoption of Parallel Ultra-low-power (PULP) IoT processors. These compute- and memory-constrained parallel architectures need to run efficiently a wide range of algorithms, including key Non-neural ML kernels that compete favorably with Deep Neural Networks in terms of accuracy under severe resource constraints. In this article, we focus on enabling efficient parallel execution of Non-neural ML algorithms on two RISCV-based PULP platforms, namely, GAP8, a commercial chip, and PULP-OPEN, a research platform running on an FPGA emulator. We optimized the parallel algorithms through a fine-grained analysis and intensive optimization to maximize the speedup, considering two alternative Floating-point (FP) emulation libraries on GAP8 and the native FPU support on PULP-OPEN. Experimental results show that a target-optimized emulation library can lead to an average 1.61× runtime improvement and 37% energy reduction compared to a standard emulation library, while the native FPU support reaches up to 32.09× and 99%, respectively. In terms of parallel speedup, our design improves the sequential execution by 7.04× on average on the targeted octa-core platforms leading to energy and latency decrease up to 87%. Last, we present a comparison with the ARM Cortex-M4 microcontroller, a widely adopted commercial solution for edge deployments, which is 12.87× slower than PULP-OPEN.

Classification of stochastic processes with topological data analysis

SummaryIn this study, we demonstrate that engineered topological features can distinguish time series sampled from different stochastic processes with different noise characteristics, in both balanced and unbalanced sampling schemes. We compare our classification results against the results of the same classification on features coming from descriptive statistics and the wavelet transform. We conclude that machine learning models built on engineered topological features alone perform consistently better than those built on the standard statistical and wavelet features for time series classification tasks. We also apply dimension reduction techniques to our engineered features and compare the result of our classification models before and after dimensionality reduction. Finally, we also show that in our calculations of the engineered topological features, employing parallel computing methods does yield significant improvements in run time and memory footprint.

Fat Pointers for Temporal Memory Safety of C

Temporal memory safety bugs, especially use-after-free and double free bugs, pose a major security threat to C programs. Real-world exploits utilizing these bugs enable attackers to read and write arbitrary memory locations, causing disastrous violations of confidentiality, integrity, and availability. Many previous solutions retrofit temporal memory safety to C, but they all either incur high performance overhead and/or miss detecting certain types of temporal memory safety bugs. In this paper, we propose a temporal memory safety solution that is both efficient and comprehensive. Specifically, we extend Checked C, a spatially-safe extension to C, with temporally-safe pointers. These are implemented by combining two techniques: fat pointers and dynamic key-lock checks. We show that the fat-pointer solution significantly improves running time and memory overhead compared to the disjoint-metadata approach that provides the same level of protection. With empirical program data and hands-on experience porting real-world applications, we also show that our solution is practical in terms of backward compatibility---one of the major complaints about fat pointers.

Rye: genetic ancestry inference at biobank scale.

Biobank projects are generating genomic data for many thousands of individuals. Computational methods are needed to handle these massive data sets, including genetic ancestry (GA) inference tools. Current methods for GA inference do not scale to biobank-size genomic datasets. We present Rye-a new algorithm for GA inference at biobank scale. We compared the accuracy and runtime performance of Rye to the widely used RFMix, ADMIXTUREand iAdmix programs and applied it to a dataset of 488221 genome-wide variant samples from the UK Biobank. Rye infers GA based on principal component analysis of genomic variant samples from ancestral reference populations and query individuals. The algorithm's accuracy is powered by Metropolis-Hastings optimization and its speed is provided by non-negative least squares regression. Rye produces highly accurate GA estimates for three-way admixed populations-African, Europeanand Native American-compared to RFMix and ADMIXTURE (${R}^2 = \ 0.998 - 1.00$), and shows 50×runtime improvement compared to ADMIXTURE on the UK Biobank dataset. Rye analysis of UK Biobank samples demonstrates how it can be used to infer GA at both continental and subcontinental levels. We discuss user consideration and options for the use of Rye; the program and its documentation are distributed on the GitHub repository: https://github.com/healthdisparities/rye.

Quantum fingerprinting for heterogeneous devices localization

In this paper, we present QHFP, a device-independent quantum fingerprint matching algorithm that addresses two of the issues for realizing worldwide ubiquitous large-scale location tracking systems: scalability of storage space and running time as well as devices heterogeneity. In particular, we present a quantum algorithm with a complexity that is exponentially better than the classical techniques, both in space and running time. QHFP also has provisions for handling the inherent localization error due to building the large-scale fingerprint using heterogeneous devices. We give the details of the entire system starting from extracting device-independent features from the raw received signal strength, mapping the classical feature vectors to their quantum counterparts, and showing a quantum cosine similarity algorithm for fingerprint matching.We implement our quantum algorithm and deploy it in a real testbed using the IBM Quantum machine simulator. We also study the effect of the different quantum error sources, such as quantum gates error and quantum measurement error, on the system accuracy. Based on this, we discuss the suitable provisions to mitigate the effect of the quantum error sources on the localization accuracy.Results confirm the ability of QHFP to obtain the correct estimated location with an exponential improvement in space and running time compared to the traditional classical counterparts. In addition, the proposed device-independent features lead to more than 20% better accuracy in median error. This highlights the promise of our algorithm for future ubiquitous large-scale worldwide device-independent fingerprinting localization systems.

ENCIDER: Detecting Timing and Cache Side Channels in SGX Enclaves and Cryptographic APIs

Confidential computing aims to secure the code and data in use by providing a Trusted Execution Environment (TEE) for applications using hardware features such as Intel SGX. Timing and cache side-channel attacks, however, are often outside the scope of the threat model, although once exploited they are able to break all the default security guarantees enforced by hardware. Unfortunately, tools detecting potential side-channel vulnerabilities within applications are limited and usually ignore the strong attack model and the unique programming model imposed by Intel SGX. This article proposes a precise side-channel analysis tool, ENCIDER, detecting both timing and cache side-channel vulnerabilities within SGX applications via inferring potential timing observation points and incorporating the SGX programming model into analysis. ENCIDER uses dynamic symbolic execution to decompose the side-channel requirement based on the bounded non-interference property and implements byte-level information flow tracking via API modeling. We have applied ENCIDER to 4 real-world SGX applications, 2 SGX crypto libraries, and 3 widely-used crypto libraries, and found 29 timing side channels and 73 code and data cache side channels. We also compare ENCIDER with three state-of-the-art side channel analysis tools using their benchmarks. ENCIDER does not only report most of the bugs with 20%-50% run time improvement and 65%-92% memory usage improvement, but also detects 9 missing bugs from these tools. We have reported our findings to the corresponding parties, e.g., Intel and ARM, who have confirmed most of the vulnerabilities detected.

Go-to-Controller is Better: Efficient and Optimal LPM Caching with Splicing

Modern data center networks are required to support huge and complex forwarding policies as they handle the traffic of the various tenants. However, these policies cannot be stored in their entirety within the limited memory available at commodity switches. The common approach in such scenarios is to have SDN controllers manage the memory available at the switch as a fast cache, updating and changing the forwarding rules in the cache according to the workloads dynamics and the global policy at hand. Many such policies, such as Longest-prefix-match (LPM) policies, introduce dependencies between the forwarding rules. Ensuring that the cache content is always consistent with the global policy often requires the switch to store (potentially many) superfluous rules, which may lead to suboptimal performance in terms of delay and throughput. To overcome these deficiencies, previous work suggested the concept of splicing, where modified Go-to-Controller rules can be inserted into the cache to improve performance while maintaining consistency. These works focused mostly on heuristics, and it was conjectured that the problem is computationally intractable. As our main result, we show that the problem of determining the optimal set of rules, with splicing, can actually be solved efficiently by presenting a polynomial-time algorithm that produces an optimal solution, i.e., for a given cache size we find an optimal set of rules, some of which are go-to-controller, which maximize the total weight of the cache while maintaining consistency. However, such optimality comes at a cost, encompassed by the fact that our algorithm has a significantly larger running time than SoTA solutions which do not employ splicing. Therefore, we further present a heuristic exhibiting close-to-optimal performance, with significantly improved running time, matching that of the best algorithm, which does not employ splicing. In addition, we present the results of an evaluation study that compares the performance of our solutions with that of SoTA approaches, showing that splicing can reduce the cache miss ratio by as much as 30%, without increasing the cache size. Lastly, we propose a simple and fast-to-compute metric (that is consistency-oblivious) in order to evaluate the potential benefits of splicing compared to classical LPM-caching, for a given policy and traffic distribution. We show that our metric is highly correlated with such benefits, thus serving as an indication of whether splicing should be incorporated within the system architecture.

Reinforcement learning in VANET penetration testing

The recent popularity of Connected and Autonomous Vehicles (CAV) corresponds with an increase in the risk of cyber-attacks. These cyber-attacks are instigated by white-coat hackers, and cyber-criminals. As Connected Vehicles move towards full autonomy the impact of these cyber-attacks also grows. The current research highlights challenges faced in cybersecurity testing of CAV, including access, the cost of representative test setup and the lack of experts in the field. Possible solutions of how these challenges can be overcome are reviewed and discussed. From these findings a software simulated Vehicular Ad Hoc NETwork (VANET) is established as a cost-effective representative testbed. Penetration tests are then performed on this simulation, demonstrating a cyber-attack in CAV. Studies have shown Artificial Intelligence (AI) to improve runtime, increase efficiency and comprehensively cover all the typical test aspects, in penetration testing in other industries. In this research a Reinforcement Learning model, called Q-Learning, is applied to automate the software simulation. The expectation from this implementation is to see improvements in runtime and efficiency for the VANET model. The results show this approach to be promising and using AI in penetration testing for VANET to improve efficiency in most cases. Each case is reviewed in detail before discussing possible ways to improve the implementation and get a truer reflection of the real-world application.

FCHM-stream: fast closed high utility itemsets mining over data streams

The high-speed, continuous and endless characteristics of data streams make it a challenging task to quickly mine high utility itemsets in limited memory space. The sliding window model, which focuses only on the most recent data, has received extensive research and attention as it can effectively adapt to the data stream environment. However, the presence of many communal batches in adjacent sliding windows causes the algorithm to repeatedly generate a large number of identical itemsets, which reduces the spatiotemporal performance of the algorithm. In order to solve these problems and provide users with a concise and lossless resultset, a new closed high utility pattern mining algorithm over data stream is proposed, named FCHM-Stream. A new utility list structure based on batch division and a resultset maintenance strategy based on skip-list structure are designed to effectively reduce identical itemsets repeatedly generated and thus reduce the running time of the algorithm. Extensive experimental results show that the proposed algorithm has a large improvement in runtime compared to the state-of-the-art algorithms.

Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges

AbstractMost machine learning algorithms are configured by a set of hyperparameters whose values must be carefully chosen and which often considerably impact performance. To avoid a time‐consuming and irreproducible manual process of trial‐and‐error to find well‐performing hyperparameter configurations, various automatic hyperparameter optimization (HPO) methods—for example, based on resampling error estimation for supervised machine learning—can be employed. After introducing HPO from a general perspective, this paper reviews important HPO methods, from simple techniques such as grid or random search to more advanced methods like evolution strategies, Bayesian optimization, Hyperband, and racing. This work gives practical recommendations regarding important choices to be made when conducting HPO, including the HPO algorithms themselves, performance evaluation, how to combine HPO with machine learning pipelines, runtime improvements, and parallelization.This article is categorized under: Algorithmic Development > Statistics Technologies > Machine Learning Technologies > Prediction

Accelerating inexact successive quadratic approximation for regularized optimization through manifold identification

AbstractFor regularized optimization that minimizes the sum of a smooth term and a regularizer that promotes structured solutions, inexact proximal-Newton-type methods, or successive quadratic approximation (SQA) methods, are widely used for their superlinear convergence in terms of iterations. However, unlike the counter parts in smooth optimization, they suffer from lengthy running time in solving regularized subproblems because even approximate solutions cannot be computed easily, so their empirical time cost is not as impressive. In this work, we first show that for partly smooth regularizers, although general inexact solutions cannot identify the active manifold that makes the objective function smooth, approximate solutions generated by commonly-used subproblem solvers will identify this manifold, even with arbitrarily low solution precision. We then utilize this property to propose an improved SQA method, $$^{+}$$ + , that switches to efficient smooth optimization methods after this manifold is identified. We show that for a wide class of degenerate solutions, $$^{+}$$ + possesses superlinear convergence not only in iterations, but also in running time because the cost per iteration is bounded. In particular, our superlinear convergence result holds on problems satisfying a sharpness condition that is more general than that in existing literature. We also prove iterate convergence under a sharpness condition for inexact SQA, which is novel for this family of methods that could easily violate the classical relative-error condition frequently used in proving convergence under similar conditions. Experiments on real-world problems support that $$^{+}$$ + improves running time over some modern solvers for regularized optimization.

Using multiple reference vectors and objective scaling in the Feasibility Pump

The Feasibility Pump (FP) is one of the best-known primal heuristics for mixed-integer programming (MIP): more than 15 papers suggested various modifications of all of its steps. So far, no variant considered information across multiple iterations, but all instead maintained the principle to optimize towards a single reference integer point. In this paper, we evaluate the usage of multiple reference vectors in all stages of the FP algorithm. In particular, we use LP-feasible vectors obtained during the main loop to tighten the variable domains before entering the computationally expensive enumeration stage, a procedure we refer to as mRENS. Moreover, we consider multiple integer reference vectors to explore further optimizing directions and introduce alternative objective scaling terms to balance the contributions of the distance functions and the original MIP objective.Our computational experiments demonstrate that the new method can improve performance on general MIP test sets. In detail, our modifications provide a 29.3% solution quality improvement and 4.0% running time improvement in an embedded setting, needing 16.0% fewer iterations over a large test set of MIP instances. In addition, the method's success rate increases considerably within the first few iterations. In a standalone setting, we also observe a moderate performance improvement, which makes our version of FP suitable for the two main use-cases of the algorithm.

Hierarchical Hashing Learning for Image Set Classification.

With the development of video network, image set classification (ISC) has received a lot of attention and can be used for various practical applications, such as video based recognition, action recognition, and so on. Although the existing ISC methods have obtained promising performance, they often have extreme high complexity. Due to the superiority in storage space and complexity cost, learning to hash becomes a powerful solution scheme. However, existing hashing methods often ignore complex structural information and hierarchical semantics of the original features. They usually adopt a single-layer hashing strategy to transform high-dimensional data into short-length binary codes in one step. This sudden drop of dimension could result in the loss of advantageous discriminative information. In addition, they do not take full advantage of intrinsic semantic knowledge from whole gallery sets. To tackle these problems, in this paper, we propose a novel Hierarchical Hashing Learning (HHL) for ISC. Specifically, a coarse-to-fine hierarchical hashing scheme is proposed that utilizes a two-layer hash function to gradually refine the beneficial discriminative information in a layer-wise fashion. Besides, to alleviate the effects of redundant and corrupted features, we impose the ℓ2,1 norm on the layer-wise hash function. Moreover, we adopt a bidirectional semantic representation with the orthogonal constraint to keep intrinsic semantic information of all samples in whole image sets adequately. Comprehensive experiments demonstrate HHL acquires significant improvements in accuracy and running time. We will release the demo code on https://github.com/sunyuan-cs.

A Generalized Multistep Dynamic (GMD) TOPMODEL

AbstractThere is a lack of Ordinary Differential Equation (ODE) formulations in numerical hydrology, contributing to the lack of application of canned adaptive timestep solvers; hence the continued dominance of fixed (e.g., Euler) timestep techniques despite their fundamental problems. In this paper, we reformulate Dynamic‐TOPMODEL into a constraint‐handling ODE form and use MATLAB's advanced adaptive ODE‐solvers to solve the resulting system of equations. For wider applicability, but based on existing research and/or first principles, we developed Generalized Multistep Dynamic TOPMODEL which includes: iso‐basin spatial discretization, diffusion wave routing, depth‐dependent overland flow velocity, relaxing the assumption of water‐table parallelism to the ground surface, a power‐law hydraulic conductivity profile, new unsaturated zone flux, and a reference frame adjustment. To demonstrate the model we calibrate it to a peat catchment case study, for which we also test sensitivity to spatial discretization. Our results suggest that (a) a five‐fold improvement in model runtime can result from adaptive timestepping; (b) the additional iso‐basin discretization layer, as a way to further constrain spatial information where needed, also improves performance; and (c) the common‐practice arbitrary Topographic Index (TI) discretization substantially alters calibrated parameters. More objective and physically constrained (e.g., top‐down) approaches to TI classification may be needed.

Improving Run Time Efficiency of Semantic Video Event Classification

Improving Run Time Efficiency of Semantic Video Event Classification

Complexity Measures for IoT Network Traffic

The coming era of widespread integration of Internet of Things (IoT) devices to all areas of society has facilitated a fundamental transformation of local and global communication networks, giving rise to novel issues relating to capacity planning, network administration, and cybersecurity. Accurate network traffic prediction is one of the key enablers for addressing these challenges. While there are methods for quantifying the complexity (predictability in timing, shape, and volume) of wide-scale aggregate traffic, they cannot be directly applied to IoT traffic as they do not account for the heterogeneity of IoT devices. Lacking an effective complexity characterization for IoT traffic, network traffic administrators are under-informed on the impacts of IoT device-type traffic on their networks. In this work, the complexity of IoT traffic is examined from two novel perspectives, the information-theoretic approach of Lempel–Ziv, a foundational algorithm in lossless data compression, and in the distribution of spectral components of the Fourier transform. Based on these perspectives, two new measures of IoT network traffic complexity are proposed. Furthermore, we introduce a novel mathematical framework to permit a formal comparison of new and existing methods. The new framework additionally verifies that the new metrics satisfy desirable properties for a measure of complexity. In a comprehensive empirical study, our results, when compared with existing approaches, exceed all others in behavioral resolution, convergence rate, physical interpretability, and algorithmic stability, under severely heterogeneous conditions. Benchmark experiments demonstrate substantial run-time improvements over existing approaches, creating a strong case for their use in online, real-time settings.

Brain-inspired Cognition in Next-generation Racetrack Memories

Hyperdimensional computing (HDC) is an emerging computational framework inspired by the brain that operates on vectors with thousands of dimensions to emulate cognition. Unlike conventional computational frameworks that operate on numbers, HDC, like the brain, uses high-dimensional random vectors and is capable of one-shot learning. HDC is based on a well-defined set of arithmetic operations and is highly error resilient. The core operations of HDC manipulate HD vectors in bulk bit-wise fashion, offering many opportunities to leverage parallelism. Unfortunately, on conventional von Neumann architectures, the continuous movement of HD vectors among the processor and the memory can make the cognition task prohibitively slow and energy intensive. Hardware accelerators only marginally improve related metrics. In contrast, even partial implementations of an HDC framework inside memory can provide considerable performance/energy gains as demonstrated in prior work using memristors. This article presents an architecture based on racetrack memory (RTM) to conduct and accelerate the entire HDC framework within memory. The proposed solution requires minimal additional CMOS circuitry by leveraging a read operation across multiple domains in RTMs called transverse read (TR) to realize exclusive-or ( XOR ) and addition operations. To minimize the CMOS circuitry overhead, an RTM nanowire-based counting mechanism is proposed. Using language recognition as the example workload, the proposed RTM HDC system reduces the energy consumption by 8.6× compared to the state-of-the-art in-memory implementation. Compared to dedicated hardware design realized with an FPGA, RTM-based HDC processing demonstrates 7.8× and 5.3× improvements in the overall runtime and energy consumption, respectively.

Improved Evaluation and Generation Of Grid Layouts Using Distance Preservation Quality and Linear Assignment Sorting

AbstractImages sorted by similarity enables more images to be viewed simultaneously, and can be very useful for stock photo agencies or e‐commerce applications. Visually sorted grid layouts attempt to arrange images so that their proximity on the grid corresponds as closely as possible to their similarity. Various metrics exist for evaluating such arrangements, but there is low experimental evidence on correlation between human perceived quality and metric value. We propose distance preservation quality (DPQ) as a new metric to evaluate the quality of an arrangement. Extensive user testing revealed stronger correlation of DPQ with user‐perceived quality and performance in image retrieval tasks compared to other metrics. In addition, we introduce Fast linear assignment sorting (FLAS) as a new algorithm for creating visually sorted grid layouts. FLAS achieves very good sorting qualities while improving run time and computational resources.

Communication-Efficient Secure Federated Statistical Tests from Multiparty Homomorphic Encryption

The power and robustness of statistical tests are strongly tied to the amount of data available for testing. However, much of the collected data today is siloed amongst various data owners due to privacy concerns, thus limiting the utility of the collected data. While frameworks for secure multiparty computation enable functions to be securely evaluated on federated datasets, they depend on protocols over secret shared data, which result in high communication costs even in the semi-honest setting.In this paper, we present methods for securely evaluating statistical tests, specifically the Welch’s t-test and the χ2-test, in the semi-honest setting using multiparty homomorphic encryption (MHE). We tested and evaluated our methods against real world datasets and found that our method for computing the Welch’s t-test and χ2-test statistics required 100× less communication than equivalent protocols implemented using secure multiparty computation (SMPC), resulting in up to 10× improvement in runtime. Lastly, we designed and implemented a novel protocol to perform a table lookup from a secret shared index and use it to build a hybrid protocol that switches between MHE and SMPC representations in order to calculate the p-value of the statistics efficiently. This hybrid protocol is 1.5× faster than equivalent protocols implemented using SMPC alone.

Maritime inventory routing with transshipment: the case of Yamal LNG

The LNG-ADP problem is a tactical planning problem for creating an annual delivery program (ADP) for a liquefied natural gas (LNG) producer. An ADP specifies the departure dates of the LNG carriers as well as the delivery dates at the different customers for a period of 12–18 months. The problem can be formulated as maritime inventory routing problem, as it is an important requirement to plan the deliveries such that inventory levels are kept within minimum and maximum limits at the customers as well as the LNG production facility. Inspired by the case of Yamal LNG, we propose a novel discrete-time formulation for the LNG-ADP problem with transshipment and intermediate storage. Our formulation also allows for waiting at the unloading ports. The problem is solved using a rolling horizon heuristic (RHH) for a case based on the Yamal LNG project. We study the impact of different RHH configurations on run time and solution quality. The results show that using a central period that is shorter than the forecast period provides the best objective function, whereas a central period that is longer than the forecast period improves run time. We also explore the effect of allowing waiting at the unloading ports. Waiting does not necessarily improve the objective function value, despite increasing the solution space. However, we observe a reduction in run time for instances where waiting is allowed.