Runtime Cost Research Articles

FEVERLESS: Fast and Secure Vertical Federated Learning Based on XGBoost for Decentralized Labels

Vertical Federated Learning (VFL) enables multiple clients to collaboratively train a global model over vertically partitioned data without leaking private local information. Tree-based models, like XGBoost and LightGBM, have been widely used in VFL to enhance the interpretation and efficiency of training. However, there is a fundamental lack of research on how to conduct VFL securely over distributed labels. This work is the first to fill this gap by designing a novel protocol, called FEVERLESS, based on XGBoost. FEVERLESS leverages secure aggregation via information masking technique and global differential privacy provided by a fairly and randomly selected noise leader to prevent private information from being leaked in the training process. Furthermore, it provides label and data privacy against honest-but-curious adversaries even in the case of collusion of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n - 2$</tex-math></inline-formula> out of n clients. We present a comprehensive security and efficiency analysis for our design, and the empirical results from our experiments demonstrate that FEVERLESS is fast and secure. In particular, it outperforms the solution based on additive homomorphic encryption in runtime cost and provides better accuracy than the local differential privacy approach.

Statistical Testing of Quantum Programs via Fixed-Point Amplitude Amplification

We present a new technique for accelerating quantum program testing. Given a quantum circuit with an input/output specification, our goal is to check whether executing the program on the input state produces the expected output. In quantum computing, however, it is impossible to directly check the equivalence of the two quantum states. Instead, we rely on statistical testing, which involves repeated program executions, state measurements, and subsequent comparisons with the specified output. To guarantee a high level of assurance, however, this method requires an extensive number of measurements. In this paper, we propose a solution to alleviate this challenge by adapting Fixed-Point Amplitude Amplification (FPAA) for quantum program testing. We formally present our technique, demonstrate its ability to reduce the required number of measurements as well as runtime cost without sacrificing the original statistical guarantee, and showcase its runtime effectiveness through case studies.

Workload Placement on Heterogeneous CPU-GPU Systems

The popularity of heterogeneous CPU-GPU processing has increased considerably in recent years. To efficiently utilize heterogeneous resources, data processing systems depend on an appropriate workload placement strategy to assign the right amount of compute to the right processor. However, finding an optimal placement strategy is not trivial due to various complex and conflicting tradeoffs related to the characteristics of processors, the nature of the workload, and data locality. In addition, placement decisions impact workload runtime and performance cost, and also depend on the availability of potentially different implementations for CPUs and GPUs, which adds extra complexity in such heterogeneous environments. In this tutorial, we review and compare state-of-the-art strategies for workload placement on heterogeneous CPU-GPU architectures, along with runtime prediction techniques and methods to support multi-device code. We also discuss open issues and identify potentially promising future research directions.

Physics-Based Artificial Neural Network Assisting in Extracting Transient Properties of Extrinsically Triggering Photoconductive Semiconductor Switches.

In this paper, a physics-based ANN assisting method for extracting transient properties of extrinsically triggering photoconductive semiconductor switches (ET-PCSSs) is proposed. It exploits the nonlinear mapping of ANN between transient current (input) and doping concentration (output). According to the basic laws of photoelectric device operating, two types of ANN models are constructed by gaussian and polynomial fitting. The mean absolute error (MAE) of forecasting transient photocurrent can be less than 10 A under low triggering optical powers, which verifies the feasibility of ANN assisting TCAD applied to PCSSs. The results are comparable to computation by Mixed-Mode simulation, yet even thousands of seconds of CPU runtime cost are saved in every period. To improve the robustness of the Poly-ANN predictor, Bayesian optimization (BO) is implemented for minimizing the curl deviation of photocurrent-time curves.

Massively Parallel Evasion Attacks and the Pitfalls of Adversarial Retraining

Even with widespread adoption of automated anomaly detection in safety-critical areas, both classical and advanced machine learning models are susceptible to first-order evasion attacks that fool models at run-time (e.g. an automated firewall or an anti-virus application). Kernelized support vector machines (KSVMs) are an especially useful model because they combine a complex geometry with low run-time requirements (e.g. when compared to neural networks), acting as a run-time lower bound when compared to contemporary models (e.g. deep neural networks), to provide a cost-efficient way to measure model and attack run-time costs. To properly measure and combat adversaries, we propose a massively parallel projected gradient descent (PGD) evasion attack framework. Through theoretical examinations and experiments carried out using linearly-separable Gaussian normal data, we present (i) a massively parallel naive attack, we show that adversarial retraining is unlikely to be an effective means to combat an attacker even on linearly separable datasets, (ii) a cost effective way of evaluating models defences and attacks, and an extensible code base for doing so, (iii) an inverse relationship between adversarial robustness and benign accuracy, (iv) the lack of a general relationship between attack time and efficacy, and (v) that adversarial retraining increases compute time exponentially while failing to reliably prevent highly-confident false classifications.

Detection of Crabs and Lobsters Using a Benchmark Single-Stage Detector and Novel Fisheries Dataset

Crabs and lobsters are valuable crustaceans that contribute enormously to the seafood needs of the growing human population. This paper presents a comprehensive analysis of single- and multi-stage object detectors for the detection of crabs and lobsters using images captured onboard fishing boats. We investigate the speed and accuracy of multiple object detection techniques using a novel dataset, multiple backbone networks, various input sizes, and fine-tuned parameters. We extend our work to train lightweight models to accommodate the fishing boats equipped with low-power hardware systems. Firstly, we train Faster R-CNN, SSD, and YOLO with different backbones and tuning parameters. The models trained with higher input sizes resulted in lower frames per second (FPS) and vice versa. The base models were highly accurate but were compromised in computational and run-time costs. The lightweight models were adaptable to low-power hardware compared to the base models. Secondly, we improved the performance of YOLO (v3, v4, and tiny versions) using custom anchors generated by the k-means clustering approach using our novel dataset. The YOLO (v4 and it’s tiny version) achieved mean average precision (mAP) of 99.2% and 95.2%, respectively. The YOLOv4-tiny trained on the custom anchor-based dataset is capable of precisely detecting crabs and lobsters onboard fishing boats at 64 frames per second (FPS) on an NVidia GeForce RTX 3070 GPU. The Results obtained identified the strengths and weaknesses of each method towards a trade-off between speed and accuracy for detecting objects in input images.

A Fast GPU Schedule For À-Trous Wavelet-Based Denoisers

Given limitations of contemporary graphics hardware, real-time ray-traced global illumination is only estimated using a few samples per pixel. This consequently causes stochastic noise in the resulting frame sequences which requires wide filter support during denoising for temporally stable estimates. The edge avoiding à-trous wavelet transform amortizes runtime cost by hierarchical filtering using a constant number of increasingly dilated taps in each iteration. While the number of taps stays constant, the runtime of each iteration increases in these usually memory-throughput bound shaders with increasing dilation, because the increasing non-locality negatively impacts cache hit rates. We present a scheduling approach that optimizes usage of the memory subsystem by permutating global invocation indices in such a way that each wavelet filter iteration is applied through undilated taps. In contrast to prior approaches, our method has identical performance characteristics in each iteration, effectively decreasing maintenance cost and improving performance predictability. Furthermore, we are able to leverage on-chip memory and hardware texture interpolation. Our permutation strategy is trivial to integrate into existing wavelet filters as a permutation before and after each level of the wavelet filter. We achieve speedups between 1.3 and 3.8 for usual wavelet configurations in Monte Carlo denoising and computational photography.

ArcheType: A Novel Framework for Open-Source Column Type Annotation Using Large Language Models

Existing deep-learning approaches to semantic column type annotation (CTA) have important shortcomings: they rely on semantic types which are fixed at training time; require a large number of training samples per type; incur high run-time inference costs; and their performance can degrade when evaluated on novel datasets, even when types remain constant. Large language models have exhibited strong zero-shot classification performance on a wide range of tasks and in this paper we explore their use for CTA. We introduce ArcheType, a simple, practical method for context sampling, prompt serialization, model querying, and label remapping, which enables large language models to solve CTA problems in a fully zero-shot manner. We ablate each component of our method separately, and establish that improvements to context sampling and label remapping provide the most consistent gains. ArcheType establishes a new state-of-the-art performance on zero-shot CTA benchmarks (including three new domain-specific benchmarks which we release along with this paper), and when used in conjunction with classical CTA techniques, it outperforms a SOTA DoDuo model on the fine-tuned SOTAB benchmark.

Morphological analysis of ballast particles: Characterization and simplified analysis of particle morphology using imaging data

This paper introduces a novel algorithm for the rigorous characterization of three-dimensional (3D) particles, particularly for railway ballast. Degraded railway ballast must be replaced with fresh material for efficient functioning. This study examined the shape and form of degraded (used) ballast to guide future maintenance efforts. Laboratory-generated used ballast, obtained via the Los Angeles abrasion test, was compared to fresh ballast. Thirteen fundamental morphological parameters of fresh and used ballasts were investigated by utilizing the shape information obtained through 3D scanning. The algorithm efficiently processed datasets comprising multiple irregular particles and monitored the morphological characteristics of ballasts based on the shape of the particles. The trimesh library was imported for 3D processing, facilitating the mathematical calculation of diverse parameters using the developed algorithm. The algorithm also incorporated mechanisms for simultaneously storing parameters provided in various 3D configuration models. With the support of the trimesh library, a morphology analyzer was used to analyze various 3D model file formats, such as .stl, .obj, and csg. This method demonstrated its efficacy with reduced runtime and computation cost. Thus, the proposed algorithm has emerged as a valuable resource for researchers investigating the influence of ballast particle shape on the mechanical behavior of granular assemblies.

Self-supervised feature-gate coupling for dynamic network pruning

Gating modules have been widely explored in dynamic network pruning (DNP) to reduce the run-time computational cost of deep neural networks while keeping the features representative. Despite the substantial progress, existing methods remain ignoring the consistency between feature and gate distributions, which may lead to distortion of gated features. In this paper, we propose a feature-gate coupling (FGC) approach aiming to align distributions of features and gates. FGC is a plug-and-play module that consists of two steps carried out in an iterative self-supervised manner. In the first step, FGC utilizes the k-Nearest Neighbor algorithm in the feature space to explore instance neighborhood relationships, which are treated as self-supervisory signals. In the second step, FGC exploits contrastive self-supervised learning (CSL) to regularize gating modules, leading to the alignment of instance neighborhood relationships within the feature and gate spaces. Experimental results validate that the proposed FGC method improves the baseline approach with significant margins, outperforming state-of-the-art methods with a better accuracy-computation trade-off. Code is publicly available at github.com/smn2010/FGC-PR.

Enhanced privacy-preserving distributed deep learning with application to fog-based IoT

Generally, privacy-preserving distributed deep learning (PPDDL) solutions provided so far present a trade-off between privacy and efficiency/effectiveness, especially, in terms of high communication and run-time costs (i.e. efficiency) and declined accuracy (i.e. effectiveness). Additionally, not much attention has been paid to proposing privacy-preserving solutions that are multi-key compliant, able to extend to new participants and well fit for an unbounded number of participants as well as being post-quantum robust. Also, application of DDL to fog-based IoT needs more attention. To this effect, we present enhanced PPDDL by proposing two solutions (basic and advanced). The basic combines LWE-based additive homomorphic encryption and partial sharing, making it communication cost friendly. The advanced, however, uses a lattice-based fully dynamic multi-key fully homomorphic encryption (FHE) scheme and partial sharing with the additional advantages of being multi-key compliant, able to extend to new participants and well fit for an unbounded number of participants. Furthermore, we put our work in the context of Fog-Based IoT. Extensive experimental evaluations for the basic solution which also best suits the Fog-Based IoT, show that the PPDDL accuracy is maintained, recording accuracy of about 97% for the MNIST dataset. For a much deeper architecture, accuracy could reach about 99%. To assess the generalizability of the accuracy trend observed in the PPDDL, the proposed solution was also evaluated using the CIFAR-10 dataset with interesting results recorded in this study. This paper is innovative as it combines partial sharing and homomorphic encryption in DDL . Solutions are post-quantum robust and communication cost efficient.

Compiler Autotuning through Multiple-phase Learning

Widely used compilers like GCC and LLVM usually have hundreds of optimizations controlled by optimization flags, which are enabled or disabled during compilation to improve the runtime performance (e.g., small execution time) of the compiler program. Due to the large number of optimization flags and their combination, it is difficult for compiler users to manually tune compiler optimization flags. In the literature, a number of autotuning techniques have been proposed, which tune optimization flags for a compiled program by comparing its actual runtime performance with different optimization flag combinations. Due to the huge search space and heavy actual runtime cost, these techniques suffer from the widely recognized efficiency problem. To reduce the heavy runtime cost, in this article we propose a lightweight learning approach that uses a small number of actual runtime performance data to predict the runtime performance of a compiled program with various optimization flag combinations. Furthermore, to reduce the search space, we design a novel particle swarm algorithm that tunes compiler optimization flags with the prediction model. To evaluate the performance of the proposed approach, CompTuner, we conduct an extensive experimental study on two popular C compilers, GCC and LLVM, with two widely used benchmarks, cBench and PolyBench. The experimental results show that CompTuner significantly outperforms the six compared techniques, including the state-of-the-art technique BOCA.

Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Models

Toxic content detection is crucial for online services to remove inappropriate content that violates community standards. To automate the detection process, prior works have proposed varieties of machine learning (ML) approaches to train Language Models (LMs) for toxic content detection. However, both their accuracy and transferability across datasets are limited. Recently, Large Language Models (LLMs) have shown promise in toxic content detection due to their superior zero-shot and few-shot in-context learning ability as well as broad transferability on ML tasks. However, efficiently designing prompts for LLMs remains challenging. Moreover, the high run-time cost of LLMs may hinder their deployments in production. To address these challenges, in this work, we propose BD-LLM, a novel and efficient approach to bootstrapping and distilling LLMs for toxic content detection. Specifically, we design a novel prompting method named Decision-Tree-of-Thought (DToT) to bootstrap LLMs' detection performance and extract high-quality rationales. DToT can automatically select more fine-grained context to re-prompt LLMs when their responses lack confidence. Additionally, we use the rationales extracted via DToT to fine-tune student LMs. Our experimental results on various datasets demonstrate that DToT can improve the accuracy of LLMs by up to 4.6%. Furthermore, student LMs fine-tuned with rationales extracted via DToT outperform baselines on all datasets with up to 16.9% accuracy improvement, while being more than 60x smaller than conventional LLMs. Finally, we observe that student LMs fine-tuned with rationales exhibit better cross-dataset transferability.

The economic and resource burden of e-scooter-related orthopaedic injuries: A district general hospital's experience

PurposeElectric scooters (e-scooters) are an increasingly popular method of transportation worldwide. However, there are concerns regarding their safety, specifically with regards to orthopaedic injuries. We aimed to investigate the overall burden and financial impact on orthopaedic services as a result of e-scooter-related orthopaedic injuries. MethodsWe retrospectively identified all e-scooter-related injuries requiring orthopaedic admission or surgical intervention in a large District General Hospital in England over a 16-month period between September 2020 and December 2021. Injuries sustained, surgical management, inpatient stay and resources used were calculated. ResultsSeventy-nine patients presented with orthopaedic injuries as a result of e-scooter transportation with a mean age of 30.1 years (SD 11.6), of which 62 were males and 17 were females. A total of 86 individual orthopaedic injuries were sustained, with fractures being the most common type of injury. Of these, 23 patients required 28 individual surgical procedures. The combined theatre and recovery time of these procedures was 5500 min, while isolated operating time was 2088 min. The total cost of theatre running time for these patients was estimated at £77,000. A total of 17 patients required hospital admission under Trauma and Orthopaedics, which accounted for total combined stay of 99 days with a mean length of stay of 5.8 days. ConclusionWhile there are potential environmental benefits to e-scooters, we demonstrate the risks of injury associated with their use and the associated increased burden to the healthcare system through additional emergency attendances, frequent outpatient clinic appointments, surgical procedures, and hospital inpatient admissions.

Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization

Features in images’ backgrounds can spuriously correlate with the images’ classes, representing background bias. They can influence the classifier’s decisions, causing shortcut learning (Clever Hans effect). The phenomenon generates deep neural networks (DNNs) that perform well on standard evaluation datasets but generalize poorly to real-world data. Layer-wise Relevance Propagation (LRP) explains DNNs’ decisions. Here, we show that the optimization of LRP heatmaps can minimize the background bias influence on deep classifiers, hindering shortcut learning. By not increasing run-time computational cost, the approach is light and fast. Furthermore, it applies to virtually any classification architecture. After injecting synthetic bias in images’ backgrounds, we compared our approach (dubbed ISNet) to eight state-of-the-art DNNs, quantitatively demonstrating its superior robustness to background bias. Mixed datasets are common for COVID-19 and tuberculosis classification with chest X-rays, fostering background bias. By focusing on the lungs, the ISNet reduced shortcut learning. Thus, its generalization performance on external (out-of-distribution) test databases significantly surpassed all implemented benchmark models.

Multimanifold Bistructured Low Rank Representation of hyperspectral images

Hyperspectral images (HSIs) are obtained by imaging surface materials using a spectral imager. In practical applications, the hyperspectral remote sensing images are inevitably contaminated by noise, leading to errors in image processing results. Gaussian noise, Poisson noise, and stripe noise are common in HSIs. This article presents an HSI denoising model based on the low rank representation of multi manifold double constructions. Introducing multiple manifolds into the field of hyperspectral image denoising for the first time, comprehensively obtaining complex information on hyperspectral surfaces. We extended the graph-regularized low-rank representation model (GLRR) by using maximum data variance to construct global information. GLRR obtains local information, and the parallel effects of nearest neighbor and maximum variance graphs take into account both global and local consistency in the denoising process. Our model is not only applicable to the case of a single manifold, but also to the case of multiple manifolds, especially in the case of complex ground target images with multiple manifold spectra. Use the LADMAP iterative algorithm to iteratively solve the proposed algorithm. Avoid introducing auxiliary variables to reduce runtime costs and algorithm complexity. Both on simulated and real datasets, the runtime is relatively fast. We experiments on both real and simulated datasets, as well as real datasets, to verify that our method has good denoising performance when Gaussian, Poisson, and stripe noise occur simultaneously.

Densest Multipartite Subgraph Search in Heterogeneous Information Networks

Cohesive multipartite subgraphs (CMS) in heterogeneous information networks (HINs) uncover closely connected vertex groups of multiple types, enhancing real applications like community search and anomaly detection. However, existing works for HINs pay less attention to searching CMS. In this paper, we leverage well-established concepts of meta-path and densest subgraph to propose a novel CMS model called the densest P -partite subgraph. Given a multipartite subgraph of an HIN induced by i =| P | types of vertices defined in a query meta-path P (i.e., a P -partite subgraph), we devise a novel density function which is the number of the instances of P over the geometric mean of the sizes of i different types of vertex sets in the subgraph. A P -partite subgraph with the highest density serves as the optimum result. To find the densest P -partite subgraph in an HIN with n vertices, we first design an exact algorithm with a runtime cost equivalent to solving Θ(|M|) instances of the min-cut problem where |M|= O (( n/i ) i ). Then, we attempt a more efficient approximation algorithm that achieves a ratio of 1/ i but still incurs the cost of solving Θ(|M|) instances of our proposed peeling problem. Both approaches struggle with scalability due to Θ(|M|). To overcome this bottleneck, we improve the exact algorithm with novel pruning rules that non-trivially reduce the number of min-cut problem instances to solve to O (|M|). Empirically, 70-90% instances are pruned, making the improved exact algorithm significantly faster than the approximation algorithm. Extensive experiments on real datasets demonstrate the effectiveness of the proposed model and the efficiency of our algorithms.

T-depth-optimized quantum search with quantum data-access machine

Abstract Quantum search algorithms offer a remarkable advantage of quadratic reduction in query complexity using quantum superposition principle. However, how an actual architecture may access and handle the database in a quantum superposed state has been largely unexplored so far; the quantum state of data was simply assumed to be prepared and accessed by a black-box operation—so-called oracle, even though this process, if not appropriately designed, may adversely diminish the quantum query advantage. Here, we introduce an efficient quantum data-access process, dubbed as quantum data-access machine (QDAM), and present a general architecture for quantum search algorithm. We analyze the runtime of our algorithm in view of the fault-tolerant quantum computation (FTQC) consisting of logical qubits within an effective quantum error correction code. Specifically, we introduce a measure involving two computational complexities, i.e. quantum query and T-depth complexities, which can be critical to assess performance since the logical non-Clifford gates, such as the T (i.e. π / 8 rotation) gate, are known to be costliest to implement in FTQC. Our analysis shows that for N searching data, a QDAM model exhibiting a logarithmic, i.e. O ( log N ) , growth of the T-depth complexity can be constructed. Further analysis reveals that our QDAM-embedded quantum search requires O ( N × log N ) runtime cost. Our study thus demonstrates that the quantum data search algorithm can truly speed up over classical approaches with the logarithmic T-depth QDAM as a key component.

GPU Database Systems Characterization and Optimization

GPUs offer massive parallelism and high-bandwidth memory access, making them an attractive option for accelerating data analytics in database systems. However, while modern GPUs possess more resources than ever before (e.g., higher DRAM bandwidth), efficient system implementations and judicious resource allocations for query processing are still necessary for optimal performance. Database systems can save GPU runtime costs through just-enough resource allocation or improve query throughput with concurrent query processing by leveraging new GPU resource-allocation capabilities, such as Multi-Instance GPU (MIG). In this paper, we do a cross-stack performance and resource-utilization analysis of four GPU database systems, including Crystal (the state-of-the-art GPU database, performance-wise) and TQP (the latest entry in the GPU database space). We evaluate the bottlenecks of each system through an in-depth microarchitectural study and identify resource underutilization by leveraging the classic roofline model. Based on the insights gained from our investigation, we propose optimizations for both system implementation and resource allocation, using which we are able to achieve 1.9x lower latency for single-query execution and up to 6.5x throughput improvement for concurrent query execution.

Evaluating the accuracy of predicted bridge condition using machine learning: the role of condition history

Effective maintenance decisions for bridges depend on accurate performance prediction. Machine learning (ML) models use historical bridge performance data to learn and predict performance. However, in many agencies, the condition history of bridges is limited and does not go beyond a few years. The question, therefore, is, to what extent does condition history help us make better predictions? To address this question, a ML model was developed that analysed more than 600,000 bridge decks with 27 years of condition history. Two data selection methods were designed: non-overlapping and overlapping data. The non-overlapping data are typically used to train the model. The overlapping data introduced in this study uses the data more efficiently for model training recognising that strings of historical data convey more information. Longer term predictions were found to be positively impacted by every additional year of condition history. Short-term condition prediction (one or two years) does not need significant historical data. It was also found that overlapping data, compared to non-overlapping data, produced larger training samples and had higher prediction accuracy in the majority of experiments, but at the cost of higher running time due to a larger sample size.