Machine Learning As A Service Research Articles

Private-preserving language model inference based on secure multi-party computation

With the exponential expansion of Internet information, technology that combines big data and artificial intelligence has gradually developed. Pre-trained large-scale language models with the transformer architecture as the core have begun to be used in daily life, resulting in the huge market of MLaaS. leading to the significant market of Machine Learning as a Service (MLaaS). Although MLaaS brings huge benefits to users, it requires receiving users’ data for processing, which includes many sensitive data. While MLaaS offers considerable benefits, it necessitates processing users’ data, which includes much sensitive data.Therefore, the problem of privacy data leakage has also been exposed. In this article paper, we propose a novel language model secure inference scheme based on secure multi-party computation (MPC) technology. This solution involves three non-colluding parties: the data provider, the model provider, and the computing power provider. Compared with direct inference on pre-trained large models, the proposed security inference framework improves the inference speed by 1.55–6.25 times. Our findings demonstrate that, when compared to conventional inference methods on pre-trained large-scale models, our approach significantly enhances inference efficiency, achieving speed improvements ranging from 1.55 to 6.25 times.

A Machine Learning as a Service (MLaaS) Approach to Improve Marketing Success

The exponential growth of data in the digital age has led to a significant demand for innovative approaches to assess data in a manner that is both effective and efficient. Machine Learning as a Service (MLaaS) is a category of services that offers considerable potential for organisations to extract valuable insights from their data while reducing the requirement for heavy technical expertise. This article explores the use of MLaaS within the realm of marketing applications. In this study, we provide a comprehensive analysis of MLaaS implementations and their benefits within the domain of marketing. Furthermore, we present a platform that possesses the capability to be customised and expanded to address marketing’s unique requirements. Three modules are introduced: Churn Prediction, One-2-One Product Recommendation, and Send Frequency Prediction. When applied to marketing, the proposed MLaaS system exhibits considerable promise for use in applications such as automated detection of client churn prior to its occurrence, individualised product recommendations, and send time optimisation. Our study revealed that AI-driven campaigns can improve both the Open Rate and Click Rate. This approach has the potential to enhance customer engagement and retention for businesses while enabling well-informed decisions by leveraging insights derived from consumer data. This work contributes to the existing body of research on MLaaS in marketing and offers practical insights for businesses seeking to utilise this approach to enhance their competitive edge in the contemporary data-oriented marketplace.

Data-Free Hard-Label Robustness Stealing Attack

The popularity of Machine Learning as a Service (MLaaS) has led to increased concerns about Model Stealing Attacks (MSA), which aim to craft a clone model by querying MLaaS. Currently, most research on MSA assumes that MLaaS can provide soft labels and that the attacker has a proxy dataset with a similar distribution. However, this fails to encapsulate the more practical scenario where only hard labels are returned by MLaaS and the data distribution remains elusive. Furthermore, most existing work focuses solely on stealing the model accuracy, neglecting the model robustness, while robustness is essential in security-sensitive scenarios, e.g, face-scan payment. Notably, improving model robustness often necessitates the use of expensive techniques such as adversarial training, thereby further making stealing robustness a more lucrative prospect. In response to these identified gaps, we introduce a novel Data-Free Hard-Label Robustness Stealing (DFHL-RS) attack in this paper, which enables the stealing of both model accuracy and robustness by simply querying hard labels of the target model without the help of any natural data. Comprehensive experiments demonstrate the effectiveness of our method. The clone model achieves a clean accuracy of 77.86% and a robust accuracy of 39.51% against AutoAttack, which are only 4.71% and 8.40% lower than the target model on the CIFAR-10 dataset, significantly exceeding the baselines. Our code is available at: https://github.com/LetheSec/DFHL-RS-Attack.

SAME: Sample Reconstruction against Model Extraction Attacks

While deep learning models have shown significant performance across various domains, their deployment needs extensive resources and advanced computing infrastructure. As a solution, Machine Learning as a Service (MLaaS) has emerged, lowering the barriers for users to release or productize their deep learning models. However, previous studies have highlighted potential privacy and security concerns associated with MLaaS, and one primary threat is model extraction attacks. To address this, there are many defense solutions but they suffer from unrealistic assumptions and generalization issues, making them less practical for reliable protection. Driven by these limitations, we introduce a novel defense mechanism, SAME, based on the concept of sample reconstruction. This strategy imposes minimal prerequisites on the defender's capabilities, eliminating the need for auxiliary Out-of-Distribution (OOD) datasets, user query history, white-box model access, and additional intervention during model training. It is compatible with existing active defense methods. Our extensive experiments corroborate the superior efficacy of SAME over state-of-the-art solutions. Our code is available at https://github.com/xythink/SAME.

United We Stand: Accelerating Privacy-Preserving Neural Inference by Conjunctive Optimization with Interleaved Nexus

Privacy-preserving Machine Learning as a Service (MLaaS) enables the powerful cloud server to run its well-trained neural model upon the input from resource-limited client, with both of server's model parameters and client's input data protected. While computation efficiency is critical for the practical implementation of privacy-preserving MLaaS and it is inspiring to witness recent advances towards efficiency improvement, there still exists a significant performance gap to real-world applications. In general, state-of-the-art frameworks perform function-wise efficiency optimization based on specific cryptographic primitives. Although it is logical, such independent optimization for each function makes noticeable amount of expensive operations unremovable and misses the opportunity to further accelerate the performance by jointly considering privacy-preserving computation among adjacent functions. As such, we propose COIN: Conjunctive Optimization with Interleaved Nexus, which remodels mainstream computation for each function to conjunctive counterpart for composite function, with a series of united optimization strategies. Specifically, COIN jointly computes a pair of consecutive nonlinear-linear functions in the neural model by reconstructing the intermediates throughout the whole procedure, which not only eliminates the most expensive crypto operations without invoking extra encryption enabler, but also makes the online crypto complexity independent of filter size. Experimentally, COIN demonstrates 11.2x to 29.6x speedup over various function dimensions from modern networks, and 6.4x to 12x speedup on the total computation time when applied in networks with model input from small-scale CIFAR10 to large-scale ImageNet.

BFS2Adv: Black-box adversarial attack towards hard-to-attack short texts

The advent of Machine Learning as a Service (MLaaS) and deep learning applications has increased the susceptibility of models to adversarial textual attacks, particularly in black-box settings. Prior work on black-box adversarial textual attacks generally follows a stable strategy that involves leveraging char-level, world-level, and sentence-level perturbations, as well as using queries to the target model to find adversarial examples in the search space. However, existing approaches prioritize query efficiency by reducing the search space, thereby overlooking hard-to-attack textual instances. To address this issue, we propose BFS2Adv, a brute force algorithm that generates adversarial examples for both easy-to-attack and hard-to-attack textual inputs. BFS2Adv, starting with an original text, employs word-level perturbations and synonym substitution to construct a comprehensive search space, with each node representing a potential adversarial example. The algorithm systematically explores this space through a breadth-first search, combined with queries to the target model, to effectively identify qualified adversarial examples. We implemented and evaluated a prototype of BFS2Adv against renowned models such as ALBERT and BERT, utilizing the SNLI and MR datasets. Our results demonstrate that BFS2Adv outperforms state-of-the-art algorithms and effectively improves the success rate of short-text adversarial attacks. Furthermore, we provide detailed insights into the robustness of BFS2Adv by analyzing those hard-to-attack examples.

Practical Privacy-Preserving MLaaS: When Compressive Sensing Meets Generative Networks

The Machine-Learning-as-a-Service (MLaaS) framework allows one to grab low-hanging fruit of machine learning techniques and data science, without either much expertise for this sophisticated sphere or provision of specific infrastructures. However, the requirement of revealing all training data to the service provider raises new concerns in terms of privacy leakage, storage consumption, efficiency, bandwidth, etc. In this paper, we propose a lightweight privacy-preserving MLaaS framework by combining Compressive Sensing (CS) and Generative Networks. It’s constructed on the favorable facts observed in recent works that general inference tasks could be fulfilled with generative networks and classifier trained on compressed measurements, since the generator could model the data distribution and capture discriminative information which are useful for classification. To improve the performance of the MLaaS framework, the supervised generative models of the server are trained and optimized with prior knowledge provided by the client. In order to prevent the service provider from recovering the original data as well as identifying the queried results, a noise-addition mechanism is designed and adopted into the compressed data domain. Empirical results confirmed its performance superiority in accuracy and resource consumption against the state-of-the-art privacy preserving MLaaS frameworks.

A flexible and privacy-preserving federated learning framework based on logistic regression

Machine Learning as a Service (MLaaS) has been widely used, such as logistic regression models trained in the cloud to provide people with classification predictions. However, these services allow the cloud to collect data from a large number of users, which could raise privacy concerns. With sharing of local models rather than user data, Federated Learning (FL) alleviates data privacy. But, in FL setting the difference between the predictive value provided by the optimal model and the shared global model (called maturing model) that approximates it is not significant, which still raises a security risk. In this paper, we design a flexible and privacy-preserving FL system to tackle these issues. This system makes the training process divide into two stages by analyzing whether the metrics of the iterative global gradient reach a set threshold or not. The two stages are implemented with Re-Encryption and CKKS homomorphism, which not only enables a secure aggregation in the cloud, but also maintains the security of the maturing model. Through detailed analysis, we prove the security of our protocol. Moreover, quantitative comparisons in the final experiments demonstrates that our scheme can achieve trade-off between accuracy, training time cost and communication overhead.

Random forest evaluation using multi-key homomorphic encryption and lookup tables

In recent years, machine learning (ML) has become increasingly popular in various fields of activity. Cloud platforms have also grown in popularity, as they offer services that are more secure and accessible worldwide. In this context, cloud-based technologies emerged to support ML, giving rise to the machine learning as a service (MLaaS) concept. However, the clients accessing ML services in order to obtain classification results on private data may be reluctant to upload sensitive information to cloud. The model owners may also prefer not to outsource their models in order to prevent model inversion attacks and to protect intellectual property. The privacy-preserving evaluation of ML models is possible through multi-key homomorphic encryption (MKHE), that allows both the client data and the model to be encrypted under different keys. In this paper, we propose an MKHE evaluation method for decision trees and we extend the proposed method for random forests. Each decision tree is evaluated as a single lookup table, and voting is performed at the level of groups of decision trees in the random forest. We provide both theoretical and experimental evaluations for the proposed method. The aim is to minimize the performance degradation introduced by the encrypted model compared to a plaintext model while also obtaining practical classification times. In our experiments with the proposed MKHE random forest evaluation method, we obtained minimal (less than 0.6%) impact on the main ML performance metrics considered for each scenario, while also achieving reasonable classification times (of the order of seconds).

A Comprehensive Defense Framework Against Model Extraction Attacks

As a promising service, Machine Learning as a Service (MLaaS) provides personalized inference functions for clients through paid APIs. Nevertheless, it is vulnerable to model extraction attacks, in which an attacker can extract a functionally-equivalent model by repeatedly querying the APIs with crafted samples. While numerous works have been proposed to defend against model extraction attacks, existing efforts are accompanied by limitations and low comprehensiveness. In this paper, we propose AMAO, a comprehensive defense framework against model extraction attacks. Specifically, AMAO consists of four interlinked successive phases: adversarial training is first exploited to weaken the effectiveness of model extraction attacks. Then, malicious query detection is used to detect malicious queries and mark malicious users. After that, we develop a label-flipping poisoning attack to instruct the adaptive query responses to malicious users. Besides, the image pHash algorithm is employed to ensure the indistinguishability of the query responses. Finally, the perturbed results are served as a backdoor to verify the ownership of any suspicious model. Extensive experiments demonstrate that AMAO outperforms existing defenses in defending against model extraction attacks and is also robust against the adaptive adversary who is aware of the defense.

Respecting Partial Privacy of Unstructured Data via Spectrum-Based Encoder

Since the popularity of Machine Learning as a Service (MLaaS) has been increasing significantly, users are facing the risk of exposing sensitive information that is not task-related. The reason is that the data uploaded by users may include some information that is not useful for inference but can lead to privacy leakage. One straightforward approach to mitigate this issue is to filter out task-independent information to protect user privacy. However, this method is feasible for structured data with naturally independent entries, but it is challenging for unstructured data. Therefore, we propose a novel framework, which employs a spectrum-based encoder to transform unstructured data into the latent space and a task-specific model to identify the essential information for the target task. Our system has been comprehensively evaluated on three benchmark visual datasets and compared to previous works. The results demonstrate that our framework offers superior protection for task-independent information and maintains the usefulness of task-related information.

Machine Learning as a Service Cloud Selection: An MCDM Approach for Optimal Decision Making

Machine Learning as a Service, MLaaS refers to cloud-based platforms that provide machine learning tools and infrastructure to users, allowing them to access and utilize machine learning capabilities without managing the underlying hardware and software. Cloud service providers offer scalable and flexible resources, enabling businesses to train and deploy machine learning models efficiently and cost-effectively. Multi-Criteria Decision making, MCDM helps individuals or organizations make choices when multiple criteria or objectives are involved. It aims to find the best alternative among a set of available options by considering various criteria or factors that are relevant to the decision. Machine learning as a service (MLaaS) and MCDM are two distinct concepts that can be combined to create powerful decision-making solutions. The present paper proposed an integrated approach based on the Analytic Hierarchy Process and TOPSIS method for choosing the best MLaaS cloud. Four alternatives for MLaaS cloud were analyzed based on seventeen shortlisted criteria. Based on the calculations, conclusions have been reached and potential study areas have been identified.

VeriTrain: Validating MLaaS Training Efforts via Anomaly Detection

Machine learning as a service (MLaaS) offers users the benefit of training state-of-the-art neural network models on fast hardware with low costs. However, it also brings security concerns since the user does not fully trust the cloud. To prove to the user that the ML training results are legitimate, existing approaches mainly adopt cryptographic techniques such as secure multi-party computation, which incur large overheads. In this paper, we model the problem of verifying ML training efforts as an anomaly detection problem. We design a verification system, dubbed <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VeriTrain</small> , which combines unsupervised anomaly detection approaches and hypothesis testing techniques to verify the legitimacy of training efforts on the MLaaS cloud. <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VeriTrain</small> is run inside trusted execution environments (TEEs) on the same cloud machine to ensure the integrity of its execution. We consider a threat model where the cloud model trainer is a lazy attacker and tries to fool <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VeriTrain</small> with minimum training effort. We perform extensive evaluations on multiple neural network models and datasets, which shows that <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VeriTrain</small> performs well in detecting parameter updates crafted by the attacker. We also implement <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VeriTrain</small> with Intel SGX and show that it only incurs moderate overheads.

Understanding images of surveillance devices in the wild

Closed-circuit television (CCTV) systems and surveillance devices have been widely used for monitoring the real world (e.g., streets, lanes, buildings). Illumination of the nature of surveillance devices’ images can help in deterring criminal activity and provide insights for improving the infrastructure safety of CCTV systems. However, millions of outdated surveillance devices are still in use today, which suffer noise, the field of view, low-light conditions, and low visual quality. In this work, we conduct a large-scale empirical study of images of surveillance devices in the wild. For data collection, we deploy a web crawler to gather webcam images in the wild covering 100 websites that distribute the live streams to the public, such as a plaza and an industrial area. To enhance the low-light quality of images, we propose a new framework called ImCam for enhancing the quality of webcam images in the wild, including the retinex model and generative adversarial network (GAN). We develop a prototype of ImCam and evaluate its effectiveness over three classification systems, AlexNet, ResNet, and Vision Transformer (ViT), and Machine Learning as a Service (MLaaS) in three cloud service providers (Azure, Baidu, and Tencent). Our results show that ImCam is capable of improving low-light images for CCTV systems, making them acceptable in practice. Finally, leveraging 203,786 live streams and our ImCam, we conduct an analysis of webcam images in the wild and shed light on the nature of security usage for CCTV systems.

DB-COVIDNet: A Defense Method against Backdoor Attacks

With the emergence of COVID-19 disease in 2019, machine learning (ML) techniques, specifically deep learning networks (DNNs), played a key role in diagnosing the disease in the medical industry due to their superior performance. However, the computational cost of deep learning networks (DNNs) can be quite high, making it necessary to often outsource the training process to third-party providers, such as machine learning as a service (MLaaS). Therefore, careful consideration is required to achieve robustness in DNN-based systems against cyber-security attacks. In this paper, we propose a method called the dropout-bagging (DB-COVIDNet) algorithm, which works as a robust defense mechanism against poisoning backdoor attacks. In this model, the trigger-related features will be removed by the modified dropout algorithm, and then we will use the new voting method in the bagging algorithm to achieve the final results. We considered AC-COVIDNet as the main inducer of the bagging algorithm, which is an attention-guided contrastive convolutional neural network (CNN), and evaluated the performance of the proposed method with the malicious COVIDx dataset. The results demonstrated that DB-COVIDNet has strong robustness and can significantly reduce the effect of the backdoor attack. The proposed DB-COVIDNet nullifies backdoors before the attack has been activated, resulting in a tremendous reduction in the attack success rate from 99.5% to 3% with high accuracy on the clean data.

I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences

Machine-Learning-as-a-Service (MLaaS) has become a widespread paradigm, making even the most complex Machine Learning models available for clients via, e.g., a pay-per-query principle. This allows users to avoid time-consuming processes of data collection, hyperparameter tuning, and model training. However, by giving their customers access to the (predictions of their) models, MLaaS providers endanger their intellectual property such as sensitive training data, optimised hyperparameters, or learned model parameters. In some cases, adversaries can create a copy of the model with (almost) identical behaviour using the the prediction labels only. While many variants of this attack have been described, only scattered defence strategies that address isolated threats have been proposed. To arrive at a comprehensive understanding why these attacks are successful and how they could be holistically defended against, a thorough systematisation of the field of model stealing is necessary. We address this by categorising and comparing model stealing attacks, assessing their performance, and exploring corresponding defence techniques in different settings. We propose a taxonomy for attack and defence approaches and provide guidelines on how to select the right attack or defence strategy based on the goal and available resources. Finally, we analyse which defences are rendered less effective by current attack strategies.

DisGUIDE: Disagreement-Guided Data-Free Model Extraction

Recent model-extraction attacks on Machine Learning as a Service (MLaaS) systems have moved towards data-free approaches, showing the feasibility of stealing models trained with difficult-to-access data. However, these attacks are ineffective or limited due to the low accuracy of extracted models and the high number of queries to the models under attack. The high query cost makes such techniques infeasible for online MLaaS systems that charge per query. We create a novel approach to get higher accuracy and query efficiency than prior data-free model extraction techniques. Specifically, we introduce a novel generator training scheme that maximizes the disagreement loss between two clone models that attempt to copy the model under attack. This loss, combined with diversity loss and experience replay, enables the generator to produce better instances to train the clone models. Our evaluation on popular datasets CIFAR-10 and CIFAR-100 shows that our approach improves the final model accuracy by up to 3.42% and 18.48% respectively. The average number of queries required to achieve the accuracy of the prior state of the art is reduced by up to 64.95%. We hope this will promote future work on feasible data-free model extraction and defenses against such attacks.

XRand: Differentially Private Defense against Explanation-Guided Attacks

Recent development in the field of explainable artificial intelligence (XAI) has helped improve trust in Machine-Learning-as-a-Service (MLaaS) systems, in which an explanation is provided together with the model prediction in response to each query. However, XAI also opens a door for adversaries to gain insights into the black-box models in MLaaS, thereby making the models more vulnerable to several attacks. For example, feature-based explanations (e.g., SHAP) could expose the top important features that a black-box model focuses on. Such disclosure has been exploited to craft effective backdoor triggers against malware classifiers. To address this trade-off, we introduce a new concept of achieving local differential privacy (LDP) in the explanations, and from that we establish a defense, called XRand, against such attacks. We show that our mechanism restricts the information that the adversary can learn about the top important features, while maintaining the faithfulness of the explanations.

Decentralized and secure deduplication with dynamic ownership in MLaaS

Machine Learning as a Service (MLaaS) these days has been an integral offering of many Internet giants, shaping people’s lives in an intelligent manner. Both academic and industrial communities are dedicated to exploring a variety of functional MLaaS platforms, where massive data is typically required for model training to achieve better capability. As the widely used training data can be repeatedly stored, data deduplication has been deemed essential. Meanwhile, the proliferation of ML-aimed attacks has raised security awareness. In light of these, we propose a secure and robust deduplication scheme over encrypted data for decentralized MLaaS platforms. We utilize message-locked encryption for privacy protection on blockchain over decentralized cloud storage. To balance the overhead from blockchain, we offload the computation off-chain and only maintain the system state on-chain. We remedy the potential leakages from the transparency of blockchain, through carefully tailored cross-user deduplication workflow. Our proposed scheme is also robust against short-information and brute-force attacks. Furthermore, we apply binary tree based key distribution to support dynamic ownership updates. We implement a prototype on Ethereum, and comprehensive experiments show that our design can function as intended with modest on-chain update gas cost (i.e., 1.67×10−4 ETH), and the blockchain-related operations run less than 6 s.

Towards realistic privacy-preserving deep learning over encrypted medical data.

Cardiovascular disease supposes a substantial fraction of healthcare systems. The invisible nature of these pathologies demands solutions that enable remote monitoring and tracking. Deep Learning (DL) has arisen as a solution in many fields, and in healthcare, multiple successful applications exist for image enhancement and health outside hospitals. However, the computational requirements and the need for large-scale datasets limit DL. Thus, we often offload computation onto server infrastructure, and various Machine-Learning-as-a-Service (MLaaS) platforms emerged from this need. These enable the conduction of heavy computations in a cloud infrastructure, usually equipped with high-performance computing servers. Unfortunately, the technical barriers persist in healthcare ecosystems since sending sensitive data (e.g., medical records or personally identifiable information) to third-party servers involves privacy and security concerns with legal and ethical implications. In the scope of Deep Learning for Healthcare to improve cardiovascular health, Homomorphic Encryption (HE) is a promising tool to enable secure, private, and legal health outside hospitals. Homomorphic Encryption allows for privacy-preserving computations over encrypted data, thus preserving the privacy of the processed information. Efficient HE requires structural optimizations to perform the complex computation of the internal layers. One such optimization is Packed Homomorphic Encryption (PHE), which encodes multiple elements on a single ciphertext, allowing for efficient Single Instruction over Multiple Data (SIMD) operations. However, using PHE in DL circuits is not straightforward, and it demands new algorithms and data encoding, which existing literature has not adequately addressed. To fill this gap, in this work, we elaborate on novel algorithms to adapt the linear algebra operations of DL layers to PHE. Concretely, we focus on Convolutional Neural Networks. We provide detailed descriptions and insights into the different algorithms and efficient inter-layer data format conversion mechanisms. We formally analyze the complexity of the algorithms in terms of performance metrics and provide guidelines and recommendations for adapting architectures that deal with private data. Furthermore, we confirm the theoretical analysis with practical experimentation. Among other conclusions, we prove that our new algorithms speed up the processing of convolutional layers compared to the existing proposals.