Commercial Cloud Research Articles

IATT: Interpretation Analysis based Transferable Test Generation for Convolutional Neural Networks

Convolutional neural networks (CNNs) have been widely used in various fields. However, it is essential to perform sufficient testing to detect internal defects before deploying CNNs, especially in security-sensitive scenarios. Generating error-inducing inputs to trigger erroneous behavior is the primary way to detect CNN model defects. However, in practice, when the model under test is a black-box CNN model without accessible internal information, in some scenarios it is still necessary to generate high-quality test inputs within a limited testing budget. In such a new scenario, a potential approach is to generate transferable test inputs by analyzing the internal knowledge of other white-box CNN models similar to the model under test, and then use transferable test inputs to test the black-box CNN model. The main challenge in generating transferable test inputs is how to improve their error-inducing capability for different CNN models without changing the test oracle. We found that different CNN models make predictions based on features of similar important regions in images. Adding targeted perturbations to important regions will generate transferable test inputs with high realism. Therefore, we propose the Interpretable Analysis based Transferable Test Generation method for CNNs (IATT), which employs interpretation methods of CNN models to explain and localize important regions in test inputs, using backpropagation optimizer and perturbation mask process to add targeted perturbations to these important regions, thereby generating transferable test inputs. This process is repeated to iteratively optimize the transferability and realism of the test inputs. To verify the effectiveness of IATT, we perform experimental studies on nine deep learning models, including ResNet-50 and Vit-B/16, and commercial computer vision system Google Cloud Vision , and compared our method with four state-of-the-art baseline methods. Experimental results show that transferable test inputs generated by IATT can effectively cause black-box target models to output incorrect results. Compared to existing testing and adversarial attack methods, the average error-inducing success rate (ESR) in different testing scenarios is 18.1% \(\sim\) 52.7% greater than the baseline methods. Additionally, the test inputs generated by IATT achieve high ESR while maintaining high realism.

Developing remote patient monitoring infrastructure using commercially available cloud platforms.

Wearable sensor devices for continuous patient monitoring produce a large volume of data, necessitating scalable infrastructures for efficient data processing, management and security, especially concerning Patient Health Information (PHI). Adherence to the Health Insurance Portability and Accountability Act (HIPAA), a legislation that mandates developers and healthcare providers to uphold a set of standards for safeguarding patients' health information and privacy, further complicates the development of remote patient monitoring within healthcare ecosystems. This paper presents an Internet of Things (IoT) architecture designed for the healthcare sector, utilizing commercial cloud platforms like Microsoft Azure and Amazon Web Services (AWS) to develop HIPAA-compliant health monitoring systems. By leveraging cloud functionalities such as scalability, security, and load balancing, the architecture simplifies the creation of infrastructures adhering to HIPAA standards. The study includes a cost analysis of Azure and AWS infrastructures and evaluates data processing speeds and database query latencies, offering insights into their performance for healthcare applications.

Bio-mimetic deconstruction of the digital business ecosystem

The advent of the Internet Plus era, digital technologies, and the digital economy has instigated profound transformations in the commercial landscape, particularly evident in the systematic reshaping of the Digital Business Ecosystem (DBE), encompassing innovations in business models, norms of commercial conduct, and the exploration of business value. This paper delves into the panoramic view of digital business operations of typical companies to uncover the fundamental structural framework of digital commerce. Through deductive reasoning and drawing upon the theoretical framework of natural niche, we construct a niche model for the digital business ecosystem, thereby achieving a bionic deconstruction of the digital business ecosystem. The significance of this research lies in offering a novel research perspective for enterprises, economic regulatory bodies, and scholars in the field of business management, proposing a systemic approach rooted in niche theory models to competition. This approach provides a fresh theoretical framework for enterprises to devise their own ecological and sustainable development strategies. The key findings are as follows: (1) Most business firms establish competitive advantages by constructing commercial cloud platforms that facilitate internal digital transformation and enable digital synergy with external economic entities; (2) Within the digital business ecosystem, enterprises extend their digital capabilities externally through four modalities: data development, data application, data services, and data manufacturing. Externally, six primary forces and roles shape the ecosystem: suppliers, governments, social institutions, consumers, as well as external and internal industry players; (3) The digital business niche is a multidimensional and hyper volumetric relationship positioning between enterprises and the digital business environment. The niche factors include six dimensions: market, personnel, resources, social relationships, technology, and institutions; (4) Given limited ecological factors, the non-exclusivity between static resource allocation and dynamic technological investments in digital enablement leads to the generalization of property rights boundaries and industrial values within the digital business ecosystem. Consequently, this fosters extensive business applications and diversified business models, thereby resulting in less competition and more cooperation, symbiosis, and complementarity within the digital business niche.

Multi-scenario failure diagnosis for lithium-ion battery based on coupling PSO-SA-DBSCAN algorithm

The battery failure diagnosis methods are the main algorithms for online battery management system, but the conventional methods are mainly developed from calibrated thresholds detection, which will suffer from the degradation for full-lifespan operation and have no consideration of utilization for battery system information. Here we show innovative diagnosis methods for detecting battery failure both from online battery management system and cloud monitoring platform based on a particle swarm optimization-simulated annealing- density based spatial clustering of applications with noise (PSO-SA-DBSCAN) algorithm. By using the battery electrochemical impedance spectroscopy and voltage, the proposed method can solve the problem of battery abnormal degradation diagnosis, thermal runaway diagnosis and sampling failure diagnosis. Compared against conventional methods, the proposed method has less reliance on parameterization with better precision, which is validated from commercial prismatic cells and cloud data from a 117 serial-cell pack, with high accuracy and lower misinformation compared against conventional detecting methods, confirming the actual effectiveness for online battery pack.

On the Analysis of Inter-Relationship between Auto-Scaling Policy and QoS of FaaS Workloads

A recent development in cloud computing has introduced serverless technology, enabling the convenient and flexible management of cloud-native applications. Typically, the Function-as-a-Service (FaaS) solutions rely on serverless backend solutions, such as Kubernetes (K8s) and Knative, to leverage the advantages of resource management for underlying containerized contexts, including auto-scaling and pod scheduling. To take the advantages, recent cloud service providers also deploy self-hosted serverless services by facilitating their on-premise hosted FaaS platforms rather than relying on commercial public cloud offerings. However, the lack of standardized guidelines on K8s abstraction to fairly schedule and allocate resources on auto-scaling configuration options for such on-premise hosting environment in serverless computing poses challenges in meeting the service level objectives (SLOs) of diverse workloads. This study fills this gap by exploring the relationship between auto-scaling behavior and the performance of FaaS workloads depending on scaling-related configurations in K8s. Based on comprehensive measurement studies, we derived the logic as to which workload should be applied and with what type of scaling configurations, such as base metric, threshold to maximize the difference in latency SLO, and number of responses. Additionally, we propose a methodology to assess the scaling efficiency of the related K8s configurations regarding the quality of service (QoS) of FaaS workloads.

Operational experience and R&D results using the Google Cloud for High-Energy Physics in the ATLAS experiment

The ATLAS experiment at CERN relies on a Worldwide Distributed Computing Grid infrastructure to support its physics program at the Large Hadron Collider. ATLAS has integrated cloud computing resources to complement its Grid infrastructure and conducted an R&D program on Google Cloud Platform. These initiatives leverage key features of commercial cloud providers: lightweight configuration and operation, elasticity and availability of diverse infrastructures. This paper examines the seamless integration of cloud computing services as a conventional Grid site within the ATLAS workflow management and data management systems, while also offering new setups for interactive, parallel analysis. It underscores pivotal results that enhance the on-site computing model and outlines several R&D projects that have benefited from large-scale, elastic resource provisioning models. Furthermore, this study discusses the impact of cloud-enabled R&D projects in three domains: accelerators and AI/ML, ARM CPUs and columnar data analysis techniques.

EFraS: Emulated framework to develop and analyze dynamic Virtual Network Embedding strategies over SDN infrastructure

The integration of Software-Defined Networking (SDN) into Network Virtualization (NV) significantly enhances network management, isolation, and troubleshooting capabilities. However, it brings forth the intricate challenge of allocating Substrate Network (SN) resources for various Virtual Network Requests (VNRs), a process known as Virtual Network Embedding (VNE). It encompasses solving two intractable sub-problems: embedding Virtual Machines (VMs) and embedding Virtual Links (VLs). While the research community has focused on formulating embedding strategies, there has been less emphasis on practical implementation at a laboratory scale, which is crucial for comprehensive design, development, testing, and validation policies for large-scale systems. However, conducting tests using commercial providers presents challenges due to the scale of the problem and associated costs. Moreover, current simulators lack accuracy in representing the complexities of communication patterns, resource allocation, and support for SDN-specific features. These limitations result in inefficient implementations and reduced adaptability, hindering seamless integration with commercial cloud providers. To address this gap, this work introduces EFraS (Emulated Framework for Dynamic VNE Strategies over SDN). The goal is to aid developers and researchers in iterating, testing, and evaluating VNE solutions seamlessly, leveraging a modular design and customized reconfigurability. EFraS offers various functionalities, including generating real-world SN topologies and VNRs. Additionally, it integrates with a diverse set of evaluation metrics to streamline the testing and validation process. EFraS leverages Mininet, Ryu controller, and OpenFlow switches to closely emulate real-time setups. Moreover, we integrate EFraS with various state-of-the-art VNE schemes, ensuring the effective validation of embedding algorithms.

Scalable and efficient DNA sequencing analysis on different compute infrastructures aiding variant discovery.

DNA variation analysis has become indispensable in many aspects of modern biomedicine, most prominently in the comparison of normal and tumor samples. Thousands of samples are collected in local sequencing efforts and public databases requiring highly scalable, portable, and automated workflows for streamlined processing. Here, we present nf-core/sarek 3, a well-established, comprehensive variant calling and annotation pipeline for germline and somatic samples. It is suitable for any genome with a known reference. We present a full rewrite of the original pipeline showing a significant reduction of storage requirements by using the CRAM format and runtime by increasing intra-sample parallelization. Both are leading to a 70% cost reduction in commercial clouds enabling users to do large-scale and cross-platform data analysis while keeping costs and CO2 emissions low. The code is available at https://nf-co.re/sarek.

Digital twin framework for smart greenhouse management using next-gen mobile networks and machine learning

Due to the increase in world population, arable land has been reduced. Consequently, the concept of urban greenhouses is on the rise. Smart greenhouses need to monitor physical parameters for the healthy growth of plants from remote locations. A digital twin is a representation of physical assets in the digital world, and this emerging technology has opened up opportunities for efficient system development for Industry 4.0. The digital twin receives real-time operational data to monitor the asset in the digital domain. It performs real-time processing, data analysis, and machine learning to predict optimized decisions. In the era of next-generation mobile networks, IoT devices can communicate and perform their remote operations in a timely manner. In smart greenhouse technology, the digital twin could be a revolutionary substitute for real-time remote monitoring and process management. However, there has been limited work on digital twin-driven smart greenhouse technology. In this paper, a process management framework is developed that can be interpreted as a machine learning and cloud-based data-driven digital twin for smart greenhouses. The proposed framework consists of three layers: the physical, fog, and cloud layers. The physical greenhouse measurements are monitored using a highly immersive cloud-based, real-time 3D environment. We present an example architecture using commercial cloud and open-source tools to verify the proof of concept. Additionally, different ML techniques are utilized to predict the operational requirements for smart greenhouses.

Secure medical data on cloud storage via DNA homomorphic encryption technique

BackgroundCloud computing offers considerable flexibility and financial savings to data owners for outsourcing their complicated data management systems from local sites to the commercial public cloud. MethodsHowever, an effective encryption scheme is needed to protect the private medical data of the user in the cloud without leaking it. Therefore, in this paper an efficient Deoxyribonucleic Acid Homomorphic Encryption algorithm (DNA-HE) has been proposed to encrypt and store the medical images in the cloud securely. Initially, the data is mapped to DNA sequences and then encrypted using homomorphic encryption. The key for the homomorphic algorithm is generated using Rider Optimization Technique which ensures security. It acts as a double encryption technique. The same procedure tends to decrypt the encrypted data. ResultsThe efficiency of the proposed technique is assessed employing several parameters such as execution time, encryption time, and decryption time. The experimental results shows that the proposed DNA-HE system reduces encryption time of 10 %, 20 %, and 35 % than OHE, HE and ECC algorithms.

CyVerse: Cyberinfrastructure for open science.

CyVerse, the largest publicly-funded open-source research cyberinfrastructure for life sciences, has played a crucial role in advancing data-driven research since the 2010s. As the technology landscape evolved with the emergence of cloud computing platforms, machine learning and artificial intelligence (AI) applications, CyVerse has enabled access by providing interfaces, Software as a Service (SaaS), and cloud-native Infrastructure as Code (IaC) to leverage new technologies. CyVerse services enable researchers to integrate institutional and private computational resources, custom software, perform analyses, and publish data in accordance with open science principles. Over the past 13 years, CyVerse has registered more than 124,000 verified accounts from 160 countries and was used for over 1,600 peer-reviewed publications. Since 2011, 45,000 students and researchers have been trained to use CyVerse. The platform has been replicated and deployed in three countries outside the US, with additional private deployments on commercial clouds for US government agencies and multinational corporations. In this manuscript, we present a strategic blueprint for creating and managing SaaS cyberinfrastructure and IaC as free and open-source software.

Joint Request Updating and Elastic Resource Provisioning With QoS Guarantee in Clouds

In a commercial cloud, service providers (e.g., video streaming service provider) rent resources from cloud vendors (e.g., Google Cloud Platform) and provide services to cloud users, making a profit from the price gap. Cloud users acquire services by forwarding their requests to corresponding servers. In practice, as a common scenario, traffic dynamics will cause server overload or load-unbalancing. Existing works mainly deal with the problem by two methods: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">elastic resource provisioning</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">request updating</i> . Elastic resource provisioning is a fast and agile solution but may cost too much since service providers need to buy extra resources from cloud vendors. Though request updating is a free solution, it will cause a significant delay, resulting in a bad users’ QoS. In this paper, we present a new scheme, called real-time request updating with elastic resource provisioning (TRUST), to help service providers pay less cost with users’ QoS guarantee in clouds. In addition, we propose an efficient algorithm for TRUST with a bounded approximation factor based on progressive-rounding. Both small-scale experiment results and large-scale simulation results show the superior performance of our proposed algorithm compared with state-of-the-art benchmarks.

Secured VM Deployment in the Cloud: Benchmarking the Enhanced Simulation Model

Cloud computing has gained widespread recognition for facilitating myriad online services and applications. However, the current stages of commercial cloud computing employ a moderate design, wherein computational resources like storage and servers are housed in a few sizable worldwide data centers. System reliability, efficiency, and low latency are all goals of virtual machine (VM) placement. Load balancing has emerged as a crucial challenge for attaining energy efficiency in a fictitious grid computing architecture where a variety of users’ workloads are distributed across several virtual machines. We propose a more effective optimization technique known as the twin fold moth flame algorithm. This algorithm considers multiple constraints, including computation time, stability, and placement cost. The proposed model’s effectiveness will be evaluated based on relocation costs, reaction times, and stability assessments. The most significant gains of the presented work are 4.24%, 9.73%, 11.10%, 28.83%, 7.63%, and 10.62% for 20 count data of nodes for artificial bee colony–bat algorithm, ant colony optimization, crow search algorithm, krill herd, whale optimization genetic algorithm, and improved Lévy-based whale optimization algorithm, respectively.

Utilizing Distributed Heterogeneous Computing with PanDA in ATLAS

Utilizing Distributed Heterogeneous Computing with PanDA in ATLAS

Accelerating science: The usage of commercial clouds in ATLAS Distributed Computing

The ATLAS experiment at CERN is one of the largest scientific machines built to date and will have ever growing computing needs as the Large Hadron Collider collects an increasingly larger volume of data over the next 20 years. ATLAS is conducting R&amp;D projects on Amazon Web Services and Google Cloud as complementary resources for distributed computing, focusing on some of the key features of commercial clouds: lightweight operation, elasticity and availability of multiple chip architectures. The proof of concept phases have concluded with the cloud-native, vendoragnostic integration with the experiment’s data and workload management frameworks. Google Cloud has been used to evaluate elastic batch computing, ramping up ephemeral clusters of up to O(100k) cores to process tasks requiring quick turnaround. Amazon Web Services has been exploited for the successful physics validation of the Athena simulation software on ARM processors. We have also set up an interactive facility for physics analysis allowing endusers to spin up private, on-demand clusters for parallel computing with up to 4 000 cores, or run GPU enabled notebooks and jobs for machine learning applications. The success of the proof of concept phases has led to the extension of the Google Cloud project, where ATLAS will study the total cost of ownership of a production cloud site during 15 months with 10k cores on average, fully integrated with distributed grid computing resources and continue the R&amp;D projects.

CarbonScaler: Leveraging Cloud Workload Elasticity for Optimizing Carbon-Efficiency

Cloud platforms are increasing their emphasis on sustainability and reducing their operational carbon footprint. A common approach for reducing carbon emissions is to exploit the temporal flexibility inherent to many cloud workloads by executing them in periods with the greenest energy and suspending them at other times. Since such suspend-resume approaches can incur long delays in job completion times, we present a new approach that exploits the elasticity of batch workloads in the cloud to optimize their carbon emissions. Our approach is based on the notion of "carbon scaling," similar to cloud autoscaling, where a job dynamically varies its server allocation based on fluctuations in the carbon cost of the grid's energy. We develop a greedy algorithm for minimizing a job's carbon emissions via carbon scaling that is based on the well-known problem of marginal resource allocation. We implement a CarbonScaler prototype in Kubernetes using its autoscaling capabilities and an analytic tool to guide the carbon-efficient deployment of batch applications in the cloud. We then evaluate CarbonScaler using real-world machine learning training and MPI jobs on a commercial cloud platform and show that it can yield i) 51% carbon savings over carbon-agnostic execution; ii) 37% over a state-of-the-art suspend-resume policy; and iii) 8 over the best static scaling policy.

Leveraging OGC API for cloud-based flood modeling campaigns

Cloud technology and services provide a convenient platform for managing large scale watershed modeling studies, such as probabilistic studies or modeling campaigns requiring thousands of simulations. As federal and state agencies undertake large scale projects that leverage commercial cloud platforms for computing and storage of model data, ensuring that the Findable, Accessible, Interoperable, Reusable (FAIR) principles are incorporated early in the design stage is crucial. This paper is the first attempt to explore the feasibility of implementing the OGC API - Processes specification for use as an interoperability layer in an environmental modeling study for performing large scale environmental studies consistent with the FAIR principles. Specifically, an evaluation of the suitability for implementing the OGC API - Processes specification in the commercial cloud for managing the fundamental processes that are involved in the setup, execution, and post-processing of hydraulic modeling in a large-scale watershed study is performed. The experimental results show that the implementation of the OGC API - Processes specification can play a valuable role in providing an interoperable service layer that can be used to request geoprocessing services and chain model development and simulation processes.

Understanding Embodied Emissions

Decarbonizing societal infrastructure, including computing, has emerged as a grand challenge for the 21st century, and thus has seen increasing research attention in recent years. Reducing the carbon emissions of societal infrastructure, whether it be a commercial building or cloud datacenter, generally has two important aspects: reducing operational emissions---the carbon emissions from day-to-day operations---and reducing embodied emissions---the carbon emissions from construction and deployment. These different types of emissions are formally defined for companies in the Greenhouse Gas Protocol (GHG) as (i) Scope 1 emissions, which represent a company's direct emissions, e.g., from directly burning fossil fuels, (ii) Scope 2 emissions, which represent a company's indirect emissions from purchasing and using energy, primarily in the form of electricity, and (iii) Scope 3 emissions, which are indirect emissions that result from all other upstream and downstream activities, such as from purchased goods and services. Collectively, Scopes 1 and 2 capture a company's operational emissions, while Scope 3 captures its embodied emissions.

The 50-year Landsat collection 2 archive

The Landsat global consolidated data archive now exceeds 50 years. In recognition of the need for consistently processed data across the Landsat satellite series, the U.S. Geological Survey (USGS) initiated collection-based processing of the entire archive that was processed as Collection 1 in 2016. In preparation for the data from the now successfully launched Landsat 9, the USGS reprocessed the Landsat archive as Collection 2 in 2020. This paper describes the rationale for, and the contents and advancements provided by Collection 2, and highlights the differences between the Collection 1 and Collection 2 products. Notably, the Collection 2 products have improved geolocation and, for the first time, the USGS provides a global inventory of Level 2 surface reflectance and surface temperature products. Also for the first time, the USGS used a commercial cloud computing architecture to efficiently process the archive and enable direct cloud access of the Landsat products. The paper concludes with discussion of likely improvements expected in Collection 3 in preparation for the Landsat Next mission that is planned for launch in the early 2030s.

Flexible Resource Allocation for Relational Database-as-a-Service

Oversubscription is an essential cost management strategy for cloud database providers, and its importance is magnified by the emerging paradigm of serverless databases. In contrast to general purpose techniques used for oversubscription in hypervisors, operating systems and cluster managers, we develop techniques that leverage our understanding of how DBMSs use resources and how resource allocations impact database performance. Our techniques are designed to flexibly redistribute resources across database tenants at the node and cluster levels with low overhead. We have implemented our techniques in a commercial cloud database service: Azure SQL Database. Experiments using microbenchmarks, industry-standard benchmarks and real-world resource usage traces show that using our approach, it is possible to tightly control the impact on database performance even with a relatively high degree of oversubscription.