Cloud Resources Research Articles

Distributed Implementation of Computing Language Model in Cloud Computing Network

In this paper, the distributed implementation of Computing Language Model (CLM) in cloud computing network is studied and discussed. With the advent of the era of big data, the application of CLM in natural language processing (NLP), machine translation and other fields is increasingly extensive, and the demand for computing resources is also increasing. As an effective way to manage computing resources, distributed computing can make full use of resources in cloud computing environment and realize efficient execution of computing tasks. In this paper, the application of distributed computing methods such as data parallelism, model parallelism and hybrid parallelism in cloud computing environment is studied and analyzed, and the advantages and disadvantages of different methods in CLM implementation are discussed. The experimental results show that the hybrid parallel method can effectively combine the advantages of data parallelism and model parallelism, and improve the training efficiency and performance of CLM. This study provides important theoretical guidance and technical support for the distributed implementation of CLM in cloud computing networks, and is of great significance for further promoting the development of big data and AI technology.

Reusable tutorials for using cloud-based computing environments for the analysis of bacterial gene expression data from bulk RNA sequencing.

This manuscript describes the development of a resource module that is part of a learning platform named "NIGMS Sandbox for Cloud-based Learning" https://github.com/NIGMS/NIGMS-Sandbox. The overall genesis of the Sandbox is described in the editorial NIGMS Sandbox at the beginning of this Supplement. This module delivers learning materials on RNA sequencing (RNAseq) data analysis in an interactive format that uses appropriate cloud resources for data access and analyses. Biomedical research is increasingly data-driven, and dependent upon data management and analysis methods that facilitate rigorous, robust, and reproducible research. Cloud-based computing resources provide opportunities to broaden the application of bioinformatics and data science in research. Two obstacles for researchers, particularly those at small institutions, are: (i) access to bioinformatics analysis environments tailored to their research; and (ii) training in how to use Cloud-based computing resources. We developed five reusable tutorials for bulk RNAseq data analysis to address these obstacles. Using Jupyter notebooks run on the Google Cloud Platform, the tutorials guide the user through a workflow featuring an RNAseq dataset from a study of prophage altered drug resistance in Mycobacterium chelonae. The first tutorial uses a subset of the data so users can learn analysis steps rapidly, and the second uses the entire dataset. Next, a tutorial demonstrates how to analyze the read count data to generate lists of differentially expressed genes using R/DESeq2. Additional tutorials generate read counts using the Snakemake workflow manager and Nextflow with Google Batch. All tutorials are open-source and can be used as templates for other analysis.

Mathematical Model and Statistical Assessment of Non Linear Scheduling Optimization in Cloud Environment using Improved Cuckoo Search Algorithm

The scheduling optimization subject is indeed one of the most effective solutions in cloud system area to ensure the high systems performance and resource usage. The novelty presented in this study is a mathematical model that draws on an Improved Cuckoo Search Algorithm as a scheduling non-linear optimizer for the cloud computing environment. The model under consideration takes into account the underlying complications as well as the varying nature of the use of cloud resources, primarily addressing the latency issue with priority given to the increase of the throughput. Improved Cuckoo Search (ICS) Algorithm, developed on 'cuckoo search' algorithm, uses adaptive step size and probabilistic switching, which, in turn, ensure that the solution space is both being properly examined and exploited. The applications’ utilization of the scheduling mathematical model is based on an elaborately designed theoretical background representing the task as a non-linear optimization problem that meets all the specific constraints and objectives important in cloud computing. Statistical reassessment involves the comparison of the proposed model with standard benchmarks. The proposed model superiority is evident in terms of convergence speed, solution quality and robustness against diverse workloads, as shown in varied tests. The findings of this research contributes to the field by deriving a specialized tool commonly used by the cloud service provider in designing resource scheduling, enhancing the service delivery and operational efficiency.

From raw seismic data to quantitative interpretation via cloud-based FWI

Full-waveform inversion (FWI) has evolved in recent years from a technique for recovering long- to intermediate-wavelength updates of acoustic overburden velocities as part of a broader model building workflow to a standalone tool for high-fidelity seismic imaging using raw seismic data – with ever greater resolution and ever more sophisticated physics being sought. In doing so, it has moved notably closer to fulfilling the vision of its original inventors in the late 70s and early 80s. In this work, two novel approaches to FWI along with enabling technologies for their effective deployment are considered. The first of these allows for robust long-wavelength updates to be generated from raw seismic data in the absence of diving wave energy, providing reliable estimates of acoustic velocity at depths equivalent to or greater than the longest offsets available in the data – thereby enabling more effective imaging with data of limited quality. The second allows for the estimation of elastic amplitude-versus-angle behaviour while still using an acoustic wave equation, thereby mitigating the significant computational burden typically associated with elastic FWI. Finally, it is demonstrated that these approaches can be deployed in a reliable and cost-effective manner via the considered use of public cloud resources, with learnings relevant for analogous massively parallel scientific applications. Case studies from two marine environments are provided for discussion, one being from deep-water US Gulf of Mexico and the other from intermediate water depths offshore Western Australia.

Review of AI-driven Cloud Optimization

Cloud automation is the key to realization a fully-optimized performance of modern cloud platforms while cloud resources utilization. Resource allocation efficiency is valuable. We are however faced with increasing pressure for computational resources. The Long Short-Term Memory (LSTM) algorithms have found a great use case in the dynamic resource allocation problem when the problem is solved by the proactive provisioning of resources based on historical usage patterns taking advantage of recurrent neural networks. Furthermore, the concern over quality-of-service delivery (QoS) and energy efficiency is now almost as challenging for cloud providers when implementing the utilization of cloud resources, especially in a dynamic setting. Deep Reinforcement Learning (DRL) allows to pursue that end by developing agents, which might guide the work of optimizing resource allocation and reducing energy expenses at the same time. It improves the result and adaptability of the applications running on clouds. Artificial intelligence, in its diverse form, for the instance machine learning and optimization algorithms, brings in a great influence in the areas of cloud operations, resource management, and security. Furthermore, the 3rd generation of FPGAs (Reconfigurable Digital Computing-In-Memory or ReDCIM processors) and the bitwise parallelism via-memory core multipliers also improve the efficiency of computing in cloud environments. Adopting these inputting methodologies is a consequence of cloud systems achieving top performance, high scalability, and low costing. Keywords - Cloud Automation, Long Short-Term Memory (LSTM) algorithms, Deep Reinforcement Learning (DRL), Reconfigurable Digital Computing-In-Memory (ReDCIM) processors, Resource sAllocation, Energy Efficiency.

Predictive optimization of DDoS attack mitigation in distributed systems using machine learning

In recent years, cloud computing has been widely used. This paper proposes an innovative approach to solve complex problems in cloud computing resource scheduling and management using machine learning optimization techniques. Through in-depth study of challenges such as low resource utilization and unbalanced load in the cloud environment, this study proposes a comprehensive solution, including optimization methods such as deep learning and genetic algorithm, to improve system performance and efficiency, and thus bring new breakthroughs and progress in the field of cloud computing resource management.Rational allocation of resources plays a crucial role in cloud computing. In the resource allocation of cloud computing, the cloud computing center has limited cloud resources, and users arrive in sequence. Each user requests the cloud computing center to use a certain number of cloud resources at a specific time.

Application of machine learning optimization in cloud computing resource scheduling and management

In recent years, cloud computing has been widely used. Cloud computing refers to the centralized computing resources, users through the access to the centralized resources to complete the calculation, the cloud computing center will return the results of the program processing to the user. Cloud computing is not only for individual users, but also for enterprise users. By purchasing a cloud server, users do not have to buy a large number of computers, saving computing costs. According to a report by China Economic News Network, the scale of cloud computing in China has reached 209.1 billion yuan.Rational allocation of resources plays a crucial role in cloud computing. In the resource allocation of cloud computing, the cloud computing center has limited cloud resources, and users arrive in sequence. Each user requests the cloud computing center to use a certain number of cloud resources at a specific time.

MCMPI: A library with elasticity for multi‐domain and public cloud environments

SummaryThis article introduces a new library that leverages and extends the MPI (message passing interface) standard, capable of aggregating servers and clusters located in multiple domains, as well as cloud resources. The execution platform is created and presented to the MPI application transparently, without the need of recompiling any code. The solution also provides functions for provisioning, adding, and removing nodes at runtime, bringing elasticity to the application. Through benchmarks, its performance was compared with the native execution of the application using the native MPI library. A prototype of an elastic application was also developed, with positive results within the expected range.

Optimizing Cloud Computing Resources: An Energy Efficient Multi-QoS Factor-Based VM Placement Strategy

Cloud computing is experiencing unprecedented demand, offering scalable and flexible resources for a wide range of applications. However, this surge in demand has raised concerns about energy consumption and the need for environmentally sustainable solutions. Green computing has emerged as a critical consideration in this context. Virtual Machine Placement (VMP) is a key component of optimizing cloud resources, aiming to allocate virtual machines efficiently while minimizing energy consumption, cost, and load balancing. This paper addresses the VMP problem by introducing a novel approach based on multifactor optimization, specifically the Diversity Constraint Digger Snake Optimizer (DCD-SO). It offers an innovative perspective on optimizing virtual machine placement by considering energy efficiency, load balancing, and resource utilization simultaneously with the aim to reduce VM migration count, time and cost. Proposed method provides a more comprehensive and sustainable solution, aligning with the principles of green computing. Through extensive simulations and experiments, we have rigorously evaluated the performance of DCD-SO in comparison to traditional optimization techniques such as Particle Swarm Optimization (PSO) and Snake Optimization. In our analysis of actual cloud environments, we compared the results of our method with existing state-of-the-art techniques. Result outcomes determine showed that proposed approach has reduced migration count of 5 and 3 for scheduling 42VMs and 84VMs on 16 and 32 host units respectively than traditional MOGANS, GA-S, GA-N and GA-NN methods. This comprehensive evaluation reinforces the effectiveness and practicality of our approach in addressing the intricate challenges of Virtual Machine Placement (VMP) in dynamic cloud computing settings. As cloud computing continues to evolve, our study contributes to more sustainable and efficient resource management, addressing both current demands and future needs.

PMRNA: Parameter matching of realtime and non‐realtime applications for resource provisioning in fog‐integrated cloud

SummaryFog computing, an emerging technology, extends Cloud computing services to the network's edge in the proximity of the application request. This extension yields improvement in Bandwidth (BW) utilization, faster responses to Real‐Time (RT) and Internet of Things (IoT) requests, and the provision of the heterogeneous resource services. While extensive work has been conducted on resource allocation for RT and Non‐Real‐Time (NRT) requests separately in Fog as well as Cloud computing, there is limited focus on resource provisioning for mixed RT and NRT requests in the Fog‐integrated Cloud (FiC) environment. Moreover, the majority of the existing provisioning methods primarily consider parameters from the system's perspective, overlooking crucial user aspects such as deadline and request size. To address the gap, this work introduces a resource provisioning method named “Parameter Matching of Realtime and Non‐Realtime Applications (PMRNA),” which considers user parameters and resource information, gathered by the broker. The performance evaluation of the proposed model is done in CloudSim, using various combinations of RT and NRT requests along with diverse Fog and Cloud resource configurations. Evaluation metric includes average Execution Time (ET), average Waiting Time (WT), average Turn Around Time (TAT), and resource utilization. The experimental results demonstrate a significant reduction in both average WT and average TAT for the diverse pool of RT and NRT requests in the FiC compared to the Cloud‐only environment.

An Efficient Cloud-Powered Bidding Marketplace

Cloud Bid: Bidding for Basics - An Efficient Cloud-Powered Bidding Marketplace" presents an innovative approach to online commerce by merging cloud computing and traditional bidding systems. This study explores the potential of “Cloud Bids” to revolutionize online auctions and negotiations. Harnessing cloud resources and cutting-edge algorithms to offer real-time scalability, efficiency, and data-driven decision-making. With applications across e-commerce, supply chain management, and public procurement, Cloud Bids promises increased transparency, cost- effectiveness, and strategic insights. Addressing security and privacy concerns, this research underscores the significance of understanding Cloud Bids in the evolving digital marketplace, marking a transformative shift in the way goods and services are traded online to get a full overview of the status of the products. This real time bidding application will ensure both the selling and buying seamless and hasslefree for the user. It’s time to explore the cloud based bidding marketplace to ensure your new way of shopping for your daily necessary stuff.

BLOOM : CLOUD OPTIMIZATION TOOL

Bloom: Cloud Optimization tool" endeavors to harness the power of the Ant Colony Optimization (ACO) algorithm to address the challenge of organizing unstructured input data in cloud storage systems. The primary objective is to optimize cloud storage utilization and minimize associated billing expenses. Through the utilization of ACO, the solution aims to transform unorganized and unstructured data into a structured format, facilitating efficient storage allocation and enhancing data management practices. By leveraging predictive insights derived from the structured data, the solution empowers both cloud service providers and users to make informed decisions regarding data storage optimization. Ultimately, this approach not only streamlines data organization processes but also aids in maximizing cost-effectiveness by ensuring that cloud resources are allocated optimally based on usage patterns and predicted future demands. The integration of ACO into the solution offers a promising avenue for addressing the complexities inherent in managing vast amounts of unstructured data in cloud environments, thereby paving the way for improved efficiency and cost savings.

An offloading and pricing mechanism based on virtualization in edge–cloud computing

Integrate users, edge servers and cloud into a system can improve resource utilization and quality of service (QoS). The users can send the same request to multiple Edge Servers. However, edge servers have limited scope, which may result in repeated offloading tasks for the same user. In this paper, we consider the cloud resource migration in the form of virtual machines (VMs) to achieve high computing efficiency and low latency requirements. Obviously, the migration of VMs causes changes in the number and price of edge resources, which further affects the task offloading strategy. Therefore, it is necessary to optimize the VMs migration strategy and resource pricing strategy successively to achieve the optimal offloading strategy suitable for users. First, based on the competition between edge servers, a static VMs migration algorithm (SVMMA) is designed to formulate the strategy and price of VMs migration. Then, according to the users’ resource needs and the resource possession of different edge servers, a resource partitioning approach (RPA) have been proposed. Finally, two different QoS functions are evaluated, representing the QoS matching degree between tasks and multiple resource combination blocks (MRCBs). Wherein, a dynamic matching and pricing of tasks algorithm (DMPT) is conducted to obtain the optimal offloading strategy and price of the task. Experiments show that our algorithms can improve the offloading rate of tasks and achieve the relative balance among user, cloud server, and edge server benefits.

FinOps-driven optimization of cloud resource usage for high-performance computing using machine learning

Cloud computing is gaining popularity in high-performance computing applications. Its utilization enables advanced simulations when local computing resources are limited. However, cloud usage may increase costs and entail resource unavailability risks. This article presents an original approach that employs machine learning to predict long-term cloud resource usage. This enables optimizing resource utilization through appropriate reservation plans, reducing the associated costs. The solution developed utilizes statistical models, XGBoost, neural networks and the Temporal Fusion Transformer. Long-term prediction of cloud resource consumption, especially the Cloud Resource Usage Optimization System that is critical for prolonged simulations, involves using prediction results to dynamically create resource reservation plans across various virtual machine types for HPC on the Google Cloud Platform. Experiments using real-life production data demonstrate that the TFT prediction model improved prediction quality (by 31.4%) compared to the best baseline method, particularly in adapting to chaotic changes in resource consumption. However, it should be noted that the best prediction model in terms of error magnitude might not be the most suitable for resource reservation planning. This was validated by the neural network-based method, introducing an FR metric for forecast evaluation. Resource reservation plans were assessed both qualitatively and quantitatively, focusing on various aspects like a service-level agreement compliance and potential downtime. This paper is an extension of work originally presented during the International Conference on Computational Science — ICCS 2023, entitled “Long-Term Prediction of Cloud Resource Usage in High-Performance Computing”.

Cloud-based urgent computing for forest fire spread prediction

Forest fires cause every year damages to biodiversity, atmosphere, and economy activities. Forest fire simulation have improved significantly, but input data describing fire scenarios are subject to high levels of uncertainty. In this work the two-stage prediction scheme is used to adjust unknown parameters. This scheme relies on an input data calibration phase, which is carried over following a genetic algorithm strategy. The calibrated inputs are then pipelined into the actual prediction phase. This two-stage prediction scheme is leveraged by the cloud computing paradigm, which enables high level of parallelism on demand, elasticity, scalability and low-cost. In this paper, all the models designed to properly allocate cloud resources to the two-stage scheme in a performance-efficient and cost-effective way are described. This Cloud-based Urgent Computing (CuCo) architecture has been tested using, as study case, an extreme wildland fire that took place in California in 2018 (Camp Fire).

Signature-based Adaptive Cloud Resource Usage Prediction Using Machine Learning and Anomaly Detection

One of the challenges in managing cloud computing clusters is assigning resources based on the customers’ needs. For this mechanism to work efficiently, it is imperative that there are sufficient resources reserved to maintain continuous operation, but not too much to avoid overhead costs. Additionally, to avoid the overhead of acquisition time, it is important to reserve resources sufficiently in advance. This paper presents a novel reliable general-purpose mechanism for prediction-based resource usage reservation. The proposed solution should be capable of operating for long periods of time without drift-related problems, and dynamically adapt to changes in system usage. To achieve this, a novel signature-based ensemble prediction method is presented, which utilizes multiple distinct prediction algorithms suited for various use-cases, as well as an anomaly detection mechanism used to improve prediction accuracy. This ensures that the mechanism can operate efficiently in different real-life scenarios. Thanks to a novel signature-based selection algorithm, it is possible to use the best available prediction algorithm for each use-case, even over long periods of time, which would typically lead to drifts. The proposed approach has been evaluated using real-life historical data from various production servers, which include traces from more than 1,500 machines collected over more than a year. Experimental results have demonstrated an increase in prediction accuracy of up to 21.4 percent over the neural network approach. The evaluation of the proposed approach highlights the importance of choosing the appropriate prediction method, especially in diverse scenarios where the load changes frequently.

Key Challenges of Cloud Computing Resource Allocation in Small and Medium Enterprises

Although cloud computing offers many benefits, such as flexibility, scalability, and profitability, some small and medium enterprises (SMEs) are still unable to fully utilize cloud resources, such as memory, computing power, storage, and network bandwidth. This reduces their productivity and increases their expenses. Therefore, the central objective of this paper was to examine the key challenges related to the allocation of cloud computing resources in small and medium enterprises. The method used for this study is based upon qualitative research using 12 interviews with 12 owners, managers, and experts in cloud computing in four countries: the United States of America, the United Kingdom, India, and Pakistan. Our results, based on our empirical data, show 11 key barriers to resource allocation in cloud computing that are classified based on the technology, organization, and environment (TOE) framework. Theoretically, this research contributes to the body of knowledge concerning cloud computing technology and offers valuable understanding of the cloud computing resource allocation approaches employed by small and medium enterprises (SMEs). In practice, this research is useful to aid SMEs in implementing successful and sustainable strategies for allocating cloud computing resources.

Presenting a meta-heuristic solution for optimal resource allocation in fog computing

Given that cloud computing is a relatively new field of study, there is an urgent need for comprehensive approaches to resource provisioning and the allocation of Internet of Things (IoT) services across cloud infrastructure. Other challenging aspects of cloud computing include IoT resource virtualization and disseminating IoT services among available cloud resources. To meet deadlines, optimize application execution times, efficiently use cloud resources, and identify the optimal service location, service placement plays a crucial role in installing services on existing virtual resources within a cloud-based environment. To achieve load balance in the fog computing infrastructure and ensure optimal resource allocation, this work proposes a meta-heuristic approach based on the cat swarm optimization method. For more clarity in the difference between the work presented in this research and other similar works, we named the proposed technique MH-CSO. The algorithm incorporates a resource check parameter to determine the accessibility and suitability of resources in different situations. This conclusion was drawn after evaluating the proposed solution in the ifogsim environment and comparing it with particle swarm and ant colony optimization techniques. The findings demonstrate that the proposed solution successfully optimizes key parameters, including runtime and energy usage.

Dynamic resource allocation for virtual machine migration optimization using machine learning

This article delves into the importance of applying machine learning and deep reinforcement learning techniques in cloud resource management and virtual machine migration optimization, highlighting the role of these advanced technologies in dealing with the dynamic changes and complexities of cloud computing environments. Through environment modeling, policy learning, and adaptive enhancement, machine learning methods, especially deep reinforcement learning, provide effective solutions for dynamic resource allocation and virtual intelligence migration. These technologies can help cloud service providers improve resource utilization, reduce energy consumption, and improve service reliability and performance. Effective strategies include simplifying state space and action space, reward shaping, model lightweight and acceleration, and accelerating the learning process through transfer learning and meta-learning techniques. With the continuous progress of machine learning and deep reinforcement learning technologies, combined with the rapid development of cloud computing technology, it is expected that the application of these technologies in cloud resource management and virtual machine migration optimization will be more extensive and in-depth. Researchers will continue to explore more efficient algorithms and models to further improve the accuracy and efficiency of decision making. In addition, with the integration of edge computing, Internet of Things and other technologies, cloud computing resource management will face more new challenges and opportunities, and the application scope and depth of machine learning and deep reinforcement learning technology will also expand, opening new possibilities for building a more intelligent, efficient and reliable cloud computing service system.

Multi-objective comprehensive container scheduling and resource allocation for container cloud using tuna swarm optimization algorithm

The paper addresses the optimization challenges in cloud resource task execution within the container paradigm, introducing the Multi-Objective Comprehensive Container Scheduling and Resource Allocation (MOCCSRA) scheme. It aims to enhance cost-effectiveness and efficiency by utilizing the Tuna Swarm Optimization (TSO) technique to optimize task planning and resource allocation. This novel approach considers various objectives for task scheduling optimization, including energy efficiency, compliance with service level agreements (SLAs), and quality of service (QoS) metrics like CPU utilization, memory usage, data transmission time, container-VM correlation, and container grouping. Resource allocation decisions are guided by the VM cost and task completion period factors. MOCCSRA distinguishes itself by tackling the multi-objective optimization challenge for task scheduling and resource allocation, producing non-dominated Pareto-optimal solutions. It effectively identifies optimal tasks and matches them with the most suitable VMs for deploying containers, thereby streamlining the overall task execution process. Through comprehensive simulations, the results demonstrate MOCCSRA’s superiority over traditional container scheduling methods, showcasing reductions in resource imbalance and notable enhancements in response times. This research introduces an innovative and practical solution that notably advances the optimization field for cloud-based container systems, meeting the increasing demand for efficient resource utilization and enhanced performance in cloud computing environments.