Grid Computing Research Articles

Content delivery network solutions for the CMS experiment: The evolution towards HL-LHC

The Large Hadron Collider at CERN in Geneva is poised for a transformative upgrade, preparing to enhance both its accelerator and particle detectors. This strategic initiative is driven by the tenfold increase in proton-proton collisions anticipated for the forthcoming high-luminosity phase scheduled to start by 2029. The vital role played by the underlying computational infrastructure, the World-Wide LHC Computing Grid, in processing the data generated during these collisions underlines the need for its expansion and adaptation to meet the demands of the new accelerator phase. The provision of these computational resources by the worldwide community remains essential, all within a constant budgetary framework. While technological advancements offer some relief for the expected increase, numerous research and development projects are underway. Their aim is to bring future resources to manageable levels and provide cost-effective solutions to effectively handle the expanding volume of generated data. In the quest for optimized data access and resource utilization, the LHC community is actively investigating Content Delivery Network (CDN) techniques. These techniques serve as a mechanism for the cost-effective deployment of lightweight storage systems that support both, traditional and opportunistic compute resources. Furthermore, they aim to enhance the performance of executing tasks by facilitating the efficient reading of input data via caching content near the end user. A comprehensive study is presented to assess the benefits of implementing data cache solutions for the Compact Muon Solenoid (CMS) experiment. This in-depth examination serves as a use-case study specifically conducted for the Spanish compute facilities, playing a crucial role in supporting CMS activities. Data access patterns and popularity studies suggest that user analysis tasks benefit the most from CDN techniques. Consequently, a data cache has been introduced in the region to acquire a deeper understanding of these effects. In this paper, the details of the implementation of a data cache system in the PIC Tier-1 compute facility are presented. It includes insights into the developed monitoring tools and discusses the positive impact on CPU usage for analysis tasks executed in the region. The study is augmented by simulations of data caches, with the objective of discerning the most optimal requirements in both size and network connectivity for a data cache serving the Spanish region. Additionally, the study delves into the cost benefits associated with deploying such a solution in a production environment. Furthermore, it investigates the potential impact of incorporating this solution into other regions of the CMS computing infrastructure.

SAS: Speculative Locality Aware Scheduling for I/O intensive scientific analysis in clouds

The execution of data intensive analysis workflows in a multi-cloud environment, such as the World Large hadron collider Computing Grid (WLCG) at CERN, requires a large amount of input data, which is stored in multiple storage elements. The turnaround time taken by an individual analysis workflow running on an edge machine is mostly affected by the data reading time. Minimizing the data reading time can improve the overall efficiency of the data analysis process. To overcome this problem, we have used Speculative Scheduling to optimize the multi-cloud analysis workflows by intelligently streaming data before a task arrives for execution at the edge machine. We propose an Event System (ES) which is an in-memory Serverless process responsible for proactively providing input data to the workflow processes. It prefetches the data from the storage elements to the memory of the edge machine, which executes the workflow. Using locality aware scheduling and prefetching algorithms, it performs Speculative Scheduling on the basis of the evaluation of historic execution logs using the Bayesian Inference model. The Serverless ES learns about the incoming jobs ahead of time and makes use of intelligent data streaming to supply data to these jobs, thus reducing the overall scheduling and data access latencies and leading to significant improvements in the overall turnaround time. We have evaluated the proposed system using a large analysis workflow from High Energy Physics (HEP) by emulating the WLCG infrastructure in a controlled environment. The results have shown that by using speculative and locality aware scheduling techniques, significant improvements (i.e. over 30%) can be achieved in the execution of data intensive workflows in the cloud environment.

GEOSPATIAL BIG DATA PROCESSING USING THE HIGH-PERFORMANCE COMPUTING TECHNOLOGY

A significant portion of the data used for different purposes includes location information. As a result, there is an increasing effort to find a general framework by utilizing crucial technological drivers. Similarly, as geospatial data becomes more accessible; the possibilities for changing scientific discoveries and methods in modern society are endless. Naturally, many applications depend on the accurate and efficient geospatial data processing. However, using the traditional method to process and compute geospatial data frequently presents several difficulties. The explanation for this is the large volume, heterogeneity, and distributed nature of these data. Consequently, it is imperative to implement contemporary techniques that can manage the computational and analytical demands of this massive amount of geospatial data. Many of these systems—particularly those related to High-Performance Computing (HPC) technology—have been described in the literature. Therefore, the goal of this article is to review the application of HPC technology to big data processing in geospatial applications. The review's findings show that predictive data science tools like parallel and grid computing are available in geographic big data applications, enabling quick and effective processing that will help create a sustainable ecosystem. Also, the efficient management of large data sets requires storage, visualization, analytics, and analysis. Of course, this review demonstrated how recent advancements in computing have impacted geospatial data handling.

Model for Anticipating Frank Failures in Computing Grids

Computing grids are infrastructures that provide almost infinite computing capacities, they are now used in all fields, from the study of pandemics to the monitoring of rocket trajectories and the study of meteorological and climatic phenomena. They have a distributed and heterogeneous architecture that gives them unlimited computing performance. They are made up of several computing nodes that are subject to failures like frank failures. A frank failure in a computing grid is an abnormal and unexpected interruption of a node. Many frank failure tolerance protocols have been proposed in the literature but none of these protocols integrate the anticipation of frank failures. The objective of this article is to propose a model based on the PDEVS formalism of frank failure tolerance in a computing grid that allows the anticipation of frank failures. The proposed model relies on the temperature variation of electronic components and on the state of the hard disk through the values provided by SMART data to predict a probable frank failure of a node. The results of the simulations on the different scenarios that we have carried out show that our results provide better performances than those proposed in the literature when the number of nodes to tolerate is greater than 200.

The role of digital transformation in enhancing clean energy startups' success: An analysis of it integration strategies

This review paper explores the pivotal role of digital transformation in enhancing the success of clean energy startups. It delves into how integrating digital technologies such as IoT, AI, big data, and blockchain can foster innovation, improve scalability, and boost operational efficiency. The paper also examines various IT integration strategies, including cloud computing, data analytics, and smart grid technologies, and their implementation models, highlighting the challenges and potential solutions. Additionally, it discusses the impact of digital transformation on market competitiveness and provides strategic recommendations for startups to leverage these technologies effectively. The paper concludes by predicting future trends in digital technologies and outlining policy implications to support the digital transformation of the clean energy sector.

A Systematic Survey on the Application of Artificial Intelligence (ai) Baseline Networks on Grid Computing Techniques - Challenges, Novelty and Prospects

A smart grid is a contemporary electrical system that supports two-way communication and utilizes the concept of demand response. To increase the smart grid's dependability and enhance the consistency, efficiency, and efficiency of the electrical supply, stability prediction is required. The true test for smart grid system designers and specialists will therefore be the increase of renewable energy. To integrate the electric utility infrastructure into the advanced communication era of today, both in terms of function and architecture, this program has made great strides toward modernizing and expanding it. The study reviews how a smart grid applied different deep learning techniques and how renewable energy can be integrated into a system where grid control is essential for energy management. The article discusses the idea of a smart grid and how reliable it is when renewable energy sources are present. Globally, a change in electric energy is needed to reduce greenhouse gas emissions, prevent global warming, reduce pollution, and boost energy security.

FUZZY MODEL OF ELECTRICITY CONTROL WITH WIRELESS INFORMATION PROCESSED ON GPU

Methods of processing information transmitted through wireless networks with software development are investigated in the work. Innovative methods of data transmission such as optical technologies, quantum data transmission and wireless data transmission technologies are disclosed. It is noted that in the modern understanding, the concept of distributed computing defines the process of convergence (convergence) of distributed processing methods, such as GRID, cloud and fog computing, with the combination of virtual cluster systems (grid clusters, cloud clusters and fog clusters) into a single information communication and computing system . It is emphasized that, unlike cellular modems, ZigBee technology nodes have microcontrollers with a pre-installed operating system and flash memory, which allows solving simple computational tasks in real time before sending data. It is advisable to solve such tasks within the framework of a multi-agent approach, which will increase the efficiency of the use of sensor nodes and the entire sensor network. The advantages of the multi-agent technology of fog computing based on sensor nodes of the wireless network of the ZigBee standard are revealed. The method of multi-agent processing of sensory information and its main components are described. The architecture of the system of distributed sensor data processing is outlined, which includes 4 hardware and software levels: Terminal sensor nodes and controllers of measuring devices and automation devices that implement fuzzy calculations; Coordinators, sensor segment routers and cellular modems that collect, protect and transmit sensor data to the processing center; A data processing center that includes a cluster of servers for GRID calculations and a cloud data storage server; Client devices to access cloud storage, computing cluster servers, and distributed fog computing terminals. It is emphasized that indicators and forecast results can be stored on distributed sensor nodes or transmitted for accumulation in cloud storage for further extraction and intelligent processing in the GRID cluster of the data center.

Green Plant Landscape Design for Urban Air Quality Purification with Computer Image Processing in Cloud, Grid, and Cluster Computing

Green Plant Landscape Design for Urban Air Quality Purification with Computer Image Processing in Cloud, Grid, and Cluster Computing

Operational experience and R&D results using the Google Cloud for High-Energy Physics in the ATLAS experiment

The ATLAS experiment at CERN relies on a Worldwide Distributed Computing Grid infrastructure to support its physics program at the Large Hadron Collider. ATLAS has integrated cloud computing resources to complement its Grid infrastructure and conducted an R&D program on Google Cloud Platform. These initiatives leverage key features of commercial cloud providers: lightweight configuration and operation, elasticity and availability of diverse infrastructures. This paper examines the seamless integration of cloud computing services as a conventional Grid site within the ATLAS workflow management and data management systems, while also offering new setups for interactive, parallel analysis. It underscores pivotal results that enhance the on-site computing model and outlines several R&D projects that have benefited from large-scale, elastic resource provisioning models. Furthermore, this study discusses the impact of cloud-enabled R&D projects in three domains: accelerators and AI/ML, ARM CPUs and columnar data analysis techniques.

Optimizing Resource Allocation in Cloud Computing Using Machine Learning

Purpose: A key component of large-scale distributed computing is the allocation of resources, as computer networks cooperate to address complex optimization problems. To get the most out of computers in general, or throughput, is the goal of resource allocation in this case. When it comes to distributed computing, there are two main varieties: grid computing and cloud computing. In grid computing, many geographically dispersed clusters are linked and made available to the general public. Materials and Methods: We looked at Monte Carlo Tree Search and Long-Short Term Memory and examined how efficient they were. By maintaining consistent traffic patterns, the simulation demonstrated that MCTS performance was improved. However, such a strategy is difficult to implement due to the potential speed with which traffic patterns may alter. A successful service level agreement (SLA) was achieved, and the issue was shown to be fixable using LSTM. We compare the suggested model to different load-balancing algorithms to find the one that best allocates resources. Findings: The results show that compared to the state-of-the-art models, the suggested model achieves an accuracy rate that is 10-15% higher. The suggested model lowers the error percentage rate of the average request blocking likelihood of traffic load by around 9.5-10.2% when compared to the predictions of existing models. Therefore, the proposed method has the potential to enhance network utilization by reducing the amount of time required by memory and the central processing unit. Implications to Theory, Practice and Policy: One advantage of the new method is a more robust forecasting strategy in comparison to earlier models. Using firefly algorithms, future research will construct a cloud data center that employs a variety of heuristics and machine learning methodologies to load balance the energy cloud (Oshawa et al., 2022).

Hybrid Agent-Based Load-Balancing Approach Used in an IaaS Platform

Cloud computing has received a resounding welcome. It was created following methodical and thorough study in web services, distributed computing, utility computing, and virtualization, and it offers several benefits, including lower costs, less space required, and easier management. These advantages bring in a significant number of new users to the cloud platform every day. In addition, because cloud computing is an Internet-based computing paradigm, it must deal with the issue of overwhelming demands through effective load-balancing. A very small number of studies only focus on load-balancing problems in cloud computing platforms, while the majority of load-balancing research is accessible in many domains, including parallel, distributed, and grid computing. Infrastructure as a Service (IaaS), Software as a Service, and Platform as a Service are the three basic categories under which cloud computing falls. For these models, there are notable differences in the load-balancing techniques used. This work compared the outcome with the current method and presented a hybrid agent-based load-balancing approach for the IaaS platform.

Exploring Cloud Security Concerns and Solutions: A Comprehensive Analysis

Cloud computing has emerged as a prominent computing paradigm, offering a range of resources and services accessible over networks such as the internet. It draws upon techniques like grid computing and distributed computing to provide users, both in industry and academia, with virtual resources accessed remotely. However, as cloud computing gains traction, ensuring its security poses significant challenges. Users entrusting their data to the cloud necessitates robust security measures. This paper aims to explore the various security issues inherent in cloud computing, including concerns surrounding multi-tenancy, elasticity, and availability. Moreover, the paper will delve into existing security measures and approaches aimed at enhancing the security of cloud environments. By discussing these techniques and models, the paper aims to provide researchers and professionals with insights into effectively addressing security threats in the cloud

Review of Edge Computing for the Internet of Things (EC-IoT): Techniques, Challenges and Future Directions

The volume of edge data has reached ZB level, placing enormous strain on the core network bandwidth due to the rapid development of contemporary technology and the quick expansion of the number of smart devices accessing wireless networks under IoT. On the other hand, numerous new applications, such as autonomous vehicles, location recognition, augmented and virtual reality, etc., have proposed stricter standards for network latency, jitter, data security, etc. Edge computing (EC) emerged as a response to the shortcomings of traditional cloud computing in these areas. Edge computing is a new computing paradigm that performs computation at the network’s edge and can meet the critical needs of various industries undergoing digital transformation thanks to its ability to provide intelligent interconnection services in close proximity to the edge. It follows in the footsteps of previous computing paradigms such as distributed computing, grid computing, and cloud computing. This article presents a thorough analysis of current findings and accomplishments in the field of edge computing, as well as a prediction of potential directions for its future growth.

SA-Based QoS Aware Workflow Scheduling of Collaborative Tasks in Grid Computing

Scheduling workflow tasks in grid computing is a complex process, especially if it is associated with satisfying the user's requirements to complete tasks within a specified time, with lowest possible cost. This paper presents a proposed Simulated Annealing (SA) based Grid Workflow Tasks Scheduling Approach (SA-GWTSA) that takes into account users’ QoS (quality of service) constraints in terms of cost and time. For a given set of inter-dependent workflow tasks, it generates an optimal schedule, which minimizes the execution time and cost, such that the optimized time is within the time constraints (deadline) imposed by the user. In SA-GWTSA, the workflow tasks, which are modeled as a DAG, are divided into task divisions, each of which consists of a set of sequential tasks. Then, the optimal sub-schedules of all task divisions are computed applying SA algorithm, and used to obtain the execution schedule of the entire workflow. In the proposed algorithm, the sub-schedule of each branch division is represented by a vector, in which each element holds the ID of the service provider chosen from a list of service providers capable of executing the corresponding task in the branch. The algorithm uses a fitness function that is formulated as a multi-objective function of time and cost, which gives users the ability to determine their requirements of time against cost, by changing the weighting coefficients in the objective function. The paper also exhibits the experimental results of assessing the performance of SA-GWTSA with workflows samples of different sizes, compared to different scheduling algorithms: Greedy-Time, Greedy-Cost, and Modified Greedy-Cost.

Embracing Distributed Systems for Efficient Cloud Resource Management: A Review of Techniques and Methodologies

The development of parallel computing, distributed computing, and grid computing has introduced a new computing model, combining elements of grid, public computing, and SaaS. Cloud computing, a key component of this model, assigns computing to distributed computers rather than local computers or remote servers. Research papers from 2017 to 2023 provide an overview of the advancements and challenges in cloud computing and distributed systems, focusing on resource management and the integration of advanced technologies like machine learning, AI-centric strategies, and fuzzy meta-heuristics. These studies aim to improve operational efficiency, scalability, and adaptability in cloud environments, focusing on energy efficiency and cost reduction. However, these advancements also present challenges, such as implementation complexity, adaptability in diverse environments, and the rapid pace of technological advancements. These issues necessitate practical, efficient, and forward-thinking solutions in real-world settings. The research conducted between 2017 and 2023 highlights the dynamic and rapidly evolving field of cloud computing and distributed systems, providing valuable guidance for ongoing and future research. This body of work serves as a crucial reference point for advancing the field and emphasizing the need for practical, efficient, and forward-thinking solutions in the ever-evolving landscape of cloud computing and distributed systems.

Research Paper on Cloud Computing

Abstract: We are in the middle of a powerful era of cloud computing in IT. Rooted in the wide reach of the Internet, this technology provides a robust computing architecture that combines networked hardware, software, and Internet infrastructure. Its advantages go beyond traditional computing methods and go beyond grid computing and others. A key moment in the maturation of cloud computing occurred in 2006 when Amazon introduced the first cloud services. Hong Kong is particularly suitable for the deployment of cloud services due to its extensive daily data processing across various sectors.

GA-based QOS-aware workflow scheduling of deadline tasks in grid computing

Grid computing is the aggregation of the power of heterogeneous, geographically distributed computing resources to provide high-performance computing. To benefit from the grid computing capabilities, effectual scheduling algorithms are primarily essential. This paper presents a GA-based approach, called Grid Workflow Tasks Scheduling Algorithm (GWTSA), for scheduling workflow tasks on grid services based on users’ QoS (quality of service) constraints in terms of cost and time. For a given set of inter-dependent workflow tasks, it generates an optimal schedule, which minimizes the execution time and cost, such that the optimized time be within the time constraints (deadline) imposed by the user. In GWTSA, the workflow tasks are modeled as a DAG, which is divided, then the optimal sub-schedules of all task divisions are computed and used to obtain the execution schedule of the entire workflow. A GA-based technique is employed in GWTSA to compute the optimal execution sub-schedule for each branch division that consists of a set of sequential tasks. In this technique, the chromosome represents a branch division, where each gene holds the id of the service provider chosen to execute the corresponding task in the branch; and the fitness function is formulated as a multi-objective function of time and cost, this gives users the ability to determine their requirements if speed against cost or vice versa, by changing the weighting coefficients in the fitness function. The paper also exhibits the experimental results of assessing the performance of GWTSA with workflow samples of different sizes.

Modifyng modern power systems quality by integrating grid computing technology

The need for reliable, powerful, and clean power generation in power systems is becoming more importance. This need requires geographically-distributed power systems to be integrated as a single entity where among the main features of this integration are large data base and computing intensive. Hence the current power systems are not able to handle this huge datasets required for that integration, Grid Computing is a gateway to virtual storage media and processing power. This paper describes the research work on why grid computing is needed for power systems, what are the major challenges and problem to implement grid computing into power systems, and how grid computing can be utilized to fulfil the requirement for efficient power generation.

Fine-grained grid computing model for Wi-Fi indoor localization in complex environments

The fingerprinting-based approach using the wireless local area network (WLAN) is widely used for indoor localization. However, the construction of the fingerprint database is quite time-consuming. Especially when the position of the access point (AP) or wall changes, updating the fingerprint database in real-time is difficult. An appropriate indoor localization approach, which has a low implementation cost, excellent real-time performance, and high localization accuracy and fully considers complex indoor environment factors, is preferred in location-based services (LBSs) applications. In this paper, we proposed a fine-grained grid computing (FGGC) model to achieve decimeter-level localization accuracy. Reference points (RPs) are generated in the grid by the FGGC model. Then, the received signal strength (RSS) values at each RP are calculated with the attenuation factors, such as the frequency band, three-dimensional propagation distance, and walls in complex environments. As a result, the fingerprint database can be established automatically without manual measurement, and the efficiency and cost that the FGGC model takes for the fingerprint database are superior to previous methods. The proposed indoor localization approach, which estimates the position step by step from the approximate grid location to the fine-grained location, can achieve higher real-time performance and localization accuracy simultaneously. The mean error of the proposed model is 0.36 ​m, far lower than that of previous approaches. Thus, the proposed model is feasible to improve the efficiency and accuracy of Wi-Fi indoor localization. It also shows high-accuracy performance with a fast running speed even under a large-size grid. The results indicate that the proposed method can also be suitable for precise marketing, indoor navigation, and emergency rescue.

Czech national e-infrastructure services for HEP

Czech national e-infrastructure services for HEP