Cluster Throughput Research Articles

FRCR: Raft Consensus Scheme Based on Semi Asynchronous Federal Reconstruction

Today, Raft’s distributed cluster scale is growing rapidly and cluster throughput is declining. The Raft consensus algorithm needs to be continuously optimized to adapt to a complex and changeable application environment. To solve the above problems, we propose a federal reconstruction Committee Raft consensus algorithm FRCR. Based on the Federation reconstruction technology, the algorithm trains, updates and evaluates the model of the characteristic data set of the Raft node, runs the model to get the nodes with better performance, constructs the committee mechanism, and improves the quality and speed of the election. We also design a semi asynchronous buffer mechanism and a strategy to resist malicious node attacks to solve the inconsistency and security problems of federation aggregation. Finally, seven aspects of FRCR are tested and analyzed to verify the effectiveness of FRCR in the consensus cluster.

Optimizing Speculative Execution in Spark Heterogeneous Environments

The execution time of a stage is extended by a few slow running tasks in Spark computing environments. To tackle this so-called straggler problem, Spark adopts speculative execution mechanism under which the scheduler speculatively launch additional backup for the straggler with the hope to complete early. However, due to the characteristics of tasks and the complexity of runtime environments, the Spark original speculative execution strategy and its improved versions cannot deal with this problem effectively. In this paper, we propose a novel strategy called ETWR to improve the efficiency of speculative execution in Spark. We consider the heterogeneous environment when we around to tackle the three key points of speculative execution: straggler identification, backup node selection and effectiveness guarantee. Based on the task type classification, first, we divide the task into sub-phases and use both the process speed and progress rate within a phase to find the straggler promptly. Second, we use the Locally Weighted Regression model to estimate the execution time of the task, which will be used to calculate the task's remaining time and backup time. Third, we present iMCP model to guarantee the effectiveness of speculative tasks, which can additionally keep load balancing for nodes. Finally, the factors of fast node and better location are considered when choosing proper backup nodes. Extensive experiments show that ETWR can reduce the job execution time by 23.8 percent, and improve the cluster throughput by 33.2 percent compared with Spark-2.2.0.

Impact of the gas dynamics on the cluster flux in a magnetron cluster-source: Influence of the chamber shape and gas-inlet position.

Although producing clusters by physical methods offers many benefits, low deposition rates have prevented cluster-beam deposition techniques from being adopted more widely. The influence of the gas aerodynamics inside the condensation chamber of a magnetron cluster-source on the cluster throughput is reported, leading to an improved understanding of the influence of gas aerodynamics on cluster transport. In the first part of this paper, the influence of the carrier gas's inlet position on the cluster flux is studied. In particular, two inlet configurations were investigated, i.e., from the rear of the chamber and from within the magnetron sputtering source. It was found experimentally that the latter configuration can lead to an increased cluster flux, under the same conditions of gas pressure and power applied to the magnetron. This behavior is explained with the help of simulations. In the second part of this paper, the gas dynamics behavior inside four chamber shapes, namely, two cylindrical shapes with different cross-sectional diameters and two conical shapes with different apex angles, was simulated. The modeling showed that the fraction of clusters successfully leaving the aggregation zone can be increased by up to eight times from the worst to the best performing chamber geometries studied. Finally, the cluster throughput was determined experimentally using a quartz microbalance in two of the four chamber designs. It was found that the cluster flux increased up to one order of magnitude, reaching ∼20 mg/h for a condensation chamber with a smaller cross section and a conical exit.

Clustering Optimization of LoRa Networks for Perturbed Ultra-Dense IoT Networks

Long Range (LoRa) communication is widely adapted in long-range Internet of Things (IoT) applications. LoRa is one of the powerful technologies of Low Power Wide Area Networking (LPWAN) standards designed for IoT applications. Enormous IoT applications lead to massive traffic results, which affect the entire network’s operation by decreasing the quality of service (QoS) and minimizing the throughput and capacity of the LoRa network. To this end, this paper proposes a novel cluster throughput model of the throughput distribution function in a cluster to estimate the expected value of the throughput capacity. This paper develops two main clustering algorithms using dense LoRa-based IoT networks that allow clustering of end devices according to the criterion of maximum served traffic. The algorithms are built based on two-common methods, K-means and FOREL. In contrast to existing methods, the developed method provides the maximum value of served traffic in a cluster. Results reveal that our proposed cluster throughput model obtained a higher average throughput value by using a normal distribution than a uniform distribution.

Cluster load based content distribution and speculative execution for geographically distributed cloud environment

The scale of big data has shown an explosive growth, which makes the processing of big data put forward higher requirements on data centers, and a single data center can no longer meet the needs of big data processing. To deal with this situation, a geographically distributed cloud system needs to be built. However, in the geographically distributed cloud system, each data center is distributed in different geographic locations, which makes the data placement operations in the geographically distributed cloud system lead to greater overhead. To solve this problem, this paper proposes a data placement strategy. This strategy comprehensively considers the data transmission latency, bandwidth cost, cloud server storage capacity, and load capacity during the data placement process, and formulates a data placement problem that minimizes the energy consumption of data transmission. Then the minimum set cover method based on Lagrangian relaxation is used to solve this problem and obtain the optimal data placement scheme. On the other hand, in a geographically distributed cloud data center, the execution progress of the job submitted by the user will be affected by the straggler task. To solve this problem, this paper proposes a speculative execution strategy for the geographically distributed cloud system. This strategy performs different speculative execution operations according to the state of the cluster load, and then calculates the load capacity of the nodes in the cluster. The node with the strongest load capacity in the cluster is used to perform speculative execution operations. Experimental results show that the proposed data placement strategy can effectively improve the performance of the energy consumption, the data storage cost, the network transmission cost and the data transmission time. The proposed speculative execution strategy can effectively improve the performance of the job completion time, cluster throughput and QoS satisfaction rate.

Deep Reinforcement Learning for Throughput Improvement of the Uplink Grant-Free NOMA System

Facing the dramatic increase of mobile devices and the scarcity of spectrum resources, grant-free nonorthogonal multiple access (NOMA) emerges as an enabling technology for massive access, which also reduces signaling overhead and access latency effectively. However, in grant-free NOMA systems, the collisions resulting from uncoordinated resource selection can cause severe interference and reduce system throughput. In this article, we apply deep reinforcement learning (DRL) in the decision making for grant-free NOMA systems, to mitigate collisions and improve the system throughput in an unknown network environment. To reduce collisions in the frequency domain and the computational complexity of DRL, subchannel and device clustering are first designed, where a cluster of devices compete for a cluster of subchannels following grant-free NOMA. Furthermore, discrete uplink power control is proposed to reduce intracluster collisions. Then, the long-term cluster throughput maximization problem is formulated as a partially observable Markov decision process (POMDP). To address the POMDP, a DRL-based grant-free NOMA algorithm is proposed to learn about the network contention status and output subchannel and received power-level selection with less collisions. The numerical results verify the effectiveness of the proposed algorithm and reveal that DRL-based grant-free NOMA outperforms slotted ALOHA NOMA with 32.9% and 156% performance gain on the system throughput when the number of devices is twice and five times that of the subchannels, respectively. When the number of devices is five times that of the subchannels, the success access probability of DRL-based grant-free NOMA is above 85%, compared to 33% in the slotted ALOHA NOMA system.

Performance drop at executing communication-intensive parallel algorithms

This work summarizes the results of a set of executions completed on three fat-tree network supercomputers: Stampede at TACC (USA), Helios at IFERC (Japan) and Eagle at PSNC (Poland). Three MPI-based, communication-intensive scientific applications compiled for CPUs have been executed under weak-scaling tests: the molecular dynamics solver LAMMPS; the finite element-based mini-kernel miniFE of NERSC (USA); and the three-dimensional fast Fourier transform mini-kernel bigFFT of LLNL (USA). The design of the experiments focuses on the sensitivity of the applications to rather different patterns of task location, to assess the impact on the cluster performance. The accomplished weak-scaling tests stress the effect of the MPI-based application mappings (concentrated vs. distributed patterns of MPI tasks over the nodes) on the cluster. Results reveal that highly distributed task patterns may imply a much larger execution time in scale, when several hundreds or thousands of MPI tasks are involved in the experiments. Such a characterization serves users to carry out further, more efficient executions. Also researchers may use these experiments to improve their scalability simulators. In addition, these results are useful from the clusters administration standpoint since tasks mapping has an impact on the cluster throughput.

How relative mobility affects the aggregate throughput of VANET cluster?

Clustering is considered an important issue in Vehicular Ad‐hoc Networks (VANETs) to ensure the network robustness and throughput. Nevertheless, in the literature, there is still a lack of understanding of how relative mobility between cluster members and the cluster head affects the throughput of VANET clusters. In this letter, we present a model to characterize the throughput of VANET clusters, by taking into account the relative mobility effect, which implicitly accounts for the Doppler effect. The numerical analysis shows that there is a large gap between the throughput of the model with/without consideration of relative mobility, which indicates the necessity to develop clustering algorithms that minimize intra‐cluster mobility.

Optimizing the parameters of the Lustre-file-system-based HPC system for reverse time migration

The article is dedicated to optimizing the computing cluster performance and throughput of the storage with the Lustre file system to perform the reverse time migration task. Optimization is needed because increasing the number of processor cores at first helps to speed up the calculation process, but at some point the calculations start to take more time despite adding more cores with the same storage throughput. This behaviour is caused by the fact that performance gain achieved through parallel operation is offset by increasing delays in the storage that grow nonlinearly as the load increases. To take into account the particularities of this case study, the specific task complexity notion that defines the number of floating point operations per input/output byte was used. This notion and the queuing theory fundamentals were applied to produce a simple formula that links the Lustre file system storage throughput, quantity of storage nodes and the cluster performance in the optimal configuration. LINPACK and SPC-2 benchmark results were used as initial data.

Statistical Analysis of the Development of the Port Cluster Economy in Beijing, Tianjin and Hebei

Li, Z.-W., 2019. Statistical analysis of the development of the port cluster economy in Beijing, Tianjin and Hebei. In: Gong, D.; Zhu, H., and Liu, R. (eds.), Selected Topics in Coastal Research: Engineering, Industry, Economy, and Sustainable Development. Journal of Coastal Research, Special Issue No. 94, pp. 692–695. Coconut Creek (Florida), ISSN 0749-0208.This paper aims to explore the development strategies of Beijing-Tianjin-Hebei port cluster. Based on the data of Beijing-Tianjin-Hebei port cluster in 1980∼2014, a VAR model was established with such four variables as total handling capacity, import and export, GDP and the employment. Then, the economic relationship between the port cluster and the hinterland was analysed by impulse response, and the four variables were subjected to the Grainger causality test. The results show that the total handling capacity of the port cluster has a positive effect on the import and export, GDP, and employment, but the increase in GDP is not the Granger-cause of the increase in the port cluster throughput. The GDP growth only has a short-term driving effect on the port cluster throughput. The long-term growth of port throughput largely depends on international economy. What is worse, the positive effect of port throughput on hinterland economy is rather limited. The research findings and suggestions provide meaningful reference for the research on port cluster economy.

Improving the management efficiency of GPU workloads in data centers through GPU virtualization

SummaryGraphics processing units (GPUs) are currently used in data centers to reduce the execution time of compute‐intensive applications. However, the use of GPUs presents several side effects, such as increased acquisition costs and larger space requirements. Furthermore, GPUs require a nonnegligible amount of energy even while idle. Additionally, GPU utilization is usually low for most applications. In a similar way to the use of virtual machines, using virtual GPUs may address the concerns associated with the use of these devices. In this regard, the remote GPU virtualization mechanism could be leveraged to share the GPUs present in the computing facility among the nodes of the cluster. This would increase overall GPU utilization, thus reducing the negative impact of the increased costs mentioned before. Reducing the amount of GPUs installed in the cluster could also be possible. However, in the same way as job schedulers map GPU resources to applications, virtual GPUs should also be scheduled before job execution. Nevertheless, current job schedulers are not able to deal with virtual GPUs. In this paper, we analyze the performance attained by a cluster using the remote Compute Unified Device Architecture middleware and a modified version of the Slurm scheduler, which is now able to assign remote GPUs to jobs. Results show that cluster throughput, measured as jobs completed per time unit, is doubled at the same time that the total energy consumption is reduced up to 40%. GPU utilization is also increased.

Hybrid Cloud Adaptive Scheduling Strategy for Heterogeneous Workloads

With the advent of the era of big data, many companies have taken the most important steps in the hybrid cloud to handle large amounts of data. In a hybrid cloud environment, cloud burst technology enables applications to be processed at a lower cost in a private cloud and burst into the public cloud when the resources of the private cloud are exhausted. However, there are many challenges in hybrid cloud environment, such as the heterogeneous jobs, different cloud providers and how to deploy a new application with minimum monetary cost. In this paper, the efficient job scheduling approach for heterogeneous workloads in private cloud is proposed to ensure high resource utilization. Moreover, the task scheduling method based on BP neural network in hybrid cloud is proposed to ensure that the tasks can be completed within the specified deadline of the user. The experimental results show that the efficient job scheduling approach can veffectively reduce the job response time and improve the throughput of cluster. The task scheduling method can reduce the response time of tasks, improve QoS satisfaction rate and minimize the cost of public cloud.

Dependency-Aware Network Adaptive Scheduling of Data-Intensive Parallel Jobs

Datacenter clusters often run data-intensive jobs in parallel for improving resource utilization and cost efficiency. The performance of parallel jobs is often constrained by the cluster's hard-to-scale network bisection bandwidth. Various solutions have been proposed to address the issue, however, most of them do not consider inter-job data dependencies and schedule jobs independently from one another. In this work, we find that aggregating and co-locating the data and tasks of dependent jobs offer an extra opportunity for data locality improvement that can help to greatly enhance the performance of jobs. We propose and design Dawn, a dependency-aware network-adaptive scheduler that includes an online plan and an adaptive task scheduler. The online plan, taking job dependencies into consideration, determines where (i.e., preferred racks) to place tasks in order to proactively aggregate dependent data. The task scheduler, based on the output of online plan and dynamic network status, adaptively schedules tasks to co-locate with the dependent data in order to take advantage of data locality. We implement Dawn on Apache Yarn and evaluate it on physical and virtual clusters using various machine learning and query workloads. Results show that Dawn effectively improves cluster throughput by up to 73 and 38 percent compared to Fair Scheduler and ShuffleWatcher, respectively. Dawn not only significantly enhances the performance of jobs with dependency, but also works well for jobs without dependency.

Effect of MPI tasks location on cluster throughput using NAS

In this work the Numerical Aerodynamic Simulation (NAS) benchmarks have been executed in a systematic way on two clusters of rather different architectures and CPUs, to identify dependencies between MPI tasks mapping and the speedup or resource occupation. To this respect, series of experiments with the NAS kernels have been designed to take into account the context complexity when running scientific applications on HPC environments (CPU, I/O or memory-bound, execution time, degree of parallelism, dedicated computational resources, strong- and weak-scaling behaviour, to cite some). This context includes scheduling decisions, which have a great influence on the performance of the applications, making difficult to achieve an optimal exploitation with cost-effective strategies of the HPC resources. An analysis on how task grouping strategies under various cluster setups drive the execution time of jobs and the infrastructure throughput is provided. As a result, criteria for cluster setup arise linked to maximize performance of individual jobs, total cluster throughput or achieving better scheduling. To this respect, a criterion for execution decisions is suggested. This work is expected to be of interest on the design of scheduling policies and useful to HPC administrators.

Analysis of energy efficiency in cloud based heterogeneous RAN with large-scale antenna systems

To address the capacity requirements resulting from huge growth in mobile data traffic, the mobile network operators (MNOs) are densifying their networks with more base stations, and with more spectrum layers. The addition of base stations and spectrum layers increases the energy consumption of the access network. Hence the network needs to be optimized to lower the power consumption (e.g.) through cloud, or deployment of spectral and power efficient radio units like Large-Scale Antenna Systems (LSAS) or massive Multiple Input Multiple Output (MIMO) radio units.In this paper, we analyse the network evolution towards Cloud based Radio Access Network (C-RAN) for a mix of base stations with Macro and LSAS Remote Radio Units (RRU). We derive the computational complexity of the base station components using a flexible power model. We evaluate the need for base station architecture split, when C-RAN is used with LSAS radio units and large antenna configurations. We use a combination of simulators viz.NS3 (for LTE radio network simulation) and CloudSIM Plus (for Cloud based base station simulation). For each base station type, we compare the computational complexity of different sub-components. Further, based on the selected power model, we study the effect of base station's scheduling algorithm (viz.) Proportionate-Fair or Round-Robin, on the computational complexity of the various base station sub-components. We consider Time and Wavelength Division Multiplexed Passive Optical Network (TWDM-PON) for the high capacity low latency fronthaul network. Using the overall radio cluster throughput and power consumption information from Cloud base station, Radio site, and fronthaul network, we assess the energy efficiency metrics for the Cloud RAN and LSAS (with different antenna configurations) with Macro RRUs as reference.Through the simulations, we observe that, the computation complexity was relatively higher for RR scheduling (compared to PF scheduling). But the variations of energy efficiency metric were similar for both the scheduling schemes for different base station configurations.We observe that compared to Macro2TR, LSAS variants are highly energy efficient. This gain is mainly associated with higher throughput due to MU-MIMO (where multiple UEs are accommodated in the same set of radio resources) and lower power consuming components in LSAS. Between LSAS32TR, LSAS64TR and LSAS128TR, we find that LSAS32TR had lower energy efficiency. Further, we observe that the energy efficiency of LSAS64TR is almost same as LSAS128TR, though LSAS64TR supports just half the number of MU-MIMO layers. This implies that the additional number of layers that LSAS128TR system supports, is compensated by power consumed to support the higher antenna chains and spatially multiplexed layers.Thus, MNOs interested in deploying high spectral efficiency solution along with high energy efficiency, may prefer to deploy LSAS64 systems for tower-top deployments of Radio unit deployment, since it offers higher energy efficiency with lower wind-load effect due to smaller antenna size.

Investigation of the influence of the number of modules of a multicluster technological complex and the operations performed by them on the overall performance of the installation using simulation simulations

Variety of compositions of multicluster tools determines the development of methods aimed at analysis and search of feasible compositions, which meet the requirements and optimal performance (throughput) criteria. Very often in the industry, there is a task to find optimum utilization of the cluster tool. To find a solution various approaches are used, such as input in the system several flows of the semifinished products, increasing overall number of the arms of the robot, etc. In this article cluster tool and throughput dependence on quantity of performed operations of the least loaded module are investigated. It is shown, that with the help of this approach one can easily simplify both finding the solution of the task to composite cluster tool with flows of two different types of wafers and determining schedule of the wafers for maximum and minimum throughput. Relevance of the article is stipulated by substantive and complex choices of both finding optimal composition of the cluster tool and scheduling semifinished products flows (in accordance with variations of the operations in modules) at both the design and exploitation stage.

Productivity Comparison of Wafer Transport Architectures in PVD Tools Used for Fan-Out Packaging RDL Barrier/Seed Formation

Abstract Physical Vapor Deposition (PVD) systems are widely used in the semiconductor fabrication industry, both for front-end applications in the wafer fab and for back-end applications at device packaging houses. In fan-out wafer level packaging (FOWLP), and in fan-out panel level packaging (FOPLP), sputter deposited Ti and Cu are the base on which electroplated copper Redistribution Layers (RDLs) are built. For these RDL barrier/seed layers, PVD cluster tools, wafer transport architectures that have been widely used since the mid-1980s, are the current Process of Record (POR) in advanced packaging; however, these tools typically operate in a regime where wafer transport is robot-limited to approximately 50 wafers per hour, which limits overall throughput and greatly influences Cost of Ownership (COO) for the sputter deposition step(s), because the central handling robot occupied with a transfer from the Ti PVD module to the Cu PVD module, for example, has no opportunity to be doing anything other than that specific transfer. Other wafer transport architectures are more efficient from a wafer handling perspective. In linear transport carrier-based PVD tools, wafers or panels passing through the system benefit from a mechanical transfer time budget that is considerably less than for a cluster tool. Transport time overhead per wafer on linear transport systems is quite low, and scheduler software optimization becomes less onerous too, as a result of the simpler wafer transport architecture. We analyzed the relative throughput of cluster and linear transport PVD tools for a typical FOWLP barrier/seed layer (1000Å Ti / 2000Å Cu) sputter deposition process, and present details here of how the time spent moving wafers to various processing chambers affects overall system productivity. In the case of the cluster tool architecture, with its central wafer handling robot, wafer throughputs are approximately 50 wafers per hour, while on the linear transport system wafer throughputs as high as 240 wafers per hour are possible. The significant difference in system throughputs greatly affects the relative Cost of Ownership (COO) per wafer processed, with the linear transport system returning COO results that are less than half those of the typical cluster PVD tool.

On the benefits of the remote GPU virtualization mechanism: The rCUDA case

SummaryGraphics processing units (GPUs) are being adopted in many computing facilities given their extraordinary computing power, which makes it possible to accelerate many general purpose applications from different domains. However, GPUs also present several side effects, such as increased acquisition costs as well as larger space requirements. They also require more powerful energy supplies. Furthermore, GPUs still consume some amount of energy while idle, and their utilization is usually low for most workloads. In a similar way to virtual machines, the use of virtual GPUs may address the aforementioned concerns. In this regard, the remote GPU virtualization mechanism allows an application being executed in a node of the cluster to transparently use the GPUs installed at other nodes. Moreover, this technique allows to share the GPUs present in the computing facility among the applications being executed in the cluster. In this way, several applications being executed in different (or the same) cluster nodes can share 1 or more GPUs located in other nodes of the cluster. Sharing GPUs should increase overall GPU utilization, thus reducing the negative impact of the side effects mentioned before. Reducing the total amount of GPUs installed in the cluster may also be possible. In this paper, we explore some of the benefits that remote GPU virtualization brings to clusters. For instance, this mechanism allows an application to use all the GPUs present in the computing facility. Another benefit of this technique is that cluster throughput, measured as jobs completed per time unit, is noticeably increased when this technique is used. In this regard, cluster throughput can be doubled for some workloads. Furthermore, in addition to increase overall GPU utilization, total energy consumption can be reduced up to 40%. This may be key in the context of exascale computing facilities, which present an important energy constraint. Other benefits are related to the cloud computing domain, where a GPU can be easily shared among several virtual machines. Finally, GPU migration (and therefore server consolidation) is one more benefit of this novel technique.

Optimising the deployment of virtual machine image replicas in cloud storage clusters

With the immense proliferation of cloud-based applications, the diversity of cloud services exacerbates the complexity of virtual machine provisioning. Addressing the performance issue, a series of solutions have been proposed to optimise the VM replica deployment. However, the majority of the existing mechanisms concentrate on optimising the locations of VM replicas, or resizing the scales of different replica sets; without an overall consideration of both the platform and application features. The purpose of this paper is to optimise the deployment of VM image replicas in cloud storage clusters. We study the impacts of both application and platform features on the VM instance installation rate, and further propose a novel mechanism to optimise the deployment of VM image replicas. Our deployment mechanism has been implemented in a real cloud storage cluster. Experimental results show that our mechanism improves the overall throughput of a storage cluster and speeds up the image installation operations.

Using Coordinate based Mobile Sink Technique to Improve the Performance of Wireless Sensor Network

Objectives: The main objectives of this research are to improve performance of LEACH protocol for data gathering by deploying multiple sinks. The movement of the sinks are depends upon the signal strength and bee colony algorithm. By using the proposed technique performance of LEACH protocol is increased. Method Analysis: To improve performance of LEACH protocol is improved by deploying multiple sinks and movement of the sinks are decided using bee colony. Findings: By Appling bee colony with LEACH protocol for multiple sink movement various energy parameters are analyzed and compared with existing algorithm. It is seen that energy consumption of the network reduced and network is increased. The improvement leads to batter data gathering from cluster heads and throughput of the network is increased.