Large-scale Cluster Research Articles

Towards an Energy Efficient Computing With Coordinated Performance-Aware Scheduling in Large Scale Data Clusters

Many prior works have investigated on how to increase the job processing performance and energy efficient computing in large scale clusters. However, they employ serialized scheduling approaches encompassed with task straggler “hunting” techniques which launches speculative tasks after detecting slow tasks. These slow tasks are detected through node instrumentation which collects system level information whilst tracking the task execution progress. Such approaches are however detrimental towards achieving maximum processing performance and preserving cluster energy as they increase communication overheads. In this paper, we observe that node instrumentation and serialized scheduling in existing works does not only degrade the job processing performance, but also increase cluster energy consumption. To alleviate this, we propose EPPADS, a light-weight scheduler which eradicates the need for instrumentation modules for job scheduling purposes. EPPADS schedules tasks in two stages, the slow-start phase (SSP) and accelerate phase (AccP). The SSP schedules initial tasks in the queue using baseline FIFO scheduling and records the initial execution times of the processing nodes, whilst tagging the effective and straggling nodes. The AccP uses the initial execution times to compute the processing nodes task distribution ratio of remaining tasks and schedules them in parallel using a single scheduling I/O, boosting up the processing performance. To amortize the computing energy costs, EPPADS implements a power management module that coordinates with the scheduling module and leverage on node tagging information, to place nodes in two different power transition pools, i.e., high and low state power pools. A single power transition signal per pool is then broadcasted to lower or raise the energy state in the low-power state pool and high-power state pool. Our evaluation using a Hadoop cluster shows that EPPADS achieves 30% and 22% performance improvement and 15% to 20% energy savings as compared to the FIFO and DynMon schedulers, respectively.

Research on the RC Panel Experiments with Cushion Layer Upon Rockfall Impact

Reinforced concrete (RC) panels are widely used in the prevention and control of rockfall disasters in mountainous areas. In order to avoid the rigid damage caused by rockfalls that directly impact RC panels, the energy dissipation layer is often added to cushion the impact of rockfalls. In order to study the dynamic response of RC panels to different cushion layers under impact, a series of rockfall tests are carried out based on outdoor test platform. At the same thickness, the maximum impact force of EPS and sand composite cushion reduces by 50% compared with that of sand cushion, and the impact contact time is obviously longer than that of sand. Under the same impact condition, the center position of EPS material is crushed and collapsed, and a large number of radial cracks occur. The composite cushion layer can effectively reduce the mid-span displacement of RC Panel. At the falling height of 3 m, 5 m and 7 m, respectively, the mid-span displacement of RC Panel decreases by 37% to 46%. When sand is used as the cushion layer, the visible cracks in the middle of RC span increase from the bottom to the top. At the falling height of 7 m, the strain rate of concrete ranges from 10¹ ms^-1 to 10² ms^-1, while the strain rate of Reinforced ranges from 10² ms^-1 to 10^3m s^-1. Therefore, EPS and sand composite cushion layer is superior to sand in terms of energy dissipation effect. Compared with large-scale cluster rockfall prevention, the composite cushion layer has more advantages in preventing and treating single rockfall because of its easy damage, high cost of maintenance and replacement.

The Impact of a Maternal Education Program Through Text Messaging in Rural China: Cluster Randomized Controlled Trial.

BackgroundIn recent years, attempts have been made to use mobile phone text messaging (short message service, SMS) to achieve positive results for a range of health issues. Reports on the impact of maternal education programs based on this widely available, inexpensive, and instant communication tool are sparse.ObjectiveThis study aimed to explore the impact of a maternal education program through text messaging.MethodsWe conducted a cluster randomized trial in a remote region in the Chinese province of Hunan between October 1, 2011, and December 31, 2012. We used county as the unit of randomization (a total of 10 counties), with half of the counties randomly allocated to the intervention arm (with maternal education material adapted from the World Health Organization being delivered by text messaging to village health workers and pregnant women alike) and the other half to the control arm (normal care without text messaging). Data on maternal and infant health outcomes and health behaviors were collected and compared between the 2 arms, with maternal and perinatal mortality as the primary outcomes.ResultsA total of 13,937 pregnant women completed the follow-up and were included in the final analysis. Among them, 6771 were allocated to the intervention arm and 6966 were allocated to the control arm. At the county level, the mean (SD) of maternal mortality and perinatal mortality rate were 0.0% (0.1) and 1.3% (0.6), respectively, in the intervention arm and 0.1% (0.2) and 1.5% (0.4), respectively, in the control arm. However, these differences were not statistically significant. At the individual level, there were 3 maternal deaths (0.04%) and 84 perinatal deaths (1.24%) in the intervention arm and 6 maternal deaths (0.09%) and 101 perinatal deaths (1.45%) in the control arm. However, the differences were again not statistically significant.ConclusionsAdequate resources should be secured to launch large-scale cluster randomized trials with smaller cluster units and more intensive implementation to confirm the benefits of the text messaging–based maternal education program suggested by this trial.Trial RegistrationClinicalTrials.gov NCT01775150; https://clinicaltrials.gov/ct2/show/NCT01775150 (Archived by WebCite at http://www.webcitation.org/74cHmUexo)

An Empirical Performance Analysis on Hadoop via Optimizing the Network Heartbeat Period

To support a large-scale Hadoop cluster, Hadoop heartbeat messages are designed to deliver the significant messages, including task scheduling and completion messages, via piggybacking to reduce the number of messages received by the NameNode. Although Hadoop is designed and optimized for high-throughput computing via batch processing, the real-time processing of large amounts of data in Hadoop is increasingly important. This paper evaluates Hadoop’s performance and costs when the heartbeat period is controlled to support latency sensitive applications. Through an empirical study based on Hadoop 2.0 (YARN) [1] architecture, we improve Hadoop’s I/O performance as well as application performance by up to 13 percent compared to the default configuration. We offer a guideline that predicts the performance, costs and limitations of the total system by controlling the heartbeat period using simple equations. We show that Hive performance can be improved by tuning Hadoop’s heartbeat periods through extensive experiments.

Exact WCRT Analysis for Message-Processing Tasks on Gateway-Integrated In-Vehicle CAN Clusters

A typical automotive integrated architecture is a controller area network (CAN) cluster integrated by a central gateway. This study proposes a novel and exact worst-case response time (WCRT) analysis method for message-processing tasks in the gateway. We first propose a round search method to obtain lower bound on response time (LBRT) and upper bound on response time (UBRT), respectively. We then obtain the exact WCRT belonging to the scope of the LBRT and UBRT with an effective non-exhaustive exploration. Experimental results on a real CAN message set reveal that the proposed exact analysis method can reduce 99.99999% combinations on large-scale CAN clusters.

Self-organizing control for satellite clusters using artificial potential function in terms of relative orbital elements

Self-organizing control for satellite clusters is a challenging and promising problem which has drawn considerable attention in the recent past. The artificial potential function method has been widely used for self-organizing control due to its elegant mathematical analysis and simplicity. This paper proposes a set of self-organizing control rules for satellite clusters described by artificial potential functions, so that the reconfiguration, uniform distribution and collision avoidance operations can be achieved spontaneously. It may work regardless of the failure of satellites, attendance of new members or existence of space debris, and the corresponding communication and control system are completely distributed. Particularly, the artificial potential functions are written in terms of relative orbital elements derived from T–H equations, so that the self-organizing control can reflect relative motion dynamics, and guide the satellite along a fuel-efficient trajectory. The proposed method is especially suitable for highly distributed micro-satellite clusters and large-scale clusters to track moving targets, with few fuel cost and relatively high control accuracy, and it is applicable to clusters in either deep space or near-earth space. The stability of the control was proved by Lyapunov second method, and verified by Monte Carlo simulation. Finally, the comparison between self-organizing control and fuel-optimal control was made to demonstrate the performance properties.

Investigation of Variability of Wind/Solar Resources Properties

Renewable Energy Sources (RES) such as wind and solar energy depend on the climatic conditions, power grid and other comprehensive factors. Areas with sufficient RES also have large-scale wind power clusters and PV plants connected to the grid. In order to reduce the impact of fluctuating energy on power system, it is necessary to investigate variation of wind and solar energy resources in a region. This paper proposes an integrated method based on the xgboost model to study regional renewable energy resources. Characteristics of annual and monthly variation of aggregated powers in a large area are analyzed. The proposed method can be further used for reliability analysis and planning of intermittent power generation in regional grids.

BiGNoC: Accelerating Big Data Computing with Application-Specific Photonic Network-on-Chip Architectures

In the era of big data, high performance data analytics applications are frequently executed on large-scale cluster architectures to accomplish massive data-parallel computations. Often, these applications involve iterative machine learning algorithms to extract information and make predictions from large data sets. Multicast data dissemination is one of the major performance bottlenecks for such data analytics applications in cluster computing, as terabytes of data need to be distributed frequently from a single data source to hundreds of computing nodes. To overcome this bottleneck for big data applications, we propose BiGNoC , a manycore chip platform with a novel application-specific photonic network-on-chip (PNoC) fabric. BiGNoC is designed for big data computing and exploits multicasting in photonic waveguides. For high performance data analytics applications, BiGNoC improves throughput by up to ${{9.9}}\times$ while reducing latency by up to 88 percent and energy-per-bit by up to 98 percent over two state-of-the-art PNoC architectures as well as a broadcast-optimized electrical mesh NoC architecture, and a traditional electrical mesh NoC architecture.

Coordinated control for medium voltage DC distribution centers with flexibly interlinked multiple microgrids

From flexible interconnection among feeders to hybrid alternating current (AC) and direct current (DC) distribution structures of future smart distribution systems, medium-voltage DC distribution centers with flexibly interlinked multiple microgrids (MGs) will have wide applications on the demand side. A generic coordinated control framework based on a distributed cooperation scheme is proposed for such DC centers, as opposed to centralized control structures. A novel unified control only using local measurements is proposed for these interlinking converters. During normal power disturbances, automatic coordinated power control and mutual support among sub-systems can be realized, thereby improving DC voltage and AC frequency stability to enable multiple MGs to be treated as a real unified cluster. Moreover, with this method, interlinking converters can realize seamless transition in power dispatching mode, common DC bus voltage control mode, and MG support mode without communication and control system switching. A simplified dynamic model has been developed to verify the proposed control strategy. This work is expected to provide a new solution for flexible interconnection and operational control of large-scale MG clusters.

Microstructural Analysis of Granular Metal-Ceramic Composite Materials of Matrix Type

The cluster structure of microphotographs of the metal-ceramic composite materials surface samples has been analyzed. Cluster analysis has been performed with the help of a self-organizing feature map in a four-dimensional feature space, built from textural visualizations of a microphotograph. As a result of our studies, two types of cluster structure elements were found: small-scale and large-scale clusters. The assumption has been made that two main features of the deformation graph can be associated with the destruction of the cluster elements of both kinds. As a result of multifractal analysis, it was found out that the difference in technology does not significantly affect the multifractal spectra, but it proves to be significant for multifractal dimensions.

A Review: Map Reduce Framework for Cloud Computing

In this generation of Internet, information and data are growing continuously. Even though various Internet services and applications. The amount of information is increasing rapidly. Hundred billions even trillions of web indexes exist. Such large data brings people a mass of information and more difficulty discovering useful knowledge in these huge amounts of data at the same time. Cloud computing can provide infrastructure for large data. Cloud computing has two significant characteristics of distributed computing i.e. scalability, high availability. The scalability can seamlessly extend to large-scale clusters. Availability says that cloud computing can bear node errors. Node failures will not affect the program to run correctly. Cloud computing with data mining does significant data processing through high-performance machine. Mass data storage and distributed computing provide a new method for mass data mining and become an effective solution to the distributed storage and efficient computing in data mining.

Parallel Algorithms of Multi-relaxation-time Lattice Boltzmann Method

The lattice Boltzmann method has become an attractive and promising approach in computational fluid dynamics. In this paper, the D3Q19 multi-relaxation-time lattice Boltzmann method is employed to simulate complex fluid flow and its parallel algorithm is presented including Cartesian grid generation, domain decomposition method, and data exchange strategy on clusters. Considering load balancing on large scale cluster, details of domain decomposition method are presented. The numerical results show that the presented algorithm have considerable scalability on 2048 cores and the efficiency can achieve 92.01%.

Fail-Slow at Scale

Fail-slow hardware is an under-studied failure mode. We present a study of 114 reports of fail-slow hardware incidents, collected from large-scale cluster deployments in 14 institutions. We show that all hardware types such as disk, SSD, CPU, memory, and network components can exhibit performance faults. We made several important observations such as faults convert from one form to another, the cascading root causes and impacts can be long, and fail-slow faults can have varying symptoms. From this study, we make suggestions to vendors, operators, and systems designers.

The interaction of multiple stellar winds in stellar clusters: potential flow

Context. While several studies have investigated large-scale cluster winds resulting from an intra-cluster interaction of multiple stellar winds, as yet they have not provided details of the bordering flows inside a given cluster. Aims. The present work explores the principal structure of the combined flow resulting from the interaction of multiple stellar winds inside stellar clusters. Methods. The theory of complex potentials is applied to analytically investigate stagnation points, boundaries between individual outflows, and the hydrodynamic structure of the asymptotic large-scale cluster wind. In a second part, these planar considerations are extended to fully three-dimensional, asymmetric configurations of wind-driving stars. Results. We find (i) that one can distinguish regions in the large-scale cluster wind that are determined by the individual stellar winds, (ii) that there are comparatively narrow outflow channels, and (iii) that the large-scale cluster wind asymptotically approaches spherical symmetry at large distances. Conclusions. The combined flow inside a stellar cluster resulting from the interaction of multiple stellar winds is highly structured.

Large-scale Au nanoparticle cluster arrays with tunable particle numbers evolved from colloidal lithography

The assembly of metal nanoparticles (NPs) can regulate their plasmon resonance properties to pursue the best properties for applications. However, the controllable assembly of large-scale metal NP cluster arrays remains a significant challenge. This paper presents a novel strategy to prepare large-scale Au NP cluster arrays based on colloidal lithography and template-guided self-assembly technique. The NPs arrays are fabricated by introducing Au NPs onto the quaternized poly(2-(dimethylamino)ethyl methacrylate) (PDMAEMA) brush templates via electrostatic interaction. The number of Au NPs in cluster can be arbitrarily tuned by changing the surface area of the polymer templates created by colloidal lithography, which resulted in tunable plasmonic properties. The prepared Au NP cluster arrays were used for surface enhanced Raman scattering (SERS) and the SERS properties of the Au NP cluster arrays were studied.

Influence of newborn health messages on care-seeking practices and community health behaviors among participants in the Zambia Chlorhexidine Application Trial.

BackgroundIdentifying and understanding traditional perceptions that influence newborn care practices and care-seeking behavior are crucial to developing sustainable interventions to improve neonatal health. The Zambia Chlorhexidine Application Trial (ZamCAT), a large-scale cluster randomized trial, assessed the impact of 4% chlorhexidine on neonatal mortality and omphalitis in Southern Province, Zambia. The main purpose of this post-ZamCAT qualitative study was to understand the impact of newborn care health messages on care-seeking behavior for neonates and the acceptability, knowledge, and attitudes towards chlorhexidine cord care among community members and health workers in Southern Province.Methods & findingsFive focus group discussions and twenty-six in-depth interviews were conducted with mothers and health workers from ten health centers (5 rural and 5 peri-urban/urban). Community perceptions and local realities were identified as fundamental to care-seeking decisions and influenced individual participation in particular health-seeking behaviors. ZamCAT field monitors (data collectors) disseminated health messages at the time of recruitment at the health center and during subsequent home visits. Mothers noted that ZamCAT field monitors were effective in providing lessons and education on newborn care practices and participating mothers were able to share these messages with others in their communities. Although the study found no effect of chlorhexidine cord washes on neonatal mortality, community members had positive views towards chlorhexidine as they perceived that it reduced umbilical cord infections and was a beneficial alternative to traditional cord applications.ConclusionThe acceptability of health initiatives, such as chlorhexidine cord application, in community settings, is dependent on community education, understanding, and engagement. Community-based approaches, such as using community-based cadres of health workers to strengthen referrals, are an acceptable and potentially effective strategy to improve care-seeking behaviors and practices.

On the Power-of-d-choices with Least Loaded Server Selection

Motivated by distributed schedulers that combine the power-of-d-choices with late binding and systems that use replication with cancellation-on-start, we study the performance of the LL(d) policy which assigns a job to a server that currently has the least workload among d randomly selected servers in large-scale homogeneous clusters. We consider general job size distributions and propose a partial integro-differential equation to describe the evolution of the system. This equation relies on the earlier proven ansatz for LL(d) which asserts that the workload distribution of any finite set of queues becomes independent of one another as the number of servers tends to infinity. Based on this equation we propose a fixed point iteration for the limiting workload distribution and study its convergence. For exponential job sizes we present a simple closed form expression for the limiting workload distribution that is valid for any work-conserving service discipline as well as for the limiting response time distribution in case of first-come-first-served scheduling. We further show that for phase-type distributed job sizes the limiting workload and response time distribution can be expressed via the unique solution of a simple set of ordinary differential equations. Numerical and analytical results that compare response time of the classic power-of-d-choices algorithm and the LL(d) policy are also presented and the accuracy of the limiting response time distribution for finite systems is illustrated using simulations.

On the Power-of-d-choices with Least Loaded Server Selection

Motivated by distributed schedulers that combine the power-of-d-choices with late binding and systems that use replication with cancellation-on-start, we study the performance of the LL(d) policy which assigns a job to a server that currently has the least workload among d randomly selected servers in large-scale homogeneous clusters. We consider general job size distributions and propose a partial integro-differential equation to describe the evolution of the system. This equation relies on the earlier proven ansatz for LL(d) which asserts that the workload distribution of any finite set of queues becomes independent of one another as the number of servers tends to infinity. Based on this equation we propose a fixed point iteration for the limiting workload distribution and study its convergence.

Bi‐level decentralised active power control for large‐scale wind farm cluster

This study presents a bi-level decentralised active power control (DAPC) for a large-scale wind farm cluster (WFC), consisting of several wind farms for better active power dispatch. In the upper level, a distributed active power control scheme based on the distributed consensus is designed to achieve fair active power sharing among multiple wind farms, which generates the power reference for each wind farm. A distributed estimator is used to estimate the total available power of all wind farms. In the lower level, a centralised control scheme based on the model predictive control is proposed to regulate active power outputs of all wind turbines (WTs) within a wind farm, which reduces the fatigue loads of WTs while tracking the power reference obtained from the upper-level control. A WFC with 8 wind farms and totally 160 WTs, was used to test the control performance of the proposed DAPC scheme.

High-Threshold Fault-Tolerant Quantum Computation with Analog Quantum Error Correction

To implement fault-tolerant quantum computation with continuous variables, the Gottesman-Kitaev-Preskill (GKP) qubit has been recognized as an important technological element. However,it is still challenging to experimentally generate the GKP qubit with the required squeezing level, 14.8 dB, of the existing fault-tolerant quantum computation. To reduce this requirement, we propose a high-threshold fault-tolerant quantum computation with GKP qubits using topologically protected measurement-based quantum computation with the surface code. By harnessing analog information contained in the GKP qubits, we apply analog quantum error correction to the surface code.Furthermore, we develop a method to prevent the squeezing level from decreasing during the construction of the large scale cluster states for the topologically protected measurement based quantum computation. We numerically show that the required squeezing level can be relaxed to less than 10 dB, which is within the reach of the current experimental technology. Hence, this work can considerably alleviate this experimental requirement and take a step closer to the realization of large scale quantum computation.