Large-scale Heterogeneous Systems Research Articles

Blockchain for Managing Heterogeneous Internet of Things: A Perspective Architecture

IoT has the potential to transform the way we think about information and communication technology. IoT has been studied extensively across many disciplines such as the networking, communication, security, business, and management communities. However, many unsolved challenges, especially in managing heterogeneous IoTs, remain to be discussed. Recent studies propose using blockchain, an emerging technology that enables decentralized coordination, to address inherent challenges in IoT. This article presents a preliminary study on an architecture that implements blockchain in managing heterogeneous IoT systems. We start by pointing out the limitations of prior IoT systems and the difficulties of integrating IoT and blockchain. Then we outline an architecture to manage a large-scale heterogeneous IoT system. Our main goal is to stimulate further effort and cross-disciplinary collaboration by providing guidance and reference for future studies.

Dynamic evaluation of ecosystem integrity in Greater Khingan Range area, China.

The impacts of global climate change and human disturbance on terrestrial ecosystems has been expanding in intensity and scope. However, there is a lack of evaluation methods for large-scale heterogeneous landscape systems during ecosystem integrity assessment. From the perspective of landscape ecology, this study chose the indicators of landscape structure and landscape stability to comprehensively evaluate regional landscape change characteristics and ecosystem integrity dynamics of Greater Hinggan Mountains area using land use data and NPP data of 2005, 2010 and 2015 with GIS data processing and analysis platform. The results showed that Greater Hinggan Mountains area was dominated by forest land. The dominance degree of forest land was stable at 64.9%, which was much higher than the grassland that with the second dominance. The landscape diversity index was 0.59 in all three periods, accounting for 42.4% of the maximum value of landscape diversity index. Such a result indicated that the landscape resistance was stable in this area. From the view of landscape biomass, farmland and residential land had increased year by year, while forest area had decreased year by year, indicating that the effects of human disturbance on Greater Hinggan Mountains area were increasing.

An intelligent water drops-based workflow scheduling for IaaS cloud

Cloud computing is an emerging technology in a distributed environment with a collection of large-scale heterogeneous systems. One of the challenging issues in the cloud data center is to select the minimum number of virtual machine (VM) instances to execute the tasks of a workflow within a time limit. The objectives of such a strategy are to minimize the total execution time of a workflow and improve resource utilization. However, the existing algorithms do not guarantee to achieve high resource utilization although they have abilities to achieve high execution efficiency. The higher resource utilization depends on the reusability of VM instances. In this work, we propose a new intelligent water drops based workflow scheduling algorithm for Infrastructure-as-a-Service (IaaS) cloud. The objectives of the proposed algorithm are to achieve higher resource utilization and minimize the makespan within the given deadline and budget constraints. The first contribution of the algorithm is to find multiple partial critical paths (PCPs) of a workflow which helps in finding suitable VM instances. The second contribution is a scheduling strategy for PCP-VM assignment for assigning the VM instances. The proposed algorithm is evaluated through various simulation runs using synthetic datasets and various performance metrics. Through comparison, we show the superior performance of the proposed algorithm over the existing ones.

The impact of workload variability on the energy efficiency of large-scale heterogeneous distributed systems

Previous studies have shown that the workload variability has a serious impact on the performance of large-scale distributed architectures, since it may cause significant fluctuations in service demands. Energy efficiency is one of the aspects of such platforms that are of paramount importance and therefore it is imperative to investigate how it may also be affected by this factor. Towards this direction, in this paper we investigate via simulation the impact of workload variability, in terms of computational volume and interarrival times, on the energy consumption of a large-scale heterogeneous distributed system. The workload consists of real-time bag-of-tasks jobs that arrive dynamically at the system. The execution rate and power consumption characteristics of the processors are modeled after real-world processors, according to the Standard Performance Evaluation Corporation (SPEC) Power benchmark. Four heuristics are employed for the scheduling of the workload, two commonly used baseline policies and two proposed energy-aware heuristics. The simulation results reveal that the workload variability has a significant impact on the energy consumption of the system and that the severity of the impact depends on the employed scheduling technique.

A green policy to schedule tasks in a distributed cloud

In the last years, demand and availability of computational capabilities experienced radical changes. Desktops and laptops increased their processing resources, exceeding users’ demand for large part of the day. On the other hand, computational methods are more and more frequently adopted by scientific communities, which often experience difficulties in obtaining access to the required resources. Consequently, data centers for outsourcing use, relying on the cloud computing paradigm, are proliferating. Notwithstanding the effort to build energy-efficient data centers, their energy footprint is still considerable, since cooling a large number of machines situated in the same room or container requires a significant amount of power. The volunteer cloud, exploiting the users’ willingness to share a quote of their underused machine resources, can constitute an effective solution to have the required computational resources when needed. In this paper, we foster the adoption of the volunteer cloud computing as a green (i.e., energy efficient) solution even able to outperform existing data centers in specific tasks. To manage the complexity of such a large scale heterogeneous system, we propose a distributed optimization policy to task scheduling with the aim of reducing the overall energy consumption executing a given workload. To this end, we consider an integer programming problem relying on the Alternating Direction Method of Multipliers (ADMM) for its solution. Our approach is compared with a centralized one and other non-green targeting solutions. Results show that the distributed solution found by the ADMM constitutes a good suboptimal solution, worth to be applied in a real environment.

Large-scale heterogeneous service systems with general packing constraints

Abstract A service system with multiple types of customers, arriving according to Poisson processes, is considered. The system is heterogeneous in that the servers can also be of multiple types. Each customer has an independent, exponentially distributed service time, with the mean determined by its type. Multiple customers (possibly of different types) can be placed for service into one server, subject to `packing' constraints, which depend on the server type. Service times of different customers are independent, even if served simultaneously by the same server. The large-scale asymptotic regime is considered such that the customer arrival rates grow to ∞. We consider two variants of the model. For the infinite-server model, we prove asymptotic optimality of the greedy random (GRAND) algorithm in the sense of minimizing the weighted (by type) number of occupied servers in steady state. (This version of GRAND generalizes that introduced by Stolyar and Zhong (2015) for homogeneous systems, with all servers of the same type.) We then introduce a natural extension of the GRAND algorithm for finite-server systems with blocking. Assuming subcritical system load, we prove existence, uniqueness, and local stability of the large-scale system equilibrium point such that no blocking occurs. This result strongly suggests a conjecture that the steady-state blocking probability under the algorithm vanishes in the large-scale limit.

Achieving dynamicity in security policies enforcement using aspects

The dynamic configuration and evolution of large-scale heterogeneous systems has made the enforcement of security requirements one of the most critical phases throughout the system development lifecycle. In this paper, we propose a framework architecture to associate the security policies with the specification and the execution phases of applications defined for these systems. Our proposed framework is based on an aspect-oriented programming approach and on the organization-based access control model to dynamically enforce and manage the access and the usage control. The deployment of the framework modules, proposed in this paper, takes into account the changes that may occur in the security policy during the application execution. We also present the implementation as well as the evaluation of our proposition.

Robust scalable stabilisability conditions for large-scale heterogeneous multi-agent systems with uncertain nonlinear interactions: towards a distributed computing architecture

ABSTRACTLarge-scale dynamic systems are becoming highly pervasive in their occurrence with applications ranging from system biology, environment monitoring, sensor networks, and power systems. They are characterised by high dimensionality, complexity, and uncertainty in the node dynamic/interactions that require more and more computational demanding methods for their analysis and control design, as well as the network size and node system/interaction complexity increase. Therefore, it is a challenging problem to find scalable computational method for distributed control design of large-scale networks. In this paper, we investigate the robust distributed stabilisation problem of large-scale nonlinear multi-agent systems (briefly MASs) composed of non-identical (heterogeneous) linear dynamical systems coupled by uncertain nonlinear time-varying interconnections. By employing Lyapunov stability theory and linear matrix inequality (LMI) technique, new conditions are given for the distributed control design of large-scale MASs that can be easily solved by the toolbox of MATLAB. The stabilisability of each node dynamic is a sufficient assumption to design a global stabilising distributed control. The proposed approach improves some of the existing LMI-based results on MAS by both overcoming their computational limits and extending the applicative scenario to large-scale nonlinear heterogeneous MASs. Additionally, the proposed LMI conditions are further reduced in terms of computational requirement in the case of weakly heterogeneous MASs, which is a common scenario in real application where the network nodes and links are affected by parameter uncertainties. One of the main advantages of the proposed approach is to allow to move from a centralised towards a distributed computing architecture so that the expensive computation workload spent to solve LMIs may be shared among processors located at the networked nodes, thus increasing the scalability of the approach than the network size. Finally, a numerical example shows the applicability of the proposed method and its advantage in terms of computational complexity when compared with the existing approaches.

623 Tflop/s HPCG run on Tianhe-2: Leveraging millions of hybrid cores

In this article, we present a new hybrid algorithm to enable and scale the high-performance conjugate gradients (HPCG) benchmark on large-scale heterogeneous systems such as the Tianhe-2. Based on an inner–outer subdomain partitioning strategy, the data distribution between host and device can be balanced adaptively. The overhead of data movement from both the MPI communication and the PCI-E transfer can be significantly reduced by carefully rearranging and fusing operations. A variety of parallelization and optimization techniques for performance-critical kernels are exploited and analyzed to maximize the performance gain on both host and device. We carry out experiments on both a small heterogeneous computer and the world’s largest one, the Tianhe-2. On the small system, a thorough comparison and analysis has been presented to select from different optimization choices. On Tianhe-2, the optimized implementation scales to the full-system level of 3.12 million heterogeneous cores, with an aggregated performance of 623 Tflop/s and a parallel efficiency of 81.2%.

Pull-based load distribution in large-scale heterogeneous service systems

The model is motivated by the problem of load distribution in large-scale cloud-based data processing systems. We consider a heterogeneous service system, consisting of multiple large server pools. The pools are different in that their servers may have different processing speed and/or different buffer sizes (which may be finite or infinite). We study an asymptotic regime in which the customer arrival rate and pool sizes scale to infinity simultaneously, in proportion to some scaling parameter $n$. Arriving customers are assigned to the servers by a "router", according to a {\em pull-based} algorithm, called PULL. Under the algorithm, each server sends a "pull-message" to the router, when it becomes idle; the router assigns an arriving customer to a server according to a randomly chosen available pull-message, if there are any, or to a random server, otherwise. Assuming sub-critical system load, we prove asymptotic optimality of PULL. Namely, as system scale $n\to\infty$, the steady-state probability of an arriving customer experiencing blocking or waiting, vanishes. We also describe some generalizations of the model and PULL algorithm, for which the asymptotic optimality still holds.

Reducing the ambiguity of karst aquifer models by pattern matching of flow and transport on catchment scale

Abstract. Assessing the hydraulic parameters of karst aquifers is a challenge due to their high degree of heterogeneity. The unknown parameter field generally leads to a high ambiguity for flow and transport calibration in numerical models of karst aquifers. In this study, a distributed numerical model was built for the simulation of groundwater flow and solute transport in a highly heterogeneous karst aquifer in south-western Germany. Therefore, an interface for the simulation of solute transport in one-dimensional pipes was implemented into the software COMSOL Multiphysics® and coupled to the three-dimensional solute transport interface for continuum domains. For reducing model ambiguity, the simulation was matched for steady-state conditions to the hydraulic head distribution in the model area, the spring discharge of several springs and the transport velocities of two tracer tests. Furthermore, other measured parameters such as the hydraulic conductivity of the fissured matrix and the maximal karst conduit volume were available for model calibration. Parameter studies were performed for several karst conduit geometries to analyse the influence of the respective geometric and hydraulic parameters and develop a calibration approach in a large-scale heterogeneous karst system. Results show that it is possible not only to derive a consistent flow and transport model for a 150 km2 karst area but also to combine the use of groundwater flow and transport parameters thereby greatly reducing model ambiguity. The approach provides basic information about the conduit network not accessible for direct geometric measurements. The conduit network volume for the main karst spring in the study area could be narrowed down to approximately 100 000 m3.

High-performance emulation of heterogeneous systems using adaptive time dilation

Building a testbed for evaluating the performance of large-scale heterogeneous systems can be costly and inefficient. Emulation is often used to evaluate the performance of a system in a controlled environment. Time dilation allows virtual machines (VMs) to emulate higher performance than that of their physical machine. We present an approach using adaptive time dilation to emulate large-scale distributed systems composed of heterogeneous machines and Operating Systems (OSs). In our implementation, VMs are globally synchronized. To evaluate our system, distributed VMs running Linux, Windows, FreeBSD, and Junos are emulated on general-purpose physical machines.

Construction of Mobile Learning Network System Based on Cloud Computing

Distributed computing and Resource sharing in large-scale heterogeneous system are the key to realize mobile learning system. This paper sets up a mobile learning service system through the construction of teaching cloud platform and cloud resources, cloud resource sharing, application and service of cloud resource, system and mechanism construction. The mobile learning network system is established by building models of it, which realizes mobile learning, collaborative learning and the situated learning anytime anywhere with freedom, individuality and diversification.

A study on combinational effects of job and resource characteristics on energy consumption

By becoming more popular and complex, HPC systems like computational grids, clusters, clouds and the supporting data centers are now changed to remarkable energy consumers. A wide variety of researches, ranging from power-aware hardware design to developing optimized programs and to power-aware job scheduling, have been done hitherto in order to reduce their energy consumption. However, the success of these approaches highly depends to having a precise knowledge about power consumption behavior of the target system. In this paper, some neglected facts are shown about combinational effects of jobs' and resources' characteristics on energy consumption rate and define corresponding parameters which make these facts practically utilizable by formulating them as functions of job-machine characteristics. These facts are supported by the experimental analyses on real machines and analytical studies. The outcome of this paper can be exploited to have more energy efficient task mapping process in large-scale heterogeneous computational systems.

Beyond Context Awareness

Much of the past pervasive computing research has been devoted to composed of a small number of devices that interact with a individual or a small group of users. Body-worn sensors and sensors embedded in the user's environment were used to infer user location, activity, and information about the user's immediate surroundings, shaping the concept of context awareness. This issue is motivated by the fact that as technology becomes truly pervasive, we must proceed from the single user, system perspective to large-scale heterogeneous systems involving many devices and many individuals collaborating over different spatial and temporal scales.

A mean field model of work stealing in large-scale systems

In this paper, we consider a generic model of computational grids, seen as several clusters of homogeneous processors. In such systems, a key issue when designing efficient job allocation policies is to balance the workload over the different resources. We present a Markovian model for performance evaluation of such a policy, namely work stealing (idle processors steal work from others) in large-scale heterogeneous systems. Using mean field theory, we show that when the size of the system grows, it converges to a system of deterministic ordinary differential equations that allows one to compute the expectation of performance functions (such as average response times) as well as the distributions of these functions.

SEMANTIC INTEROPERABILITY AND INFORMATION FLUIDITY

Ontologies are developed to describe data semantics on the Semantic Web. Given the distributed nature and scale of the Semantic Web, a large number of ontologies with different terminologies and structures will be created to describe the same concepts and domains. Without semantic mapping, information fluidity within the Web could be blocked at the boundaries of these ontologies. Therefore, ontology mapping is needed to translate datasets represented by disparate ontologies. We believe that over time communities will incrementally build an ontology mapping between select ontologies based on their own communication interests. How will these interest-driven mapping activities eventually change semantic interoperability and information fluidity across the Web? This paper proposes metrics to quantify information fluidity and builds an analytical model with "small-world" graph theory to analyze the growth of the Semantic Web. Further with this model, we analyze how information fluidity can evolve by "market-driven" semantic mapping activities occurring across the Web. Our results can be useful in evaluating mapping efforts needed for large-scale heterogeneous information systems. One conclusion, based on this model, is that the development of decentralized ontology mappings can lead to significant information fluidity within the Semantic Web.

Integrating multimedia archives: the architecture and the content layer

In the last few years, numerous multimedia archives have made extensive use of digitized storage and annotation technologies. Still, the development of single points of access, providing common and uniform access to their data, despite the efforts and accomplishments of standardization organizations, has remained an open issue as it involves the integration of various large-scale heterogeneous and heterolingual systems. This paper describes a mediator system that achieves architectural integration through an extended three-tier architecture and content integration through semantic modeling. The described system has successfully integrated five multimedia archives, quite different in nature and content from each other, while also providing easy and scalable inclusion of more archives in the future.