Low Migration Costs Research Articles

Nereus: A Distributed Stream Band Join System With Adaptive Range Partitioning

Many real-time applications in consumer electronics rely on stream band join as a fundamental operation. With two streams, the band join operation targets at obtaining the pairs of tuples which are separately included in the two streams and have close values within a user specified range. Range partitioning keeps tuples with close values in a partition. For band join, the cost of employing range partitioning is less join cost than that of employing other partitioning strategies. However, the distribution of real-world data from consumer electronics over the range is skewed, causing severe load imbalance among instances in the distributed system that employs existing static range partitioning. Load migration can alleviate the load imbalance. For range partitioning, the migration has two kinds of objectives. Migrating load to the instance with adjacent partition controls the number of partitions. However, it causes an unacceptable high cost for migrating span multiple instances. While directly migrating a split partition to the lightest instance is low cost. However, it leads to an uncontrollable number of partitions. The system for consumer electronic applications cannot tolerate high migration cost and an uncontrollable number of partitions, which result in high latency and low throughput. In this work, we propose an adaptive range partitioning strategy to ensure a controllable number of partitions and load balancing with low cost. We implement Nereus, a distributed stream band join system. Nereus designs a migration benefit model using queuing theory measure, which integrates the benefits of partition’s change and load balancing. Such a design can obtain the most beneficial migration, which achieves low migration cost and an appropriate number of partitions. We conduct comprehensive experiments using large-scale datasets from real-world applications to evaluate this design. The results show that Nereus improves the throughput by 51% and reduces the processing latency by 99%, compared to existing designs.

Virtual Network Embedding for NGSO Systems: Algorithmic Solution and SDN-Testbed Validation

Non-Geostationary Orbit satellite (NGSO) is an essential element in 5G Non-Terrestrial Networks (NTNs), which can operate either independently or as complementary parts to terrestrial systems to boost the network capacity, coverage and resilience. Due to the highly dynamic topologies, one of the challenges in NGSO is how to harmonize the network virtualized resources to satisfy diverse quality of service requirements in an efficient manner. In this paper, we investigate Virtual Network Embedding (VNE) for integrated NGSO-terrestrial systems while considering dynamic topologies. We propose a Mixed Binary Linear Programming (MBLP) formulation for a Dynamic Topology-Aware VNE (DTA-VNE) algorithm. Given priori information about the network’s evolution over time, DTA-VNE plans the embedding for each Virtual Network Request (VNR) over its lifetime. In a highly dynamic environment, the VNE decision can be varying for different VNRs at the expense of a considerable cost of migrating traffic and reconfiguring resources. DTA-VNE aims at minimizing this migration cost to avoid unnecessary re-mappings. To tackle the exponential complexity of the MBLP, we propose an efficient algorithm based on relaxation approaches (DTA-R) to solve large-scale problems. In numerical results, the effectiveness of the proposed DTA-R is demonstrated with much lower migration cost than the conventional implementations. The trade-off between the computation time and migration cost of DTA-R is studied. Finally, we test DTA-R and the baselines in our developed MultI-layer awaRe SDN-based testbed for SAtellite-Terrestrial networks (MIRSAT) to precisely quantify the packet lost for each migration. DTA-R proved to reduce the packet lost by ~2.5-5% compared to baselines.

A two-phase heuristic algorithm for power-aware offline scheduling in IaaS clouds

The paper aims at mitigating hot-spots during Offline Scheduling in IaaS (Infrastructure-as-a-Service) cloud systems. Unlike previous studies, the research focuses on identifying and resolving hot-spots not at servers, but at server racks. A two-phase algorithm for performing power-aware offline scheduling is proposed. The first phase aims at identifying and mitigating hot-spots at racks, while the second phase performs VM consolidation, i.e. minimization of the number of occupied servers while maintaining a feasible VM mapping and low migration costs. The proposed algorithm takes into account the dynamic nature of VM's resource consumption: it does not only resolve detected hot-spots, but also tries to avoid hot-spots in a reasonable future time period. The algorithm was tested with the data from a real IaaS cloud with different sets of algorithm's parameters. Experimental evaluation showed that the statistical estimates of the future VM's resource consumption provide the most reliable mapping, which is a result of minimization of the number of new hot-spot occurrences.

LightWarner: Predicting Failure of 3D NAND Flash Memory Using Reinforcement Learning

NAND flash memory has gained popularity in a wide variety of digital storage systems. Although with excellent performance, NAND flash memory suffers various reliability problems. In recent years, researchers try to predict flash failure by using machine-learning models. However, the application of machine-learning based failure prediction method faces the following problems: imbalance between robustness and portability. When applying on different flash chips, the performance of prediction model degrades with the variation of error characteristics. In order to adapt to the variation, the machine-learning model needs to be re-built to ensure performance of failure prediction. The overheads of re-building model result in challenges when adjusting prediction model to adapt to the variation of error characteristics. To overcome these challenges, we present LightWarner, an easily applicable predictor based on model-free Reinforcement learning algorithms. LightWarner learns error characteristics dynamically during flash lifetime without pre-training. We evaluate the performance of LightWarner on six types of 3D flash chips. The evaluation result shows that LightWarner achieves over 93% F1 score on different flash chips, which is about 10% higher than supervised machine learning methods. And LightWarner can adapt to the variation of error characteristics with low migration costs.

Migration cost optimization for service provider legacy network migration to software‐defined IPv6 network

SummaryThis paper studies a problem for seamless migration of legacy networks of Internet service providers to a software‐defined networking (SDN)‐based architecture along with the transition to the full adoption of the Internet protocol version 6 (IPv6) connectivity. Migration of currently running legacy IPv4 networks into such new approaches requires either upgrades or replacement of existing networking devices and technologies that are actively operating. The joint migration to SDN and IPv6 network is considered to be vital in terms of migration cost optimization, skilled human resource management, and other critical factors. In this work, we first present the approaches of SDN and IPv6 migration in service providers' networks. Then, we present the common concerns of IPv6 and SDN migration with joint transition strategies so that the cost associated with joint migration is minimized to lower than that of the individual migration. For the incremental adoption of software‐defined IPv6 (SoDIP6) network with optimum migration cost, a greedy algorithm is proposed based on optimal path and the customer priority. Simulation and empirical analysis show that a unified transition planning to SoDIP6 network results in lower migration cost.

Understanding the Gap Between De Facto and De Jure Urbanization in China: A Perspective from Rural Migrants’ Settlement Intention

This paper tries to analyze the determinants and driving mechanisms of both settlement intention and hukou transfer intention for rural migrants in Chinese cities, which can help to understand the gap between de facto and de jure urbanization in China. Based on China Labor-force Dynamics Survey (CLDS) in 2014, 1145 samples with their settlement intention, hukou transfer intention, individual demographic characteristics, urban working and living conditions, rural resources and attachment, and geographic characteristics were collected. It suggested that compared with settlement intention, the rural migrants’ hukou transfer intention were much weaker. The rural migrants preferred small and medium cities for urban settlement but large and megacities for urban hukou conversion. By logistic regression analysis, a set of complex determinants of settlement intention was identified, including age, education attainment, marital status and spouse living together, as well as the trade-off between urban working and living conditions in the current host cities and rural landholdings and attachment in the hometown. In contrast, the hukou transfer intention was mainly determined by age, personal income, rural landholdings and the size of current host city, which highlighted the personal citizenization capacity and the trade-off between benefits related to urban and rural hukou. Moreover, by examining the characteristics of four sub-types of rural migrants with different settlement intention and hukou transfer intention, it was found that the rural migrants who intended to settle down and convert hukou at the same time usually had high personal citizenization capacity and preferred megacities; those who intended to settle down but rejected hukou conversion usually had high citizenization capacity and low migration cost; those who intended to convert hukou but rejected settling down in the cites preferred megacities instead of small cities; those who did not intend to settle down or convert hukou at all usually had low citizenization capacity and high migration cost. Based on these findings, it is recommended to promote the complete citizenization of rural migrants by improving their livelihood and well-being in the cities through kinds of policy reform about hukou, land, and social insurance.

Migration, Commuting, or a Second Home? Insights from an Experiment Among Academics.

In the last decades, more complex types of residential mobility, such as second homes, arose while one-way relocations declined. Existing studies that address migration and commuting as being substitutes often face problems of endogeneity or lack of data regarding decision-making on different types of residential mobility. In contrast, we investigate decision-making on migration, commuting, and establishing a second home by using a factorial survey experiment, which allows for the accounting of both endogeneity bias through randomisation and complexity in decision-making through treatment variation. Hypothetical job offers were presented to a sample of academic staff of a Swiss university (ETH Zurich) in order to examine the intended types of residential mobility and their drivers. Referring to the concept of loss aversion, the utility of a second home can exceed the joint utilities of migration and commuting particularly when the total loss of the current residence bears intolerable costs caused by location-specific capital, and daily commuting bears prohibitive costs caused by a lengthy distance. Analyses show that the migration intention is mainly caused by low migration and high transition costs and the commuting intention is mainly caused by low transition costs and high migration costs. Establishing a second home is indeed more intended with simultaneously higher migration and transition costs. In sum, a second home may be understood as a substitute for one-way relocations and daily commuting, yet primarily under conditions of extremely high or irreversible migration costs and unsustainable transition costs.

EASM: Efficiency-aware switch migration for balancing controller loads in software-defined networking

Distributed multi-controller deployment is a promising method to achieve a scalable and reliable control plane of Software-Defined Networking (SDN). However, it brings a new challenge for balancing loads on the distributed controllers as the network traffic dynamically changes. The unbalanced load distribution on the controllers will increase response delay for processing flows and reduce the controllers'throughput. Switch migration is an effective approach to solve the problem. However, existing schemes focus only on the load balancing performance but ignore migration efficiency, which may result in high migration costs and unnecessary control overheads. This paper proposes Efficiency-Aware Switch Migration (EASM) to balance the controllers'loads and improve migration efficiency. We introduce load difference matrix and trigger factor to measure load balancing on controllers. We also introduce the migration efficiency problem, which considers load balancing rate and migration cost simultaneously to optimally migrate switches. We propose EASM to efficiently solve to the problem. The simulation results show that EASM outperforms baseline schemes by reducing the controller response time by about 21.9%, improving the controller throughput by 30.4% on average, maintaining good load balancing rate, low migration costs and migration time, when the network scale changes.

Decentralized cloud datacenter reconsolidation through emergent and topology-aware behavior

Consolidation of multiple applications on a single Physical Machine (PM) within a cloud data center can increase utilization, minimize energy consumption, and reduce operational costs. However, these benefits come at the cost of increasing the complexity of the scheduling problem.In this paper, we present a topology-aware resource management framework. As part of this framework, we introduce a Reconsolidating PlaceMent scheduler (RPM) that provides and maintains durable allocations with low maintenance costs for data centers with dynamic workloads. We focus on workloads featuring both short-lived batch jobs and latency-sensitive services such as interactive web applications. The scheduler assigns resources to Virtual Machines (VMs) and maintains packing efficiency while taking into account migration costs, topological constraints, and the risk of resource contention, as well as the variability of the background load and its complementarity to the new VM.We evaluate the model by simulating a data center with over 65,000 PMs, structured as a three-level multi-rooted tree topology. We investigate trade-offs between factors that affect the durability and operational cost of maintaining a near-optimal packing. The results show that the proposed scheduler can scale to the number of PMs in the simulation and maintain efficient utilization with low migration costs.

Effect of migration based on strategy and cost on the evolution of cooperation

Humans consider not only their own ability but also the environment around them during the process of migration. Based on this fact, we introduce migration based on strategy and cost into the Spatial Prisoner’s Dilemma Game on a two-dimensional grid. The migration means that agents cannot move when all of the neighbors are cooperators; otherwise, agents move with a probability related to payoff and cost. The result obtained by the computer simulation shows that the moving mechanism based on strategy and cost improves the level of cooperation in a wide parameter space. This occurs because movement based on strategy effectively keeps the cooperative clusters and because movement based on cost effectively regulates the rate of movement. Both types of movement provide a favorable guarantee for the evolution of stable cooperation under the mutation rate q=0.0. In addition, we discuss the effectiveness of the migration mechanism in the evolution of cooperation under the mutation rate q=0.001. The result indicates that a higher level of cooperation is obtained at a lower migration cost, whereas cooperation is suppressed at a higher migration cost. Our work may provide an effective method for understanding the emergence of cooperation in our society.

International migration and human capital in Mexico: Networks or parental absence?

This article discusses the effect of international migration on the accumulation of human capital among Mexican youths aged 15–18 who are left behind. Evidence indicates the existence of a negative impact of sibling and parental migration on school attendance among young males but not on the measure of cognitive ability. Migration of extended family members has no significant effect. There is no evidence of a robust effect among females. The negative effect of sibling migration suggests that lower migration costs and differences in return to Mexican formal education between the labor markets of the United States and Mexico could largely explain the negative effect of international migration on human capital.

Partitioning functions for stateful data parallelism in stream processing

In this paper, we study partitioning functions for stream processing systems that employ stateful data parallelism to improve application throughput. In particular, we develop partitioning functions that are effective under workloads where the domain of the partitioning key is large and its value distribution is skewed. We define various desirable properties for partitioning functions, ranging from balance properties such as memory, processing, and communication balance, structural properties such as compactness and fast lookup, and adaptation properties such as fast computation and minimal migration. We introduce a partitioning function structure that is compact and develop several associated heuristic construction techniques that exhibit good balance and low migration cost under skewed workloads. We provide experimental results that compare our partitioning functions to more traditional approaches such as uniform and consistent hashing, under different workload and application characteristics, and show superior performance.

Preserving the original MPI semantics in a virtualized processor environment

Processor virtualization is a technique in which a programmer divides a computation into many entities, which are mapped to the available processors. The number of these entities, referred to as virtual processors, is typically larger than the number of physical processors. For an MPI program, the user decomposes the computation into more MPI tasks than physical processors. This approach allows overlapping computation and communication, and enables load balancing. User-level threads are often used to implement these virtual processors because they are generally faster to create, manage and migrate than heavy processes or kernel threads. However, these threads present issues concerning private data because they break the private address space assumption typically made by MPI programs. In this paper, we propose a new approach to privatize data in user-level threads. This approach is based on thread-local storage (TLS), which is often used by kernel threads. We apply this technique so that MPI programs can be executed in a virtualized environment while preserving their original semantics. We show that this alternative has a more efficient context switch and lower migration cost and is simpler to implement than other approaches.

Execution migration in a heterogeneous-ISA chip multiprocessor

Prior research has shown that single-ISA heterogeneous chip multiprocessors have the potential for greater performance and energy efficiency than homogeneous CMPs. However, restricting the cores to a single ISA removes an important opportunity for greater heterogeneity. To take full advantage of a heterogeneous-ISA CMP, however, we must be able to migrate execution among heterogeneous cores in order to adapt to program phase changes and changing external conditions (e.g., system power state). This paper explores migration on heterogeneous-ISA CMPs. This is non-trivial because program state is kept in an architecture-specific form; therefore, state transformation is necessary for migration. To keep migration cost low, the amount of state that requires transformation must be minimized. This work identifies large portions of program state whose form is not critical for performance; the compiler is modified to produce programs that keep most of their state in an architecture-neutral form so that only a small number of data items must be repositioned and no pointers need to be changed. The result is low migration cost with minimal sacrifice of non-migration performance. Additionally, this work leverages binary translation to enable instantaneous migration. When migration is requested, the program is immediately migrated to a different core where binary translation runs for a short time until a function call is reached, at which point program state is transformed and execution continues natively on the new core. This system can tolerate migrations as often as every 100 ms and still retain 95% of the performance of a system that does not do, or support, migration.

Execution migration in a heterogeneous-ISA chip multiprocessor

Prior research has shown that single-ISA heterogeneous chip multiprocessors have the potential for greater performance and energy efficiency than homogeneous CMPs. However, restricting the cores to a single ISA removes an important opportunity for greater heterogeneity. To take full advantage of a heterogeneous-ISA CMP, however, we must be able to migrate execution among heterogeneous cores in order to adapt to program phase changes and changing external conditions (e.g., system power state). This paper explores migration on heterogeneous-ISA CMPs. This is non-trivial because program state is kept in an architecture-specific form; therefore, state transformation is necessary for migration. To keep migration cost low, the amount of state that requires transformation must be minimized. This work identifies large portions of program state whose form is not critical for performance; the compiler is modified to produce programs that keep most of their state in an architecture-neutral form so that only a small number of data items must be repositioned and no pointers need to be changed. The result is low migration cost with minimal sacrifice of non-migration performance. Additionally, this work leverages binary translation to enable instantaneous migration. When migration is requested, the program is immediately migrated to a different core where binary translation runs for a short time until a function call is reached, at which point program state is transformed and execution continues natively on the new core. This system can tolerate migrations as often as every 100 ms and still retain 95% of the performance of a system that does not do, or support, migration.

Integrated approach towards adaptive state-tracking job migration for maximising performance benefit

Presented is a new state-tracking migration policy which is adaptively integrated with immediate restart and selective preemption techniques for risk-resilient job allocation in high-throughput computing environments. The proposed approach guarantees the maximum reduction of inevitable delays with relatively low migration costs. Simulation results show that the integrated migration policy averagely outperforms the conventional mechanisms by 26% in makespan and 33% in throughput, respectively, under various job's burstiness.

Optimal Fiscal Policy When Migration Is Feasible

Abstract This paper investigates how the feasibility of migration affects governments' optimal fiscal policies. We assume that households migrate toward economies where their welfare is higher, governments choose taxes and public expenditures to maximize a weighted sum of the households' welfare, welfare is increasing in public expenditures, and only distortionary labor income taxes are available. In isolated economies, the optimal fiscal policy implies that some households are net fiscal contributors, while other households are net fiscal beneficiaries. When households can migrate, however, governments compete for the households which are net fiscal contributors, and modify the fiscal policy in their favor, lowering their taxes and net fiscal contribution, and increasing their welfare. The magnitude of the effect increases with the sensitivity of migration to welfare. In the limiting case of free mobility, all households are zero net fiscal contributors. As to the patterns of migration, the model predicts that, with high migration costs, all households migrate toward the same high-productivity countries, which benefits low-productivity households, whereas with low migration costs, households with different productivities migrate toward different countries, which benefits high-productivity households.

Education, redistribution and the threat of brain drain

We analyze how the threat of brain drain affects redistributive government policies and net incomes of skilled and unskilled workers. Our analysis is based on a model that captures human capital formation, credit market constraints and tax avoidance activities. We characterize how decreasing migration costs for skilled workers shape the time-consistent policies of a government that wants to shift resources from skilled to unskilled workers. Starting from a closed economy, declining migration costs first increase net incomes of both skilled and unskilled workers, and then decrease net incomes of all households. There is a conflict of interest only for very low migration costs. In this case, skilled workers start to benefit from a further rise in their mobility, but now at the expense of the unskilled labor force.

Why is Employment Protection Stricter in Europe than in the United States?

I argue that the reason why the United States prefers a lower level of employment protection than the European countries lies in the differences in gains and costs from geographical mobility. I present a model in which labour migration and employment protection are both determined endogenously. The labour market is modelled within a matching framework, where the employment protection reduces both the job finding and job firing rates. Countries with low migration costs and high economic heterogeneity may prefer no employment protection so that workers can move quickly to better horizons rather than being maintained in low productive activities.

Why is Employment Protection Stricter in Europe than in the US?

In this paper, we argue that the reason why the United States prefer a lower level of employment protection than the European countries lies in the differences in gains and costs from geographical mobility.We present a model where labor migration and employment protection are both determined endogenously.The labor market is modeled within a matching framework, where the employment protection reduces both the job finding and firing rates.Countries with low migration costs and high economic heterogeneity may prefer no employment protection so that workers can move quickly to better horizons rather than being maintained in low productive activities.