Bandwidth Cost Research Articles

Mitigating the Impact of Data Sampling on Social Media Analysis and Mining

The last decade has witnessed the explosive growth of online social media in users and contents. Due to the unprecedented scale and the cascading power of the underlying social networks, social media has created a new paradigm for sharing information, broadcasting breaking news, and reporting real-time events by any user from anywhere at any time. Many popular social media sites including Twitter provide streaming data services by standard APIs to the broad researcher and developer communities. Given the sheer data volume, rapid velocity, and feature variety of online social media, these sites often supply only a sampled set of streaming data, rather than the full data set to reduce the resource cost of computations, storage, and network bandwidth. In light of the substantial impact of sampling in Twitter data stream, this article explores a combination of spectral clustering, locality-sensitive hashing (LSH), latent Dirichlet allocation (LDA) topic modeling, and differential equation modeling to mitigate the impact of sampling on social media data analysis, in particular on detecting real-world events and predicting information diffusion. Our extensive experiments demonstrate that our proposed method is able to detect effectively the real-time emerging events and predict accurately the cascading pattern of these events from the 1% sampled Twitter data stream. To the best of our knowledge, this article is the first effort to introduce a systematic methodology to study and mitigate the impact of data sampling on social media analysis and mining.

Verifying Rule Enforcement in Software Defined Networks With REV

Software defined networking (SDN) reshapes the ossified network architectures, by decoupling the control plane and data plane. Due to such a decoupling, SDN assumes that rules issued by the control plane are always correctly enforced by the data plane. However, this assumption breaks as an adversary can prevent the data plane from enforcing the rules, by exploiting the vulnerabilities of switch OS and control channel. The serious consequence is that packets may deviate from their original paths, thereby violating critical security policies like access control. To this end, this paper introduces rule enforcement verification (REV) , which enables the controller to check whether switches have correctly enforced the rules that it issues. Since using message authentication code (MAC) can incur heavy switch-to-controller traffic, we propose the compressive MAC , which lets switches compress MACs before reporting to the controller, thereby significantly reducing the bandwidth cost. Finally, we propose a heuristic flow selection algorithm, which allows the controller to verify much less flows for rule coverage. We implement REV based on Open vSwitch with DPDK, and use experiments to show: (1) by using compressive MAC, REV achieves a 97% reduction in switch-to-controller traffic, and an $8\times $ increase in verification throughput; (2) by using the heuristic flow selection algorithm, REV can reduce the number of flows to verify by 40%–60%.

Optimizing Geo-Distributed Data Analytics with Coordinated Task Scheduling and Routing

Recent trends show that cloud computing is growing to span more and more globally distributed datacenters. For geo-distributed datacenters, there is an increasingly need for scheduling algorithms to place tasks across datacenters, by jointly considering WAN traffic and computation. This scheduling must deal with situations such as wide-area distributed data, data sharing, WAN bandwidth costs and datacenter capacity limits, while also minimizing makespan. However, this scheduling problem is NP-hard. We propose a new resource allocation algorithm called HPS + , an extension to Hypergraph Partition-based Scheduling. HPS + models the combined task-data dependencies and data-datacenter dependencies as an augmented hypergraph, and adopts an improved hypergraph partition technique to minimize WAN traffic. It further uses a coordination mechanism to allocate network resources closely following the guidelines of task requirements, for minimizing the makespan. Evaluation across the real China-Astronomy-Cloud model and Google datacenter model show that HPS + saves the amount of data transfers by upto 53 percent and reduces the makespan by 39 percent compared to existing algorithms.

Silence

The cost of communication is a substantial factor affecting the scalability of many distributed applications. Every message sent can incur a cost in storage, computation, energy, and bandwidth. Consequently, reducing the communication costs of distributed applications is highly desirable. The best way to reduce message costs is by communicating without sending any messages whatsoever. This article initiates a rigorous investigation into the use of silence in synchronous settings, in which processes can fail. We formalize sufficient conditions for information transfer using silence, as well as necessary conditions for particular cases of interest. This allows us to identify message patterns that enable communication through silence. In particular, a pattern called a silent choir is identified, and shown to be central to information transfer via silence in failure-prone systems. The power of the new framework is demonstrated on the atomic commitment problem (AC). A complete characterization of the tradeoff between message complexity and round complexity in the synchronous model with crash failures is provided, in terms of lower bounds and matching protocols. In particular, a new message-optimal AC protocol is designed using silence, in which processes decide in three rounds in the common case. This significantly improves on the best previously known message-optimal AC protocol, in which decisions were performed in Θ( n ) rounds. And in the naked light I saw Ten thousand people, maybe more People talking without speaking … People writing songs that voices never share And no one dared Disturb the sound of silence Paul Simon, 1964

Latency-Constrained Cost-Minimized Request Allocation for Geo-Distributed Cloud Services

Latency to end-users and regulatory requirements push cloud providers to operate many datacenters all around the globe to host their cloud services. An emerging problem under such geo-distributed architecture is to assign each user request to an appropriate datacenter to benefit both cloud providers (e.g., low bandwidth cost) and end-users (e.g., low latency)—known as request allocation. However, prior request allocation solutions have significant limitations: they either focus only on optimizing the benefits for one entity (e.g., providers or users), or ignore some practical yet indispensable factors (e.g., heterogeneous latency requirements of different users and diverse per unit bandwidth cost among different datacenters) when optimizing benefits for both entities. In this paper, we study the problem of minimizing the total bandwidth cost for cloud service providers while guaranteeing the latency requirement for end-users. Specifically, we formulate an integer programming with consideration of the diversities in both the delay of requests and per unit bandwidth cost of datacenters. To efficiently and practically solve this problem, we first relax the integer programming into a continuous convex optimization and then take the advantages of random sampling to enforce the solution to be a feasible one for the original integer programming. We have conducted rigorous theoretical analysis to prove that our algorithm can provide a considerable good competitive ratio. Extensive simulations demonstrate that our proposed algorithm can reduce the total bandwidth cost by 30% while guaranteeing the latency requirements of all requests, as compared to conventional methods.

A Novel Strategy to Achieve Bandwidth Cost Reduction and Load Balancing in a Cooperative Three-Layer Fog-Cloud Computing Environment

Recently, IoT (Internet of Things) has been an attractive area of research to develop smart home, smart city environment. IoT sensors generate data stream continuously and majority of the IoT based applications are highly delay sensitive. The initially used cloud based IoT services suffers from higher delay and lack of efficient resources utilization. Fog computing is introduced to improve these problems by bringing cloud services near to edge in small scale and distributed nature. This work considers an integrated fog-cloud environment to minimize resource cost and reduce delay to support real-time applications at a lower operational cost. We first present a cooperative three-layer fog-cloud computing environment, and propose a novel optimization model in this environment. This model has a composite objective function to minimize the bandwidth cost and provide load balancing. We consider balancing load in both links' bandwidth and servers' CPU processing capacity level. Simulation results show that our framework can minimize the bandwidth cost and balance the load by utilizing the cooperative environment effectively. We assign weight factors to each objective of the composite objective function to set the level of priority. When minimizing bandwidth cost gets higher priority, at first, the demand generated from the traffic generator sensors continues to be satisfied by the regional capacity of layer-1 fog. If the demand of a region goes beyond the capacity of that region, remaining demand is served by other regions layer-1 fog, then by layer-2 fog, and finally by the cloud. However, when load balancing is the priority, the demand is distributed among these resources to reduce delay. Link level load balancing can reduce the queueing delay of links while server level load balancing can decrease processing delay of servers in an overloaded situation. We further analyzed how the unit bandwidth cost, the average and maximum link utilization, the servers' resources utilization, and the average number of servers used vary with different levels of priority on different objectives. As a result, our optimization formulation allows tradeoff analysis in the cooperative three-layer fog-cloud computing environment.

Video Stream Distribution Scheme Based on Edge Computing Network and User Interest Content Model

In view of the inherent defects of traditional network protocols, unreasonable video content distribution process, content scheduling and routing mechanism need to be optimized in the current edge computing (EC) network. A centralized control EC network architecture of named data networking (NDN) is proposed in this paper. EC nodes are deployed on the edge of the network to provide caching and computing capabilities, and distribute video streams. So that transmission delay is reduced, bandwidth cost is saved, and the quality of experience (QoE) is improved. Firstly, the communication mechanism of NDN is introduced to distribute content and improve the efficiency of content distribution. In the scheme, the content distribution mechanism of video stream is improved in a distributed way. Different bit rate versions of video clips are distributed and buffered on the distributed EC nodes, so as to avoid redundant cache and save storage resources. In addition, the user interest content model is constructed. That is, video is divided into different categories according to content and user content preference model. Finally, with the limited cache and computing resources of EC nodes, the software defined network (SDN) mechanism is introduced for centralized management and control. And it provides a solution for content scheduling problem under resource constraints. In the scheme, the content distribution mode is decided by the effective arrangement of network resources, and the user's request is processed and responded cooperatively to avoid redundant cache and transmission. Based on the selection of forwarding response mode, the distributed characteristics of EC nodes and the routing and forwarding mechanism of NDN are fully utilized to provide users with response. It is shown by the experimental results that the proposed scheme optimizes the load on the network side and the experience of the client to a certain extent.

Computation Migration and Resource Allocation in Heterogeneous Vehicular Networks: A Deep Reinforcement Learning Approach

With the development of 5G technology, the requirements for data communication and computation in emerging 5G-enabled vehicular networks are becoming increasingly stringent. Computation-intensive or delay-sensitive tasks generated by vehicles need to be processed in real time. Mobile edge computing (MEC) is an appropriate solution. Wireless users or vehicles can offload computation tasks to the MEC server due to it has strong computation ability and is closer to the wireless users or vehicles. However, the communication and computation resources of the single MEC are not sufficient for executing the continuously generated computation-intensive or delay-sensitive tasks. We consider migrating computation tasks to other MEC servers to reduce the computation and communication pressure on current MEC server. In this article, we construct an MEC-based computation offloading framework for vehicular networks, which considers time-varying channel states and stochastically arriving computation tasks. To minimize the total cost of the proposed MEC framework, which consists of the delay cost, energy computation cost, and bandwidth cost, we propose a deep reinforcement learning-based computation migration and resource allocation (RLCMRA) scheme that requires no prior knowledge. The RLCMRA algorithm can obtain the optimal offloading and migration policy by adaptive learning to maximize the average cumulative reward (minimize the total cost). Extensive numerical results show that the proposed RLCMRA algorithm can adaptively learn the optimal policy and outperform four other baseline algorithms.

Unmanned-Aerial-Vehicle-Assisted Computation Offloading for Mobile Edge Computing Based on Deep Reinforcement Learning

Users in heterogeneous wireless networks may generate massive amounts of data that are delay-sensitive or require computation-intensive processing. Owing to computation ability and battery capacity limitations, wireless users (WUs) cannot easily process such data in a timely manner, and mobile edge computing (MEC) is increasingly being used to resolve this issue. Specifically, data generated by WUs can be offloaded to the MEC server for processing, which has greater computing power than WUs. However, as the location of MEC servers is fixed, unmanned aerial vehicles (UAVs) have been considered a promising solution in heterogeneous wireless networks. In this study, we design an UAV-assisted computation offloading scheme in an MEC framework with renewable power supply. The proposed model considers the instability of energy arrival, stochastic computation tasks generated by WUs, and a time-varying channel state. Owing to the complexity of the state, it is difficult to use traditional Markov decision process (MDP) with complete prior knowledge for offloading optimization. Accordingly, we propose UAV-assisted computation offloading for MEC based on deep reinforcement learning (UACODRL) to minimize the total cost, which is the weighted sum of the delay, energy consumption, and bandwidth cost. We first use the K-Means algorithm for classification to reduce the dimension of the action space. Subsequently, we use UACODRL to find the near-optimal offloading scheme to minimize the total cost. Simulations demonstrate that UACODRL converges satisfactorily and performs better than four baseline schemes with different parameter configurations.

МЕТОДИКА ОЦЕНКИ КОЭФФИЦИЕНТА ГОТОВНОСТИ СИСТЕМЫ С РЕЗЕРВИРОВАНИЕМ. ПАССИВНЫЙ РЕЖИМ

The use of fiber-optic communication lines allows for the transfer of huge amounts of information per unit of time. Even short-term failures significantly affect the quality of services provided. The level of network reliability depends on how resilient the network infrastructure is to failures and how quickly recovery occurs after them. Reliability guarantees are the main component of quality of service agreements between operators and consumers. Most often, redundancy is used to ensure the required level of reliability indicators. The disadvantage of this approach is the high cost of network bandwidth, the advantage - is a short time to switch at the reserve after a failure is detected. In order to reduce costs, in some cases, the 1:N backup option is used (one backup path is allocated for N working paths). In this paper, we consider the case of 1:2, when one backup path is allocated for two working paths. In this case active and passive switching modes are possible. When switching is active, if both working paths fail, the traffic of one of them switches to the backup path. After restoring the working path, traffic switches to the restored path, i.e. the backup path is freed. It is replaced by traffic from another failed path. A special feature of passive mode consists in that after the backup path is released, it does not deal with the traffic of the second failed path. There are several ways to restore traffic over a failed work path:. the second working path starts to recover after the first one is restored; the second working path starts to recover simultaneously with the first one. In either case, the traffic of the first path is switched back from the backup to the working path after it is restored. This paper presents a method for calculating the conditional probability that the backup path will be used for transmitting traffic from one of the working paths if both paths fail. The formula for calculating the unavailability factor of working paths is given.

Transmitting extra bits by rotating signal constellations

In this Letter, the authors propose a novel low-density parity-check (LDPC) coding scheme to transmit extra bits aided by rotated signal constellations without any additional cost in transmission power or bandwidth. In the proposed scheme, the LDPC coded data are modulated by a rotated two-dimensional signal constellation, in which the rotation angle is specified by the given extra bits. At the receiver, the rotation angle is estimated with the aid of the statistical learning of the syndrome of the LDPC code. After recovering the rotation angle, the coded payload data can be decoded by the LDPC decoder. The simulation results show that, for an LDPC code of length 2304, up to four extra bits can be transmitted with negligible influence on the reliability of the LDPC coded data.

Investigation on optimisation, prioritising and weight allocation techniques for load balancing and controlling multimedia traffic in wireless mesh network

Wireless mesh network (WMN) is a type of wireless ad hoc network where radio nodes are wirelessly connected within sizable geographic area through mesh topology. The best path is identified by routing metrics to ensure efficient data transmission from source node to destination node. The performance of routing is concern with set of routing constraints such as bandwidth, network delay, energy consumption, load balancing and communication cost. The routing constraints are needed to be resolved to improve the WMN performance. In WSN, network load balancing plays a significant role to achieve effective routing without traffic. The energy consumption is an additional factor to provide high-quality data delivery at the destination end. Our research work is concentrated on developing the energy efficient and load balanced routing in WMN.

A Resource Usage Intensity Aware Load Balancing Method for Virtual Machine Migration in Cloud Datacenters

To provide robust infrastructure as a service (IaaS), clouds currently perform load balancing by migrating virtual machines (VMs) from heavily loaded physical machines (PMs) to lightly loaded PMs. The unique features of clouds pose formidable challenges to achieving effective and efficient load balancing. First, VMs in clouds use different resources (e.g., CPU, bandwidth, memory) to serve a variety of services (e.g., high performance computing, web services, file services), resulting in different overutilized resources in different PMs. Also, the overutilized resources in a PM may vary over time due to the time-varying heterogeneous service requests. Second, there is intensive network communication between VMs. However, previous load balancing methods statically assign equal or predefined weights to different resources, which lead to degraded performance in terms of speed and cost to achieve load balance. Also, they do not strive to minimize the VM communications between PMs. We propose a Resource Intensity Aware Load balancing method (RIAL). For each PM, RIAL dynamically assigns different weights to different resources according to their usage intensity in the PM, which significantly reduces the time and cost to achieve load balance and avoids future load imbalance. It also tries to keep frequently communicating VMs in the same PM to reduce bandwidth cost, and migrates VMs to PMs with minimum VM performance degradation. We also propose an extended version of RIAL with three additional algorithms. First, it optimally determines the weights for considering communication cost and performance degradation due to VM migrations. Second, it has a more strict migration triggering algorithm to avoid unnecessary migrations while still satisfying Service Level Objects (SLOs). Third, it conducts destination PM selection in a decentralized manner to improve scalability. Our extensive trace-driven simulation results and real-world experimental results show the superior performance of RIAL compared to other load balancing methods.

Bulk Savings for Bulk Transfers: Minimizing the Energy-Cost for Geo-Distributed Data Centers

With the fast proliferation of cloud computing, major cloud service providers, e.g., Amazon, Google, Facebook, etc., have been deploying more and more geographically distributed data centers to provide customers with better reliability and quality of services. A basic demand in such a geo-distributed data center system is to transfer bulk volumes of data from one data center to another. Geographic distribution and large delay-tolerance of such inter-data-center bulk data transfers provide cloud service providers opportunities to optimize the operating cost. Most existing studies on inter-data-center bulk data transfers focus on minimizing the network bandwidth cost. However, the energy-cost of the bulk data transfers, which also accounts for a large proportion of operating cost in the data centers, still remains unexplored. This is an important problem, especially in the multi-electricity-market environment, where the electricity price exhibits both spatial and temporal diversities. In this paper, we systematically study the problem of how to route and schedule inter-data-center bulk data transfers to minimize the energy-cost for geo-distributed data centers. We model this problem as a min-cost multi-commodity flow problem and develop an efficient two-stage optimization method to solve it. Extensive evaluations with real-life inter-data-center network and electricity prices show that our method brings significant energy-cost savings over existing bulk data transfer methods.

Cooperative Perception for 3D Object Detection in Driving Scenarios Using Infrastructure Sensors

3D object detection is a common function within the perception system of an autonomous vehicle and outputs a list of 3D bounding boxes around objects of interest. Various 3D object detection methods have relied on fusion of different sensor modalities to overcome limitations of individual sensors. However, occlusion, limited field-of-view and low-point density of the sensor data cannot be reliably and cost-effectively addressed by multi-modal sensing from a single point of view. Alternatively, cooperative perception incorporates information from spatially diverse sensors distributed around the environment as a way to mitigate these limitations. This article proposes two schemes for cooperative 3D object detection using single modality sensors. The early fusion scheme combines point clouds from multiple spatially diverse sensing points of view before detection. In contrast, the late fusion scheme fuses the independently detected bounding boxes from multiple spatially diverse sensors. We evaluate the performance of both schemes, and their hybrid combination, using a synthetic cooperative dataset created in two complex driving scenarios, a T-junction and a roundabout. The evaluation shows that the early fusion approach outperforms late fusion by a significant margin at the cost of higher communication bandwidth. The results demonstrate that cooperative perception can recall more than 95% of the objects as opposed to 30% for single-point sensing in the most challenging scenario. To provide practical insights into the deployment of such system, we report how the number of sensors and their configuration impact the detection performance of the system.

A Stackelberg game-based approach for interactions among Internet service providers and content providers

This paper studies non-neutral market share where Internet service providers (ISPs) charge content providers (CPs) for the content delivery. Each provider seeks to maximize its own profit by determining its price and their service qualities (quality of service and credibility of content). First, we utilize a Stackelberg game to study the interactions between ISPs and CPs. We formulate the interactions among multiple ISPs (multiple CPs) as a non-cooperative game. In turn, the subscribers’ demand for the service of a provider depends not only on their strategies, but also upon those proposed by all of its competitors. Then we provide some interesting results regarding the Nash equilibrium of this game. More precisely, we show existence and uniqueness of the Nash equilibrium under some conditions. An iterative and distributed algorithm based on best response dynamics is proposed to achieve the equilibrium point; additionally, in order to quantify how efficient the Nash equilibrium point is, a detailed analysis of the Price of Anarchy is adopted to evaluate the performance of the system at equilibrium. Finally, extensive simulations show convergence of a proposed schema to the Nash equilibrium and show the effect of side payment on providers’ policies and the effect of bandwidth cost on ISPs’ policies.

Optimization-based Offloading Method for Mobile Cloud Computing Environment

Mobile cloud computing (MCC) is a program that should be applied to defeat the hurdles of computing in the mobile environment. Though developing data-intensive purposes, such as Natural Language Processing (NLP) and face recognition, takes more difficulties on mobile cloud computing stages because of data location and high bandwidth cost issues. To overcome these issues, this paper proposes a dynamic task (resources) allocation model to schedule data-intensive applications on mixed resources (public cloud, cloudlets, and mobile devices) computing environments. Efficient task allocation strategy requires to develop by estimating the number of intentions while performing the decisions of allocation, such as fast response and reduced consumption of energy, to obtain the most reliable task allocation providing the requirements of cloud users and increasing the MCC environment performance. In this paper, Cultural Algorithm (CA) based offloading strategy is proposed for obtaining the minimized task execution time by causing smart decisions for allocation. This proposed algorithm has been implemented using a cloudsim toolkit, and the performance is estimated by analyzing with Genetic and greedy algorithm allocation techniques on a collection of parameters like throughput and makespan for scheduling the resource.

Flow monitoring scheme design in SDN

An SDN controller can collect all flow statistics by periodically polling switches in the network. Each polling request contains a match structure identifying one or many flow entries at a switch. A flow is covered if its statistics is collected. In this paper, a new polling mechanism called poll-some is introduced. As compared to the two existing mechanisms, poll-single and poll-all, poll-some aims at collecting the statistics of multiple not-yet-covered flows at a switch using a minimum number of match structures. The problem of finding the minimum number of match structures to cover all the not-yet-covered flows at a switch, or Minimum Match Structure (MMS) problem, is then formulated. An efficient algorithm called Critical Column First (CCF) is designed to solve it. Based on CCF, a new flow monitoring scheme called Lowest-cost Switch First (LSF) is proposed to minimize the controller-switch communication bandwidth cost incurred in collecting flow statistics. LSF consists of two phases, finding the switch to be polled next and determining the best polling mechanism. As compared with an existing greedy algorithm, simulation results show that our LSF not only runs faster but also cut down the bandwidth cost by about 30%.

LFastqC: A lossless non-reference-based FASTQ compressor.

The cost-effectiveness of next-generation sequencing (NGS) has led to the advancement of genomic research, thereby regularly generating a large amount of raw data that often requires efficient infrastructures such as data centers to manage the storage and transmission of such data. The generated NGS data are highly redundant and need to be efficiently compressed to reduce the cost of storage space and transmission bandwidth. We present a lossless, non-reference-based FASTQ compression algorithm, known as LFastqC, an improvement over the LFQC tool, to address these issues. LFastqC is compared with several state-of-the-art compressors, and the results indicate that LFastqC achieves better compression ratios for important datasets such as the LS454, PacBio, and MinION. Moreover, LFastqC has a better compression and decompression speed than LFQC, which was previously the top-performing compression algorithm for the LS454 dataset. LFastqC is freely available at https://github.uconn.edu/sya12005/LFastqC.

Joint beamforming and space‐time coding in limited‐feedback cooperative networks with threshold‐based MIMO selection relaying

AbstractIn this paper, a limited‐feedback decode‐and‐forward cooperative network with multiple relays is considered where each relay is equipped with multiple antennas. To combat error propagation, each multiple‐input multiple‐output relay participates in relaying phase, according to its received signal‐to‐noise ratio. In our scheme, space‐time coding is independently employed at each relay. At relay, the detected symbols are forwarded using a novel beamforming method proposed in this paper. In the proposed technique, relays employ only two‐bit quantized phase information of the channel between relay and destination, which reduces the overhead of the feedback channel. Our results highlight that using a simple maximal ratio combining at destination, the received SNR after combining the space‐time coded signals is enhanced significantly compared with other existing methods. We further investigate the bit error rate performance of the general family of M‐ary modulation schemes and show that the proposed design attains full diversity order without any rate or bandwidth cost. Also, closed form expressions are derived for optimal threshold and end‐to‐end bit error rate of the system. Numerical results illustrate a good agreement between the analytical and simulation results.