Dynamic Requests Research Articles

Online Microservice Orchestration for IoT via Multiobjective Deep Reinforcement Learning

By providing loosely coupled, lightweight, and independent services, the microservice architecture is promising for large-scale and complex service provision requirements in the Internet of Things (IoT). However, it requires more fine-grained resource management and orchestration for service provision. Most of the existing microservice orchestration solutions are based on those designed for the traditional cloud. They can only provide coarse-grained resource allocation using possibly conflicting weighted objectives. In this article, we present a fine-grained microservice orchestration approach to provide services online for dynamic requests of IoT applications. By using a fine-grained resource model of energy cost and service end-to-end response time of orchestrated microservices, we formulate the microservice orchestration problem as a multiobjective Markov decision process. We then propose a multiobjective optimization solution based on deep reinforcement learning (DRL) to simultaneously reduce energy consumption and response time. Through extensive experiments, our proposed algorithm presents significant performance results than the state of the art. To the best of our knowledge, this is the first work that addresses microservice orchestration using DRL for multiple conflicting objectives.

Profit Driven Service Provisioning in Edge Computing via Deep Reinforcement Learning

With the integration of Mobile Edge Computing (MEC) and Network Function Virtualization (NFV), service providers are able to provide low-latency services to mobile users for profit. In this paper, we study the online service placement and request assignment problem in an MEC network, where service requests arrive one by one without the knowledge of future arrivals, and each arrived request demands a specific service with a tolerable service delay requirement with the aim to maximize the profit of the service provider, through admitting as many service requests as possible for a given monitoring period. This optimization objective is achieved by assigning service requests to appropriate cloudlets in the MEC network, pre-installing service instances into cloudlets to shorten service delays, and accommodating new services by revoking some idle service instances from cloudlets due to limited computing resources in MEC networks. In this paper, we first show that the problem is NP-hard. We then devise an efficient deep reinforcement learning algorithm for the online service placement and request assignment problem that consists of a deep reinforcement learning-based prediction mechanism for dynamic service placement, followed by a dynamic request assignment procedure to assign requests to cloudlets. We finally evaluate the performance of the proposed algorithms by conducting experiments through simulations. Simulation results demonstrate that the proposed algorithm is promising, improving performance by 46.8% compared with that of the comparison algorithms.

The real-time on-demand bus routing problem: The cost of dynamic requests

The real-time on-demand bus routing problem (ODBRP) supports the online routing of buses in a large-scale ride-sharing system. Given are a set of buses with fixed capacity, a set of bus stations and a set of transportation requests, only part of which are known before the planning horizon. A request consists of a set of possible departure and arrival stations, as well as an earliest departure and latest arrival time. The aim is to (1) assign each passenger to a departure and arrival bus station and (2) develop a set of bus routes to fulfill each request within its time window while minimizing the sum of the total waiting time and the total user ride time.Including the possibility for requests to be issued after the start of the planning horizon, i.e., when buses have already started servicing other requests, requires a dynamic re-optimization of a partially executed solution. Compared to the case in which all requests are known beforehand, the solution quality, expressed as the total waiting and user ride time, is expected to decline. This decline in objective function value can be seen as the “cost” of the dynamic requests. In this paper, we introduce the real-time ODBRP as an extension of its static problem variant and present a heuristic to deal with dynamic requests. In addition, an extensive set of experiments allows us to conclude that dynamic requests indeed lead to higher waiting and user ride times, especially for passengers who submit their request at the last minute. Passengers are therefore encouraged, if possible, to send their request well in advance, as this results in lower and more stable service promises, higher customer satisfaction, and higher revenues for the operating on-demand bus company.

Solving large-scale dynamic vehicle routing problems with stochastic requests

Dynamic vehicle routing problems (DVRPs) arise in several applications such as technician routing, meal delivery, and parcel shipping. We consider the DVRP with stochastic customer requests (DVRPSR), in which vehicles must be routed dynamically with the goal of maximizing the number of served requests. We model the DVRPSR as a multi-stage optimization problem, where the first-stage decision defines route plans for serving scheduled requests. Our main contributions are knapsack-based linear models to approximate accurately the expected reward-to-go, measured as the number of accepted requests, at any state of the stochastic system. These approximations are based on representing each vehicle as a knapsack with a capacity given by the remaining service time available along the vehicle’s route. We combine these approximations with optimal acceptance and assignment decision rules and derive efficient and high-performing online scheduling policies. We further leverage good predictions of the expected reward-to-go to design initial route plans that facilitate serving dynamic requests. Computational experiments on very large instances based on a real street network demonstrate the effectiveness of the proposed methods in prescribing high-quality offline route plans and online scheduling decisions.

A cost-driven online auto-scaling algorithm for web applications in cloud environments

Today, many web application service providers rely on clouds to deploy applications to serve users. Generally, request arrivals faced by web applications are dynamic and uncertain. When a service provider deploys web applications in clouds, for saving costs, it needs to flexibly rent cloud VM (Virtual Machine) instances based on dynamic request arrivals. However, renting an instance too early may incur more rental fees for the new instance being added incorrectly due to few future requests, and renting an instance too late may incur more penalty fees for SLA (service-level agreement) violations due to too many future requests, which indicates that an arbitrary instance scaling decision will incur more costs. For making optimal instance scaling decisions, future request arrival rate curves are needed, but it is generally very hard to predict them precisely. To solve this problem, in this paper, we propose a cost-driven online auto-scaling algorithm which can make optimized instance rental decisions without requiring future knowledge. We show theoretically that the proposed algorithm can achieve a guaranteed competitive ratio which is less than 2. Eventually, we verify the effectiveness of our online auto-scaling algorithm via extensive experiments using workload data which can simulate real end users.

A green dynamic TSP with detailed road gradient dependent fuel consumption estimation

The rapid transition to the digitalization of firms directs companies to speed up their decision-making processes and make technology-based decisions. The increasing environmental concerns and fossil-fuel scarcity force companies to employ new logistical aims, such as reducing fuel consumption and greenhouse gas emission in addition to the traditional cost minimization and profit maximization objectives. Traditional assumptions in fuel consumption calculation have been left not only to reach the objectives of the company and make consistent delivery plans but also to increase the solution quality of real-life problems. From this point of view, this study contributes to the related literature by developing a decision support model for a Travelling Salesman Problem that considers dynamic customer requests and realistic road gradients of the entire road network while calculating fuel consumption from transportation operations. The applicability of the developed model has been shown with a real-life case study in which laboratory samples have been collected and transferred from local family clinics to a central laboratory. Several objective functions are employed to demonstrate the benefits of considering realistic road gradients. The results of the numerical analyses on static and dynamic instances illustrate the benefits of respecting realistic road gradients in fuel consumption and addressing dynamic requests during delivery operations.

A Research on Routing and Spectrum Allocation Algorithm for Elastic Optical Networks Based on Deep Learning

Dynamic allocation request and spectrum release will lead to spectrum fragmentation, which will affect the allocation of subsequent services and spectrum resource utilization of elastic optical network. This paper proposes a new routing and spectrum allocation algorithm based on deep learning, which will find the best routing and spectrum allocation method for a specific network, so as to improve the overall network performance. Simulation results show that compared with the traditional resource allocation strategy, the neural network model used in this paper can improve the degree of spectrum fragmentation and reduce the network blocking probability.

Dynamic Allocation of SDN Controllers in NFV-Based MEC for the Internet of Vehicles

The expected huge amount of connected cars and applications with varying Quality of Service (QoS) demands still depend on agile/flexible networking infrastructure to deal with dynamic service requests to the control plane, which may become a bottleneck for 5G and Beyond Software-Defined Network (SDN) based Internet of Vehicles (IoV). At the heart of this issue is the need for an architecture and optimization mechanisms that benefit from cutting edge technologies while granting latency bounds in order to control and manage the dynamic nature of IoV. To this end, this article proposes an autonomic software-defined vehicular architecture grounded on the synergy of Multi-access Edge Computing (MEC) and Network Functions Virtualization (NFV) along with a heuristic approach and an exact model based on linear programming to efficiently optimize the dynamic resource allocation of SDN controllers, ensuring load balancing between controllers and employing reserve resources for tolerance in case of demand variation. The analyses carried out in this article consider: (a) to avoid waste of limited MEC resources, (b) to devise load balancing among controllers, (c) management complexity, and (d) to support scalability in dense IoV scenarios. The results show that the heuristic efficiently manages the environment even in highly dynamic and dense scenarios.

Distributed Multi-Agent Meta Learning for Trajectory Design in Wireless Drone Networks

In this paper, the problem of the trajectory design for a group of energy-constrained drones operating in dynamic wireless network environments is studied. In the considered model, a team of drone base stations (DBSs) is dispatched to cooperatively serve clusters of ground users that have dynamic and unpredictable uplink access demands. In this scenario, the DBSs must cooperatively navigate in the considered area to maximize coverage of the dynamic requests of the ground users. This trajectory design problem is posed as an optimization framework whose goal is to find optimal trajectories that maximize the fraction of users served by all DBSs. To find an optimal solution for this non-convex optimization problem under unpredictable environments, a value decomposition based reinforcement learning (VD-RL) solution coupled with a meta-training mechanism is proposed. This algorithm allows the DBSs to dynamically learn their trajectories while generalizing their learning to unseen environments. Analytical results show that, the proposed VD-RL algorithm is guaranteed to converge to a locally optimal solution of the non-convex optimization problem. Simulation results show that, even without meta-training, the proposed VD-RL algorithm can achieve a 53.2% improvement of the service coverage and a 30.6% improvement in terms of the convergence speed, compared to baseline multi-agent algorithms. Meanwhile, the use of the meta-training mechanism improves the convergence speed of the VD-RL algorithm by up to 53.8% when the DBSs must deal with a previously unseen task.

Simulating Extraocular Muscle Dynamics. A Comparison between Dynamic Implicit and Explicit Finite Element Methods

The finite element method has been widely used to investigate the mechanical behavior of biological tissues. When analyzing these particular materials subjected to dynamic requests, time integration algorithms should be considered to incorporate the inertial effects. These algorithms can be classified as implicit or explicit. Although both algorithms have been used in different scenarios, a comparative study of the outcomes of both methods is important to determine the performance of a model used to simulate the active contraction of the skeletal muscle tissue. In this work, dynamic implicit and dynamic explicit solutions are presented for the movement of the eye ball induced by the extraocular muscles. Aspects such as stability, computational time and the influence of mass-scaling for the explicit formulation were assessed using ABAQUS software. Both strategies produced similar results regarding range of movement of the eye ball, total deformation and kinetic energy. Using the implicit dynamic formulation, an important amount of computational time reduction is achieved. Although mass-scaling can reduce the simulation time, the dynamic contraction of the muscle is drastically altered.

Passenger-centric vehicle routing for first-mile transportation considering request uncertainty

First-mile transportation provides convenient transit service for passengers to travel from their homes, workplaces, or public institutions to a public transit station that is located beyond comfortable walking distance. This paper studies the Passenger-Centric Vehicle Routing for First-Mile Transportation (PCVR-FMT) problem to plan optimal vehicle routes that provide a high quality of service (QoS) to enhance passenger experience. We consider a practical scenario with 1) deterministic requests, consisting of travel requests that are known in advance (e.g., submitted by passengers through a mobile application), and 2) uncertain requests (e.g., new travel requests generated during service operation). We formally formulate the PCVR-FMT problem to maximize the QoS in terms of the passenger waiting and riding time (for both deterministic and uncertain requests) while jointly considering the traditional constraints on the pickup time window as well as vehicle capacity. We developed an Ant-Colony Optimization algorithm based on a novel Dynamic Request Driven scheme for vehicle route construction, denoted as DRD-ACO, to efficiently solve PCVR-FMT. DRD-ACO relies on a novel request-location graph that models both deterministic and uncertain requests, which enable the ants to share information across different generations via pheromone for dealing with uncertain requests. A dynamic seat reservation mechanism is devised to determine a suitable number of reserved seats to deal with uncertain requests. A time window expansion mechanism is developed to selectively expand passengers’ pickup time window if the number of vehicles is insufficient. The effectiveness of the proposed methods is evaluated using Singapore’s road network and synthetic travel requests generated from real bus travel demands. Some of the travel requests are treated as uncertain requests. The results show that on relatively small size instances, our methods obtain solutions that are close to the optimal ones computed by CPLEX, with deviations ranging from only 2.55% to 8.79%. In comparison to other baselines, our methods achieve better results in the objective function, ratio of served uncertain passengers, as well as the expansion in waiting time due to request uncertainty, for both small-scale and large-scale problem instances.

Learning-Based Cognitive Hitless Spectrum Defragmentation for Dynamic Provisioning in Elastic Optical Networks

The rapid evolution of data transmission has posed unprecedented challenges to the dynamic resource provisioning in optical networks. In this letter, we investigate the dynamic spectrum defragmentation problem in elastic optical networks, and propose a novel deep reinforcement learning based solution Deep-DF to achieve self-adaptive spectrum optimization. Compared to the existing pre-fixed heuristics tailored to only immediate optimizations for the current network state, the key advantage of Deep-DF is that it can be trained to identify and perform hitless defragmentation operations that lead to longer-term optimal dynamic request provisioning, and continuously reoptimize and readapt as traffic changes. The simulation results validate the superior performance of the proposed Deep-DF over the benchmark heuristics, and its robustness against heterogeneous and non-stationary traffic distributions and traffic changes.

Vehicle Routing and Scheduling of Flex-Route Transit under a Dynamic Operating Environment

To improve the reliability, responsiveness, and productivity of the flex-route transit service, this paper investigates the vehicle scheduling and routing problem under a dynamic operating environment. First, we discuss the new operating polices after the introduction of intelligent transportation systems (ITSs), including automatic vehicle location (AVL) system, mobile data terminal (MDT), and computer-aided dispatch (CAD) system. Second, a mixed integer programming (MIP) formulation is employed to solve the offline routing problem. Third, an online scheduling scheme is presented to tackle different dynamic events, such as dynamic requests, travel time fluctuations, cancellations of requests, and customer no-shows. Finally, simulation experiments based on a real-life flex-route transit service are conducted to assess the influence of different dynamic events. The results demonstrate that the proposed scheduling scheme is reliable for coping with various dynamic events, and our findings can be used to guide the policy making of flex-route transit services.

Bi-objective optimization model for the heterogeneous dynamic dial-a-ride problem with no rejects

This work proposes a bi-objective mathematical optimization model and a two-stage heuristic for a real-world application of the heterogeneous Dynamic Dial-a-Ride Problem with no rejects, i.e., a patient transportation system. The problem consists of calculating route plans to meet a set of transportation requests by using a given heterogeneous vehicle fleet. These transportation requests can be either static or dynamic, and all of them must be attended to. In the first stage of the proposed heuristic, the problem’s static part is solved by applying a General Variable neighborhood Search based algorithm. In the second stage, the dynamic requests are dealt with by implementing a simple insertion heuristic. We create different instances based on the real data provided by a Brazilian city’s public health care system and test the proposed approach on them. The analysis of the results shows that the higher the level of dynamism, i.e., the number of urgent requests on each instance, the smaller the objective function value will be in the static part. The results also demonstrate that a higher level of dynamism increases the chance of a time window violation happening. Besides, we use the weighted sum method of the two conflicting objectives to analyze the trade-off between them and create an approximation for the Pareto frontier.

Dynamic SDN Control Plane Request Assignment in NFV Datacenters

Owing to powerful programmability, software defined networking (SDN) is well aligned with the requirements of NFV datacenters. Although the mainstream SDN frameworks can effectively respond to the requests of control plane, the load distribution among controllers is not balanced due to the request dynamics and request diversity. The existing solutions balance the load of control plane by migrating switches between controllers. However, these solutions are based on the binding between switches and controllers, so they are difficult to adapt to the dynamic and diverse requests. In this paper, we propose a new framework to decouple the binding. The new framework performs modular management for request queues. A complete request queue is provided for each type of request between each switch controller pair, so that the assignment between requests is independent of each other. Based on the proposed framework, we transform the request assignment problem into a variant of the scheduling problem in a Stochastic Processing Network (SPN), and propose a Maximum Pressure Policy (MPP) which can provide runtime guarantees on request throughput and response latency. To fit with the constraints inherent to large-scale deployment, we propose a distributed version of MPP, named DMPP. DMPP runs in local state on each switch and performs scheduling logic for batch requests. We implement a protosystem of our solution and evaluate it on the settings representing real-world scenarios. The results show that our solution can provide guarantees on request throughput and response latency, and significantly outperforms state-of-the-art solutions through a more efficient resource usage

The online vehicle routing problem with occasional drivers

We investigate a vehicle routing problem in which customer requests are either known in advance with respect to the planning of the distribution, or they arrive online during the distribution process. Each request is associated with a time window. The company managing the distribution has a given fleet of vehicles to serve the customers, and, in addition, occasional drivers are available to perform the service, i.e., private citizens who are willing to distribute some customer orders in exchange for a compensation. Each occasional driver specifies the time window in which he/she is available. A penalty is incurred when violating time windows as well as when a request is not served. The objective of the company is to determine the distribution plan that minimises the distribution cost, which is given by the sum of the cost of regular vehicles, the compensation paid to the occasional drivers and the penalty cost. In addition, we design and implement a solution method which is based on an insertion algorithm evaluating each request singularly. Albeit being simple, this approach allows to handle dynamic requests and to adjust the distribution plan in real-time. Then, we propose a re-optimisation approach in which the solution constructed through the insertion algorithm is periodically passed to a variable neighborhood search algorithm. In a detailed computational study, we analyse the behavior of the proposed solution approaches. In addition, we evaluate the impact of dynamic information through a comparison of online approaches with an a priori information scenario.

A Workflow-Aided Internet of Things Paradigm with Intelligent Edge Computing

In this article, we propose a workflow-aided Internet of things (WIoT) paradigm with intelligent edge computing (IEC) to automate the execution of IoT applications with dependencies. Our design primarily targets at reducing the latency of the IoT systems from two perspectives. To reduce the latency from an application perspective, we develop a WIoT paradigm to orchestrate various IoT applications in a programming way. To reduce the latency from a computation perspective, we propose a novel IEC framework to execute latency-sensitive IoT tasks at the edge network. We put forth a deep reinforcement learning algorithm to adaptively allocate the edge resources to the dynamic requests, aiming to provide the best quality of service for terminal users in real-time. Furthermore, we design a software platform to implement the proposed WIoT with IEC. Experimental results demonstrate that WIoT with IEC can significantly reduce the service latency and improve the network throughput, compared with the traditional cloud-based IoT systems.

Dynamic multi-objective evolutionary algorithm for IoT services

The primary goal of the Internet of things(IoT) is to provide people with anywhere services in real life. But intelligent IoT shouldn’t only provide services, but also consider how to allocate heterogeneous resources reasonably, which has become a very challenging problem. To obtain the best resource allocation scheme, it is crucial to minimize the service cost and service time. Since the two objectives are contradictory, we have modelled IoT services as a dynamic multi-objective optimization problem. Then a dynamic multi-objective evolutionary algorithm for dynamic IoT services(dMOEA/DI) is proposed. In dMOEA/DI, we have designed operators such as the appropriate encoding method, dynamic detection operator, filtering strategy, differential evolution, and polynomial mutation. Based on the single service strategy and collaborative service strategy, experimental research is performed on the agricultural IoT services with dynamic requests under different distributions. The simulation experimental results prove that dMOEA/DI performs better than the contrasted algorithms on the IoT service optimization problems.

An infrastructure for embedded systems using task scheduling

Task scheduling in heterogeneous environments such as cloud data centers is considered to be an NP-complete problem. Efficient task scheduling will lead to balance the load on the virtual machines (VMs) thereby achieving effective resource utilization. Hence there is a need for a new scheduling framework to perform load balancing amid considering multiple quality of service (QoS) metrics such as makespan, response time, execution time, and task priority. Multi-core Web server is difficult to achieve dynamic balance in the process of remote dynamic request scheduling, so it is necessary to improve it based on the traditional scheduling algorithm to enhance the actual effect of the algorithm. This article do research on the multi-core Web server, Focusing on multi-core Web server queuing model. On this basis, the author draws the drawbacks of the multi-core Web server in the remote dynamic request scheduling algorithm, and improves the traditional algorithm with the demand analysis. Not only it overcomes the drawbacks of traditional algorithms, but also promotes the system threads carrying the same amount of tasks, and promotes the server being always in a dynamic balance. On the basis of this, it achieves an effective solution to customer requests.

Smart Edge Caching-Aided Partial Opportunistic Interference Alignment in HetNets

The development of the next-generation wireless networks are regarded as the essentials to embrace of Internet of Things (IoT) and edge computing in heterogeneous networks (HetNets). Due to the the spectrum scarcity problem and the large number of connectivity demand of IoT users, intelligent interference management for IoT is worthy of thorough investigation and should be well discussed with consideration on edge computing in heterogeneous networks (HetNets). Two crucial challenges in the context are: 1) placing edge cache based on dynamic request of IoT users, and 2) cache-enabled interference management with time-varying wireless channels. In this paper, we proposed smart edge caching-aided partial opportunistic interference alignment(POIA) with deep reinforcement learning for IoT downlink system in HetNets. Towards this end, the proposed scheme can update the base station (BS) cache dynamically, and then select the optimal cache-enabled POIA user group considering the time-varying user’s requests and time-varying wireless channels. To solve this problem efficiently, the reinforcement learning is exploited that can take advantage of a deep Q-learing to replace the system action. Extensive evaluations demonstrate that the proposed method is effectiveness according to sum rate and energy efficiency of IoT downlink transmission for HetNets.