Efficient Information Updates in Compute-First Networking via Reinforcement Learning With Joint AoI and VoI

Cell-free massive multiple-input multiple-output (CF-mMIMO) is an emerging beyond fifth-generation (5 G) technology that improves energy efficiency (EE) and removes cell structure limitation by using multiple access points (APs). This study investigates the EE maximization problem. Forming proper cooperation clusters is crucial when optimizing EE, and it is often done by selecting AP–user pairs with good channel quality or aligning AP cache contents with user requests. However, the result can be suboptimal if we determine the clusters based solely on either aspect. This motivates our joint design of user association and content caching. Without knowing the user content preferences in advance, two deep reinforcement learning (DRL) approaches, i.e., single-agent reinforcement learning (SARL) and multi-agent reinforcement learning (MARL), are proposed for different scenarios. The SARL approach operates in a centralized manner which has lower computational requirements on edge devices. The MARL approach requires more computation resources at the edge devices but enables parallel computing to reduce the computation time and therefore scales better than the SARL approach. The numerical analysis shows that the proposed approaches outperformed benchmark algorithms in terms of network EE in a small network. In a large network, the MARL yielded the best EE performance and its computation time was reduced significantly by parallel computing.

In this paper, we consider the stochastic network with dynamic traffic. The spatial distribution of access points (APs) and users are modeled as mutually independent Poisson point processes. Different from most of the previous literature works, which assume that all the APs are fully loaded, we consider the fact that APs having no data to transmit do not generate interference to users. The APs opportunistically share the channel according to the existence of the packet to be transmitted and the proposed interference suppression strategy. In the interference suppression region, only one AP can be active at a time to transmit the packet on the channel and the other adjacent APs keep silent to reduce serious interference. The idle probability of any AP, influenced by the traffic load and availability of the channels, is analyzed. The density of simultaneously active APs in the network is obtained, and the packet loss rate is further elaborated. We reveal the impacts of network features (e.g., AP density, user density, and channel state) and service features (e.g., user request and packet size) on the network performance.

A large scale, distributed video-on-demand (VOD) system allows geographically dispersed residential and business users to access video services, such as movies and other multimedia programs or documents on demand from video servers on a high speed network. In this paper we demonstrate through analysis and simulation the need for a hierarchical architecture for the video-on-demand distribution network. We assume a hierarchical architecture, which fits the existing tree topology used in today's cable TV (CATV) hybrid fiber/coaxial (HFC) distribution networks. We develop a model for the design, configuration, program placement and performance evaluation of such systems. The model takes into account the user behavior, the fact that the user re quests are transmitted over a shared channel before reaching the video server containing the requested program, the finite input/output (I/O) capacity of the video servers, the exponential storage cost of the multimedia programs imposed by the adopted architecture of the video-on-demand distribution network, and finally the communication cost. In addition, our model allows batching of user requests at the video server and we study the effect of batching on the performance of the video server and on the delivered to the user Quality of Service (QoS). The design and evaluation is based on an extensive analytical and simulation study. The results in this paper contribute to the understanding of the tradeoffs between quality of service, server I/O, storage required, program placement, and commu nication cost. They are helpful in configuring the VOD dis tribution network, optimally placing the programs across the hierarchy of the video servers, and maximizing the performance of the overall system.

Distributed VOD server is a solution to the problem of the capacity limit posed by single server system. There needs to be an algorithm for the head-end to use to evaluate the current status of traffic loading in the whole VOD system, so that it can decide whether a new user request should be served locally, forwarded to a remote server, or simply be rejected. The computation is further complicated by the fact that server capacities and communication costs between different servers may be variab1e. In this paper, we first define a mathematical model used to analyze the performance of our VOD systems. We then present an algorithm to compute the number of copies each film should be duplicated, as well as where to p1ace them in order to gain maximum revenue. Numerical results show that our algorithm offers improvement over Chen's algorithm [7]. We also present a method to evaluate whether a new user request should be served locally, or forwarded to a remote server, or simply be rejected. It can be seen under the prerequisite of maximal revenue that while a local video server is too busy to serve a local request, it is a good idea to send it to a remote video server which has extra capacity to process it.

기존의 보안 Wi-Fi 네트워크는 사용자가 AP(Access Point)의 암호에 맞춰야 하므로 사용이 불편하며, 사용자들이 암호를 공유하므로 시간이 갈수록 보안성이 낮아지는 문제점을 갖고 있다. 이를 해결하기 위해 사용자마다 별도의 가상 Wi-Fi 네트워크를 할당하는 방안을 제안한다. 이 방법에서는 각 사용자가 자신만의 Wi-Fi 네트워크를 가지므로 사용자 중심의 네트워크 설정이 가능하다. 사용자는 자신의 기기에 나름대로의 SSID(Service Set IDentifier)와 암호를 미리 설정해 두며 AP는 자신의 공개키를 적절한 방법으로 공개한다. AP는 또한 사용자들이 언제나 접속할 수 있는 공개채널을 유지한다. 사용자 요청이 있을 때 사용자 기기는 연결 요청 메시지를 보내는데 이 메시지에는 AP의 공개키로 암호화된 사용자 기기의 SSID와 암호가 실려 있다. 연결 요청 메시지를 받은 AP는 해당 SSID 및 암호가 설정된 새로운 가상의 AP를 생성하는데 이 가상 AP는 해당 사용자만 사용할 수 있는 전용 AP라고 할 수 있다. 이렇게 만들어지는 가상 네트워크는 비밀번호를 여러 사용자가 공유하지 않으므로 보안성이 높다. 또 이 가상 네트워크는 네트워크가 사용자 기기에 맞춰 스스로를 설정하기 때문에 사용의 편의성이 높다. 새 방법이 제공하는 보안성과 편리성에도 불구하고 기존의 Wi-Fi 네트워크에 비해 별다른 전송 능력 저하는 나타나지 않음을 실험을 통해 확인하였다. Existing Wi-Fi networks require users to follow network settings of the AP (Access Point), resulting in inconveniences for users, and the password of the AP is shared by all users connected to the AP, causing security information leaks as time goes by. We propose, in this work, a personalized secure Wi-Fi network, in which each user is assigned her own virtual Wi-Fi network. One virtual Wi-Fi per user makes the user-centric network configuration possible. A user sets a pair of her own SSID and password on her device a priori, and the AP publishes its public key in a suitable way. The AP also maintains an open Wi-Fi channel, to which users can connect anytime. On user's request, the user device sends a connection request message containing a pair of SSID and password encrypted with the AP's public key. Receiving the connection request message, the AP instantiates a new virtual AP secured with the pair of SSID and password, which is dedicated to that single user device. This virtual network is securer because the password is not shared among users. It is more convenient because the network adapts itself to the user device. Experiments show that these advantages are obtained with negligible degradation in the throughput performance.

Multi-cloud service composition using Formal Concept Analysis

Recently, a Hybrid Coded MapReduce for server-rack architecture has been proposed. In the data shuffling phase, the total communication cost is divided into intra-rack communication cost and cross-rack communication cost. Compared with the general Coded MapReduce, intra-rack communication cost is reduced with increased cross-rack communication cost. In this paper we focus on the three-tier tree topology under the Fog Computing environment, in which fog nodes are connected wirelessly by access points. The communication cost is divided into uplink and downlink cost, and computing and storage can be performed by access points. We introduce Access Point Decoding Hybrid Coding MapReduce scheme where access points instead of fog nodes participate in decoding. Theoretical analyses and simulations show that the total communication cost of secondary access points during the data shuffling phase is reduced further compared with the Hybrid Coded MapReduce in the case of redundancy factor of 2.

In a public WLAN hotspot, a roaming mobile terminal (MT) may be within radio range of more than one access point (AP), each of which may or may not have roaming agreements with the service provider of the user of the MT. In this case, the MT may need to discover some <i>service information</i> before it can make an intelligent <i>network-selection</i> decision. The most critical is <i>roaming information</i>; while other information such as <i>security policies</i>, <i>price</i>, <i>AP workload</i> may also be useful. Currently, roaming information is typically provisioned on the MTs as static <i>roaming tables</i> or <i>roaming lists</i>. However, this approach may not scale well when there are millions of hotspots globally. Addressing this shortcoming, recently several solutions have been proposed by different groups. In this paper, we contrast these solutions, and propose our own solution called <i>Roaming Information Code (RIC)</i>, which can be transported as SSID or a new Information Element of the 802.11 standard. RIC is scalable and can be fully backward compatible with existing APs (if transported as SSID). Furthermore, it does not hinder fast handoffs. In the second half of the paper, we will also discuss two other schemes addressing the other service information: a scheme called <i>RIC-VAP</i> for provider-specific security information; and a scheme that allows an AP to announce price and workload information.

An application that uses reinforcement learning (RL) agents for traffic control along an arterial under high traffic volumes is presented. RL agents were trained using Q learning and a modified version of the state representation that included information on the occupancy of the links from neighboring intersections. The proposed structure also includes a reward that considers potential blockage from downstream intersections (due to saturated conditions), as well as pressure to coordinate the signal response with the future arrival of traffic from upstream intersections. Experiments using microscopic simulation software were conducted for an arterial with 5 intersections under high conflicting volumes, and results were compared with the best settings of coordinated pre-timed phasing. Data showed lower delays and less number of stops with RL agents, as well as a more balanced distribution of the delay among all vehicles in the system. Evidence of coordinated-like behavior was found as the number of stops to traverse the 5 intersections was on average lower than 1.5, and also since the distribution of green times from all intersections was very similar. As traffic approached to capacity, however, delays with the pre-timed phasing were lower than with RL agents, but the agents produced lower maximum delay times and lower maximum number of stops per vehicle. Future research will analyze variable coefficients in the state and reward structures for the system to better cope with a wide variety of traffic volumes, including transitions from oversaturation to undersaturation and vice versa.

This paper discusses a various customer heterogeneous server (VCHS) queuing system strategy for the optimal assignment of mobile users to wireless LAN access points (APs) to optimize user quality of service (QoS). The users and APs have heterogeneous types of service requests and services, such as types of service providers, respectively. It is assumed that users arrive in a stochastic manner and that the proportions of user traffic volumes to each service request are provided. The user only connects to an AP that matches the service with the user request. Then, the user leaves after downloading a specific volume of data. A “group” assignment strategy is proposed to match the service resources to the service request demands. The strategy includes how to make a group from a number of APs proportional to the volume of user traffic to each service request. The evaluations show that the “group” assignment strategy is significantly better than a “random” assignment strategy in terms of mean waiting time. The proposed strategy reduces the mean waiting time by 25%, with more significant decreases as the number of APs with more types of services increases.

The energy consumption of communications networks has been steadily increasing, becoming one of the key factors restricting the development of mobile communications. Cell-free massive multiple-input multiple-output (CF-mMIMO) systems offer wide coverage, high spectral efficiency (SE) and excellent energy efficiency (EE), and by integrating renewable energy sources, can greatly enhance their energy-saving capabilities. However, differences in energy harvesting (EH) efficiency and communications conditions across various deployment locations bring significant challenges. In this paper, we investigate the access point (AP) deployment strategy for CF-mMIMO systems with hybrid energy supply. We formulate an optimization problem aimed at maximizing grid EE and propose an AP deployment optimization method based on the deep deterministic policy gradient (DDPG) algorithm. By considering the large number of potential AP locations, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i>-means-based preliminary AP selection method is introduced to significantly reduce the action space dimensionality in the DDPG framework. Simulation results indicate that the proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i>-means-based approach effectively reduces the action space dimensionality and enhances the training efficiency of the DDPG algorithm. Furthermore, the proposed method shows great effectiveness in improving the grid EE.

Cognitive radio is an efficient spectrum sharing mechanism to solve the contradiction between spectrum shortage and spectrum underutility, where secondary users (SUs) are allowed to access the spectrum licensed to primary users (PUs) in an opportunistic manner. In cognitive radio networks with multiple access points (APs), due to the information exchange cost and system flexibility, APs may not cooperate with each other and there usually does not exist a central controller in practice. We propose a distributed user association scheme based on multi-agent reinforcement learning to achieve load balancing for cognitive radio networks with multiple independent APs. In our proposed scheme, APs execute reinforcement learning process independently to derive optimal policies on user association. In each iteration, APs make decisions on choosing SUs for association and then SUs choose the optimal AP for association based on the offers of all APs, the behaviors of APs and SUs is modeled as a dynamic matching game. Simulation results show that the proposed multi-agent reinforcement learning approach can highly improve the system performance with excellent robustness, compared to the conventional max-SINR method.

General sharing of information resources on the Internet has already be achieved through Internet technology. But the goal of grid is trying to achieve a comprehensive connectivity and sharing for all resources on the Internet, providing high-performance computing services for users. In this paper, grid resource scheduling was studied and according to the current research about it some innovation and improvement for it were made from the following aspects. The first was innovation in architecture. Because of that the architecture of the Internet is formed of a large number of autonomous system (AS) and each AS is run by different organizations, in order to be appropriate to the architecture of the Internet's autonomous system, a grid resource scheduling system architecture model based on multi-polar autonomous system was designed, where distributed management and centralized management were combined in the model. In internal AS, resource management and scheduling could make use of two-stage structure in the form of region and backbone region. A distributed structure was used among AS, where all AS worked together to solve collaboratively problems. The second was innovation in work mode. A work mode from lowlevel to high-level was presented. Resource scheduling for users was firstly taken within their own region. Only when the resources in internal region can not meet user's requests in the region, was it possible to schedule resources in the backbone one. When user's resource request could also not be met in the backbone area, AS broadcasted an urgent appeal to the entire grid for resources via an external communication protocol and the AS that could meet the conditions would suddenly respond to the appeal. The third was agent-based system design. Because multi-agent has autonomous, interactive and collaborative features, multi-agent technology was introduced into multi-polar autonomous system grid resource scheduling mode and a detailed design for its implementation was made. The fourth was that task distribution and scheduling algorithm was improved, a design objective based on the trinity of economic cost, time cost, communication cost was proposed and genetic algorithm was used to optimize globally task assignments. It was believed that multi-polar autonomous system grid resource scheduling model based on multi-agent and genetic algorithm would take possession of intelligent, dynamic self-adaptive and unlimited expanding features, which could be well positioned to meet the new requirements of grid technology development.

For better road safety and driving experience, content distribution for vehicle users through roadside Access Points (APs) becomes an important and promising complement to 3G and other cellular networks. In this paper, we introduce Cooperative Content Distribution System for Vehicles (CCDSV) which operates upon a network of infrastructure APs to collaboratively distribute contents to moving vehicles. CCDSV solves several important issues in a practical system, like the robustness to mobility prediction errors, limited resources of APs and the shared content distribution. Our system organizes the cooperative APs into a novel structure, namely, the contact map which is based on the vehicular contact patterns observed by APs. To fully utilize the wireless bandwidth provided by APs, we propose a representative-based prefetching mechanism, in which a set of representative APs are carefully selected and then share their prefetched data with others. The selection process explicitly takes into account the AP's storage capacity, storage status, inter-APs bandwidth and traffic loads on the backhaul links. We apply network coding in CCDSV to augment the distribution of shared contents. The selection of shared contents to be prefetched on an AP is based on the storage status of neighboring APs in the contact map in order to increase the information utility of each prefetched data piece. Through extensive simulations, CCDSV proves its effectiveness in vehicular content distribution under various scenarios.

In this study, the authors consider the benefits of mobile terminals (MTs) with different access points (APs) for uplink and downlink transmission to conserve MTs' energy. In traditional cellular networks, an MT is typically associated with a single AP. However, as wireless networks evolve, heterogeneous and/or overlay deployment scenario become viable and an MT can be associated with different APs for uplink and downlink transmission. The authors call this 'dual APs association'. The authors show that allowing dual APs association provides a significant gain on the uplink system capacity and/or the uplink transmit power savings. As a specific example of the use of dual APs, the authors focus on relay networks and show that considering the relay cost further increases the benefits of this approach. Based on extensive simulations using IEEE 802.16m relay network Evaluation Methodology, the authors demonstrate that dual APs can improve the uplink harmonic capacity by 350% or reduce the uplink transmit power by 7 dB. The authors note, however, that there exists a signalling cost in implementing dual APs association, which needs be overcome to achieve these substantial performance improvements.