Real Traces Research Articles

Double Q-learning based routing protocol for opportunistic networks

Opportunistic Delay Tolerant Networks also referred to as Opportunistic Networks (OppNets) are a subset of wireless networks having mobile nodes with discontinuous opportunistic connections. As such, developing a performant routing protocol in such an environment remains a challenge. Most research in the literature have shown that reinforcement learning-based routing algorithms can achieve a good routing performance, but these algorithms suffer from under-estimations and/or over-estimations. Toward addressing these shortcomings, in this paper, a Double Q-learning based routing protocol for Opportunistic Networks framework named Off-Policy Reinforcement-based Adaptive Learning (ORAL) is proposed, which selects the most suitable next-hop node to transmit the message toward its destination without any bias by using a weighted double Q-estimator. In the next-hop selection process, a probability-based reward mechanism is involved, which considers the node’s delivery probability and the frequency of encounters among the nodes to boost the protocol’s efficiency. Simulation results convey that the proposed ORAL protocol improves the message delivery ratio by maintaining a trade-off between underestimation and overestimation. Simulations are conducted using the HAGGLE INFOCOM 2006 real mobility data trace and synthetic model, showing that when time-to-live is varied, (1) the proposed ORAL scheme outperforms DQLR by 14.05%, 9.4%, 5.81% respectively in terms of delivery probability, overhead ratio and average delay; (2) it also outperforms RLPRoPHET by 16.17%, 9.2%, 6.85%, respectively in terms of delivery ratio, overhead ratio and average delay.

Reliability-Aware Flow Distribution Algorithm in SDN-Enabled Fog Computing for Smart Cities

In order to improve the performance of the communication infrastructure in smart cities, integration of two emerging technologies, namely Fog Computing (FC) and Software-Defined Networking (SDN), has been proposed and is gaining momentum. The resulting SDN-based FC integrated architecture is expected to meet the Internet-of-Things (IoT) applications' requirements, especially in terms of easy manageability, high scalability, increased reliability, and low latency. Existing traffic engineering approaches proposed for SDN-based FC for IoT compute the route between an IoT device and fog server subject to some Quality of Service (QoS) constraints. However, these approaches ignore the link reliability level in the route computation process. Unlike them, this paper proposes a Reliability-Aware Flow Distribution Algorithm (RAFDA) and two associated optimization algorithms called Reactive Reliability-Aware Heuristic Algorithms (RRAHA-1 and RRAHA-2), which distribute the flows on the links based on the links' reliability levels, subject to additional constraints like traffic load on the link, bandwidth allocation, link utilization, and end-to-end delay. The proposed algorithms minimize the impact of link failure occurrences on the ongoing time-critical flows (applications/services) of smart cities. The proposed algorithms, evaluated using both real network traces and simulations, outperform existing approaches in terms of performance for delay-sensitive services in smart cities.

Bounding Memory Access Times in Multi-Accelerator Architectures on FPGA SoCs

Modern FPGA System-on-Chips (SoCs) embed large FPGA logics capable of hosting multiple hardware accelerators. Typically, hardware accelerators require direct access to the shared DRAM memory for reaching the high performance demanded by modern applications. In commercial FPGA SoCs, this goal is achieved by interconnecting the hardware accelerators on an interconnect based on AMBA AXI, which is the de-facto industrial standard for on-chip communications. The AXI standard provides great flexibility in the definition of the network topology. Nevertheless, such flexibility generates a significant unpredictability when attempting to bound the hardware accelerators’ response time when executing under contention. This work focus on bounding the worst-case memory access time of hardware accelerators deployed on commercial FPGA SoCs. We propose a modeling and analysis technique to bound the response time of the hardware accelerators and evaluate the schedulability of a system applicable to arbitrary AXI-based bus structures deployed on FPGA SoCs. Our results are validated on real execution traces collected on two popular FPGA SoCs belonging to the Xilinx ZYNQ-7000 and Zynq-Ultrascale+ families and by simulated results.

Tripartite Graph Aided Tensor Completion For Sparse Network Measurement

Network measurements provide critical inputs for a wide range of network management. Existing network-wide monitoring methods face the challenge of incurring a high measurement cost. Some recent studies show that network-wide measurement data such as end-to-end latency and flow traffic, have hidden spatio-temporal correlations and thus low-rank features. Taking advantage of the low-rank feature, enlightened by tensor model's strong capability of information representation and extracting, this paper studies a novel sparse measurement scheduling problem which selects a proportion of Origin and Destination (OD) pairs to take measurements in the future time slots, while ensuring the data of the remaining un-measured OD pairs be accurately inferred through tensor completion. It is challenging to find the optimal sampling points (OD pairs) without knowing the structure of the future data and also infer the un-measured data in the presence of noise in the measurement samples. To conquer the challenges, we propose several techniques: a tripartite graph to illustrate the relationship between sample locations and tensor factorization, a graph-based sample selection algorithm, and a graph-based robust tensor completion algorithm. We have conducted extensive experiments based on two real network latency monitoring traces (PlanetLab and Harvard) and two other network monitoring traces (including a traffic trace Abilene and a throughput trace WS-Dream). Our results demonstrate that, even with a sampling ratio of less than 5%, our scheme can accurately obtain the complete network-wide monitoring data by inferring the missing ones based on the samples taken. To achieve similar recovery performance, the best peer tensor completion algorithm needs a significantly larger number of samples, with the sampling ratio up to 25-150 times ours.

LiveSR: Enabling Universal HD Live Video Streaming With Crowdsourced Online Learning

The high-definition (HD) live video streaming has gained significant popularity due to the rapid growth of 4G/5G and social media. However, for devices with constrained bandwidth, they still have no sufficient bandwidth to support HD live video streaming. In this paper, we propose a neural-enhanced HD live video streaming framework called LiveSR to provide universal HD live video streaming for both bandwidth-constrained and bandwidth-rich devices. For bandwidth-constrained devices, LiveSR delivers low-quality video streams and then boosts video quality at the device side with super-resolution (SR) techniques. The difficulty lies in how to train the SR model with low cost and conduct quality enhancement in real time. To address these challenges, we design a crowdsourced online training method by exploiting computation resources and HD video data on bandwidth-rich devices in the same video channel. We also propose an imitation learning-based decision making algorithm to make downloading decisions for video chunks and SR models under limited bandwidth. We implement and evaluate our proposed LiveSR framework using real network traces, and the experiment results show that LiveSR outperforms all the other baseline approaches, with 65.5% improvement in terms of the average QoE and 5.7% in terms of video quality (i.e., PSNR), and the achieved frame rate can be as high as 30 frames per second.

Right heart catherisation – a virtual reality

Abstract Introduction Right heart catheterisation (RHC) is the gold standard for assessing patients with pulmonary hypertension. Doctors require training in this procedure in a safe and friendly environment with minimal risk to patients. Due to the Covid pandemic, formal RHC teaching workshops were cancelled in our country, so we sought to develop a Virtual Reality Right Heart Catheterisation (VRRHC) training program to fulfil this area of need without the need for face to face contact. The aim was to improve training, competency and confidence in this technique with improved diagnostic skills and reduction of procedural errors. Method We approached a health technology company to design a VRRHC training module based on our current RHC simulation workshops. Phase 1 required virtual insertion of RHC via the right internal jugular vein using micro-puncture, double Seldinger technique under ultrasound guidance, followed by insertion of the RHC to the right atrium, right ventricle and pulmonary artery with pulmonary artery occlusion using real time pressure tracings and fluoroscopy. Thermodilution cardiac outputs and chamber saturations were also performed. The proprietary platform technology was delivered via a laptop and VR headset. Clinicians perform the VRRHC with imaging, monitoring and haptic feedback with the collection of real time performance tracking allowing user data (e.g. failed steps and proficiency scores) to be captured and subsequently visualised in the learning management system. We collected analytics and data on user engagement, experience and retention, targeted learning outcomes and learning curve, reduction in operating costs, reduction in procedure times due to higher proficiency, early diagnosis of pulmonary hypertension, reduced complications, improved interpretation and diagnosis. Results The program was launched in October 2021. Preliminary data shows a learning curve is associated with both using VR (10–15 minutes) and the RHC procedure itself. Initial time to completion of the RHC was 30–40 mins, reducing to 20–30 minutes with experience and 15 minutes in experts. Completion rates increase with experience from 40–50% to 100% and error rates reduce with frequency of completion. Conclusion A Virtual Reality Right Heart Catheter training program is safe, feasible and non-invasive. Increased experience results in increased completion rates, reduced procedure time and reduced errors. Using this program will potentially have beneficial effects on doctor training, outcomes, patient safety and health economics with no risk to a real patient. Funding Acknowledgement Type of funding sources: Private grant(s) and/or Sponsorship. Main funding source(s): Janssen Pharmaceuticals

Predicting consecutive spectrum opportunities with hidden Markov models

SummaryThe increasing demands for wireless channels to accommodate the surge of internet of things devices and the associated services exacerbated the need for flexible channel allocation strategies. Opportunistic spectrum sharing is expected to provide a more reasonable use of the limited radio frequencies by allowing the coexistence of licensed users and unlicensed users in the same frequency. This arrangement is called opportunistic channel allocation, where unlicensed users explore the channel when the licensed user is not transmitting. The challenge in opportunistic spectrum allocation is to find transmission opportunities. Statistical and machine learning techniques have been used to forecast spectrum opportunities on time slotted channels based on acquired transmission data from licensed users. However, opportunity forecast is usually limited to one slot ahead of time. This work explores the use of Hidden Markov Model (HMM) training and predicting procedures to forecast a consecutive sequence of idle slots that can be explored for opportunistic transmission. To improve prediction accuracy and speedup the training process, the proposed training procedures reduce the classification alphabet, which is achieved by balancing the number of training sequences to limit the influence of outliers and provide opportunity forecast even when the training process is executed over a limited number of observed sequences. While an aggressive predictor may explore more opportunities as compared to a conservative predictor, the former may incur in higher collision rates while the latter may explore fewer opportunities. Hence, this work proposes a metric to assist in the process of selecting the best predictor's parameters to balance collision and opportunity rates. The proposed HMM predictors are evaluated on both synthetic and real traffic traces. The results on synthetic traffic shows that, for channel load from up to , the proposed predictors identified over of channel opportunities with the collision rates below . On real traffic traces, where the licensed users' transmissions have deterministic inter‐packet delays, the HMM predictors identified over of the channel opportunities with collision rates below .

Analytical ray tracing based on Hamilton principal function and conjugate variable pairs.

A transformation method based on optical Hamilton equations is proposed for 3D ray tracing in axially inhomogeneous gradient-index (GRIN) media with cylindrical symmetry. For a given GRIN field, the optical conjugate variable pairs of physical space can be transformed into a virtual space by applying canonical transformation. The virtual trace can be simply solved as a uniform expression regardless of what the GRIN field is, and one can inversely transform it into the physical space. The transformation is intimately related to a Hamilton principal function, called the S function, which simultaneously gives the real ray trace and its conjugate. The "conjugate trace" displays the direction cosines of the ray trace and thus shows the information about propagation direction at every point, and it can be independently derived from the S function without knowing the real trace. In addition, as two special dimension-reduction cases, the S function is also applicable for 2D structures with only axial inhomogeneity or cylindrical symmetry.

ExtendedSketch+: Super host identification and network host trust evaluation with memory efficiency and high accuracy

Host cardinality estimation is one crucial task in network traffic measurement. Super host is the host that exhibits anomalies in host cardinality and it is usually related to network abnormal events. Therefore, accurate host cardinality estimation is the premise of super host identification and it can be used to measure the trust degree of a network. Sketches have been widely used in super host identification. However, expanding network scale and increasing link rate bring challenges to the efficiency and accuracy of sketch-based super host identification. The size of the counters in most sketches requires a trade-off between memory resources and accuracy. To meet the need of measuring high cardinalities, large size counters are applied, which leads to memory waste in monitoring low cardinality hosts. On the other hand, most sketches have high computational overhead when tracking superhosts, resulting in inefficiency. In order to address these issues, we propose a novel memory efficient and reversible sketch, named ExtendedSketch+, to provide accurate host cardinality estimation with the purpose of super host identification and network host trust evaluation. ExtendedSketch+ achieves both high memory utilization efficiency and high accuracy, in addition to high super host identification efficiency. It applies extensible counters to record unbalanced host cardinality distribution, by adaptively expanding the counters with the increase of cardinality. It adopts a lossless traffic information transfer strategy during counter extension to ensure the accuracy of cardinality estimation. It can directly tracks the host with the highest cardinality in each bucket, which greatly improves the identification efficiency of super hosts. Based on accurate cardinality estimation of ExtendedSketch+, we can further accurately evaluate host trust degree. We theoretically analyze ExtendedSketch+ with regard to its space and time complexities and estimation accuracy. We conduct performance evaluation based on real network traffic traces. Experimental results show that ExtendedSketch+ performs better than state-of-the-art sketches regarding super host identification and provides accurate host trust evaluation.

A Controller-Driven Approach for Opportunistic Networking

Opportunistic networks (OppNets) leverage opportunistic contacts to flow data across an infrastructure-free network. As of yet, OppNets’ performance depends on applying the most suitable forwarding strategy based on the OppNet typology. On the other hand, software-defined networking (SDN) is a paradigm for wired networks that decouples the control and data planes. The control plane oversees the network to configure the data plane optimally. Our proposal uses SDN-like controllers to build a partial overview of the opportunistic network. The forwarding strategy uses this context information to achieve better network performance. As a use case of our proposal, in the context of an OppNet quota-based forwarding algorithm, we present a controller-driven architecture to tackle the congestion problem. Particularly, the controller-driven architecture uses the context information on the congestion of the network to dynamically determine the message replication limit used by the forwarding algorithm. A simulation based on real and synthetic mobility traces shows that using context information provided by the controller to configure the forwarding protocol increments the delivery ratio and keeps a good latency average and a low overhead compared with the baseline forwarding protocols based on message replication. These results strengthen the benefits of using supervised context information in the forwarding strategy in OppNets.

Making Cache Monotonic and Consistent

We propose monotonic consistent caching (MCC), a cache scheme for applications that demand consistency and monotonicity. MCC warrants that a transaction-like request always sees a consistent view of the backend database and observed writes over the cache will not be lost. We show that the complexity of MCC ranges from P time to N p -C omplete . We characterize MCC via a notion of obsolete items, based on which we abstract a principle for designing competitive MCC policies. By applying the principle, we develop an optimal MCC policy for the batch model, where requests in a batch are known in advance. For the online and semi-online models, we develop ML-augmented policies that benefit from blackbox ML models for classifying obsolete items, while being provably competitive even if the ML is arbitrarily bad. Using benchmark and real-life traces, we show that MCC policies reduce 39.09% of database reads for Redis atop HBase and improve their throughput by 77.15%.

Truthful Incentive Mechanism for Budget-Constrained Online User Selection in Mobile Crowdsensing

Mobile crowdsensing (MCS) has attained much attention for gathering distributed mobile users to complete large-scale sensing tasks. To ensure the task completion, enough users should be motivated to participate in sensing tasks. Thus, much research in MCS focuses on proposing incentive mechanism, and these works usually focus on the offline scenario, where the information of all users is available to the platform. However, we argue that the actual MCS is usually an online scenario, where the platform does not know the user's information until they establish connections with the platform. Meanwhile, mobile users connect to the platform randomly and will cut off the connection at any time. Hence, when accessed by a user, the platform needs to make an irrevocable decision instantly about whether to select the user or not, and decides a remuneration for the user without knowing future information. In this paper, we first propose a reverse-auction framework to model the interaction between the platform and mobile users. Then, we present an online truthful incentive mechanism (OTIM) to motivate users, including online winner selection and remuneration determination strategies. Finally, massive simulations are conducted based on three real traces, and the simulation results illustrate that OTIM achieves truthfulness, individual rationality, computational efficiency and an approximately full budget utilization.

Виявлення бонет-трафіку на основі потоків, використовуючи ШІ

This paper outlines the generalized framework for building end-to-end botnet network activity detection systems using artificial intelligence (AI) techniques. The paper describes network flows reconstruction as a primary feature-extraction method and considers different AI classifiers for achieving a better detection rate. The results of the latest research by other authors in the field are incorporated to implement a more efficient approach for botnet discovery. The described intrusion detection pipeline was tested on a dataset with real botnet activity traces. The performance metrics for different AI classification models were obtained and analyzed in detail. Different data preprocessing techniques were tried and described which helped improve the results even further. Some options for future enhancement of network feature selection were proposed as well. The comparison of the obtained performance metrics was drawn against the results provided by other researchers in this field.

Federated Learning on the Road Autonomous Controller Design for Connected and Autonomous Vehicles

The deployment of future intelligent transportation systems is contingent upon seamless and reliable operation of connected and autonomous vehicles (CAVs). One key challenge in developing CAVs is the design of an autonomous controller that can accurately execute near real-time control decisions, such as a quick acceleration when merging to a highway and frequent speed changes in a stop-and-go traffic. However, the use of conventional feedback controllers or traditional learning-based controllers, solely trained by each CAV’s local data, cannot guarantee a robust controller performance over a wide range of road conditions and traffic dynamics. In this paper, a new federated learning (FL) framework enabled by large-scale wireless connectivity is proposed for designing the autonomous controller of CAVs. In this framework, the learning models used by the controllers are collaboratively trained among a group of CAVs. To capture the varying CAV participation in the FL training process and the diverse local data quality among CAVs, a novel dynamic federated proximal (DFP) algorithm is proposed that accounts for the mobility of CAVs, the wireless fading channels, as well as the unbalanced and non-independent and identically distributed data across CAVs. A rigorous convergence analysis is performed for the proposed algorithm to identify how fast the CAVs converge to using the optimal autonomous controller. In particular, the impacts of varying CAV participation in the FL process and diverse CAV data quality on the convergence of the proposed DFP algorithm are explicitly analyzed. Leveraging this analysis, an incentive mechanism based on contract theory is designed to improve the FL convergence speed. Simulation results using real vehicular data traces show that the proposed DFP-based controller can accurately track the target CAV speed over time and under different traffic scenarios. Moreover, the results show that the proposed DFP algorithm has a much faster convergence compared to popular FL algorithms such as federated averaging (FedAvg) and federated proximal (FedProx). The results also validate the feasibility of the contract-theoretic incentive mechanism and show that the proposed mechanism can improve the convergence speed of the DFP algorithm by 40% compared to the baselines.

False Negative Awareness in Indicator-Based Caching Systems

Distributed caching systems such as content distribution networks often advertise their content via lightweight approximate indicators (e.g., Bloom filters) to efficiently inform clients where each datum is likely cached. While false-positive indications are necessary and well understood, most existing works assume no false-negative indications. Our work illustrates practical scenarios where false-negatives are unavoidable and ignoring them significantly impacts system performance. Specifically, we focus on false-negatives induced by indicator staleness, which arises whenever the system advertises the indicator only periodically, rather than immediately reporting every change in the cache. Such scenarios naturally occur, e.g., in bandwidth-constraint environments or when latency impedes each client’s ability to obtain an updated indicator. Our work introduces novel false-negative aware access policies that continuously estimate the false-negative ratio and sometimes access caches despite negative indications. We present optimal policies for homogeneous settings and provide approximation guarantees for our algorithms in heterogeneous environments. We further perform an extensive simulation study with multiple real system traces. We show that our false-negative aware algorithms incur a significantly lower service cost than existing approaches or match the cost of these approaches while requiring an order of magnitude fewer resources (e.g., caching capacity or bandwidth).

SMURF: Efficient and Scalable Metadata Access for Distributed Applications

In parallel with big data processing and analysis dominating the usage of distributed and Cloud infrastructures, the demand for distributed metadata access and transfer has increased. The volume of data generated by many application domains exceeds petabytes, while the corresponding metadata amounts to terabytes or even more. This article proposes a novel solution for efficient and scalable metadata access for distributed applications across wide-area networks, dubbed SMURF. Our solution combines novel pipelining and concurrent transfer mechanisms with reliability, provides distributed continuum caching and semantic locality-aware prefetching strategies to sidestep fetching latency, and achieves scalable and high-performance metadata fetch/prefetch services in the Cloud. We incorporate the phenomenon of semantic locality awareness for increased prefetch prediction rate using real-life application I/O traces from Yahoo! Hadoop audit logs and propose a novel prefetch predictor. By effectively caching and prefetching metadata based on the access patterns, our continuum caching and prefetching mechanism significantly improves the local cache hit rate and reduces the average fetching latency. We replay approximately 20 Million metadata access operations from real audit traces, where SMURF achieves 90% accuracy during prefetch prediction and reduced the average fetch latency by 50% compared to the state-of-the-art mechanisms.

InFEDge: A Blockchain-Based Incentive Mechanism in Hierarchical Federated Learning for End-Edge-Cloud Communications

Advances in communications and networking technologies are driving the computing paradigm toward the end-edge-cloud collaborative architecture to leverage ubiquitous data and resources. Opposite to centralized intelligence, Hierarchical Federated Learning (HFL) relieves overwhelmed communication overhead and enjoys the advantages of high bandwidth as well as abundant computing resources while retaining privacy-preserving benefits of Federated Learning (FL). It is difficult to balance system overhead and model performance in the HFL framework, while it could be solved by introducing an incentive mechanism. Although the incentive mechanism can alleviate the above anxiety by compensating relevant participants, some limitations (multi-dimensional properties, incomplete information and unreliable participants) will significantly degrade the performance and efficiency of the designed mechanism. To address the challenges caused by the above limitations, we propose InFEDge, a blockchain-based incentive mechanism in the HFL. The InFEDge considers 1) multi-dimensional individual properties to model system participants and proves the uniqueness of Nash equilibrium with the closed-form solution. Meanwhile, 2) we transform the problem under incomplete information into a contract game where we obtain the optimal solution. Moreover, 3) we also leverage the blockchain to provide economic incentives, prevent unreliable participants’ disturbance and further ensure data privacy by implementing the mechanism in the smart contract to offer a credible, faster, and transparent resource trading system. Experimental evaluations on a proof-of-concept testbed along with real traces demonstrate the superiority of our mechanism. Further, our method solves a real-world user allocation problem for future communications and networking.

Attack Resilience of Cache Replacement Policies: A Study Based on TTL Approximation

Caches are pervasively used in communication networks to speed up content access by reusing previous communications, where various replacement policies are used to manage the cached contents. The replacement policy of a cache plays a key role in its performance, and is thus extensively engineered to achieve a high hit ratio in benign environments. However, some studies showed that a policy with a higher hit ratio in benign environments may be more vulnerable to cache pollution attacks that intentionally send requests for unpopular contents. To understand the cache performance under such attacks, we analyze a suite of representative replacement policies under the framework of TTL approximation in how well they preserve the hit ratios for legitimate users, while incorporating the delay for the cache to obtain a missing content. We further develop a scheme to adapt the cache replacement policy based on the perceived level of attack. Our analysis and validation on real traces show that although no single policy is resilient to all the attack strategies, suitably adapting the replacement policy can notably improve the attack resilience of the cache. Motivated by these results, we implement selected policies as well as policy adaptation in an open-source SDN switch to manage flow rule replacement, which is shown to notably improve its resilience to pollution attacks.

IOT based Social Safety Android Application using Geolocation for Real Time Tracing

Abstract: Women's security is a critical issue in today's world, and it is very crucial for every individual to be acting over such a problem. But this problem is not only limited to women but also to other citizens like old people and children. Today most of the old people and children are victim of illness and child abuse. Crime against senior citizens in the New Delhi city rose by 22% in 2021 compared to 2020 as published by National Crime Records Bureau (NCRB). According to NCRB crime against children in 2019 recorded were 1,48,185 cases, which means an average of 400 crimes were committed every day in the country. These society problems led the motivation for development of a social safety application.

ImpalaE: Towards an optimal policy for efficient resource management at the edge

Edge computing is an extension of cloud computing where physical servers are deployed closer to the users in order to reduce latency. Edge data centers face the challenge of serving a continuously increasing number of applications with a reduced capacity compared to traditional data center. This paper introduces ImpalaE, an agent based on Deep Reinforcement Learning that aims at optimizing the resource usage in edge data centers. First, it proposes modeling the problem as a Markov Decision Process, with two optimization objectives: reducing the number of physical servers used and maximize number of applications placed in the data center. Second, it introduces an agent based on Proximal Policy Optimization, for finding the optimal consolidation policy, and an asynchronous architecture with multiple workers-shared learner that enables for faster convergence, even with reduced amount of data. We show the potential in a simulated edge data center scenario with different VM sizes based on Microsoft Azure real traces, considering CPU, memory, disk and network requirements. Experiments show that ImpalaE effectively increases the number of VMs that can be placed per episode and that it quickly converges to an optimal policy.