Quality Of Service Degradation Research Articles

Log‐based anomaly detection for distributed systems: State of the art, industry experience, and open issues

AbstractDistributed systems have been widely used in many safety‐critical areas. Any abnormalities (e.g., service interruption or service quality degradation) could lead to application crashes or decrease user satisfaction. These things may cause serious economic losses. Among the various quality assurance approaches for distributed systems, log‐based anomaly detection (LAD) has become a popular research topic. Its popularity relates to system logs being able to record and reveal important run‐time information. This paper presents a general LAD framework for distributed systems. Log grouping and feature‐pattern mining are two crucial LAD components that impact on the anomaly‐detection effectiveness. We also present a systematic survey of techniques in these two directions; propose classification frameworks for log grouping and feature patterns; and summarize four log‐grouping techniques and five feature patterns (which refer to invariant relationships among logs that can be used for anomaly detection). To evaluate their applicability, we report on the findings when applying existing techniques to Ray, a popular industrial distributed system. Based on these findings, several open issues are identified, which provide potential guidance for future research and development.

Mitigating the impact of controller failures on QoS robustness for software-defined wide area networks

Emerging cloud services and applications pose different Quality of Service (QoS) requirements for the network, where Software-Defined Wide Area Networks (SD-WANs) play a crucial role in QoS provisioning by introducing network programmability into network flows to enable dynamic flow routing and ensure low data transmission latency for these applications. However, controller failures may happen in SD-WANs, and all programmable flows that the failed controller previously controlled will become offline and lose the network programmability, resulting in the degradation of QoS. Existing control recovery solutions propose to remap offline switches/flows to available active controllers but cannot promise good recovery performance due to the following two problems: (1) the recovery performance suffers from either coarse-grained remapping granularity or introducing extra processing delays, and (2) QoS robustness cannot be guaranteed in the design of recovery solution. To this end, we propose Predator, a QoS-aware network programmability recovery scheme that utilizes the P4 Runtime enabled by existing P4 switches to achieve fine-grained per-flow remapping without introducing extra delays. Specifically, our proposed Predator categorizes flows based on their QoS requirements and smartly recovers offline flows based on their priorities to guarantee the QoS robustness for high-priority flows. Simulation results under real-world topology demonstrate that our proposed Predator can improve the recovered network programmability of high-priority flows by up to 505.5%, and substantially reduce the communication overhead of high-priority flows, compared with baselines.

Optimal service provisioning in IoT fog-based environment for QoS-aware delay-sensitive application

This paper addresses the escalating challenges posed by the ever-increasing data volume, velocity, and the demand for low-latency applications, driven by the proliferation of smart devices and Internet of Things (IoT) applications. To mitigate service delay and enhance Quality of Service (QoS), we introduce a hybrid optimization of Particle Swarm (PSO) and Chemical Reaction (CRO) to improve service delay in FogPlan, an offline framework that prioritizes QoS and enables dynamic fog service deployment. The method optimizes fog service allocation based on incoming traffic to each fog node, formulating it as an Integer Non-Linear Programming (INLP) problem, considering various service attributes and costs. Our proposed algorithm aims to minimize service delay and QoS degradation. The evaluation using real MAWI Working Group traffic data demonstrates a substantial 29.34% reduction in service delay, a 66.02% decrease in service costs, and a noteworthy 50.15% reduction in delay violations compared to the FogPlan framework.

Handover Triggering Prediction with the Two-Step XGBOOST Ensemble Algorithm for Conditional Handover in Non-Terrestrial Networks

A Non-Terrestrial Network (NTN) is a network system that enables service for areas where terrestrial networks cannot cover. An NTN provides communication services using flying objects such as UAVs, HAPs, and satellites. In the case of satellites, they move in Earth’s orbit at a constant speed. Ground services from continuously moving satellites cause frequent handovers. In addition, frequent handovers may come as a load between User Equipment (UE) and the communication system, which leads to degradation of service quality. Unlike Terrestrial Networks (TN), communication services are provided to UEs at altitudes ranging from 20 km to 35,584 km, rather than from base stations close to the ground. Service at high altitudes is unreliable due to the measurement values that were previously used as quality indicators to operate terrestrial networks. Moreover, service at high altitudes demands long-distance communication, and propagation delay occurs from the long-distance communication. In the 3GPP Rel. 17 document, it is suggested that the above problems should be solved. This paper tries to solve the problem by proposing the two-step XGBOOST, a CART-based Gradient Boosting Model. Handover in TN uses measurement-based conditional handover (CHO), but the measured values in the NTN environment are not valid. Using this, the distance between the UE and the center of the cell and the elevation angle are used to construct a model that predicts the HO triggering time point. In order to overcome the propagation delay caused by communication at a high altitude, a model that predicts the distance and elevation angle between the UE and the center of the cell considering the propagation delay is proposed. The model is composed of two-step XGBOOST. The one-step model is a model in which the UE predicts the distance and elevation angle between cell centers after propagation delay at the time when satellite position information is transmitted to the UE. The two-step model predicts handover triggering occurrence based on the data predicted by the one-step result. As a result of the experiment, the model considering the propagation delay showed about 8% better performance on average than the model not considering the propagation delay, and the XGBOOST model achieved an average F1-score of 0.9891 in the propagation delay experiments.

Optimizing multimedia communication in internet of thing network for improving quality of service

The internet of things (IoT) has revolutionized the way we interact with technology, with the proliferation of interconnected devices leading to an increase in the volume of data transmitted over the network. One of the key applications of IoT is in multimedia communication, where real-time audio and video data is transmitted over the network. However, the diverse nature of applications and the sheer volume of data in IoT networks can lead to network congestion, latency, and variable quality of service (QoS) at the internet side, resulting in a degradation of the overall QoS for multimedia traffic. In this paper, we propose a cross-layer multimedia optimization solution for multi-point to point IoT networks that incorporates service differentiation and bandwidth reservation techniques to improve the QoS of multimedia traffic. We evaluate the performance of our proposed solution using simulations and compare it with existing solutions. Our results show that our proposed solution can significantly improve the QoS of multimedia traffic in IoT networks, even during periods of high network congestion or variable QoS at the internet side.

Agile Services Provisioning for Learning-Based Applications in Fog Computing Networks

Fog computing has emerged as a promising solution for service provisioning. To support the increasing number of smart end devices and their requirement for intelligent data processing, numerous learning-based applications will be deployed at the edge network to provide services with an acceptable level of latency. However, such applications assume that the training and the actual data are independent and identically distributed, which is impractical in the dynamic and heterogeneous environment of the real world. This makes service matchmaking a challenge because even if the service functionality matches user requirements, the mismatched actual data will inevitably result in the degradation of service quality. This paper targets the mismatching issue of learning-based applications and the high requirement of intelligent data processing at the network edge and proposes a novel service provisioning model. This model introduces a novel service description model to resolve data mismatch, a clustering algorithm that pre-processes requests to deal with high concurrency requirements, and a heuristic joint service searching method to reduce traffic costs. This work was evaluated through a case study and simulations. Evaluation results show that the proposed model can achieve a good quality of services and reduce response time as well as traffic costs.

Handover Decision-Making Algorithm for 5G Heterogeneous Networks

The evolution of 5G small cell networks has led to the advancement of vertical handover decision-making algorithms. A mobile terminal (MT) tends to move from one place to another and, as the 5G network coverage is small, user network access will change frequently and lead to a high probability of unnecessary handover, which is a waste of network resources and causes degradation of service quality. This paper aims to reduce the number of unnecessary handovers in 5G heterogeneous networks by proposing a handover decision-making algorithm that integrates the dwelling time prediction technique and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS). The proposed algorithm reduces the number of unnecessary handovers by estimating the connection time to a small cell network using the dwell time prediction technique. The TOPSIS evaluates the network quality and chooses the best network based on user preference. The result shows that the proposed handover algorithm reduces the number of unnecessary handovers to small cell networks in high-speed scenarios. It also saves the network connection cost by up to 27.51% compared with the TOPSIS-based handover algorithm. As for throughput achievement, the proposed algorithm yields an improvement of 5.12%. The proposed algorithm significantly reduces the number of unnecessary handovers in the high-speed scenario while fulfilling user preferences.

NSEN: Improving Session Efficiency in Distributed MEC Networks

The rapid development of 5G technology and edge computing prompts emerging edge applications with heterogeneous requirements for network services with respect to latency, reliability and throughput. Although content distribution networks (CDN) and edge computing proposals such as multiaccess edge computing (MEC) improve quality of service by deploying services close to end users at the edge, the session efficiency of service still is constrained by host name resolution which also leads to quality degradation of service in the case of user mobility. In this article, we proposed a novel network architecture, Named Service Edge Network (NSEN), on the basis of DNS protocol. We enabled NSEN functionality in the MEC reference architecture and presented high-efficiency session models to satisfy the requirements of various edge applications and enhance the mobility support in distributed MEC environment. The simulation and experiment results demonstrate the NSENenabled MEC architecture can improve session efficiency of MEC services in 5G cellular networks in terms of latency and throughput.

Research on Resource Allocation Algorithm for Satellite Uplink to Provide Differentiated QoS Support

Broadband satellite systems are built to serve different user groups ranging from enterprise and military users to common personal users, with various kinds of services calling for dissimilar quality of service (QoS). Obviously, the satellite systems are expected to distinguish the specific type of an incoming service request as well as the importance of its owner in order to perform an efficient assignment of valuable resources. In this article, the uplink resource management challenge in a multifrequency-time-division multiple access satellite system is addressed where provisioning the discriminated QoS is of vital importance. Having taken both user priority and service type into account, a differentiated QoS support principle conforming to the digital video broadcast - second generation DVB interactive satellite system (DVB-RCS2) standard is put forward, based on which the uplink timeslot assignment problem is formulated.The NP-hardness of the problem is proved, and a differentiated QoS resource allocation scheme is designed, decoupling the problem into the coarse-level of resource allocation and fine-level of timeslot assignment. The former stage involves the constant rate assignment, bandwidth adjustment, QoS degradation and user scheduling, among which the subproblem of user scheduling is an integer nonlinear program. To tackle the issue, the problem is first innovatively converted to an integer linear program, and is then modeled as the multiple-choice multiple knapsack (MCMK) problem. Furthermore, an MCMK algorithm is presented to give the optimal solution to the user scheduling problem, after which the timeslot assignment is investigated in the second phase to generate the terminal burst time plan table for return channel satellite terminals.

An intelligent scheduling framework for DNN task acceleration in heterogeneous edge networks

With the upgrade of hardware architecture and device capacities, many accelerator-based hardware platforms have been widely deployed in Mobile Edge Computing (MEC) environments. The execution time of many computation-intensive applications (e.g., face recognition and pedestrian detection) can be significantly reduced when deployed on these heterogeneous devices. Moreover, thanks to the popularity of Deep Learning (DL), most terminal applications are integrated with Deep Neural Networks (DNN) and can be divided into interdependent tasks. The structure of these applications can be represented as the Directed Acyclic Graph (DAG). Therefore, it is critical to seek the optimal scheduling order and execution placement of tasks according to the acceleration effects of edge servers and the task dependency. However, conventional scheduling strategies focus on the short-term performance, potentially leading to service quality degradation in the long term. Besides, many studies use Deep Reinforcement Learning (DRL) algorithms to seek a long-term optimal scheduling strategy but ignore the device acceleration and the task dependency. Furthermore, training a well-performed DRL agent is time-consuming, and the large scale of trial-and-error will take up tremendous computation and storage resources. In this paper, we model the scheduling process as a Markov Decision Process (MDP) and design an adaptive scheduling framework for task acceleration. Fully considering the data dependencies, resource conditions, and network conditions, the proposed scheduling algorithm called Meta-AC uses policy gradient combined with meta-learning to minimize the average task delay and the ratio of time-out tasks. As a hierarchical DRL algorithm, Meta-AC uses meta data to learn directed exploration strategies in the high-level agent, improving the learning efficiency from experience samples. Extensive simulations demonstrate the superiority of the proposed method over the counterpart methods.

Energy efficient resource optimization algorithm for cloud infrastructure

The proliferation of cloud computing infrastructure has increased the energy demand remarkably. Energy-efficient resource management is essential for running a cost effective and environment friendly data center. Virtual Machine (VM) consolidation is a well-accepted method for reducing the energy consumption of the cloud data center. Quality of service is an equally important aspect of cloud services. VM migrations caused by consolidation often cause degradation in QoS. These two parameters have been dealt with individually in most research and very few addressed both energy efficiency and QoS simultaneously. We have proposed a new Energy and QoS Efficient (EQSE) VM selection and placement method for improving the energy efficiency along with quality of service (QoS). VM selection and placement are two critical steps of VM consolidation. EQSE uses Resource Gap Minimization (RGM) algorithm for VM selection and Utilization-Aware Best-Fit Decreasing (UABFD) algorithm for placement of these VMs. EQSE along with dynamic thresholds reduces energy consumption and improves the quality of service by reducing the number of VM migrations. CloudSim simulation performed on PlanetLab data establishes the superiority of the proposed method compared to the existing state of the art methods of VM consolidation.

A multi-objective cloud energy optimizer algorithm for federated environments

Energy is a considerable portion of the cost of a cloud data center. In addition to energy, carbon emission tax imposes another cost on data centers, which have different policies in different cities around the world. Most of the cloud data centers are geographically distributed, which can have a huge advantage in reducing such costs. In addition, in recent years, cloud federation, in which multiple cloud providers share their IT infrastructures voluntarily, could play a crucial role in minimizing the energy consumption of cloud data centers. On the other hand, improper VM placement results frequent VM migration, constant turning off/on physical machines, service quality degradation and energy consumption increase. To tackle these weaknesses, we propose a multi-objective algorithm, Called OUR-ACS, to minimize data centers' energy consumption, carbon emission, and the total expenses (energy cost + carbon tax) in a federated environment considering both initial VM placement and VM consolidation approaches. We evaluate the efficiency of our proposed algorithm by using the ClousSim Plus simulator toolkit using real-world datasets. Compared to the competing algorithms, our simulation results indicate that our proposed algorithm could reduce the total energy consumption, carbon emission, and the total costs of multiple cloud data centers by an average of 37.6%, 41%, and 25%, respectively, in a federated environment.

Confidence interval‐based overload avoidance algorithm for virtual machine placement

AbstractVirtualization plays an essential role in decreasing energy consumption and optimizing resource utilization by enabling the creation of virtual machines (VM) and their consolidation through live migration. Excessive migrations and a lack of required VMs are two critical factors in QoS degradation. The current consolidation approaches impose an intensive time complexity and cannot be used in large data centers with hundreds of hosts. This article proposes a framework for dynamic consolidation divided into a QoS‐aware algorithm for overload avoidance and a power‐aware algorithm for VM placement. To compute a safe zone criterion for any VM, relations were suggested by applying an interval estimate with a confidence level. By employing this criterion, the offered algorithm could guarantee the quality of service (QoS), particularly for specific VMs, while avoiding overhead. The VM placement algorithm is developed based on the maximum utilization of active hosts. It provides the capability to control the number of active hosts for the data center manager. The simulation results with real workloads revealed that the proposed framework could decline the amount of service level agreement violations by 78% and the number of migrations by 74%, and energy consumption by up to 13% in comparison with the best results of the benchmark algorithms. Hence, the application of this framework upgrades the QoS of data centers and declines their energy costs.

Proactive Eavesdropping for Wireless Information Surveillance Under Suspicious Communication Quality-of-Service Constraint

This paper investigates a proactive eavesdropping scenario where multiple suspicious communication (SC) links are eavesdropped by a full-duplex monitor who can send jamming or constructive signals on each SC link. In order to avoid being discovered by the suspicious users, the monitor restricts the SC quality-of-service (QoS) degradation caused by itself. Under this SC QoS constraint and the average transmit power constraint, the problems of optimizing the mode and transmit power of the monitor to maximize the average successful eavesdropping probability and the average eavesdropping rate are investigated for the delay-sensitive suspicious service and delay-tolerant suspicious service, respectively. By proving that the time-sharing condition is satisfied, the optimal solutions are derived based on the Lagrange duality method. Simulation results validate the effectiveness of the proposed schemes. It is shown that the proposed schemes can provide satisfactory eavesdropping performance improvement even with zero SC QoS degradation requirement.

Realistic Prognostic Modeling of Specific Attenuation due to Rain at Microwave Frequency for Tropical Climate Region

Absorption and scattering of propagated microwave radio signals by atmospheric variables, particularly rainfall, remained a major cause of propagation attenuation losses and service quality degradation over terrestrial communication links. The International Telecommunications Union Radio (ITU-R) reports and other related works in the literature provided information on attenuation due to rain and microwave propagation data. Such propagation attenuation information in the tropical region of Nigeria is destitute, especially at lower radio waves transmission frequencies. Therefore, this study addresses this problem by employing 12-year rainfall datasets to conduct realistic prognostic modeling of rain rate intensity levels. A classification of the rainfall data into three subgroups based on the depth of rainfall in the region is presented. Additionally, an in-depth estimation of specific rain attenuation intensities based on the 12-year rainfall data at 3.5 GHz is demonstrated. On average, the three rainfall classes produced rain rates of about 29.27 mm/hr, 73.71 mm/hr, and 105.39 mm/hr. The respective attenuation values are 0.89 dB, 1.71 dB, and 2.13 dB for the vertical polarisation and 1.09 dB, 1.20 dB, and 2.78 dB for the horizontal polarisation at 0.01% time percentage computation. Generally, results indicate that higher rain attenuation of 12% is observed for the horizontal polarisation compared to the vertical polarisation. These results can provide valuable first-hand information for microwave radio frequency planning in making appropriate decisions on attenuation levels due to different rainfall depths, especially for lower frequency arrays.

Minimizing the monetary penalty and energy cost of server migration service

AbstractCurrent IaaS (infrastructure as a service) cloud service may not satisfy communication QoS (quality of service) requirements of delay‐sensitive network applications, if there is a significant physical distance between a server of the network application (NetApp server) at a data center and its network application clients (NetApp clients). In order to improve communication QoS of NetApp clients, we propose server migration service (SMS) in this article. SMS allows NetApp servers to migrate among different locations in the network (1) to optimally locate themselves in relation to NetApp clients and mitigate the QoS degradation caused by location‐related factors (ie, propagation delays on network links) and (2) to optimally distribute traffic load over routers and processing load over (physical) computers and decrease the energy consumption. We develop a mixed‐integer programming model that determines when and to which locations NetApp servers migrate to minimize the total operating cost of SMS, that is, the sum of the monetary penalty incurred due to QoS violation and energy cost incurred due to energy consumption, while preventing NetApp servers from excessively migrating and adversely impacting QoS of the non‐SMS service that share the resource of the substrate network with SMS. Simulation results show that the model developed in this article achieves up to 42% lower total operating cost of SMS compared to the model that only minimizes the monetary penalty of SMS without considering the energy cost of SMS.

EdgeDR: An Online Mechanism Design for Demand Response in Edge Clouds

The computing frontier is moving from centralized mega datacenters towards distributed cloudlets at the network edge. We argue that cloudlets are well-suited for handling power demand response to help the grid maintain stability due to more flexible workload management attributed to their distributed nature. However, they also require computing demand response to avoid overload and maintain reliability. To this end, we propose a novel online market mechanism, EdgeDR, to achieve cost efficiency in edge demand response programs. At a high level, we observe that the cloudlet operator can dynamically switch on/off entire cloudlets to compensate for the energy reduction required by the power grid or provide enough computing resources to the edge service. We formulate a long-term social cost minimization problem and decompose it into a series of one-round procurement auctions. In each auction instance, we propose to let the cloudlet tenants bid with cost functions of their two-dimension service quality degradation tolerance, and let the cloudlet operator choose the service quality, manage the workload, and schedule the cloudlet activation status. In addition, we present a dynamic payment mechanism for the operator to balance the tradeoff between short-term profit and long-term benefit in more practical scenarios. Via rigorous analysis, we exhibit that our bidding policy is individually rational and truthful; our workload management algorithm has near-optimal performance in each auction; and our overall online algorithm achieves a provable competitive ratio. We further confirm the performance of our mechanism through extensive trace-driven simulations.

A multi-agent approach to monitor and manage container-based distributed systems

The software architectures' evolution over the years has brought new challenges for the software development area. New architectural styles have emerged, such as microservice architecture and container-based distributed systems. In this sense, it can be said that service monitoring systems can be considered as an important element, since they can observe and verify the quality and progress of an application over a period of time. Automated solutions based on pre-defined intervals can result in unnecessary use of resources to identify any type of anomalies (e.g., failure or Quality of Service degradation). Considering this scenario, a multi-agent approach to monitor container-based distributed systems is proposed in this article. Such approach autonomously and dynamically decides the monitoring frequency of a container, optimizing the identification of anomalies. To evaluate the applicability of our approach, we conducted a case study on an application to manage a smart restaurant named App2SmartRest. The results reported in this article enable us to create a favorable scenario for our approach to contribute efficiently with different software development communities.

5G Network Management System With Machine Learning Based Analytics

Application of intelligent data analytics using machine learning in management of 5G networks can enable autonomous networking capabilities in 5G networks. This paper describes the design and implementation of CygNet MaSoN, a management system supporting advanced aggregation and analytics features combined with machine learning. The system supports detection of anomalous network behaviour, detection of degradation in network performance and service quality and also supports resource optimization. The main objective is to achieve self-organizing and closed loop automation functionalities expected as part of autonomous functioning of 5G networks. Details of the system architecture and components are presented. Three real-life use cases implemented on this system are then described. The features considered, machine learning models built and synthetic data generation methods adopted are presented. The results obtained using the MaSoN system are also presented to demonstrate the effectiveness of the system in 5G network operations.

Power- and QoS-Aware Job Assignment With Dynamic Speed Scaling for Cloud Data Center Computing

As the usage of mission-critical mobile applications increases in Industry 4.0, such as smart manufacturing and self-driving cars, the cloud computing paradigm and its supporting data centers have become more crucial. However, a common practice in the cloud data center computing industry tends to supply a surfeit of computing resources mainly for a robust quality-of-service (QoS). In this paper, we propose a simple real-time algorithm which combines a power-aware job assignment policy for a centralized job dispatcher and a power- and QoS-aware dynamic speed scaling policy for each physical machine (PM). The job assignment policy is called “Join the Least Power Consuming (LPC) Server” that routes an incoming cloud job to a server spending minimum power upon request. The server-side adaptive speed scaling policy expedites energy efficiency and satisfies response time-associated QoS condition. We call this policy “Minimizing Earliness (ME)” since it manages the server speed towards finishing jobs at their deadlines as precisely as possible, reducing the earliness of job completion. The design principle of LPC-ME combination supports both energy efficiency and service quality required in cloud data centers. Numerical experiments compare the proposed algorithm’s power consumption and response time with those of existing popular policies and demonstrate better energy efficiency with negligible degradation of service quality.