Learning-based Scheme Research Articles

A learning-based proactive scheme for improving distribution systems resilience against windstorms

This paper presents a proactive operation scheme for improving distribution system resiliency against natural hazards, specifically windstorms. In this context, important attributes associated with the windstorm consisting the distance from the windstorm route, the wind speed, the distance from tall trees and buildings, and cable type are used in a deep neural network (DNN) engine to identify the vulnerable branches and predict their failure during the windstorm. The DNN predictive model is integrated in the proposed scheme. Afterwards, a power flow-based optimization engine is employed to proactively enhance the grid resiliency. Grid resiliency is measured by the inevitable action of load shedding. For minimum load shedding, the optimization engine reconfigures the network topology, optimizes the droop parameter settings, and allocates mobile energy storage systems (ESSs) before the arrival of the windstorm. This optimization engine is integrated in the proposed scheme. To validate its performance, the proposed proactive scheme is tested on a 33-bus test system with a mix of diesel units (DUs), wind turbines (WTs), and photovoltaic units (PVs). The simulation results demonstrate that without the proposed learning mechanism, the load shedding can reach up to 36% for the system under study, while the learning-based scheme can reduce the load shedding to 13%. The proposed learning-based proactive operation scheme would substantially improve the distribution system resiliency during windstorms.

LSTM-Based Intrusion Detection System for VANETs: A Time Series Classification Approach to False Message Detection

In vehicular ad hoc networks (VANETs), vehicles broadcast emergency messages and beacon messages, which enable drivers to perceive traffic conditions beyond their visual range thus improve driving safety. However, internal attackers can launch a false message attack for selfish purposes by reporting a non-existent traffic incident in emergency messages. Moreover, some collusion attackers may spread bogus beacon messages cooperatively to make the bogus traffic incident more deceptive. To improve the accuracy of false emergency message detection, we propose a novel intrusion detection system (IDS) based on time series classification and deep learning. Considering that traffic parameters are highly correlated with time, we collect time series of traffic parameters closely related to traffic incidents from messages of vehicles near reported traffic incidents as time series feature vectors. To recognize the pattern of traffic parameters changing over time more accurately, a traffic incident classifier based on long short-term memory (LSTM) is designed and trained using time series feature vectors from both normal and collusion attack scenarios. Based on the classification result, the authenticity of the emergency message can be determined. Finally, we evaluate the performance of the proposed LSTM-based IDS through extensive simulation. Simulation results validate that our IDS is more accurate in false message detection compared with some well-known machine learning-based schemes.

A learning approach for multi-agent travelling problem with dynamic service requirement in mobile IoT

Introducing mobile agents into modern IoT systems for service provision such wireless charging, real-time monitoring etc. can improve the economic efficiency, reduce energy consumption and human effort. The path planning of mobile agent(s) visiting specific locations to provide the required service is thus a critical problem. Different from the existing studies, this work considers the dynamic service requirements in practical systems, such as dynamic demand/emergency in each location which changes along time, and explores the large-scale Travelling Salesman Problem (TSP) with the dynamic service requirements. A Graph Attention based Pointer Network (GAPN) is proposed to address the problem, the objective of which is to maximize the number of successfully served nodes under dynamic temporal constraints while minimizing the tour length. The experimental results show that GAPN outperforms the existing heuristic-based and learning-based schemes in terms of TSP with dynamic temporal constraints and the large-scale TSP.

A Novel Machine Learning-Based Scheme for Spectrum Sharing in Virtualized 5G Networks

Network virtualization allows the coexistence of multiple network slices over a shared physical infrastructure, each delivering a service with own requirements in terms of quality of service, coverage, and time span. Spectrum is an expensive commodity, so it must be managed among these slices in the most efficient way. However, current spectrum sharing techniques are too rigid to address all combinations of service requirements and properly exploit the new air interface flexibility and network deployment options introduced by the Fifth Generation (5G) mobile networks. In this paper, we extend the state of the art first by proposing a novel spectrum sharing scheme that supports an unlimited number of radio networks. Second, we propose a 5G compliant novel service-based network management architecture to integrate Machine Learning (ML) algorithms for Radio Resource Management (RRM), including spectrum sharing. Finally, we propose a new three-stage ML framework that exploits forecasting, clustering and reinforcement learning algorithms to implement the proposed spectrum sharing scheme. Our proposed solution can allow an arbitrary number of 5G network slices to share spectrum more effectively and more dynamically either with each other or with other radio networks.

Secondary Frequency Control of Microgrids: An Online Reinforcement Learning Approach

In this article, we present a reinforcement learning-based scheme for secondary frequency control of lossy inverter-based microgrids. Compared with the existing methods in the literature, we relax the common restrictions on the system, i.e., being lossless, and the transmission lines and loads to have known constant impedances. The proposed secondary frequency control scheme does not require <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> information about system parameters and can achieve frequency synchronization within an ultimate bound in the presence of dominantly resistive and/or inductive line and load impedances, model parameter uncertainties, and time varying loads and disturbances. First, using Lyapunov theory, a feedback control is formulated based on the unknown dynamics of the microgrid. Next, a performance function is defined based on cumulative costs toward achieving convergence to the nominal frequency. The performance function is approximated by a critic neural network in real-time. An actor network is then simultaneously learning a parameterized approximation of the nonlinear dynamics and optimizing the approximated performance function obtained from the critic network. Furthermore, using the Lyapunov approach, the uniformly ultimate boundedness of the closed-loop frequency error dynamics and the networks' weight estimation errors are shown.

Automatic Double-Auction Mechanism for Federated Learning Service Market in Internet of Things

In the past decade, the Artificial Intelligent & Internet of Things (AIoT) has become the disruptive force reshaping our lives and works. Billions of AIoT devices around the world are now connected to the Internet. This widespread deployment of AIoT devices presents huge challenges for the centralized AI training architecture (e.g., security issues, privacy issues). Recently, with the advance of federated learning (FL) technology, the integration of AIoT and FL is considered as a promising solution. FL enables distributed devices to collaboratively train a shared AI model while keeping all the training data contained in local. However, unlike the centralized architecture, the distributed learners are independent, uncontrollable, and self-interested. Designing effective incentive schemes to stimulate the distributed devices to actively participate in training tasks and contribute high-quality training models is the priority question. In this paper, we design a double auction mechanism for FL service market, where the trained models can be automatically traded between AIoT devices and FL platforms. Then, we propose an iterative double auction (IDA) algorithm, where a controller is implemented to guide participants to adjust their policies for the sake of maximizing individual participates utilities as well as social welfare. In addition, we propose a reinforcement learning-based scheme, termed as the Experience Weighted Attraction Learning-Double Auction (EWA-DA) algorithm. In EWA-DA, the participants can learn to improve their strategies directly by participating in the auction continuously, without the controller participating.

Towards reinforcement learning in UAV relay for anti-jamming maritime communications

Maritime communications with sea surface reflections and sea wave occlusions are susceptible to jamming attacks due to the wide geographical area and intensive wireless communication services. Unmanned Aerial Vehicles (UAVs) help relay messages to improve communication performance, but the relay policy that depends on the rapidly changing maritime environments is difficult to optimize. In this paper, a reinforcement learning-based UAV relay policy for maritime communications is proposed to resist jamming attacks. Based on previous transmission performance, the relay location, the received power of the transmitted signal and the received jamming power, this scheme optimizes the UAV trajectory and relay power to save the energy consumption and decrease the Bit-Error-Rate (BER) of the maritime signals. A deep reinforcement learning-based scheme is also proposed, which designs a deep neural network with dueling architecture to further improve the communication performance and computational complexity. The performance bounds regarding the signal to interference plus noise ratio, energy consumption and the communication utility are provided based on the Nash equilibrium of the game against jamming, and the computational complexity of the proposed schemes is analyzed. Simulation results show that the proposed schemes improve the energy efficiency and decrease the BER compared with the benchmark.

Resource allocation for UAV-aided energy harvesting-powered D2D communications: A reinforcement learning-based scheme

Unmanned Aerial Vehicle (UAV) has become one of the most significant component in future wireless networks since its on-demand and cost-effective deployment. Meanwhile, Device-to-Device (D2D) communications with the nature of providing proximity-based service to improve network performance has emerged as an important feature in 5th generation cellular networks. However, the explosive growth of smart devices and bandwidth-hungry applications cause enormous energy consumption. Energy Harvesting (EH) as a potential solution of improving energy efficiency has shown great importance. Consequently, in this paper, we investigate the resource allocation problem in UAV-aided EH-powered D2D Cellular Networks (UAV-EH-DCNs). Our objective is to maximize the energy efficiency while guaranteeing the satisfaction of ground users (GUs). Owning to the non-convexity of the problem, we formulate the problem as a Markov decision process. Afterward, Deep Deterministic Policy Gradient (DDPG) is proposed to find the optimal strategy. Additionally, Long Short-Term Memory (LSTM) network is employed to facilitate the convergence speed by extracting the previous information of GUs satisfaction to determine the current resource allocation strategy. Numerical results adduce the validity of the proposed DDPG+LSTM algorithm as compared to the DDPG and deep Q-learning algorithms.

Resource Allocation for Cellular Zero-Touch Deterministic Industrial M2M Networks: A Reinforcement Learning-Based Scheme

In this letter, we investigate the resource allocation problem for cellular zero-touch deterministic industrial machine-to-machine networks. We consider a scenario in which multiple self-adaptive industrial machine-to-machine (M2M) networks are deployed underlaying cellular networks to emulate a variety of vertical use cases in future Industry 4.0 with the objective of guaranteeing the determinacy of latency by learning-inspired resource allocation policy. Specifically, in order to figure out the relationship between latency determinacy and the utilization of wireless resources meanwhile modeling the uncertainty of the stochastic environment, we first formulate the resource allocation problem as a Markov game by jointly considering transmission power control, frequency spectrum allocation, and the selection of base stations. Afterward, a random graph-based sparse long short-term memory network is proposed to solve the optimization problem while reducing the computational complexity. Finally, numerical result demonstrates the effectiveness of the proposed scheme.

Intelligent and compliant dynamic software license consolidation in cloud environment

Based on the virtualization technology and pushed by the softwarization paradigm and the actual demand for services and resources, commercial cloud data centers know an unprecedented expansion. The systematic presence of software and the services generated enabled the development of the already dense application expenses. This causes, not only a cost explosion, especially when proprietary solutions protected by licenses are in hand, but also, represents a critical need in terms of software asset and resource management at the SaaS level. In addition to these costs, inefficient resource utilization, and the resulting energy represent an important part of the operational expenditure of data centers and are still a hot topic despite the consolidation initiatives put in place. The main objective of the consolidation service is to maximize resource exploitation while minimizing energy consumption and costs, among others. Even so, we have noticed that the reported literature doesn’t treat license management in the cloud environment as a whole, especially, from the resource management perspective and the overwhelming majority of the consolidation work focuses on resource optimization at the IaaS level. Therefore, we propose a reinforcement learning-based scheme that allows efficient use of resources and optimizes costs, energy consumption, and resource wastage, while remaining compliant. The experimental results show that our intelligent consolidator outperforms the baseline approaches according to the evaluation metrics used regardless of the resource heterogeneity and the data center dimensionality.

HTTP Adaptive Streaming Framework with Online Reinforcement Learning

Dynamic adaptive streaming over HTTP (DASH) is an effective method for improving video streaming’s quality of experience (QoE). However, the majority of existing schemes rely on heuristic algorithms, and the learning-based schemes that have recently emerged also have a problem in that their performance deteriorates in a specific environment. In this study, we propose an adaptive streaming scheme that applies online reinforcement learning. When QoE degradation is confirmed, the proposed scheme adapts to changes in the client’s environment by upgrading the ABR model while performing video streaming. In order to adapt the adaptive bitrate (ABR) model to a changing network environment while performing video streaming, the neural network model is trained with a state-of-the-art reinforcement learning algorithm. The proposed scheme’s performance was evaluated using simulation-based experiments under various network conditions. The experimental results confirmed that the proposed scheme performed better than the existing schemes.

Machine Learning-Based Approach Using Open Data to Estimate PM2.5 over Europe

Air pollution is currently considered one of the most serious problems facing humans. Fine particulate matter with a diameter smaller than 2.5 micrometres (PM2.5) is a very harmful air pollutant that is linked with many diseases. In this study, we created a machine learning-based scheme to estimate PM2.5 using various open data such as satellite remote sensing, meteorological data, and land variables to increase the limited spatial coverage provided by ground-monitors. A space-time extremely randomised trees model was used to estimate PM2.5 concentrations over Europe, this model achieved good results with an out-of-sample cross-validated R2 of 0.69, RMSE of 5 μg/m3, and MAE of 3.3 μg/m3. The outcome of this study is a daily full coverage PM2.5 dataset with 1 km spatial resolution for the three-year period of 2018–2020. We found that air quality improved throughout the study period over all countries in Europe. In addition, we compared PM2.5 levels during the COVID-19 lockdown during the months March–June with the average of the previous 4 months and the following 4 months. We found that this lockdown had a positive effect on air quality in most parts of the study area except for the United Kingdom, Ireland, north of France, and south of Italy. This is the first study that depends only on open data and covers the whole of Europe with high spatial and temporal resolutions. The reconstructed dataset will be published under free and open license and can be used in future air quality studies.

Micro-Safe: Microservices- and Deep Learning-Based Safety-as-a-Service Architecture for 6G-Enabled Intelligent Transportation System

In this paper, we propose a microservices and deep learning-based scheme, termed as Micro-Safe, for provisioning Safety-as-a-Service (Safe-aaS) in a 6G environment. A Safe-aaS infrastructure provides customized safety-related decisions dynamically to the registered end-users. As the decisions are time-sensitive in nature, the generation of these decisions should incur minimum latency and high accuracy. Further, scalability and extension of the coverage of the entire Safe-aaS platform are also necessary. Considering road transportation as the application scenario, we propose Safe-aaS, which is a microservices- and deep learning-based platform for provisioning ultra-low latency safety services to the end-users in a 6G scenario. We design the proposed solution in two stages. In the first stage, we develop the microservices-enabled application layer to improve the scalability and adaptability of the traditional Safe-aaS platform. Moreover, we apply the state space model to represent the decision parameters requested and the decision delivered to the end-users. During the second stage, we use deep learning models to improve the accuracy in the decisions delivered to the end-users. Additionally, we apply an assortment of activation functions to analyze and compare the accuracy of the decisions generated in the proposed scheme. Extensive simulation of our proposed scheme, Micro-Safe, demonstrates that latency is improved by 26.1 – 31.2%, energy consumption is reduced by 22.1 – 29.9%, throughput is increased by 26.1 – 31.7%, compared to the existing schemes.

Facilitating Applications of SSVEP-Based BCIs by Within-Subject Information Transfer

The steady-state visual evoked potential based brain–computer interface (SSVEP–BCI) can provide high-speed alternative and augmentative communication in real-world applications. For individuals using a long-term BCI, within-subject (i.e., cross-day and cross-electrode) transfer learning could improve the BCI performance and reduce the calibration burden. To validate the within-subject transfer learning scheme, this study designs a 40-target SSVEP–BCI. Sixteen subjects are recruited, each of whom has performed experiments on three different days and has undergone the experiments of the SSVEP–BCIs based on the dry and wet electrodes. Several transfer directions, including the cross-day directions in parallel with the cross-electrode directions, are analyzed, and it is found that the transfer learning-based approach can maintain stable performance by zero training. Compared with the fully calibrated approaches, the transfer learning-based approach can achieve significantly better or comparable performance in different transfer directions. This result verifies that the transfer learning-based scheme is well suited for implementing a high-speed zero-training SSVEP–BCI, especially the dry electrode-based SSVEP–BCI system. A validation experiment of the cross-day wet-to-dry transfer, involving nine subjects, has shown that the average accuracy is 85.97 ± 5.60% for the wet-to-dry transfer and 77.69 ± 6.42% for the fully calibrated method with dry electrodes. By leveraging the electroencephalography data acquired on different days by different electrodes via transfer learning, this study lays the foundation for facilitating the long-term usage of the SSVEP–BCI and advancing the frontier of the dry electrode-based SSVEP–BCI in real-world applications.

PCA-based sub-surface structure and defect analysis for germanium-on-nothing using nanoscale surface topography

Empty space in germanium (ESG) or germanium-on-nothing (GON) are unique self-assembled germanium structures with multiscale cavities of various morphologies. Due to their simple fabrication process and high-quality crystallinity after self-assembly, they can be applied in various fields including micro-/nanoelectronics, optoelectronics, and precision sensors, to name a few. In contrast to their simple fabrication, inspection is intrinsically difficult due to buried structures. Today, ultrasonic atomic force microscopy and interferometry are some prevalent non-destructive 3-D imaging methods that are used to inspect the underlying ESG structures. However, these non-destructive characterization methods suffer from low throughput due to slow measurement speed and limited measurable thickness. To overcome these limitations, this work proposes a new methodology to construct a principal-component-analysis based database that correlates surface images with empirically determined sub-surface structures. Then, from this database, the morphology of buried sub-surface structure is determined only using surface topography. Since the acquisition rate of a single nanoscale surface micrograph is up to a few orders faster than a thorough 3-D sub-surface analysis, the proposed methodology benefits from improved throughput compared to current inspection methods. Also, an empirical destructive test essentially resolves the measurable thickness limitation. We also demonstrate the practicality of the proposed methodology by applying it to GON devices to selectively detect and quantitatively analyze surface defects. Compared to state-of-the-art deep learning-based defect detection schemes, our method is much effortlessly finetunable for specific applications. In terms of sub-surface analysis, this work proposes a fast, robust, and high-resolution methodology which could potentially replace the conventional exhaustive sub-surface inspection schemes.

Equalization Network-Aided SCMA Codec Scheme with Deep Learning

With the rapid development of the Internet of Things, sparse code multiple access (SCMA), which aims to promote spectrum efficiency and support massive connectivity in the future beyond fifth- and sixth-generation massive machine-type communication (mMTC) scenarios, has been widely investigated. To improve the bit error rate (BER) performance of the SCMA system in the uplink Rayleigh fading channel, we propose a novel deep learning-based SCMA codec scheme. The proposed scheme consists of an equalization network-aided decoder network and a denoising autoencoder- (DAE-) based encoder network. At the decoder, an equalization network and a multiuser detection network constitute the decoder network. The equalization network, composed of two deep neural network (DNN) units, compensates for the phase shift of the signal through the fading channel, which improves the antifading capability of the system. At the encoder, a complete DAE is constructed, which introduces an extra noise layer at the input of the encoder that yields a robust encoder output representation, improving the antinoise capability of the system. We use an end-to-end training method to train the SCMA codec and optimize the parameters and structural model of the neural network. Simulation results show that our proposed scheme can reduce the detection time and improve the BER performance of the system in the uplink Rayleigh fading channel.

Perturbation Theory-Aided Learned Digital Back-Propagation Scheme for Optical Fiber Nonlinearity Compensation

Derived from the regular perturbation treatment of the nonlinear Schr&#x00F6;dinger equation, a machine learning-based scheme to mitigate the intra-channel optical fiber nonlinearity is proposed. Referred to as the perturbation theory-aided (PA) learned digital back propagation (LDBP), the proposed scheme constructs a deep neural network (DNN) in a way similar to the split-step Fourier method: linear and nonlinear operations alternate. Inspired by the perturbation analysis, the intra-channel cross phase modulation term is conveniently represented by matrix operations in the DNN. The introduction of this term in each nonlinear operation considerably improves the performance, as well as enables the flexibility of PA-LDBP by adjusting the numbers of spans per step. The proposed scheme is evaluated by numerical simulations of a single-carrier optical fiber communication system operating at 32 Gbaud with 64-quadrature amplitude modulation and 20 &#x00D7; 80 km transmission distance. The results show that the proposed scheme achieves approximately 1 dB, 1.2 dB, 1.2 dB, and 0.5 dB performance gain in terms of <inline-formula><tex-math notation="LaTeX">${\it Q}$</tex-math></inline-formula> factor over LDBP, when the numbers of spans per step are 1, 2, 4, and 10, respectively. Two methods are proposed to reduce the complexity of PA-LDBP, i.e., pruning the number of perturbation coefficients and chromatic dispersion compensation in the frequency domain for multi-span per step cases. Investigation of the performance and complexity suggests that PA-LDBP attains improved performance gains with reduced complexity when compared to LDBP in the cases of 4 and 10 spans per step.

Deep Learning-Based Destination Prediction Scheme by Trajectory Prediction Framework

Destination prediction is an important task in vehicular ad hoc networks (VANETs), which benefits to resolve traffic congestion. The traditional destination prediction methods are mainly based on statistical learning, which is difficult to fit the movement pattern of vehicles and predict destination of a new query trajectory. In recent years, deep learning attracted more and more attentions since it has a better ability to adaptively extract features from data and fit complex models. The exiting deep learning-based destination prediction methods mainly extract features from past trajectory and fail to exploit future trajectory because the future locations are unknown, which results in a large prediction error. In this paper, we propose a novel deep learning-based destination prediction model that can predict the future trajectory and make full use of the information about it. Firstly, we exploit a trajectory prediction method to predict future locations, which makes the process of destination prediction continuous. Secondly, we design a branch network to determine which future location is the destination of the trajectory. We also evaluate our model with a real-world dataset. Experimental results show that our model can predict the coordinates of destination with a low prediction error.

Deep Learning-Based Scheduling Scheme for IEEE 802.15.4e TSCH Network

IEEE 802.15.4e time-slotted channel hopping (TSCH) is one of the most reliable resources of the Industrial Internet of Things (IIoT). TSCH operates on the slot-frame structure consisting of multiple channel-offsets and multiple slot-offsets. It is gaining acceptance due to its simple architecture and consume low power in industrial applications. The performance of TSCH is mainly dominated by the media access control (MAC) mechanism, which covers the refitment, enumeration, composition, and data transmission. However, in many cases, the data transmission schedules are not accurately prescribed. Therefore, most researchers are trying to define many pragmatic scenarios of scheduling. Their fundamental approach is to schedule TSCH network in a centralized way while framing scheduling based on network performance such as throughput and delay. In this work, a deep learning (DL)-based scheme has been proposed. TSCH network schedules for links to cell assignment of a slot-frame can be constructed as a maximum edge weighted bipartite matching approach. In this paper, we design bipartite edge weight to be composed of throughput and delay, and we use the Hungarian algorithm for proper cell assignment. With the Hungarian scheduling algorithm, we generate the training data and train a deep neural network (DNN) accordingly. In the simulation, we consider a simple TSCH network with 5 nodes where 12 links are formulated, and we consider 16 cells for the link assignment. The simulation results show that the proposed deep learning-based scheduling scheme can provide performance similar to the Hungarian algorithm-based scheduling scheme with above 90% accuracy and nearly 80% execution time reduction.

A Deep Reinforcement Learning-Based Scheme for Solving Multiple Knapsack Problems

A knapsack problem is to select a set of items that maximizes the total profit of selected items while keeping the total weight of the selected items no less than the capacity of the knapsack. As a generalized form with multiple knapsacks, the multi-knapsack problem (MKP) is to select a disjointed set of items for each knapsack. To solve MKP, we propose a deep reinforcement learning (DRL) based approach, which takes as input the available capacities of knapsacks, total profits and weights of selected items, and normalized profits and weights of unselected items and determines the next item to be mapped to the knapsack with the largest available capacity. To expedite the learning process, we adopt the Asynchronous Advantage Actor-Critic (A3C) for the policy model. The experimental results indicate that the proposed method outperforms the random and greedy methods and achieves comparable performance to an optimal policy in terms of the profit ratio of the selected items to the total profit sum, particularly when the profits and weights of items have a non-linear relationship such as quadratic forms.