Unmanned Aerial Vehicle Base Research Articles

Resource Management in 5G Networks Assisted by UAV Base Stations: Machine Learning for Overloaded Macrocell Prediction Based on Users’ Temporal and Spatial Flow

The rapid growth of data traffic due to the demands of new services and applications poses new challenges to the wireless network. Unmanned aerial vehicles (UAVs) can be a solution to support wireless networks during congestion, especially in scenarios where the region has high traffic peaks due to the temporal and spatial flow of users. In this paper, an intelligent machine-learning-based system is proposed to deploy UAV base stations (UAV-BS) to temporarily support the mobile network in regions suffering from the congestion effect caused by the high density of users. The system includes two main steps, the load prediction algorithm (LPA) and the UAV-BSs clustering and positioning algorithm (UCPA). In LPA, the load history generated by the mobile network is used to predict which macrocells are congested. In UCPA, planning is performed to calculate the number of UAV BSs needed based on two strategies: naïve and optimized, in addition to calculating the optimal positioning for each device requested to support the overloaded macrocells. For prediction, we used two models, generalized regression neural networks (GRNN) and random forest, and the results showed that both models were able to make accurate predictions, and the random forest model was better with an accuracy of over 85%. The results showed that the intelligent system significantly reduced the overhead of the affected macrocells, improved the quality of service (QoS), and reduced the probability of blocking users, as well as defined the preventive scheduling for the UAV BSs, which benefited the scheduling and energy efficiency.

On the Performance of Laser-Powered UAV-Assisted SWIPT Enabled Multiuser Communication Network With Hybrid NOMA

Owing to the factors such as controllable mobility, ready-to-use technology, low cost, easy implementation, and so on, unmanned aerial vehicle (UAV) possesses tremendous potential to be one of the primary candidates for next-generation (6G) wireless networks. This paper presents a UAV-assisted multiuser communication network where a multiple antenna UAV base station (BS) serves multiple single antenna ground users (GUs). UAV-BS uses a laser source-based charging mechanism to fulfill its power requirement and applies simultaneous wireless information and power transfer (SWIPT) in the downlink in order to provide desired power to energy-constrained GUs. Also, a clustering-based hybrid multiple access technique is used that combines both orthogonal multiple access (OMA) and non-orthogonal multiple access (NOMA) signaling to transmit the information to all GUs, simultaneously. Due to the involved analytical complexity corresponding to the multiple antennas and users, we use a hybrid beamforming method for efficient communication. Next, we analyze the performance of the proposed framework in terms of user outage probabilities, their respective throughput, and average power harvested considering non-linear energy harvesting and derive expressions of these performance metrics. Moreover, we formulate an optimization problem where the throughput of one GU is maximized by optimally choosing the power allocation parameter while ensuring the desired target throughput at other GU in each cluster. We also illustrate how crucial is the optimal selection of the target rates to maximize the network performance. Simulation results are provided to validate the accuracy of derived expressions and to highlight the dominance of hybrid beamforming and hybrid NOMA compared to conventional methods on the performance of the considered network.

Joint Resource, Deployment, and Caching Optimization for AR Applications in Dynamic UAV NOMA Networks

The cache-enabling unmanned aerial vehicle (UAV) non-orthogonal multiple access (NOMA) networks for mixture of augmented reality (AR) and normal multimedia applications are investigated, which is assisted by UAV base stations. The user association, power allocation of NOMA, deployment of UAVs and caching placement of UAVs are jointly optimized to minimize the content delivery delay. A branch and bound (BaB) based algorithm is proposed to obtain the per-slot optimization. To cope with the dynamic content requests and mobility of users in practical scenarios, the original optimization problem is transformed to a Stackelberg game. Specifically, the game is decomposed into a leader level user association sub-problem and a number of power allocation, UAV deployment and caching placement follower level sub-problems. The long-term minimization was further solved by a deep reinforcement learning (DRL) based algorithm. Simulation result shows that the content delivery delay of the proposed BaB based algorithm is much lower than benchmark algorithms, as the optimal solution in each time slot is achieved. Meanwhile, the proposed DRL based algorithm achieves a relatively low long-term content delivery delay in the dynamic environment with lower computation complexity than BaB based algorithm.

Organizational Resource Allocation by Mobile Edge Computing in the Context of the Internet of Things

In recent years, the number of electronic devices and users has increased sharply with the rapid development and progress of electronic information technology. The motivation of this paper is to optimize the organization’s resource allocation strategy in the Internet of Things (IoT) environment. The optimal path planning and information processing efficiency are improved through Unmanned Aerial Vehicle (UAV) technology and migration optimization algorithm. The research method is migration optimization algorithm and UAV dynamic network. The information processing capability of traditional wireless communication systems has gradually been unable to meet the actual information processing needs. Therefore, a static network migration algorithm is constructed based on multiple-user multilateral edge computing servers. It migrates each user’s information processing task to the neighboring edge confidence processing server and uses the information processing server to perform auxiliary calculations. The simulation model adds a utility function that simulates energy consumption, delay weighting, and maximum extreme value, combined with the allocation strategy of optimizing each user’s information processing task to achieve the optimization goal. The static network migration algorithm established in this simulation has better results than other benchmark algorithms. Both scenario 1 and scenario 2 in the simulation show very close performance to the optimal solution. Meanwhile, a migration algorithm that can provide wireless charging for UAVs is built by a dynamic edge computing model based on the time associated with the UAV base station and multiple end users. Combined with completing the information processing tasks in each time slot, the energy arrival is also non-directional. The dynamic network migration algorithm can optimize the number of tasks absorbed by the end-user based on the current online status of the system without knowing the global information. The optimized target equation is related to the queue stability, and the parameter V has a linear relationship with the queue backlog length. Here, the problem of computing migration is studied in Mobile Edge Computing (MEC). The results show that the utility function of the weighted sum has an approximately linear relationship with the weights. As the value of the utility function increases, so does the weight function. The optimal data throughput of the proposed model is 70,000 bits, while the optimal data throughput of the state-of-the-art model is 68,000 bits. Therefore, the data transmission performance of the model presented here is better than that of other models. MEC can be significantly improved the efficiency of organizational resource allocation. Combining UAV and wireless charging technology, the computing and communication resource allocation issues of the UAV’s edge computing system are comprehensively discussed to improve the performance and efficiency of the network.

UAV-BS Formation Control Method Based on Loose Coupling Structure

We propose a novel formation control method considering the challenges of formation keeping, unmanned aerial vehicle (UAV) control, and position adjustment in UAV Base Station (UAV-BS) formation control scenarios. The method is based on a loose coupling structure inspired by the traditional virtual structure method. Firstly, a type of new formation is designed for UAV-BS group. When the UAV-BS group keep a stable network, the centers of all UAV-BS moving trajectory, e.g., circular, are treated as virtual points to form a rigid body. When the UAV-BS group transforms between different formations, the real positions of the UAV-BS group are used to form a rigid body. Secondly, we redesign a trajectory tracking method for UAV-BS to control the flight direction. A flight direction controller is designed for UAV-BS, based on nonlinear guidance logic. According to the empirical model of error feedback, an error corrector is designed to control the position error between the UAV-BS and the virtual point. Simulation results show that the proposed algorithm can satisfactorily complete the formation and control of UA-BS. Besides, the algorithm can reduce the difficulty of formation control of UAV-BS and the requirements of formation control for computing power and network communication.

Energy-Efficient UAV Trajectory Design with Information Freshness Constraint via Deep Reinforcement Learning

Unmanned aerial vehicle (UAV) technique with flexible deployment has enabled the development of Internet of Things (IoT) applications. However, it is difficult to guarantee the freshness of information delivery for the energy-limited UAV. Thus, we study the trajectory design in the multiple-UAV communication system, in which the massive ground devices send the individual information to mobile UAV base stations under the demand of information freshness. First, an energy-efficiency (EE) maximization optimization problem is formulated under the rest energy, safety distance, and age of information (AoI) constraints. However, it is difficult to solve the optimization problem due to the nonconvex objective function and unknown dynamic environment. Second, a trajectory design based on the deep Q-network method is proposed, in which the state space considering energy efficiency, rest energy, and AoI and the efficient reward function related with EE performance are constructed, respectively. Furthermore, to avoid the dependency of training data for the neural network, the experience replay and random sampling for batch are adopted. Finally, we validate the system performance of the proposed scheme. Simulation results show that the proposed scheme can achieve a better EE performance compared with the benchmark scheme.

Reinforcement Learning Aided UAV Base Station Location Optimization for Rate Maximization

The application of unmanned aerial vehicles (UAV) as base station (BS) is gaining popularity. In this paper, we consider maximization of the overall data rate by intelligent deployment of UAV BS in the downlink of a cellular system. We investigate a reinforcement learning (RL)-aided approach to optimize the position of flying BSs mounted on board UAVs to support a macro BS (MBS). We propose an algorithm to avoid collision between multiple UAVs undergoing exploratory movements and to restrict UAV BSs movement within a predefined area. Q-learning technique is used to optimize UAV BS position, where the reward is equal to sum of user equipment (UE) data rates. We consider a framework where the UAV BSs carry out exploratory movements in the beginning and exploitary movements in later stages to maximize the overall data rate. Our results show that a cellular system with three UAV BSs and one MBS serving 72 UE reaches 69.2% of the best possible data rate, which is identified by brute force search. Finally, the RL algorithm is compared with a K-means algorithm to study the need of accurate UE locations. Our results show that the RL algorithm outperforms the K-means clustering algorithm when the measure of imperfection is higher. The proposed algorithm can be made use of by a practical MBS–UAV BSs–UEs system to provide protection to UAV BSs while maximizing data rate.

Two-Stage Channel Adaptive Algorithm for Unmanned Aerial Vehicles Localization with Cellular Networks

Unmanned aerial vehicle (UAV) is regarded as a powerful tool to expand the existing ground wireless network into aerial space. Since high mobility is an essential characteristic for UAV, it is important to carry out an accurate, real-time, and high-precision localization in terms of safe operation and communication link maintenance. The cellular network-based localization technology has provided UAV a solution with both high coverage and seamless connection. However, the complex channel environment between the UAV and terrestrial base station (BS) would have weakened the localization performance. To solve this problem, a two-stage channel adaptive algorithm for cellular-connected UAV has been proposed. The first stage of the algorithm is to revise the observation error introduced by the complex channel environment using the model of DDPG. The second stage is to locate the UAV position with TDOA algorithm using the revised observation values. Simulation results have demonstrated that the proposed algorithm can achieve the channel adaptive effect by revising the observation errors and improve location performance greatly, especially for UAVs at a relative lower altitude.

Intelligent Reflecting Surface and UAV Assisted Secrecy Communication in Millimeter-Wave Networks

In this paper, we consider the secure transmission problem in an unmanned aerial vehicle (UAV) and intelligent reflecting surface (IRS) assisted mmWave networks in the presence of an eavesdropper. The UAV base station (UAV-BS) and IRS are deployed to overcome the blockages. Artificial noise (AN) is exploited against the eavesdropper. Aiming to maximize the secrecy rate, we jointly design the positions and beamforming of UAV-BS and IRS, where the positions here represent the deployed position of the UAV-BS and the activation position of the IRS. Meanwhile, the maximum transmits power, minimum height constraint for UAV-BS, and the legitimate receiver minimum rate constraint are required. To obtain the optimal UAV-BS position, the elevation angle-dependent probabilistic LoS air-to-ground channel is exploited in this network. To tackle the formulated non-convex problem, we divide it into two subproblems: (1) design UAV-BS and IRS positions; (2) design UAV-BS and IRS beamforming. Moreover, to reduce the complexity, we propose an ideal beamforming model to find the near-optimal positions of UAV-BS and IRS. Then, semidefinite relaxation (SDR) is utilized to cope with the highly coupled and high-dimensional variable vectors. Finally, we obtain a sub-optimal solution by an efficient and low-complexity alternating optimization algorithm. Numerical results demonstrate the improvement of secrecy rate by jointly designing the positions as well as the significant performance gains achieved by our proposed schemes over benchmark schemes.

Multi-Objective UAV Positioning Mechanism for Sustainable Wireless Connectivity in Environments with Forbidden Flying Zones

A communication system based on unmanned aerial vehicles (UAVs) is a viable alternative for meeting the coverage and capacity needs of future wireless networks. However, because of the limitations of UAV-enabled communications in terms of coverage, energy consumption, and flying laws, the number of studies focused on the sustainability element of UAV-assisted networking in the literature was limited thus far. We present a solution to this problem in this study; specifically, we design a Q-learning-based UAV placement strategy for long-term wireless connectivity while taking into account major constraints such as altitude regulations, nonflight zones, and transmit power. The goal is to determine the best location for the UAV base station (BS) while reducing energy consumption and increasing the number of users covered. Furthermore, a weighting method is devised, allowing energy usage and the number of users served to be prioritized based on network/battery circumstances. The suggested Q-learning-based solution is contrasted to the standard k-means clustering method, in which the UAV BS is positioned at the centroid location with the shortest cumulative distance between it and the users. The results demonstrate that the proposed solution outperforms the baseline k-means clustering-based method in terms of the number of users covered while achieving the desired minimization of the energy consumption.

Analytical Blind Beamforming for a Multi-Antenna UAV Base-Station Receiver in Millimeter-Wave Bands.

Over the last decade, unmanned aerial vehicles (UAVs) with antenna arrays have usually been employed for the enhancement of wireless communication in millimeter-wave bands. They are commonly used as aerial base stations and relay platforms in order to serve multiple users. Many beamforming methods for improving communication quality based on channel estimation have been proposed. However, these methods can be resource-intensive due to the complexity of channel estimation in practice. Thus, in this paper, we formulate an MIMO blind beamforming problem at the receivers for UAV-assisted communications in which channel estimation is omitted in order to save communication resources. We introduce one analytical method, which is called the analytical constant modulus algorithm (ACMA), in order to perform blind beamforming at the UAV base station; this relies only on data received by the antenna. The feature of the constant modulus (CM) is employed to restrict the target user signals. Algebraic operations, such as singular value decomposition (SVD), are applied to separate the user signal space from other interferences. The number of users in the region served by the UAV can be detected by exploring information in the measured data. We seek solutions that are expressible as one Kronecker product structure in the signal space; then, the beamformers that correspond to each user can be successfully estimated. The simulation results show that, by using this analytically derived blind method, the system can achieve good signal recovery accuracy, a reasonable system sum rate, and acceptable complexity.

On the Performance of a UAV-Aided Wireless Network Based on NB-IoT

In recent years, interest in Unmanned Aerial Vehicles (UAVs) as a means to provide wireless connectivity has substantially increased thanks to their easy, fast and flexible deployment. Among the several possible applications of UAV networks explored by the current literature, they can be efficiently employed to collect Internet-of-Things (IoT) data because the non-stringent latency and small-size traffic type is particularly suited for UAVs’ inherent characteristics. However, the implications coming from the implementation of existing technology in such kinds of nodes are not straightforward. In this article, we consider a Narrow Band IoT (NB-IoT) network served by a UAV base station. Because of the many configurations possible within the NB-IoT standard, such as the access structure and numerology, we thoroughly review the technical aspects that have to be implemented and may be affected by the proposed UAV-aided IoT network. For proper remarks, we investigate the network performance jointly in terms of the number of successful transmissions, access rate, latency, throughput and energy consumption. Then, we compare the obtained results on different and known trajectories in the research community and study the impact of varying UAV parameters such as speed and height. Moreover, the numerical assessment allows us to extend the discussion to the potential implications of this model in different scenarios. Thus, this article summarizes all the main aspects that must be considered in planning NB-IoT networks with UAVs.

Special Issue on Unmanned Aerial Vehicle (UAV)-Enabled Green Communications and Networking

Supporting explosive data traffic and massive connections is the most critical challenge of 5G and beyond networks. To overcome this challenge, an unmanned aerial vehicle (UAV) can be exploited as a promising communication platform in future wireless networks due to its inherent advantage of flexibility, mobility, hovering and low cost. In particular, the air-to-air and air-to-ground channels are much better than the ground-to-ground channels when a UAV is used as a base station. In many cases, the communication between a UAV base station and a user equipment can benefit from the line-of-sight (LoS) channel in between. Under such circumstances, mmWave transmission becomes practicable for UAV communications, which could provide much higher data rate.

Latency Optimization of UAV-Enabled MEC System for Virtual Reality Applications Under Rician Fading Channels

This letter considers an unmanned aerial vehicle (UAV)-enabled mobile edge computing (MEC) system, where UAV base stations (UBSs) are deployed to cache, process, and deliver virtual reality (VR) content from a cloud server to VR users (VRUs). Under Rician fading channel model, we optimize various resource allocation parameters, e.g., association of VRUs with UBSs, caching policy, computing-capacity allocation, and location of UBSs, with the objective of minimizing the maximum latency, subject to computing, caching, and power constraints at the UBSs. The problem is non-convex and solved by using alternating optimization and successive convex approximation techniques. Our simulation results clearly show the importance of the proposed joint optimization algorithm, in terms of performance improvement, over different benchmark schemes.

Efficient algorithms for decision making and coverage deployment of connected multi-low-altitude platforms

Unmanned aerial vehicles (UAVs) have gained significantly in popularity in recent years, and they are coupled with the emerging Internet-of-Things, 5G, and mobile edge computing. Their application domains will continue to expand. UAVs can deliver various IoT-driven applications (e.g., video surveillance UAVs-based applications) anytime and anywhere. Indeed, the rapid deployment of such applications in different environments (e.g., urban or wildlife surveillance/sensing environments) is one of the critical factors to support the emergence of new services and applications (e.g., 5G/B5G UAVs base station, civil and commercial IoT applications). However, several research challenges need to be addressed before the latter can be deployed in the real world. In this paper, we propose a novel approach that enables efficient coverage using a set of UAVs. We define two multi-objective sub-problems. The first sub-problem allows selecting a minimum number of appropriate UAVs for deployment as Low-Altitude-Platforms (LAP) over an area of interest from an extensive collection of options. We model this sub-problem as multi-criteria decision-making (MCDM) problem considering various UAV features and user constraints and solve it using a novel multi-criteria selection algorithm named SAGA. The second sub-problem is a coverage optimization problem that enables controlling UAVs’ hover locations and use gimbal-mounted rotating cameras or multi-lens cameras to increase the coverage accordingly. We propose a strategy aiming at optimizing hovering locations of UAVs and then rotating their cameras. The process comprises two techniques to solve the latter sub-problem: (1) an improved preference-guided genetic algorithm named NSPGGA; (2) a hybrid heuristic DCXGA built upon three algorithmic techniques: divide-and-conquer, greedy search, and exhaustive search. We carried out comparative analyses with seven MCDM methods and ten multi-objective evolutionary/swarm intelligence algorithms. The proposed algorithms outperformed the benchmarking techniques and showed remarkable results, e.g., the selection algorithm SAGA exhibits a high success rate, accuracy, and consistency. The preference-guided genetic algorithm NSPGGA achieves better efficiency, and it is four times faster than the famed NSGA-II. Finally, the hybrid heuristic DCXGA allows having more significant imaging coverages with few camera rotations. The aforementioned vital results are validated through diverse and intensive simulation scenarios.

Trajectory Design and Access Control for Air–Ground Coordinated Communications System With Multiagent Deep Reinforcement Learning

Unmanned-aerial-vehicle (UAV)-assisted communications has attracted increasing attention recently. This article investigates air–ground coordinated communications system, in which trajectories of air UAV base stations (UAV-BSs) and access control of ground users (GUs) are jointly optimized. We formulated this optimization problem as a mixed cooperative–competitive game, where each GU competes for the limited resources of UAV-BSs to maximize its own throughput by accessing a suitable UAV-BS, and UAV-BSs cooperate with each other and design their trajectories to maximize the defined <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fair throughput</i> to improve the total throughput and keep the GU fairness. Moreover, the action space of GUs is discrete, while that of UAV-BS is continuous. To tackle this hybrid action space issue, we transform the discrete actions into continuous action probabilities and propose a multiagent deep reinforcement learning (MADRL) approach, named air–ground probabilistic multiagent deep deterministic policy gradient (AG-PMADDPG). With well-designed rewards, AG-PMADDPG can coordinate two types of agents, UAV-BSs and GUs, to achieve their own objectives based on local observations. Simulation results demonstrate that AG-PMADDPG can outperform the benchmark algorithms in terms of throughput and fairness.

On-demand UAV base station deployment for wireless service of crowded tourism areas

With the support of wireless network, tourists in tourism areas could enjoy various tourism information search and smart tourism–related services. However, due to the limited capacity of wireless networks, in peaking seasons, tourism area crowding in local areas could result in emergency and temporary wireless network congestion. While increasing infrastructure investment (e.g., densifying base stations) is desiring for peak seasons, it can be a waste of resource for significantly shrunk tourist arrival in off seasons. In response to the temporary network congestion offloading demand, this paper proposes an on-demand coverage solution based on unmanned aerial vehicle (UAV) base stations. Firstly, taking the air-to-ground channel characteristic into account, we define the effective coverage radius, based on which the optimal altitude of UAV BS is derived. Then, to tackle the inherent challenge of irregular tourist distribution issue in tourism areas, an automatic UAV BS deployment algorithm is designed to determine the minimal number of UAV BSs and their two-dimensional coordinates simultaneously. Simulation results show that the proposed solution could realize efficient UAV BS on-demand deployment.

3D Deployment of Unmanned Aerial Vehicle‐Base Station Assisting Ground‐Base Station

Unmanned aerial vehicles (UAVs), also named as drones, have become a modern model to provide a quick wireless communication infrastructure. They have been used when conventional base stations’ capacity is suffering in some extreme cases such as congestion inside the cell or a special event. This paper proposes an efficient three‐dimension (3D) placement of a single UAV‐assisted wireless network in such cases. Our proposed model assists the ground base station (GBS) using the UAV to serve arbitrary distributed users considering the impact of the obstacle blockage over the well‐known air‐to‐ground (A2G) path model. This work is aimed at optimizing the percentage of available bandwidth that must be provided to the UAV in order to maximize the number of served users. In addition, it finds the 3D placement of the UAV base station (UAVBS) that maximizes the number of served users, each with maximum quality‐of‐service (QoS). The exhaustive search and particle swarm optimization (PSO) algorithms are used to find the problem’s solution.

On the Physical Layer Security of the Cooperative Rate-Splitting-Aided Downlink in UAV Networks

Unmanned Aerial Vehicles (UAVs) have found compelling applications in intelligent logistics, search and rescue as well as in air-borne Base Station (BS). However, their communications are prone to both channel errors and eavesdropping. Hence, we investigate the max-min secrecy fairness of UAV-aided cellular networks, in which Cooperative Rate-Splitting (CRS) aided downlink transmissions are employed by each multi-antenna UAV Base Station (UAV-BS) to safeguard the downlink of a two-user Multi-Input Single-Output (MISO) system against an external multi-antenna Eavesdropper ( $Eve$ ). Realistically, only Imperfect Channel State Information (ICSI) is assumed to be available at the transmitter. Additionally, we consider a realistic total power constraint and guarantee the specific Quality of Service (QoS) requirements of the legitimate users. To handle the worst-case channel uncertainty of the legitimate users and an external $Eve$ , we conceive a robust secure resource allocation algorithm, which maximizes the minimum worst-case secrecy rate of the legitimate users. Based on the CRS principle, the transmitter splits and encodes the messages of legitimate users into common as well as private streams and the user having stronger CSI is asked to help the cell-edge user by opportunistically forwarding its decoded common message. In contrast to the existing schemes adopted in the literature for ensuring secure transmission of the first cooperative phase only, in our proposed solution the common message has a twin-fold mission. Explicitly, apart from serving as the desired message, it also acts as Artificial Noise (AN) for drowning out $Eve$ without consuming extra power. This is in stark contrast to the conventional AN designs. In the second phase, the pure AN is directed towards the $Eve$ , deploying a robust Maximum Ratio Transmitter (MRT) beamformer at the UAV-BS. To solve the resultant non-convex optimization problem we resort to the Sequential Parametric Convex Approximation (SPCA) method together with a bespoke initialization algorithm to avoid any failure due to infeasibility. Our simulation results confirm that the proposed secure transmission scheme outperforms the existing cooperative benchmarkers.

Optimal 3D UAV base station placement by considering autonomous coverage hole detection, wireless backhaul and user demand

The rising number of technological advanced devices making network coverage planning very challenging tasks for network operators. The transmission quality between the transmitter and the end users has to be optimum for the best performance out of any device. Besides, the presence of coverage hole is also an ongoing issue for operators which cannot be ignored throughout the whole operational stage. Any coverage hole in network operators' coverage region will hamper the communication applications and degrade the reputation of the operator's services. Presently, there are techniques to detect coverage holes such as drive test or minimization of drive test. However, these approaches have many limitations. The extreme costs, outdated information about the radio environment and high time consumption do not allow to meet the requirement competently. To overcome these problems, we take advantage of Unmanned aerial vehicle (UAV) and Q-learning to autonomously detect coverage hole in a given area and then deploy UAV based base station (UAV-BS) by considering wireless back-haul with the core network and users demand. This machine learning mechanism will help the UAV to eliminate human-in-the-loop (HiTL) model. Later, we formulate an optimisation problem for 3D UAV-BS placement at various angular positions to maximise the number of users associated with the UAV-BS. In summary, we have illustrated a cost-effective as well as time saving approach of detecting coverage hole and providing on-demand coverage in this article.