Low Data Rate Research Articles

Fog Computing-Based System for Decentralized Smart Parking System by Using Firebase

The growth of vehicle number is unavoidable whilst the availability of parking is not directly proportional with this condition. Nowadays, many shopping centers do not have sufficient parking spot, causing customers to have difficulty in finding available parking spots. Research has been conducted to tackle the issue of finding available parking spots. Much of this research proposed the narrowband-Internet of things (NB-IoT) as a fog node. For communication purposes, this NB-IoT-based fog node has some shortcomings, such as security and privacy, lower data rate, higher cost in development, dependency with wireless system, and only covers one area. In this research, the fog computing was proposed to decentralize smart parking system by using Firebase to cover several areas or malls in one system and interface. Instead of using NB-IoT, this research employed decentralized local server as a fog node to deliver a fast data exchange. Firestore database (Firebase) was also used to secure, manage, and analyze the data in the cloud. Conjunctively, the Android application was created as a user interface to book and find the availability of parking spots. The Android application was built using Android Studio and implemented authentication to keep the data access secure and private. The testing scenario was done following the design unified modeling language (UML). The research results confirmed that the fog computing system successfully supported the decentralized smart parking system and was able to be implemented for covering several areas or malls in one system.

Lossless Encoding of Time-Aggregated Neuromorphic Vision Sensor Data Based on Point-Cloud Compression.

Neuromorphic Vision Sensors (NVSs) are emerging sensors that acquire visual information asynchronously when changes occur in the scene. Their advantages versus synchronous capturing (frame-based video) include a low power consumption, a high dynamic range, an extremely high temporal resolution, and lower data rates. Although the acquisition strategy already results in much lower data rates than conventional video, NVS data can be further compressed. For this purpose, we recently proposed Time Aggregation-based Lossless Video Encoding for Neuromorphic Vision Sensor Data (TALVEN), consisting in the time aggregation of NVS events in the form of pixel-based event histograms, arrangement of the data in a specific format, and lossless compression inspired by video encoding. In this paper, we still leverage time aggregation but, rather than performing encoding inspired by frame-based video coding, we encode an appropriate representation of the time-aggregated data via point-cloud compression (similar to another one of our previous works, where time aggregation was not used). The proposed strategy, Time-Aggregated Lossless Encoding of Events based on Point-Cloud Compression (TALEN-PCC), outperforms the originally proposed TALVEN encoding strategy for the content in the considered dataset. The gain in terms of the compression ratio is the highest for low-event rate and low-complexity scenes, whereas the improvement is minimal for high-complexity and high-event rate scenes. According to experiments on outdoor and indoor spike event data, TALEN-PCC achieves higher compression gains for time aggregation intervals of more than 5 ms. However, the compression gains are lower when compared to state-of-the-art approaches for time aggregation intervals of less than 5 ms.

Modeling Development of a Diamagnetically Stabilized Magnetically Levitated Gravimeter.

The aim of this work is to create a new type of gravimeter that can function effectively in the challenging conditions of space, specifically on the surfaces of planets and moons. The proposed device, called a diamagnetically stabilized magnetically levitated gravimeter (DSMLG), uses magnetic forces to balance a test mass against the force of gravity, allowing for accurate measurements. A diamagnetically stabilized levitation structure comprises a floating magnet, diamagnetic material, and a lifting magnet. The floating magnet levitates between two diamagnetic plates without the need for external energy input due to the interaction between the magnetic forces of the floating magnet and the stabilizing force of the diamagnetic material. This structure allows for stable levitation of the floating magnet without requiring additional energy. The goal is to design a gravimeter that is lightweight, requires minimal power, can withstand extreme temperatures and shocks, and has a low data rate. The authors envision this gravimeter being used on various robotic spacecraft, such as landers and rovers, to study the interiors of rocky and icy celestial bodies. This paper reports on the results of a finite element model analysis of the DSMLG and the strength of the resulting diamagnetic spring. The findings contribute to the understanding of the levitation characteristics of diamagnetically stabilized structures and provide valuable insights for their practical applications, including in the development of the proposed DSMLG.

Quantized Zeroth-Order Gradient Tracking Algorithm for Distributed Nonconvex Optimization Under Polyak-Łojasiewicz Condition.

This article focuses on distributed nonconvex optimization by exchanging information between agents to minimize the average of local nonconvex cost functions. The communication channel between agents is normally constrained by limited bandwidth, and the gradient information is typically unavailable. To overcome these limitations, we propose a quantized distributed zeroth-order algorithm, which integrates the deterministic gradient estimator, the standard uniform quantizer, and the distributed gradient tracking algorithm. We establish linear convergence to a global optimal point for the proposed algorithm by assuming Polyak-Łojasiewicz condition for the global cost function and smoothness condition for the local cost functions. Moreover, the proposed algorithm maintains linear convergence at low-data rates with a proper selection of algorithm parameters. Numerical simulations validate the theoretical results.

Deep Reinforcement Learning-Based Scheduling for NR-U/WiGig Coexistence in Unlicensed mmWave Bands

This paper investigates the coexistence of the New Radio-based access to Unlicensed spectrum (NR-U) network and the Wireless Gigabit (WiGig) network in unlicensed millimeter-wave (mmWave) bands. To enable the NR-U network to achieve equitable and harmonious spectrum sharing with WiGig systems, we develop two new classes of user equipment (UE) scheduling schemes by exploiting the deep reinforcement learning (DRL) technique. Specifically, we first propose the distributed deep reinforcement learning scheduling (DeepDS) scheme, wherein multiple deep neural networks (DNNs) are used to make decisions for different panels in an independent fashion. Thereafter, to reduce the computational cost of adopting multiple DNNs, we design the centralized deep reinforcement learning scheduling (DeepCS) scheme that introduces the shared DNN framework to perform decisions for all panels in parallel at one time. The objective of both DeepDS and DeepCS is to maximize the total data rate of the NR-U network with as little interference to WiGig systems as possible, while satisfying the quality of service (QoS) requirement for each UE. We first formulate this problem into the constrained Markov decision process framework. To address the multi-constraint issue, we put forth a new DRL algorithm that incorporates the Lagrangian primal-dual optimization into the deep Q-network framework, referred to as adaptive multi-constraint deep Q-network (AMC-DQN). With AMC-DQN, both DeepDS and DeepCS can achieve their goals even without acquiring prior operations about the WiGig network. Simulation results show that compared with the state-of-the-art omniLBT and dirLBT, both DeepDS and DeepCS yield significant performance benefits in terms of the total network data rate. We also demonstrate the ability of DeepDS and DeepCS to satisfy the QoS requirements of different UEs and their robustness against various simulation setups. Furthermore, compared with DeepDS, DeepCS can save a large amount of computational cost although at the expense of a slightly lower data rate.

Developing and exploring the validity of a patient reported experience measure for adult inpatient diabetes care.

To develop and explore the validity of a Patient Reported Experience Measure (PREM) for adult inpatient diabetes care. 27 in-depth interviews were conducted to inform the development of the 42-item PREM which was cognitively tested with 10 people. A refined 38-item PREM was piloted with 228 respondents completing a paper (n = 198) or online (n = 30) version. The performance of the PREM was evaluated by exploring (i) uptake/number of responses and (ii) survey validity by investigating whether the PREM data were of adequate quality and delivered useful information. The PREM had low drop-out or missing data rates suggesting it was appropriately constructed. Analysis of item frequencies and variances, and problem score calculations concluded that questions provided sufficient score differentiation. This new PREM allows for experiences of inpatient diabetes care to be measured, understood and reported on to help identify priority areas for improving care quality.

Comparative Analysis for Radio Channel Propagation Models in the City of Tripoli/ Libya for 4G/LTE Networks

The accurate prediction of radio channel propagation is one of the most important factors for the design and optimization of wireless communication systems. Path loss is the key element to design appropriate propagation model. There are different propagation models used to predict path loss, however, inaccurate propagation model may result in unsatisfactory services for the Global System for Mobile communications (GSM) network. These include low data rates, high co-channel interference, and wasted received power.This paper presents a comparative analysis of radio channel propagation models specifically for urban environment. The aim is to evaluate and compare the performance of various models and accurately predict path loss and signal behavior in urban settings. The research focuses on adapting the Okumura-Hata model which is used by the Libya Telecom and Technology Company (LTT) for Long Term Evolution (LTE) technology in the GSM Band. This study is based on real time measurements collected in downtown Tripoli in Libya. It suggests considering other models, such as Standard Propagation Model (SPM), Cost-231, and Ericsson. Using MATLAB for simulation, the findings indicate that SPM model is the best fit especially when distance is over ~22 Km.

A Design for Increasing the Capacity Fourfold in NB-IoT Systems using A Modified Symbol Time Compression Approach

One of the most significant obstacles facing the Internet of Things (IoT) is how to offer support for communication to an expanding number of linked items. Narrow-band Internet of Things (NB-IoT), which is currently an aspect of fifth-generation (5G), is a new narrow-band wireless communication technology that has evolved in recent years to address this issue. To save power, NB-IoT currently exploits modulation schemes that are characterized by low order to deliver low data rate. However, as the number of applications dependent on data rate increases, NB-IoT needs to use data rate improvement technologies that do not consume additional power. This paper proposes a design for quadrupling the total number of linked items in the NB-IoT system using a symbol time compression methodology. A modified symbol time compression (MSTC) approach is specifically suggested, which could preserve 75% of the bandwidth (BW) by reducing the symbol time to a quarter (25%). This study suggests a design that uses the MSTC method four times in orthogonal frequency division multiplexing (4-MSTC-OFDM) to take advantage of the unused bandwidth and has the capability to deliver a quadruple quantity of information in comparison to the standard OFDM system. Simulations show that the suggested design (4-MSTC-OFDM) lowers the signal-to-noise ratio (SNR) by 3.9 dB (at BER = 10-6) and 4.4 dB (at BER = 10-4) compared to the OFDM system with 16-QAM (16-QAM-OFDM) when sending the same amount of data over the Rayleigh fading channel and additive white Gaussian noise channel (AWGN). Moreover, the results of simulations indicate that the proposed design possesses an identical BER as the conventional OFDM technique, implying that using the suggested method does not degrade the BER.

Incentive Oriented Two-Tier Task Offloading Scheme in Marine Edge Computing Networks: A Hybrid Stackelberg-Auction Game Approach

With the increasing exploration of marine resources, various marine wireless devices have been rapidly deployed for different marine applications such as marine navigation, ocean environment monitoring, and seabed resource exploitation. However, due to long transmission delay and low data rate between marine wireless devices and the cloud, it is challenging to satisfy the service requirements of computing-intensive and delay-sensitive tasks. By migrating computing resources from cloud to the near side of ocean, the paradigm of marine edge computing networks, which integrates communication and computation capacities in marine wireless devices, is expected to support a variety of marine tasks (e.g., data collection, monitoring and processing) with low delay and high data rate. However, considering the rationality and selfishness of marine wireless devices and their limited computing-capacity, how to motivate marine wireless devices to conduct task processing becomes an important problem for improving computing efficiency. To address this issue, in this paper, we propose an incentive oriented two-tier task offloading scheme for marine edge computing networks via hybrid Stackelberg-auction game approach, with the objective of improving the offloading efficiency and maximizing marine wireless devices’ utilities. Specifically, for underwater acoustic transmission tier, we exploit multi-access task offloading scheme, in which underwater wireless sensor (UWS) uploads its workloads to an unmanned underwater vehicle (UUV) and a sea surface sink node (SN) via non-orthogonal multiple access (NOMA) transmission. We formulate the utility of each party and model the task offloading process among UWS, UUV and SN as a Stackelberg game to optimize the UWS’s offloading strategy, UUV’s and SN’s price strategies. For radio frequency transmission tier, SN can offload its partial workloads to an unmanned aerial vehicle (UAV) via frequency division multiple access (FDMA) transmission. We provide their utilities and model the offloading process between a SN and a UAV as a double auction game to optimize their bidding strategies. Extensive simulation results are provided to validate the performance of the proposed scheme. Numerical results demonstrate that the proposed algorithms can obtain the optimal solutions and increase the utilities for marine wireless devices.

ETXRE: Energy and delay efficient routing metric for RPL protocol and wireless sensor networks

AbstractInternet of Things is an emerging paradigm based on interconnecting physical and virtual objects with each other and to the Internet. Most connected things fall into the category of constrained devices, with restricted resources (processing power, memory, and energy). These low‐power and lossy networks (LLNs) are known for their instability, high loss rates and low data rates, which makes routing one of the most challenging problems in low‐cost communications. A routing protocol for low‐power and lossy networks (RPL) is a proactive dynamic routing protocol based on IPv6. This protocol defines an objective function (OF) that utilises a set of metrics to select the best possible path to the destination. Minimum rank hysteresis objective function (MRHOF) and objective function zero (OF0) are the most basic OFs, where the first one selects the path to the sink based on the expected transmission count (ETX) metric, and OF0 is based on the hop count (HC). These two metrics prioritise either brute performance (i.e. ETX) or simplicity (i.e. HC). Therefore, using a single metric with an OF can either limit the performance or have an inefficient impact on load management and energy consumption. To overcome these challenges, a routing metric based on MRHOF OF which takes into consideration the link‐based routing metric (i.e. ETX) and node‐based metric (i.e. remaining energy) for route selection is provided. Expected transmission count remaining energy (ETXRE) is evaluated through 36 scenarios with different parameters. Preliminary results show that ETXRE outperforms ETX and RE in terms of end‐to‐end delay by an average of at least 17%, packet delay by 13% and consumes 10% less energy.

Age-Optimal Scheduling Over Hybrid Channels

We consider the problem of minimizing the age of information when a source can transmit status updates over two heterogeneous channels. Our work is motivated by recent developments in 5 G mmWave technology, where transmissions may occur over an unreliable but fast (e.g., mmWave) channel or a slow reliable (e.g., sub-6 GHz) channel. The unreliable channel is modeled as a time-correlated Gilbert-Elliot channel at a high rate when the channel is in the “ON” state. The reliable channel provides a deterministic but lower data rate. The scheduling strategy determines the channel to be used for transmission in each time slot, aiming to minimize the time-average age of information (AoI). The optimal scheduling problem is formulated as a Markov Decision Process (MDP), which is challenging to solve because super-modularity does not hold in a part of the state space. We address this challenge and show that a multi-dimensional threshold-type scheduling policy is optimal for minimizing the age. By exploiting the structure of the MDP and analyzing the discrete time Markov chains (DTMCs) of the threshold-type policy, we devise a low-complexity bisection algorithm to compute the optimal thresholds. We compare different scheduling policies using numerical simulations.

Designing Acoustic Reconfigurable Intelligent Surface for Underwater Communications

The UnderWater Acoustic (UWA) communication is the foundation for oceanic information applications. However, existing UWA systems suffer from low data rate problem. Although the Multiple Input Multiple Output (MIMO) scheme is proved to be able to increase channel capacity in many terrestrial scenarios, the high cost and complexity of acoustic MIMO significantly limit its usage. Recently, the acoustic reconfigurable intelligent surfaces (acoustic RIS) concept has been proposed to address the aforementioned problem. In this paper, with the real-world constraints taken into consideration, three key components of the acoustic RIS are designed to realize the underwater RIS concept, including the new acoustic RIS hardware, the ultra-wideband (UWB) beamforming, and practical operation protocol. Specifically, a completely new hardware design of acoustic RIS is first provided, since existing electromagnetic RIS designed for terrestrial environments do not work for underwater acoustic waves. Then, the UWB beamforming solution is developed, since underwater acoustic RIS need to control the acoustic signals, whose bandwidth is comparable to its carrier frequency. Finally, the practical operation protocol is developed to realize the acoustic RIS functionalities in complex underwater environment. The acoustic RIS design is validated through both COMSOL multiphysics simulations and end-to-end Bellhop-based simulations.

Comparative Studies between Phasor Measurement Units (Synchrophasors) & SCADA System

bstract: The increasing demand of load without considerable increase in transmission resources has posed numerous constraints and challenges in the power system monitoring and performance. Modern power system grid monitoring tools use data from remote terminal units (RTUs), protective relays, and transducers to provide information to system operators. This information is vital for the operation of the power system on a daily basis and under system contingencies. However, the method presently available to retrieve data from the devices is asynchronous and relatively slow. The asynchronous nature of the data does not provide accurate angle difference information from two nodes on the network. Moreover, the low data rate may be too slow to capture many short-duration disturbances on the grid. Alternatively, Synchrophasors based technology can be used to provide high-speed and coherent real-time information of the power system that is not available from legacy supervisory control and data acquisition (SCADA) systems.

Side-emitting fiber-based distributed receiver for visible light communication uplink.

We present a distributed receiver for visible light communication based on a side-emitting optical fiber. We show that 500 kbps data rate can be captured with a bit-error rate below the forward-error correction limit of 3.8·10-3 with a light-emitting diode (LED) transmitter 25 cm away from the fiber, whereas by increasing the photodetector gain and reducing the data rate down to 50 kbps, we improve the LED-fiber distance significantly up to 4 m. Our results lead to a low-cost distributed visible-light receiver with a 360° field of view for indoor low-data rate, Internet of Things, and sensory networks.

Smart Traffic Light Prototype Implementation using LoRa Technology

Smart traffic light systems have the potential to significantly improve traffic flow and safety in both urban and suburban areas. Most smart traffic lights were deployed in a city with adequate infrastructure. However, building a smart traffic light in suburban areas can be challenging due to the limited availability of communication infrastructure. This paper aims to build and implement a smart traffic light system in a suburban area by incorporating LoRa (Long Range) technology for communication. LoRa is a low-power wireless technology that allows long-distance communication between devices and has low data rates and power consumption. It was created especially for Internet of Things (IoT) devices that run on batteries. The peer-to-peer method was used, which utilized three LoRa nodes for enabling the communication between traffic lights. Usability testing was conducted to determine the impact of the smart traffic light system on traffic flow and safety. The result of the testing shows that the system is necessary to improve traffic management in suburban areas, and the respondents were positive about its potential benefits. Received Signal Strength (RSS) was measured in two different environments: paddy field and residential areas in Kampung Alor Ara, Arau, Perlis. Results show a slight difference in RSS indicator versus distance for the placement of traffic lights in three-junction lanes. The optimum distance between any two traffic lights is 100m in the paddy field area. While for residential areas, the optimum distance is only 50m. The successful implementation of this prototype demonstrates the potential of LoRa technology in enhancing traffic management in suburban areas. Further research such as implementing artificial intelligence (AI) and utilizing advanced LoRa devices and antennas is needed to explore the scalability and long-term sustainability of the system.

Optimal Energy-efficient Resource Allocation and Fault Tolerance scheme for task offloading in IoT-FoG Computing Networks

Due to technology development in the recent past, it is observed that an exponential growth in the usage of high speed intelligent devices which includes smart objects, smart home, smart vehicles etc. Therefore, for the efficient transmission between different smart objects/devices situated across different regions, an effective communication is required and the corresponding technology is called as ‘Internet of Things (IoT)’. This leads to an issues in various aspects like, increase in complexity, low data rate & latency and security may reduce the performance of the existing technologies. Therefore, there is a huge requirement of advanced technology which may support ultra-low latency transmission for the task received from the devices. Fog computing (FC) is one of an emerging technology that can improve network performance and provide resource-constrained applications for the IoT devices. In this paper, the Optimal Energy-efficient Resource Allocation (OEeRA) algorithm is proposed based on Minimal Cost Resource Allocation (MCRA) and Fault Identification and Rectification (FIR) algorithms for effective task offloading of IoT-FoG computing networks. The MCRA algorithm is proposed to assign at least one FN and Resource Block (RB) for each device, and also it ensures that each FN is connected with one or more RBs and devices. The leftover RBs are collected and stored in the buffer to replace the faulty RBs, as proposed in the FIR algorithm, which achieves better processing and response time with higher fault detection accuracy. The energy efficiency (EE) of the proposed OEeRA algorithm is computed through MCRA and FIR algorithms by varying FN, RB, and IoT devices. The performance analysis shows that the proposed algorithm achieved the maximum EE of 6.12 × 109 bit/J, 5.69 × 1010 bit/J, and 3.019 × 1010 (bit/J) for varying RBs, IoTs, and FNs, respectively.

Opportunistic Hybrid Routing Protocol for Acoustic-Radio Cooperative Networks

The ocean contains abundant resources and has high scientific and economic values. Increasing underwater communication scenarios stimulates many research efforts on underwater wireless ad-hoc networks. Underwater acoustic communication technology supports medium to long-range wireless communication. However, it suffers from low transmission data rates and long link delays. This paper introduces an Acoustic-Radio Cooperation Network (ARCNet) model for more efficient maritime information transmission. Under this ARCNet, we propose a new Radio-Acoustic Opportunistic Hybrid (RAOH) routing protocol composed of a neighbor discovery mechanism and a hybrid routing strategy. In the neighbor discovery stage, we explore the surface radio links to aid in selecting the shortest delay path between an underwater node and the surface nodes. In the route establishment stage, we combine the advantages of opportunistic routing and on-demand routing and design an opportunistic hybrid routing strategy, which improves the success rate of data forwarding and reduces the time spent on route establishment. Simulation results show that the proposed RAOH protocol outperforms traditional terrestrial and underwater routing strategies in routing response speed, packer delivery rate, throughput, end-to-end delay, and energy efficiency.

Design of a Multi-Standard UWB-LoRa Antenna Structure and Transceiver Board for High-Accuracy and Long-Range Localization Applications

Long-Range Wide-Area Networks (LoRaWAN) allow the transmission of data via radio link from sensors, which are potentially isolated or difficult to access, to gateways and servers that are connected to cellular networks for data processing, exchange, or relay, with low transmission power. This concept employs Long-Range (LoRa) modulation and has led to the emergence of many applications for the monitoring and tracking of objects. However, due to its characteristic of a low data rate for low-power communication, the transmission of information with LoRa technology is not suitable for the fast real-time monitoring of data. Additionally, due to its narrow bandwidth, an attempt to perform localization through the LoRa modulation technique will result in very limited accuracy because of its inability to resolve multipath problems. Thus, in this paper, we propose a multi-standard Ultra-Wide Bandwidth (UWB) and LoRa end-device that is capable of measuring location with high accuracy using UWB technology and then transmitting the location information through LoRa method to gateways and the Internet of Things Network. The results of measurements in indoor and outdoor scenarios show a UWB localization accuracy that is of sub-meter level, being between 10 and 33 cm, and a UWB range of 124 m in Line-of-Sight (LOS) and 55 m in Non-Line-of-Sight (NLOS) applications, respectively.

Theoretical Validation and Hardware Implementation of Dynamic Adaptive Scheduling for Heterogeneous Systems on Chip

Domain-specific systems on chip (DSSoCs) aim to narrow the gap between general-purpose processors and application-specific designs. CPU clusters enable programmability, whereas hardware accelerators tailored to the target domain minimize task execution times and power consumption. Traditional operating system (OS) schedulers can diminish the potential of DSSoCs, as their execution times can be orders of magnitude larger than the task execution time. To address this problem, we propose a dynamic adaptive scheduling (DAS) framework that combines the advantages of a fast, low-overhead scheduler and a sophisticated, high-performance scheduler with a larger overhead. We present a novel runtime classifier that chooses the better scheduler type as a function of the system workload, leading to improved system performance and energy-delay product (EDP). Experiments with five real-world streaming applications indicate that DAS consistently outperforms fast, low-overhead, and slow, sophisticated schedulers. DAS achieves a 1.29× speedup and a 45% lower EDP than the sophisticated scheduler under low data rates and a 1.28× speedup and a 37% lower EDP than the fast scheduler when the workload complexity increases. Furthermore, we demonstrate that the superior performance of the DAS framework also applies to hardware platforms, with up to a 48% and 52% reduction in the execution time and EDP, respectively.

Secrecy Rate Maximization in THz-Aided Heterogeneous Networks: A Deep Reinforcement Learning Approach

Densely deployed femtocells in heterogeneous networks (HetNets) improve the quality-of-service (QoS) and quality-of-experience (QoE) for both next-generation networks and end users, respectively. However, users near the femtocell edge, i.e., femto edge users (FEUs), which are far away from the Femto base station (FBS) may receive a low signal-to-interference-plus-noise ratio (SINR). Moreover, due to the distributed and dynamic nature of HetNets, femtocells may not resist various eavesdropping attacks from different attackers. Thus, it is essential to ensure the wireless channel's secrecy through which FEUs communicate with FBS since they may face eavesdropping attacks and may have low SINR and achievable data rates. Aiming to enhance the secrecy rate of femtocells, in this article, we formulate a joint optimization problem of beamforming vectors, artificial noise (AN), and power control for terahertz (THz)-enabled femtocells. Addressing this non-convex optimization problem is challenging due to the system's complexity and dynamic nature of FEUs. Thus, we translate the optimization problem into a multi-agent reinforcement learning (MARL) problem using the Markov decision process (MDP). Then, to solve the MDP with continuous action space, two deep reinforcement learning (DRL) based schemes, i.e., <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deep Deterministic Policy Gradient-based Secrecy Maximization (DDPG-SM)</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Asynchronous-Advantage-Actor-Critic based Secrecy Maximization (A3C-SM)</i> are proposed to maximize the secrecy rate of femtocells. The secrecy rate performance of the proposed schemes is compared and analyzed with Advantage-Actor-Critic (A2C) and state-of-the-art Perfect Secrecy Rate Maximization (PSRM) schemes. Simulation results show that the proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A3C-SM</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DDPG-SM</i> schemes achieve 37.73% and 22.64% better average secrecy rate in comparison to the PSRM scheme. Also, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A3C-SM</i> scheme outperforms all other <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DDPG-SM</i> , A2C, and PSRM schemes in terms of average secrecy rate, received SINR, and energy-efficiency of femtocells.