Deterministic Policy Gradient Research Articles

Off-policy actor-critic deep reinforcement learning methods for alert prioritization in intrusion detection systems

Alert prioritization plays a very important role in network security as it helps security teams manage and respond to the overwhelming volume of alerts generated by intrusion detection systems (IDSs). Although current deep reinforcement learning (DRL) based deep deterministic policy gradient (DDPG) has achieved good performance for alert prioritization, there are still limitations in dealing with DDPG's instability and limited exploration capabilities. In this paper, we propose TD3-AP and SAC-AP, two DRL empowered off-policy actor-critic alert prioritization methods based on twin delayed deep deterministic policy gradient (TD3) and soft actor-critic (SAC), respectively. We model the interaction between an adversary and a defender as a zero-sum game and use a double oracle framework to obtain mixed strategy Nash equilibrium (MSNE). Our objective is to minimize the defender's loss which represents the inability of the defender to investigate alerts generated due to the attacks. To demonstrate the benefits and scalability of the proposed approaches, we conduct extensive experiments on three datasets: MQTT-IoT-IDS2020, DARPA 2000 LLDOS 1.0 and CSE-CIC-IDS2018. The results depict that TD3-AP and SAC-AP exhibit a decrease in the defender's loss by 50% and 14.28%, respectively, in comparison to the alert prioritization method based on DDPG. The improvements are significantly higher when the proposed approaches are compared with traditional alert prioritization methods including Uniform, Snort, Fuzzy and RAP. Additionally, we also provide an analysis of the interpretability of our results through the utilization of SHapley Additive exPlanations (SHAP). We assess the sensitivity of our proposed approaches to different hyperparameters and evaluate the training time and computational resources necessary for their implementation.

A deep learning optimized LQR method for enhanced formation control with embedded systems

To achieve higher accuracy throughout the formation control processes and enhance precision in dynamic environments, particularly for the formation control of follower vehicles with embedded systems, this paper proposes a method and framework for vehicle formation control. An Ackermann-model based Linear Quadratic Regulator (LQR) controller is developed for lateral distance control and a Proportional-Derivative (PD) controller for longitudinal distance control. To enhance the efficacy of the LQR controller, the Deep Deterministic Policy Gradient Derivative (DDPG) method is introduced into the control system. The DDPG networks are trained in a simulation environment and can subsequently predict LQR parameters in real-time experiments. The practical application of the method is showcased, and the concluding remarks emphasize the potential and superior performance of our proposed formation control approach by experimental comparison with other controllers. This method can be implemented in small vehicles that possess limited computational resources and is also suitable for scenarios requiring dynamic motion control with higher tracking accuracy and stability.

Optimizing UAV-UGV coalition operations: A hybrid clustering and multi-agent reinforcement learning approach for path planning in obstructed environment

One of the most critical applications undertaken by Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) is reaching predefined targets by following the most time-efficient routes while avoiding collisions. Unfortunately, UAVs are hampered by limited battery life, and UGVs face challenges in reachability due to obstacles and elevation variations, which is why a coalition of UAVs and UGVs can be highly effective. Existing literature primarily focuses on one-to-one coalitions, which constrains the efficiency of reaching targets. In this work, we introduce a novel approach for a UAV-UGV coalition with a variable number of vehicles, employing a modified mean-shift clustering algorithm (MEANCRFT) to segment targets into multiple zones. This approach of assigning targets to various circular zones based on density and range significantly reduces the time required to reach these targets. Moreover, introducing variability in the number of UAVs and UGVs in a coalition enhances task efficiency by enabling simultaneous multi-target engagement. In our approach, each vehicle of the coalitions is trained using two advanced deep reinforcement learning algorithms in two separate experiments, namely Multi-agent Deep Deterministic Policy Gradient (MADDPG) and Multi-agent Proximal Policy Optimization (MAPPO). The results of our experimental evaluation demonstrate that the proposed MEANCRFT-MADDPG method substantially surpasses current state-of-the-art techniques, nearly doubling efficiency in terms of target navigation time and task completion rate.

Digital twin-assisted service function chaining in multi-domain computing power networks with multi-agent reinforcement learning

The emerging computing power network (CPN) is believed to undergo the paradigm reformation of network function virtualization (NFV) and service function chaining (SFC). It is prerequisite to explore the performance upper bound of NFV-assisted CPN before truly deploying the NFV and SFC technologies onto physical networks. Inspired by the application of digital twin (DT) in the industry and due to its advantage in synchronizing physical objects with their virtual replicas, we propose to use the DT to assist the SFC deployment in the multi-domain CPN, with the aid of multi-agent deep deterministic policy gradient (MADDPG) framework. First, we build a dynamic SFC mapping problem in the virtual twin network layer, by modeling the computing power, link bandwidth, delay performance and the VNF ordering as DT objects and constraints, to jointly optimize the energy consumption, end-to-end delay and the VNF re-deploying cost. Then, the prioritized experience replay and re-parameterization trick-empowered centralized training and decentralized execution MADDPG framework is utilized to learn the SFC deployment, by taking each domain controller as one agent. Finally, numerical experiments are carried out to validate the effectiveness of MADDPG in the cross-domain SFC deployment. For performance verification, the deployment success rate, number of crossed domains, energy consumption, end-to-end latency and load balancing degree are all taken as metrics, to show the performance of MADDPG compared to other learning frameworks.

Spectrum-efficient user grouping and resource allocation based on deep reinforcement learning for mmWave massive MIMO-NOMA systems

Millimeter-wave (mmWave) massive multiple-input multiple-output non-orthogonal multiple access (MIMO-NOMA) is proven to be a primary technique for sixth-generation (6G) wireless communication networks. However, the great increase in users and antennas brings challenges for interference suppression and resource allocation for mmWave massive MIMO-NOMA systems. This study proposes a spectrum-efficient and fast convergence deep reinforcement learning (DRL)-based resource allocation framework to optimize user grouping and allocation of subchannel and power. First, an enhanced K-means grouping algorithm is proposed to reduce the multi-user interference and accelerate the convergence. Then, a dueling deep Q-network (DQN) structure is proposed to perform subchannel allocation, which further improves the convergence speed. Moreover, a deep deterministic policy gradient (DDPG)-based power resource allocation algorithm is designed to avoid the performance loss caused by power quantization and improve the system’s achievable sum-rate. The simulation results demonstrate that our proposed scheme outperforms other neural network-based algorithms in terms of convergence performance, and can achieve higher system capacity compared with the greedy algorithm, the random algorithm, the RNN algorithm, and the DoubleDQN algorithm.

Hierarchical path planner combining probabilistic roadmap and deep deterministic policy gradient for unmanned ground vehicles with non-holonomic constraints

Path planning plays a vital role in autonomous driving as it needs to guide the vehicle to achieve the target position without collision while satisfying the vehicle's non-holonomic constraints. A good path planner should be able to cope with sophisticated environments and try to make the planned path as short as possible. Traditional path planning methods when being applied on four-wheel vehicles may suffer from one of the following disadvantages including conflicting with non-holonomic constraints, far from close-optimal length solution, or easy to get stuck in specific environments. In this paper, a hierarchical path planning method for a four-wheel vehicle combining probabilistic roadmap (PRM) and deep deterministic policy gradient (DDPG) is proposed to address existing problems. In the upper level, PRM is used to generate a guidance path quickly, which doesn't consider the vehicle's non-holonomic constraints but gives the sketch of the path that helps the vehicle to jump out of some dilemmas in specific environments while maintaining a relatively short path distance. In the lower level, reinforcement learning, specifically DDPG, is used to optimize the guidance path generated by PRM considering multiple factors including non-holonomic constraints and obstacle avoidance. The proposed method is validated in four different environments including a simple map, an easy-to-stuck specific map, a maze-like complex map and an office-like complex map. Results show that the proposed method could generate a smooth path satisfying non-holonomic constraints while demonstrating desirable obstacle avoidance performance. The planned path of the proposed method is superior to that of both traditional and state-of-the-art methods in terms of path length and smoothness, which could be 1.3–12.9 % shorter and 2.6–52.1 % smoother in different environments.

An improved energy management strategy for hybrid electric powered aircraft based on deep reinforcement learning

Electric airplanes have attracted a great deal of attentions because of their strategic commercial values and low emissions. One of the essentials for efficient control and management of hybrid electric propulsion aircraft is a well-designed energy management strategy (EMS). A new EMS approach based on deep learning is introduced in this study to realize thrust distribution and power distribution, consequently lowering aircraft fuel consumption. A model of hybrid electric propulsion power system including turbine engines and lithium-ion battery pack is constructed to act as an application environment. An intelligent energy management strategy based on deep reinforcement learning (DRL) is then proposed in this paper. Two DRL agents based on twin delayed deep deterministic policy gradient (TD3) are combined in the hierarchical framework. The multi-agents are trained to handle the energy management problem in this paper. In a finite period of time, the training process is accomplished. After training, the simulation result of the test shows that this DRL-based EMS reduces the equivalent fuel consumption by 2.2% compared with that using the existing method, and preserves 99.9% of the battery state of health (SOH). The battery pack's state of charge (SOC) regresses precisely to the reference value at the end. Particularly, the proposed EMS is able to work properly for different flight profiles and various initial SOCs. The results of this paper provide preliminary supports for the feasibility and superiority of deep reinforcement learning applied to the energy management strategy of series hybrid electric aircraft.

Deep Reinforcement Learning-Based Task Offloading and Load Balancing for Vehicular Edge Computing

Vehicular edge computing (VEC) effectively reduces the computational burden on vehicles by offloading tasks from resource-constrained vehicles to edge nodes. However, non-uniformly distributed vehicles offloading a large number of tasks cause load imbalance problems among edge nodes, resulting in performance degradation. In this paper, we propose a deep reinforcement learning-based decision scheme for task offloading and load balancing with the optimization objective of minimizing the system cost considering the split offloading of tasks and the load dynamics of edge nodes. First, we model the mutual interaction between mobile vehicles and Mobile Edge Computing (MEC) servers using a Markov decision process. Second, the optimal task-offloading and resource allocation decision is obtained by utilizing the twin delayed deep deterministic policy gradient algorithm (TD3), and server load balancing is achieved through edge collaboration using a server selection algorithm based on the technique for order preference by similarity to the ideal solution (TOPSIS). Finally, we have conducted extensive simulation experiments and compared the results with several other baseline schemes. The proposed scheme can more effectively reduce the system cost and increase the system resource utilization.

A memory and attention-based reinforcement learning for musculoskeletal robots with prior knowledge of muscle synergies

PurposeLimited by the types of sensors, the state information available for musculoskeletal robots with highly redundant, nonlinear muscles is often incomplete, which makes the control face a bottleneck problem. The aim of this paper is to design a method to improve the motion performance of musculoskeletal robots in partially observable scenarios, and to leverage the ontology knowledge to enhance the algorithm’s adaptability to musculoskeletal robots that have undergone changes.Design/methodology/approachA memory and attention-based reinforcement learning method is proposed for musculoskeletal robots with prior knowledge of muscle synergies. First, to deal with partially observed states available to musculoskeletal robots, a memory and attention-based network architecture is proposed for inferring more sufficient and intrinsic states. Second, inspired by muscle synergy hypothesis in neuroscience, prior knowledge of a musculoskeletal robot’s muscle synergies is embedded in network structure and reward shaping.FindingsBased on systematic validation, it is found that the proposed method demonstrates superiority over the traditional twin delayed deep deterministic policy gradients (TD3) algorithm. A musculoskeletal robot with highly redundant, nonlinear muscles is adopted to implement goal-directed tasks. In the case of 21-dimensional states, the learning efficiency and accuracy are significantly improved compared with the traditional TD3 algorithm; in the case of 13-dimensional states without velocities and information from the end effector, the traditional TD3 is unable to complete the reaching tasks, while the proposed method breaks through this bottleneck problem.Originality/valueIn this paper, a novel memory and attention-based reinforcement learning method with prior knowledge of muscle synergies is proposed for musculoskeletal robots to deal with partially observable scenarios. Compared with the existing methods, the proposed method effectively improves the performance. Furthermore, this paper promotes the fusion of neuroscience and robotics.

Inspection Robot Navigation Based on Improved TD3 Algorithm.

The swift advancements in robotics have rendered navigation an essential task for mobile robots. While map-based navigation methods depend on global environmental maps for decision-making, their efficacy in unfamiliar or dynamic settings falls short. Current deep reinforcement learning navigation strategies can navigate successfully without pre-existing map data, yet they grapple with issues like inefficient training, slow convergence, and infrequent rewards. To tackle these challenges, this study introduces an improved two-delay depth deterministic policy gradient algorithm (LP-TD3) for local planning navigation. Initially, the integration of the long-short-term memory (LSTM) module with the Prioritized Experience Re-play (PER) mechanism into the existing TD3 framework was performed to optimize training and improve the efficiency of experience data utilization. Furthermore, the incorporation of an Intrinsic Curiosity Module (ICM) merges intrinsic with extrinsic rewards to tackle sparse reward problems and enhance exploratory behavior. Experimental evaluations using ROS and Gazebo simulators demonstrate that the proposed method outperforms the original on various performance metrics.

Energy-Delay Joint Optimization for Task Offloading in Digital Twin-Assisted Internet of Vehicles

Vehicle edge computing (VEC) provides efficient services for vehicles by offloading tasks to edge servers. Notably, extant research mainly employs methods such as deep learning and reinforcement learning to make resource allocation decisions, without adequately accounting for the ramifications of high-speed mobility of vehicles and the dynamic nature of the Internet of Vehicles (IoV) on the decision-making process. This paper endeavours to tackle the aforementioned issue through the introduction of a novel concept, namely, a digital twin-assisted IoV. Among them, the digital twin of IoV offers training data for computational offloading and content caching decisions, which allows edge servers to directly interact with the dynamic environment while capturing its dynamic changes in real-time. Through this collaborative endeavour, edge intelligent servers can promptly respond to vehicular requests and return results. We transform the dynamic edge computing problem into a Markov decision process (MDP), and then solve it with the twin delayed deep deterministic policy gradient (TD3) algorithm. Simulation experiments demonstrate the adaptability of our proposed approach in the dynamic environment while successfully enhancing the Quality of Service, that is, decreasing total delay and energy consumption.

Cooperative motion planning and control of a group of autonomous underwater vehicles using twin-delayed deep deterministic policy gradient

The cooperative execution of complex tasks can lead to desirable outcomes and increase the likelihood of mission success. Nevertheless, coordinating the movements of multiple autonomous underwater vehicles (AUVs) in a collaborative manner is challenging due to nonlinear dynamics and environmental disturbances. The paper presents a decentralized deep reinforcement learning algorithm for AUVs that enables cooperative motion planning and obstacle avoidance. The goal is to formulate control policies for AUVs, empowering each vehicle to create its optimal collision-free path through adjustments in speed and heading. To ensure safe navigation of multiple AUVs, COLlision AVoidance (COLAV) plays a crucial role. Therefore, the implementation of a multi-layer region control strategy enhances the AUVs’ responsiveness to nearby obstacles, leading to improved COLAV. Furthermore, a reward function is formulated to consider four criteria: path planning, obstacle- and self-COLAV, as well as feasible control signals, with the aim of strengthening the proposed strategy. Notably, the devised scheme demonstrates robustness against disturbances A comparative study is conducted with the well-established Artificial Potential Field (APF) planning method. The simulation results indicate that the proposed system effectively and safely guides the AUVs to their goals and exhibits desirable generalizability.

Deep-Reinforcement-Learning-Based Joint Energy Replenishment and Data Collection Scheme for WRSN.

With the emergence of wireless rechargeable sensor networks (WRSNs), the possibility of wirelessly recharging nodes using mobile charging vehicles (MCVs) has become a reality. However, existing approaches overlook the effective integration of node energy replenishment and mobile data collection processes. In this paper, we propose a joint energy replenishment and data collection scheme (D-JERDG) for WRSNs based on deep reinforcement learning. By capitalizing on the high mobility of unmanned aerial vehicles (UAVs), D-JERDG enables continuous visits to the cluster head nodes in each cluster, facilitating data collection and range-based charging. First, D-JERDG utilizes the K-means algorithm to partition the network into multiple clusters, and a cluster head selection algorithm is proposed based on an improved dynamic routing protocol, which elects cluster head nodes based on the remaining energy and geographical location of the cluster member nodes. Afterward, the simulated annealing (SA) algorithm determines the shortest flight path. Subsequently, the DRL model multiobjective deep deterministic policy gradient (MODDPG) is employed to control and optimize the UAV instantaneous heading and speed, effectively planning UAV hover points. By redesigning the reward function, joint optimization of multiple objectives such as node death rate, UAV throughput, and average flight energy consumption is achieved. Extensive simulation results show that the proposed D-JERDG achieves joint optimization of multiple objectives and exhibits significant advantages over the baseline in terms of throughput, time utilization, and charging cost, among other indicators.

A Multi-Objective Optimal Control Method for Navigating Connected and Automated Vehicles at Signalized Intersections Based on Reinforcement Learning

The emergence and application of connected and automated vehicles (CAVs) have played a positive role in improving the efficiency of urban transportation and achieving sustainable development. To improve the traffic efficiency at signalized intersections in a connected environment while simultaneously reducing energy consumption and ensuring a more comfortable driving experience, this study investigates a flexible and real-time control method to navigate the CAVs at signalized intersections utilizing reinforcement learning (RL). Initially, control of CAVs at intersections is formulated as a Markov Decision Process (MDP) based on the vehicles’ motion state and the intersection environment. Subsequently, a comprehensive reward function is formulated considering energy consumption, efficiency, comfort, and safety. Then, based on the established environment and the twin delayed deep deterministic policy gradient (TD3) algorithm, a control algorithm for CAVs is designed. Finally, a simulation study is conducted using SUMO, with Lankershim Boulevard as the research scenario. Results indicate that the proposed methods yield a 13.77% reduction in energy consumption and a notable 18.26% decrease in travel time. Vehicles controlled by the proposed method also exhibit smoother driving trajectories.

A Deep Reinforcement Learning Approach for Optimizing Inventory Management in the Agri-Food Supply Chain

This research aims to improve inventory management throughout the agri-food supply chain, through the use of Deep Reinforcement Learning algorithms. Three Deep Reinforcement Learning algorithms, including Deep Q-Networks, Deep Deterministic Policy Gradient, and Proximal Policy Optimization algorithms were implemented and tested in order to evaluate their ability to actively manage inventories and improve the performance of supply chains. The results of the experimental phase offered important information regarding the performance of each Deep Reinforcement Learning algorithm. The Deep Deterministic Policy Gradient algorithm was identified as a viable choice, offering the best results in terms of accuracy to set the optimal inventories for the supply chain and improving the efficiency of the supply chain. The percentage of cost efficiency improved from 92.5% to 95.8% in the case of the DDPG model. The inventory turnover was also improved, surpassing the level of 8.1 units from the original level of 7.3 units which means that the system converts the inventor into sales in less time. The metric for on-time delivery also beneficiated from several improvements, reaching 96.5% form the level of 93.2% return. The quality metrics also registered a significant improvement and reported level of 96.2% after the implementation of the DDPG algorithm and compared to the level of 94.6% prior to the implementation of the algorithm. These results suggest that using such a system will bring beneficial changes to the supply chain and will offer the possibility of implementing a data-driven inventory system based on Deep Reinforcement Learning.

Decentralized computation offloading via multi-agent deep reinforcement learning for NOMA-assisted mobile edge computing with energy harvesting devices

Supporting latency-sensitive and computation-intensive applications is hardly possible for mobile devices (MDs) with limited battery capacity and low computing resources. Therefore, mobile edge computing (MEC) and wireless power transfer (WPT) have emerged as promising technologies that allow MDs to offload part or all of their workloads to MEC servers and harvest energy for prolonging their battery lifetime. However, the MEC server’s limited computing resources, available communication channel quality, and time-limited energy harvesting (EH) challenge the computation offloading. In this paper, we study the joint problem of decentralized computation offloading and resource allocation (JDCORA) in the environment of non-orthogonal multiple access (NOMA)-assisted MEC with multiple EH-enabled MDs. To learn decentralized offloading policies, we propose a multi-agent deep reinforcement learning (MADRL)-based scheme towards minimizing energy consumption and task completion time, which considers the cooperation between MDs to adjust their strategies. In particular, we improve the multi-agent deep deterministic policy gradient (MADDPG) by applying the features of double actors, double centralized critics, soft value estimation, critic regularization and proportional-based prioritized experience replay (pPER) and propose an algorithm called multi-agent twin actors regularized critics (MATARC). Simulation results demonstrate that the proposed MATARC has a better convergence performance compared to other baseline methods and also reduces the average energy consumption, task completion time and task drop rate.

Deep reinforcement learning for PMSG wind turbine control via twin delayed deep deterministic policy gradient (TD3)

AbstractThis article proposes the use of a deep reinforcement learning method—and precisely a variant of the deep deterministic policy gradient (DDPG) method known as twin delayed DDPG, or TD3—for maximum power point tracking in wind energy conversion systems that use permanent magnet synchronous generators (PMSGs). An overview of the TD3 algorithm is provided, together with a detailed description of its implementation and training for the considered application. Simulation results are provided, also including a comparison with a model‐based control method based on feedback linearization and linear‐quadratic regulation. The proposed TD3‐based controller achieves a satisfactory control performance and is more robust to PMSG parameter variations as compared to the presented model‐based method. To the best of the authors' knowledge, this article presents for the first time an approach for generating both speed and current control loops using DRL for wind energy conversion systems.

UISA: User Information Separating Architecture for Commodity Recommendation Policy with Deep Reinforcement Learning

Commodity recommendation contributes an important part of individuals’ daily life. In this context, deep reinforcement learning methods have demonstrated substantial efficacy in enhancing recommender systems’ performance. Nevertheless, several recommender systems directly utilize original feature information as a foundational element for decision-making, which seems simplistic and low efficient. Furthermore, the incorporation of sequential decision-making adds complexity to the task of recommendation. In pursuit of maximizing the long-term sequential returns of recommender systems, our study introduces a novel architecture, named User Information Separating Architecture (UISA). This framework is tailored to align with classic reinforcement learning algorithms and aims to extract the user’s interest value through the discrete processing of both static and dynamic user information. Through integration with deep reinforcement learning, the architecture is oriented towards the maximization of long-term profit and is applicable in sequential recommendation scenarios. We conduct experimental assessments by combining the proposed architecture with proximal policy optimization (PPO) and deep deterministic policy gradient (DDPG) algorithms. The outcomes illustrate marked improvements in commodity recommendation, showcasing enhancements ranging from approximately 5% to 40% in both reward and click-through rate metrics across a self-constructed JDEnv environment and the Virtual Taobao environment. Through comparison experiments, the UISA models demonstrate comparable performance.

AoI minimization of ambient backscatter-assisted EH-CRN with cooperative spectrum sensing

To evaluate the timeliness and freshness of information from energy-constrained devices, we minimize the weighted average age of information (AoI) of the ambient backscatter-assisted energy harvesting cognitive radio network (AB-EH-CRN), where secondary transmitters (STs) follow the non-orthogonal multiple access (NOMA) strategy to transmit status updates. The secondary receiver (SR) schedules the selected actions for STs to perform spectrum access, and receives status updates from them. We formulate the AoI minimization problem where the action selection and transmission power of STs are considered as variables. To exploit spectrum resources in the AB-EH-CRN with unknown signal-to-noise ratios (SNRs) at the STs, we propose the deep neural network (DNN)-based cooperative spectrum sensing (CSS), and adopt an energy threshold approach for STs to determine the energy allocation between spectrum sensing (SS) and packet transmission. Moreover, we address the AoI minimization problem by the proposed hybrid action and energy penalty deep deterministic policy gradient (HAEP-DDPG) algorithm, which adapts to the hybrid continuous and discrete action spaces. Simulation results validate the advantage of the proposed HAEP-DDPG algorithm compared with comparison schemes in terms of AoI.

An experimental application of machine learning algorithms to optimize the FEL lasing via beam trajectory tuning at Dalian Coherent Light Source

The lasing optimization of Free-Electron Laser (FEL) facilities is a time-consuming and challenging task. Instead of operating manually by experienced operators, implementation of machine learning algorithms offers a rapid and adaptable approach for FEL lasing optimization. Recently, such an experiment has been conducted at the vacuum ultraviolet FEL facility - Dalian Coherent Light Source (DCLS). Four algorithms, namely the standard and the neural network-based genetic algorithms, the deep deterministic policy gradient and the soft actor critic reinforcement learning algorithms, have been employed to enhance the FEL intensity by optimizing the electron beam trajectory. These algorithms have shown notable efficacy in enhancing the FEL lasing, especially the reinforcement learning ones which achieved convergence within only approximately 400 iterations. This study demonstrates the validity of machine learning algorithms for FEL lasing optimization, providing a forward-looking perspective on the automatic operation of DCLS.