Reinforcement Learning Techniques Research Articles

Evaluating ChatGPT's Ability to Answer Common Patient Questions Regarding Hip Fracture.

ChatGPT is an artificial intelligence chatbot software programmed for conversational applications using reinforcement learning techniques. With its growing popularity and overall versatility, it is likely that ChatGPT's applications will expand into health care especially because it relates to patients researching their injuries. The purpose of this study was to investigate ChatGPT's ability to accurately answer frequently asked questions regarding hip fractures. Eleven frequently asked questions regarding hip fractures were posed to ChatGPT, and the responses were recorded in full. Five of these questions were determined to be high-yield based on the likelihood that a patient would ask the question to a chatbot software. The chatbot's responses were analyzed by five fellowship-trained orthopaedic trauma surgeons for their quality and accuracy using an evidence-based approach. The chatbot's answers were rated as "Excellent response requiring no clarification", "Satisfactory response requiring minimal clarification", "Satisfactory response requiring moderate clarification", or "Unsatisfactory response requiring significant clarification." Of the five high-yield questions posed to the chatbot, no question was determined to be unsatisfactory requiring significant clarification by the authors. The remaining responses were either satisfactory requiring minimal clarification (n = 3) or satisfactory requiring moderate clarification (n = 2). The chatbot was generally found to provide unbiased and evidence-based answers that would be clearly understood by most orthopaedic patients. These findings suggest that ChatGPT has the potential to be an effective patient education tool especially because it continues to grow and improve as a chatbot application. Level IV study.

Reconfigurable-Intelligent-Surface-Enhanced Dynamic Resource Allocation for the Social Internet of Electric Vehicle Charging Networks with Causal-Structure-Based Reinforcement Learning

Charging stations and electric vehicle (EV) charging networks signify a significant advancement in technology as a frontier application of the Social Internet of Things (SIoT), presenting both challenges and opportunities for current 6G wireless networks. One primary challenge in this integration is limited wireless network resources, particularly when serving a large number of users within distributed EV charging networks in the SIoT. Factors such as congestion during EV travel, varying EV user preferences, and uncertainties in decision-making regarding charging station resources significantly impact system operation and network resource allocation. To address these challenges, this paper develops a novel framework harnessing the potential of emerging technologies, specifically reconfigurable intelligent surfaces (RISs) and causal-structure-enhanced asynchronous advantage actor–critic (A3C) reinforcement learning techniques. This framework aims to optimize resource allocation, thereby enhancing communication support within EV charging networks. Through the integration of RIS technology, which enables control over electromagnetic waves, and the application of causal reinforcement learning algorithms, the framework dynamically adjusts resource allocation strategies to accommodate evolving conditions in EV charging networks. An essential aspect of this framework is its ability to simultaneously meet real-world social requirements, such as ensuring efficient utilization of network resources. Numerical simulation results validate the effectiveness and adaptability of this approach in improving wireless network efficiency and enhancing user experience within the SIoT context. Through these simulations, it becomes evident that the developed framework offers promising solutions to the challenges posed by integrating the SIoT with EV charging networks.

Machine Learning Library in C++

This project aims to develop a comprehensive and versatile machine learning library in C++ tailored to address the diverse needs of developers and researchers in the field. The library encompasses a robust set of core machine learning algorithms, encompassing supervised, unsupervised, and reinforcement learning techniques. Additionally, it incorporates essential data preprocessing tools to streamline data manipulation and feature engineering tasks, along with model evaluation capabilities crucial for assessing algorithm performance. The library's primary focus is on providing a rich suite of machine learning algorithms, This empowers users to effectively prepare data for training machine learning models. Additionally, the library provides tools for data splitting, and model evaluation to ensure reliable and robust model performance assessment.

Data-Driven Policy Learning Methods from Biological Behavior: A Systematic Review

Policy learning enables agents to learn how to map states to actions, thus enabling adaptive and flexible behavioral generation in complex environments. Policy learning methods are fundamental to reinforcement learning techniques. However, as problem complexity and the requirement for motion flexibility increase, traditional methods that rely on manual design have revealed their limitations. Conversely, data-driven policy learning focuses on extracting strategies from biological behavioral data and aims to replicate these behaviors in real-world environments. This approach enhances the adaptability of agents to dynamic substrates. Furthermore, this approach has been extensively applied in autonomous driving, robot control, and interpretation of biological behavior. In this review, we survey developments in data-driven policy-learning algorithms over the past decade. We categorized them into the following three types according to the purpose of the method: (1) imitation learning (IL), (2) inverse reinforcement learning (IRL), and (3) causal policy learning (CPL). We describe the classification principles, methodologies, progress, and applications of each category in detail. In addition, we discuss the distinct features and practical applications of these methods. Finally, we explore the challenges these methods face and prospective directions for future research.

Analysis and Implementation of Web Advertisement Optimization Using UCB Reinforcement Learning

Web advertisements are vital in digital marketing for connecting businesses with customers; yet optimizing their placement faces challenges from evolving user preferences. This project explores leveraging the Upper Confidence Bound (UCB) algorithm, a reinforcement learning technique, to automate ad placement strategies based on real-time feedback. Through analysis and implementation, this study investigates UCB's potential for web advertisement optimization. It examines existing methods, delves into reinforcement learning theory, and develops a simulation environment reflecting real-world dynamics. Experimental results comparing the UCB approach against baselines demonstrate its efficacy in improving ad placement and enhancing online advertising campaigns.

Deep Reinforcement Learning techniques for dynamic task offloading in the 5G edge-cloud continuum

The integration of new Internet of Things (IoT) applications and services heavily relies on task offloading to external devices due to the constrained computing and battery resources of IoT devices. Up to now, Cloud Computing (CC) paradigm has been a good approach for tasks where latency is not critical, but it is not useful when latency matters, so Multi-access Edge Computing (MEC) can be of use. In this work, we propose a distributed Deep Reinforcement Learning (DRL) tool to optimize the binary task offloading decision, this is, the independent decision of where to execute each computing task, depending on many factors. The optimization goal in this work is to maximize the Quality-of-Experience (QoE) when performing tasks, which is defined as a metric related to the battery level of the UE, but subject to satisfying tasks’ latency requirements. This distributed DRL approach, specifically an Actor-Critic (AC) algorithm running on each User Equipment (UE), is evaluated through the simulation of two distinct scenarios and outperforms other analyzed baselines in terms of QoE values and/or energy consumption in dynamic environments, also demonstrating that decisions need to be adapted to the environment’s evolution.

Enhanced Diabetic Retinopathy Diagnosis: A Comparative Analysis of models with an Ensemble classifier and Deep Q Learning

- A novel method that combines the strengths of different classifiers such as Naive Bayes, Multi-Layer Perceptron (MLP), and Support Vector Machine (SVM) is introduced in this paper. This method tackles the urgent need for cutting-edge diagnostic methods in the field of ophthalmology, mainly for the identification of diabetic retinopathy (DR). The approach is ensemble-based. Classical methods in retinal analysis of images often fail as they are static and are unable to adjust to the unique details that each distinct image presents. This constraint results in less accurate and precise diagnostic results, highlighting the urgent need for more adaptable and dynamic methods. The suggested model differs significantly from previous methods. Through the use of an ensemble approach, it capitalizes on the distinct advantages of each classifier: the MLP process's sophisticated feature extraction skill, Naive Bayes' probabilistic analysis, and SVM's non-linear pattern recognition capacity. By combining these techniques, the inherent drawbacks of utilizing a single strategy are addressed, guaranteeing a more thorough examination of retinal samples and images. The core of this idea is the system using Deep Q Learning (DQL) for adaptive classifier selection. Using learned Q Values for various contexts, this reinforcement learning technique selects the best classifier adaptively for each unique retinal image, hence optimizing the ensemble. This approach not only advances diagnosis accuracy and precision but also guarantees ongoing learning and adaptation to keep up with changing data patterns and advances in imaging technology. Extensive experiments on the IDRiD & EyePACS Dataset show the effectiveness of this model with a 5.5% increase in overall accuracy with other performance metrics, the results show a significant improvement over the current method. They represent a significant advancement in the timely and accurate identification of diabetic retinopathy, which will ultimately benefit patients and lessen the strain on healthcare systems. Thus, this work represents a major step forward in patient care as well as a technological advance, opening the door to more efficient supervision and treatment of retinal illnesses.

Dynamic event-triggered-based adaptive fuzzy optimized control for slowly switched nonlinear system with intermittent state constrains

This article presents a novel fuzzy event-triggered optimized strategy for slowly switched nonlinear system using reinforcement learning technique. For addressing intermittent state constraints problem, some improved shifting functions and barrier functions are designed in the adaptive backstepping process. Subsequently, by designing global performance functions with the discount term, the corresponding Hamilton-Jacobi-Bellman equation is derived and the optimal solution can be obtained under critic-actor structure. Different from the existing event-triggered methods, a novel dynamic event-triggered scheme for subsystem is developed, which not only saves communication resources, but also solves the mismatch behavior between controller and subsystem without restriction on maximum asynchronous intervals and the number of switch within a triggering interval. Furthermore, by using modified average cycle dwell time method and Lyapunov stability theorem, all signals of system are proved to be bounded. Eventually, numerical simulation results validate the superiority of the proposed approach, and this new scheme is applied to single-link robotic manipulator.

Energy-Aware Adaptive Scaling of Server Farms for NFV With Reliability Requirements

Auto-scaling techniques aim to keep the right number of active servers for the current load: if this number is too small we risk service disruption, but if it is too large we waste resources. Despite the interest in the efficient operation of this type of systems, no prior work has addressed auto-scaling techniques for Network Function Virtualization (NFV) with stringent reliability requirements such as those envisioned in 5G (5 or 6 nines). To achieve such levels of reliability, we need to account for both the activation delay until servers become available (i.e., the wake-up or activation time) and the fallible nature of servers (which may fail with some probability). In this paper, we build on control theory to design an auto-scaling technique for a server farm for NFV that guarantees certain reliability while minimizing the number of active resources. We show that the use of well-established tools from control theory results in convergence times much shorter than those obtained with state-of-the-art reinforcement learning techniques. This shows that, despite the current trend to apply machine learning to all sorts of networking problems, there may be some cases where other techniques (such as control theory) can be more suitable.

AoI-Guaranteed Incentive Mechanism for Mobile Crowdsensing With Freshness Concerns

With the explosive spread of smart mobile devices, Mobile CrowdSensing (MCS) has been becoming a promising paradigm, by which a platform can coordinate a group of workers to complete large-scale data collection tasks using their mobile devices. In this paper, we investigate the incentive mechanism design in MCS systems, taking the freshness of collected data and social benefits into consideration. First, the Age of Information (AoI) metric is introduced to measure the freshness of data. Then, we model the incentive mechanism design with AoI guarantees as an incomplete information two-stage Stackelberg game with multiple constraints. Next, we consider the scenario that all participants share the public utility function parameters of the Stackelberg game. By deriving the optimal remuneration paid by the platform and the optimal data update frequency for each worker, and proving the existence of a unique Stackelberg equilibrium, we propose an AoI-guaranteed Incentive Mechanism (AIM) that enables the platform and all workers to maximize their utilities simultaneously. Furthermore, we extend AIM to a general scenario where each participant has no prior knowledge of the utility function parameters of the game. By resorting to the Deep Reinforcement Learning (DRL) technique and modeling the two-stage Stackelberg game as a Markov decision process, we propose a DRL-based Incentive Mechanism (DIM) with AoI guarantees, which makes each participant effectively seek its optimal strategy through trial and error. Meanwhile, the system can guarantee that the AoI values of all data uploaded to the platform are not larger than a given threshold. Finally, numerical experiments on real-world traces are conducted to validate the efficacy and efficiency of AIM and DIM.

Reinforcement learning-based differential evolution algorithm for constrained multi-objective optimization problems

Many real-world problems can be established as Constrained Multi-objective Optimization Problems (CMOPs). It is still challenging to automatically set efficient parameters for Constrained Multi-Objective Evolutionary Algorithms (CMOEAs) to solve these CMOPs. A Reinforcement Learning-based Multi-Objective Differential Evolution (RLMODE) algorithm is proposed, in which the main parameters are dynamically adjusted. During the evolution process, the offspring generated is evaluated and compared with its corresponding parents, the relationship between the offspring and parent can adjust the parameters of RLMODE by the Reinforcement Learning (RL) technique. The feedback mechanism can produce the most appropriate parameters for RLMODE, which pushes the population towards feasible regions. The proposed RLMODE is evaluated on thirty functions and compared with some popular CMOEAs. The performance indicator IGD has revealed that the proposed RLMODE is competitive. Then, they are applied to solve the UAV path planning problem with three objectives and a constraint. The real application has further demonstrated the superiority of the proposed RLMODE.

Guidance Design for Escape Flight Vehicle against Multiple Pursuit Flight Vehicles Using the RNN-Based Proximal Policy Optimization Algorithm

Guidance commands of flight vehicles can be regarded as a series of data sets having fixed time intervals; thus, guidance design constitutes a typical sequential decision problem and satisfies the basic conditions for using the deep reinforcement learning (DRL) technique. In this paper, we consider the scenario where the escape flight vehicle (EFV) generates guidance commands based on the DRL technique, while the pursuit flight vehicles (PFVs) derive their guidance commands employing the proportional navigation method. For every PFV, the evasion distance is described as the minimum distance between the EFV and the PFV during the escape-and-pursuit process. For the EFV, the objective of the guidance design entails progressively maximizing the residual velocity, which is described as the EFV’s velocity when the last evasion distance is attained, subject to the constraint imposed by the given evasion distance threshold. In the outlined problem, three dimensionalities of uncertainty emerge: (1) the number of PFVs requiring evasion at each time instant; (2) the precise time instant at which each of the evasion distances can be attained; (3) whether each attained evasion distance exceeds the given threshold or not. To solve the challenging problem, we propose an innovative solution that integrates the recurrent neural network (RNN) with the proximal policy optimization (PPO) algorithm, engineered to generate the guidance commands of the EFV. Initially, the model, trained by the RNN-based PPO algorithm, demonstrates effectiveness in evading a single PFV. Subsequently, the aforementioned model is deployed to evade additional PFVs, thereby systematically augmenting the model’s capabilities. Comprehensive simulation outcomes substantiate that the guidance design method based on the proposed RNN-based PPO algorithm is highly effective.

Reinforcement Learning Algorithm to Optimize Players' Tactical Decisions and Round Planning in Tennis Matches

This paper explores the application of the Ranking Genetic Optimization Reinforcement Learning (RGORL) algorithm to optimize players' tactical decisions and round planning in tennis matches. Leveraging evolutionary principles and reinforcement learning techniques, RGORL offers a data-driven framework for enhancing on-court performance. Extensive simulations demonstrate the algorithm's effectiveness in improving match outcomes, points won percentages, and games won percentages. Results illustrate a steady improvement in fitness scores over successive generations, indicating RGORL's ability to evolve and refine strategies over time. Analysis of tactical decisions reveals the superiority of strategies such as the "Net Approach" in terms of win rates, points won percentages, and games won percentages. Through extensive simulations, RGORL demonstrates a notable improvement in match outcomes, with a maximum increase of 13% in win rates. Analysis of tactical decisions reveals significant enhancements in points won percentages, with improvements of up to 34% observed across various strategies, notably the "Net Approach." Furthermore, the algorithm achieves substantial gains in games won percentages, with increases of up to 25% recorded.

DIET PLANNING AND RECOMMENDATION SYSTEM USING ML AND MERN STACK

The "Diet Planning and Recommendation System Using ML and MERN Stack" project aims to develop an innovative solution to address the challenge of personalized diet planning and recommendation. With the rising awareness of the importance of nutrition in maintaining overall health and wellness, there is a growing demand for tools that can offer tailored dietary guidance to individuals based on their unique needs and preferences. This project leverages the power of Machine Learning (ML) algorithms and the MERN (MongoDB, Express.js, React.js, Node.js) stack to create a comprehensive and user-friendly platform. The system collects user data encompassing demographic information, health metrics, dietary habits, and goals. Using ML techniques such as regression, classification, and clustering, the system analyzes this data to generate personalized diet plans and recommendations. The backend of the system, built on Node.js and Express.js, manages data storage and processing, while the frontend, developed with React.js, provides an intuitive interface for users to interact with the system. MongoDB serves as the database, ensuring scalability and flexibility in data management. The ML models continuously learn and adapt based on user feedback and outcomes, enhancing the accuracy and effectiveness of the recommendations over time. Reinforcement learning techniques are employed to optimize diet plans based on real-world outcomes and user satisfaction. By integrating ML with the MERN stack, this project offers a novel approach to diet planning and recommendation, empowering individuals to make informed dietary choices and improve their overall health and well-being. KeyWords:Diet Planning ,Recommendation System,Machine Learning,(ML),MERNStack,Personalized,Nutrition,Health,,,Metrics,Dietary,Habits,Regression,Classification,Node.js, ,React.js,,MongoDB,User,Feedback,ReinforcementLearni,Real-world Outcomes,User Satisfaction,Wellness,Informed Dietary Choices

Manipulating Camera Gimbal Positioning by Deep Deterministic Policy Gradient Reinforcement Learning for Drone Object Detection

The object recognition technology of unmanned aerial vehicles (UAVs) equipped with “You Only Look Once” (YOLO) has been validated in actual flights. However, here, the challenge lies in efficiently utilizing camera gimbal control technology to swiftly capture images of YOLO-identified target objects in aerial search missions. Enhancing the UAV’s energy efficiency and search effectiveness is imperative. This study aims to establish a simulation environment by employing the Unity simulation software for target tracking by controlling the gimbal. This approach involves the development of deep deterministic policy-gradient (DDPG) reinforcement-learning techniques to train the gimbal in executing effective tracking actions. The outcomes of the simulations indicate that when actions are appropriately rewarded or penalized in the form of scores, the reward value can be consistently converged within the range of 19–35. This convergence implies that a successful strategy leads to consistently high rewards. Consequently, a refined set of training procedures is devised, enabling the gimbal to accurately track the target. Moreover, this strategy minimizes unnecessary tracking actions, thus enhancing tracking efficiency. Numerous benefits arise from training in a simulated environment. For instance, the training in this simulated environment is facilitated through a dataset composed of actual flight photographs. Furthermore, offline operations can be conducted at any given time without any constraint of time and space. Thus, this approach effectively enables the training and enhancement of the gimbal’s action strategies. The findings of this study demonstrate that a coherent set of action strategies can be proficiently cultivated by employing DDPG reinforcement learning. Furthermore, these strategies empower the UAV’s gimbal to rapidly and precisely track designated targets. Therefore, this approach provides both convenience and opportunities to gather more flight-scenario training data in the future. This gathering of data will lead to immediate training opportunities and help improve the system’s energy consumption.

A Comprehensive Evaluation and Comparison of Enhanced Learning Methods

This paper provides a comprehensive evaluation and comparison of current reinforcement learning methods. By analyzing the strengths and weaknesses of the main methods, such as value function-based, strategy gradient-based, and value and strategy-based methods, the differences in their performances on standard problems and the applicable scenarios are explored. Meanwhile, other methods such as Monte Carlo Tree Search (MCTS) and evolutionary methods are also briefly introduced. Through the research and analysis in this paper, it provides reference and guidance for choosing appropriate reinforcement learning methods, and promotes the development and application of reinforcement learning techniques in practical applications.

Learning robust autonomous navigation and locomotion for wheeled-legged robots.

Autonomous wheeled-legged robots have the potential to transform logistics systems, improving operational efficiency and adaptability in urban environments. Navigating urban environments, however, poses unique challenges for robots, necessitating innovative solutions for locomotion and navigation. These challenges include the need for adaptive locomotion across varied terrains and the ability to navigate efficiently around complex dynamic obstacles. This work introduces a fully integrated system comprising adaptive locomotion control, mobility-aware local navigation planning, and large-scale path planning within the city. Using model-free reinforcement learning (RL) techniques and privileged learning, we developed a versatile locomotion controller. This controller achieves efficient and robust locomotion over various rough terrains, facilitated by smooth transitions between walking and driving modes. It is tightly integrated with a learned navigation controller through a hierarchical RL framework, enabling effective navigation through challenging terrain and various obstacles at high speed. Our controllers are integrated into a large-scale urban navigation system and validated by autonomous, kilometer-scale navigation missions conducted in Zurich, Switzerland, and Seville, Spain. These missions demonstrate the system's robustness and adaptability, underscoring the importance of integrated control systems in achieving seamless navigation in complex environments. Our findings support the feasibility of wheeled-legged robots and hierarchical RL for autonomous navigation, with implications for last-mile delivery and beyond.

Efficient and practical quantum compiler towards multi-qubit systems with deep reinforcement learning

Abstract Efficient quantum compiling is essential for complex quantum algorithms realization. The Solovay-Kitaev theorem offers a theoretical lower bound on the required operations for approaching any unitary operator. However, it is still an open question that this lower bound can be actually reached in practice. Here, we present an efficient quantum compiler which, for the first time, approaches the S-K lower bound in practical implementations, both for single-qubit and two-qubit scenarios, marking a significant milestone. Our compiler leverages deep reinforcement learning (RL) techniques to address current limitations in terms of optimality and inference time. Furthermore, we show that our compiler is versatile by demonstrating comparable performance between inverse-free basis sets, which is always the case in real quantum devices, and inverse-closed sets. Our findings also emphasize the often-neglected constant term in scaling laws, bridging the gap between theory and practice in quantum compiling. These results highlight the potential of RL-based quantum compilers, offering efficiency and practicality while contributing novel insights to quantum compiling theory.

A generalisable tool path planning strategy for free-form sheet metal stamping through deep reinforcement and supervised learning

AbstractDue to the high cost of specially customised presses and dies and the advance of machine learning technology, there is some emerging research attempting free-form sheet metal stamping processes which use several common tools to produce products of various shapes. However, tool path planning strategies for the free forming process, such as reinforcement learning technique, derived from previous path planning experience are not generalisable for an arbitrary new sheet metal workpiece. Thus, in this paper, a generalisable tool path planning strategy is proposed for the first time to realise the tool path prediction for an arbitrary sheet metal part in 2-D space with no metal forming knowledge in prior, through deep reinforcement (implemented with 2 heuristics) and supervised learning technologies. Conferred by deep learning, the tool path planning process is corroborated to have self-learning characteristics. This method has been instantiated and verified by a successful application to a case study, of which the workpiece shape deformed by the predicted tool path has been compared with its target shape. The proposed method significantly improves the generalisation of tool path planning of free-form sheet metal stamping process, compared to strategies using pure reinforcement learning technologies. The successful instantiation of this method also implies the potential of the development of intelligent free-form sheet metal stamping process.

Interactive learning for multi-finger dexterous hand: A model-free hierarchical deep reinforcement learning approach

When a multi-fingered dexterous hand interacts with the external environment, it encounters various challenges, including the utilization of complex control techniques and the intricate coordination of finger motion sequences. Previous studies have primarily concentrated on investigating the interaction between multi-fingered dexterous hands and external objects, usually using model-based control or model-free reinforcement learning techniques. However, during practical implementation, various constraining factors are encountered, such as intricate modeling and limited interaction capabilities. In practical scenarios, the utilization of multi-fingered dexterous hands is imperative for the swift and efficient execution of a wide range of interactive tasks, including but not limited to throwing a ball and playing rock-paper-scissors. These tasks require skilled manual dexterity to demonstrate both precise control and quick responsiveness. To tackle this issue, we propose a hierarchical control approach for multi-fingered dexterous hands with interactive functionalities, utilizing model-free deep reinforcement learning. The complex interaction task is decomposed into simple sub-tasks using hierarchical strategy and action primitive decomposition, which effectively reduces the complexity of the action space, and achieves the motion planning and end finger trajectory control of dexterous hand. In a simulated environment, the aforementioned method has successfully executed interactive tasks, including ball throwing and playing rock-paper-scissors. It achieved a maximum normalized reward of 0.83 and an 84 % success rate. These results are noteworthy in terms of both control accuracy and response speed. This study offers novel insights into the effective resolution of the intricate challenges associated with interactions involving multi-fingered dexterous hands and human-computer interaction.