Task Success Rate Research Articles

Hybrid Artificial Intelligence Strategies for Drone Navigation

Objective: This paper describes the development of hybrid artificial intelligence strategies for drone navigation. Methods: The navigation module combines a deep learning model with a rule-based engine depending on the agent state. The deep learning model has been trained using reinforcement learning. The rule-based engine uses expert knowledge to deal with specific situations. The navigation module incorporates several strategies to explain the drone decision based on its observation space, and different mechanisms for including human decisions in the navigation process. Finally, this paper proposes an evaluation methodology based on defining several scenarios and analyzing the performance of the different strategies according to metrics adapted to each scenario. Results: Two main navigation problems have been studied. For the first scenario (reaching known targets), it has been possible to obtain a 90% task completion rate, reducing significantly the number of collisions thanks to the rule-based engine. For the second scenario, it has been possible to reduce 20% of the time required to locate all the targets using the reinforcement learning model. Conclusions: Reinforcement learning is a very good strategy to learn policies for drone navigation, but in critical situations, it is necessary to complement it with a rule-based module to increase task success rate.

Multi-UAV Cooperative Pursuit of a Fast-Moving Target UAV Based on the GM-TD3 Algorithm

Recently, developing multi-UAVs to cooperatively pursue a fast-moving target has become a research hotspot in the current world. Although deep reinforcement learning (DRL) has made a lot of achievements in the UAV pursuit game, there are still some problems such as high-dimensional parameter space, the ease of falling into local optimization, the long training time, and the low task success rate. To solve the above-mentioned issues, we propose an improved twin delayed deep deterministic policy gradient algorithm combining the genetic algorithm and maximum mean discrepancy method (GM-TD3) for multi-UAV cooperative pursuit of high-speed targets. Firstly, this paper combines GA-based evolutionary strategies with TD3 to generate action networks. Then, in order to avoid local optimization in the algorithm training process, the maximum mean difference (MMD) method is used to increase the diversity of the policy population in the updating process of the population parameters. Finally, by setting the sensitivity weights of the genetic memory buffer of UAV individuals, the mutation operator is improved to enhance the stability of the algorithm. In addition, this paper designs a hybrid reward function to accelerate the convergence speed of training. Through simulation experiments, we have verified that the training efficiency of the improved algorithm has been greatly improved, which can achieve faster convergence; the successful rate of the task has reached 95%, and further validated UAVs can better cooperate to complete the pursuit game task.

Integrated Intelligent Control of Redundant Degrees-of-Freedom Manipulators via the Fusion of Deep Reinforcement Learning and Forward Kinematics Models

Redundant degree-of-freedom (DOF) manipulators offer increased flexibility and are better suited for obstacle avoidance, yet precise control of these systems remains a significant challenge. This paper addresses the issues of slow training convergence and suboptimal stability that plague current deep reinforcement learning (DRL)-based control strategies for redundant DOF manipulators. We propose a novel DRL-based intelligent control strategy, FK-DRL, which integrates the manipulator’s forward kinematics (FK) model into the control framework. Initially, we conceptualize the control task as a Markov decision process (MDP) and construct the FK model for the manipulator. Subsequently, we expound on the integration principles and training procedures for amalgamating the FK model with existing DRL algorithms. Our experimental analysis, applied to 7-DOF and 4-DOF manipulators in simulated and real-world environments, evaluates the FK-DRL strategy’s performance. The results indicate that compared to classical DRL algorithms, the FK-DDPG, FK-TD3, and FK-SAC algorithms improved the success rates of intelligent control tasks for the 7-DOF manipulator by 21%, 87%, and 64%, respectively, and the training convergence speeds increased by 21%, 18%, and 68%, respectively. These outcomes validate the proposed algorithm’s effectiveness and advantages in redundant manipulator control using DRL and FK models.

Memory-extraction-based DRL cooperative guidance against the maneuvering target protected by interceptors

This paper presents an open and interesting issue for missiles, i.e., achieving collaborative parameters constrained cooperative guidance, despite the interference of pursing interceptors (INTs) and the maneuvering target, by the fact that the target-missile-interceptor (TMI) engagement induces their complex and time-varying relationships. The Memory-Extraction-based Soft-Actor-Critic (ME-SAC) approach is proposed, which enhances the collaborative performance of missiles by implicitly extracting coupling motion characteristics among TMI from historical state, achieving the joint optimization of situation awareness and strategy. Firstly, the cooperative guidance task is formulated as a multi-order Markov decision process (MOMDP) to better represent the dynamic evolution of engagement, and a memory-extraction process is introduced to alleviate the curse of dimensionality. Secondly, a memory-decision-oriented maximum entropy framework combined with memory update modules is designed for enhancing strategy search ability. Then, a domain-knowledge-based pre-training is implemented to improve convergence speed. Finally, in simulation evaluation with various scenarios, the proposed ME-SAC shows more promising than the typical DRL-based and model-based algorithms in task success rate, learning efficiency, and adaptability.

Learning to drive as humans do: Reinforcement learning for autonomous navigation

This article introduces an approach aimed at enabling self-driving cars to emulate human-learned driving behavior. We propose a method where the navigation challenge of autonomous vehicles, from starting to ending positions, is framed as a series of decision-making problems encountered in various states negating the requirement for high-precision maps and routing systems. Utilizing high-quality images and sensor-derived state information, we design rewards to guide an agent’s movement from the initial to the final destination. The soft actor-critic algorithm is employed to learn the optimal policy from the interaction between the agent and the environment, informed by these states and rewards. In an innovative approach, we apply the variational autoencoder technique to extract latent vectors from high-quality images, reconstructing a new state space with vehicle state vectors. This method reduces hardware requirements and enhances training efficiency and task success rates. Simulation tests conducted in the CARLA simulator demonstrate the superiority of our method over others. It enhances the intelligence of autonomous vehicles without the need for intermediate processes such as target detection, while concurrently reducing the hardware footprint, even though it may not perform as well as the currently available mature techniques.

Open-set 3D semantic instance maps for vision language navigation – O3D-SIM

Humans excel at forming mental maps of their surroundings, equipping them to understand object relationships and navigate based on language queries. Our previous work SI Maps (Nanwani L, Agarwal A, Jain K, et al. Instance-level semantic maps for vision language navigation. In: 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE; 2023 Aug.) showed that having instance-level information and the semantic understanding of an environment helps significantly improve performance for language-guided tasks. We extend this instance-level approach to 3D while increasing the pipeline's robustness and improving quantitative and qualitative results. Our method leverages foundational models for object recognition, image segmentation, and feature extraction. We propose a representation that results in a 3D point cloud map with instance-level embeddings, which bring in the semantic understanding that natural language commands can query. Quantitatively, the work improves upon the success rate of language-guided tasks. At the same time, we qualitatively observe the ability to identify instances more clearly and leverage the foundational models and language and image-aligned embeddings to identify objects that, otherwise, a closed-set approach wouldn't be able to identify. Project Page - https://smart-wheelchair-rrc.github.io/o3d-sim-webpage

Adaptive task migration strategy with delay risk control and reinforcement learning for emergency monitoring

The timely and reliable handling of post-disaster emergency monitoring tasks is crucial for effective rescue operations. UAV-assisted edge computing plays a pivotal role in the rapid deployment of such systems. However, challenges persist due to communication and computation resource bottlenecks when dealing with delay-sensitive monitoring tasks. In dynamic post-disaster environments, effective task scheduling and resource allocation decisions directly impact the system’s ability to process tasks. Therefore, this paper proposes an adaptive task migration decision-making system for emergency monitoring tasks in UAV-assisted edge computing. Firstly, We decomposed the optimization objectives based on the task processing workflow, then devised a stepwise delay risk control and resource recovery mechanism based on early discarding. Secondly, by integrating multi-agent reinforcement learning (MARL), optimal strategies for task offloading, UAV queue scheduling, and communication resource allocation are learned to enhance the decision system’s environmental awareness and maximize the successful completion of emergency monitoring tasks. Simulation experiments demonstrate that the algorithm significantly improves the success rate of migration tasks and data processing capacity, thereby validating its convergence and effectiveness.

Autonomous UAV Navigation with Adaptive Control Based on Deep Reinforcement Learning

Unmanned aerial vehicle (UAV) navigation plays a crucial role in its ability to perform autonomous missions in complex environments. Most of the existing reinforcement learning methods to solve the UAV navigation problem fix the flight altitude and velocity, which largely reduces the difficulty of the algorithm. But the methods without adaptive control are not suitable in low-altitude environments with complex situations, generally suffering from weak obstacle avoidance. Some UAV navigation studies with adaptive flight only have weak obstacle avoidance capabilities. To address the problem of UAV navigation in low-altitude environments, we construct autonomous UAV navigation in 3D environments with adaptive control as a Markov decision process and propose a deep reinforcement learning algorithm. To solve the problem of weak obstacle avoidance, we creatively propose the guide attention method to make a UAV’s decision focus shift between the navigation task and obstacle avoidance task according to changes in the obstacle. We raise a novel velocity-constrained loss function and add it to the original actor loss to improve the UAV’s velocity control capability. Simulation experiment results demonstrate that our algorithm outperforms some of the state-of-the-art deep reinforcement learning algorithms performing UAV navigation tasks in a 3D environment and has outstanding performance in algorithm effectiveness, with the average reward increasing by 9.35, the success rate of navigation tasks increasing by 14%, and the collision rate decreasing by 14%.

Black widow optimization algorithm for efficient task assignment in cloud computing

Cloud computing was developed by blending virtualization and grid computing technologies. Its purpose is to provide Internet-based, on-demand, and consumption-based access to pools of computing resources in a measurable, adaptable, and scalable manner. Task scheduling is essential to cloud computing to ensure the performance of cloud services. However, inefficient scheduling can lead to resource issues such as under-allocation and over-allocation, which wastes resources and degrades service performance. Therefore, metaheuristic algorithms are incorporated into task scheduling systems to efficiently and timely distribute complex and diverse incoming tasks to limited resources. This study aims to analyze task priorities and precisely assign them to virtual machines. This is achieved by utilizing the Black Widow Optimization (BWO) algorithm. The primary objectives are to reduce time and energy consumption, improve task success rates, and optimize turnaround efficiency. Ultimately, these improvements aim to enhance the overall trustworthiness of the system.

Heuristic evaluation and end-user testing of a machine learning-based lower-limb exercise training system for management of knee pain in individuals aged 55 years or over

ObjectiveUsing machine learning techniques, we have developed an interactive exercise training system to assist individuals aged 55 years or over with knee pain to perform lower-limb exercises to improve their knee health. The system has three features: video-based exercise demonstrations, real-time feedback on exercise movements, and tracking of exercise performance and progress. The current study aimed to evaluate the design of the computer prototype of the system, and determine its usability and end users’ intention to use it (i.e., acceptance of it). MethodsHeuristic evaluation and end-user testing of the computer-based prototype system were conducted. Three human factors practitioners identified the design deficiencies, with reference to 64 design principles. In addition, 10 individuals with knee pain were recruited to use the prototype system to complete five tasks in the study laboratory. We recorded and examined the task success rate, number of requests for assistance, difficulties encountered during tasks, and perceptions of usability and acceptance. ResultsFour design deficiencies were identified, regarding recognition and recovery of errors, navigation, auditory perception, and help documentation. Most participants had difficulty in calibrating the camera and performing exercises. However, in general, the prototype system was perceived as usable and acceptable. ConclusionsThe use of heuristic evaluation and end-user testing revealed the capacity to systematically detect design deficiencies in interactive self-help systems, allowing for effective system adjustments. Moreover, our system shows potential for individuals managing knee pain, but conducting iterative usability testing is necessary to identify additional improvements. Furthermore, several design propositions have been submitted.

To Abstract or Not to Abstract? A Comparative Study Evaluating the User Experience of Spreadsheet Programming with Sheet-Defined Functional Abstractions

Spreadsheets are a widespread functional programming paradigm that offer liveness and directness of interaction. However, spreadsheets are notoriously error-prone and difficult to debug. To overcome this limitation and improve the expressive power of spreadsheets, we propose an extension to the spreadsheet paradigm in the form of sheet-defined lambdas – user-defined functions that abstract computations on the sheet. This concept was developed and deployed in our web-based spreadsheet application named Lattice. We evaluate this approach through a user study which compared the user experience of programming in a spreadsheet with and without lambdas, as well as the difference in performance between learner (N = 12) and experienced (N = 12) programmers. The study measured participant task performance (task time, success rate and number of errors) and the quality of their user experience of using Lattice (video recordings of use, interviews and questionnaire responses). Our findings indicate that programming with lambdas is not only more efficient than writing formulas in a conventional way, but also provides a rewarding hedonic experience. However, we found that learners perceived the concept of functional abstractions with lambdas as difficult to comprehend; while experienced programmers noted potential utilitarian advantages that aid in managing the complexity of a spreadsheet program. The results obtained in this work contribute to a better understanding of human-spreadsheet interaction and can inform the future design of user-friendly computational systems.

A Hybrid Genetic Algorithm for Ground Station Scheduling Problems

In recent years, the substantial growth in satellite data transmission tasks and volume, coupled with the limited availability of ground station hardware resources, has exacerbated conflicts among missions and rendered traditional scheduling algorithms inadequate. To address this challenge, this paper introduces an improved tabu genetic hybrid algorithm (ITGA) integrated with heuristic rules for the first time. Firstly, a constraint satisfaction model for satellite data transmission tasks is established, considering multiple factors such as task execution windows, satellite–ground visibility, and ground station capabilities. Leveraging heuristic rules, an initial population of high-fitness chromosomes is selected for iterative refinement. Secondly, the proposed hybrid algorithm iteratively evolves this population towards optimal solutions. Finally, the scheduling plan with the highest fitness value is selected as the best strategy. Comparative simulation experimental results demonstrate that, across four distinct scenarios, our algorithm achieves improvements in the average task success rate ranging from 1.5% to 19.8% compared to alternative methods. Moreover, it reduces the average algorithm execution time by 0.5 s to 28.46 s and enhances algorithm stability by 0.8% to 27.7%. This research contributes a novel approach to the efficient scheduling of satellite data transmission tasks.

SMITS: Social and Mobility aware Intelligent Task Scheduling in Vehicular Fog Computing — A Federated DRL Approach

Vehicular fog computing (VFC) has become an enticing research hot spot to provide resources for the Internet of Vehicles (IoV) application requests. Moreover, with the massive expansion of services in vehicular fog networks such as augmented reality, 3-D gaming, and autonomous driving, the allocation of fog computing resources to IoV applications constraining the deadline and energy consumption of the vehicles is becoming challenging. In this paper, we investigate a task scheduling problem in vehicular fog networks to jointly optimize the success rate of vehicular tasks and energy consumption of vehicles by considering the moving paths of the vehicles and the social relationships among the vehicles. In this problem, we model the social relationship among the vehicles based on their communication patterns and social characteristics to improve the success rate of the tasks. We also incorporate the mobility of vehicles using the Markov renewal process (MRP) technique. Initially, we formulate a mixed integer nonlinear programming (MINLP) problem for the proposed problem and further prove it to be an NP-hard problem. To solve the proposed problem, we design a federated deep reinforcement learning (FDRL) mechanism by converting the problem into a Markov decision process (MDP) problem. We compare our proposed algorithm with existing scheduling approaches, and simulation results show that the proposed algorithm performs efficiently for task scheduling problems. The proposed algorithm achieved a substantial improvement of 12% in the maximization of the success rate and 36% in the minimization of the energy consumption compared to the existing algorithms.

User Experience Evaluation of a Spinal Surgery Robot: Workload, Usability, and Satisfaction Study.

Robotic spine surgery has continued to evolve since its US Food and Drug Administration approval in 2004, with products now including real-time video guidance and navigation during surgery. As the market for robotic surgical devices evolves, it is important to consider usability factors. The primary objective of this study was to determine the user experience of a surgical-assistive robotic device. The secondary objective was to evaluate workload, usability, the After-Scenario Questionnaire (ASQ), and the System Usability Scale (SUS). In addition, this study compares the workload, usability, and satisfaction survey of the device among different occupational groups using the device. Doctors (n=15) and nurses (n=15), the intended users of the surgical assistant robot, participated in the usability evaluation. Participants performed essential scenarios for the surgical assistant robot and provided scenario-specific satisfaction (ASQ), workload (NASA Task Load Index), and usability (SUS) scores. Both doctors and nurses had task success rates of 85% or higher for each scenario. ASQ results showed that both doctors and nurses were least satisfied with ease of completing the task of registration (group 1: mean 4.73, SD 1.57 and group 2: mean 4.47, SD 1.8), amount of time it took (group 1: mean 4.47, SD 1.63 and group 2: mean 4.40, SD 2.09), and support information satisfaction (group 1: mean 5.13, SD 1.50 and group 2: mean 5.13, SD 1.89). All participants had low workloads, and the overall Task Load Index score had a P value of .77, which is greater than .05. The SUS results showed that the overall usability mean for doctors was 64.17 (SD 16.52) and the mean for nurses was 61.67 (SD 19.18), with a P value of .84, which is greater than .05, indicating no difference between the 2 groups. In this study, doctors and nurses evaluated the interaction of the device in a simulated environment, the operating room. By evaluating the use experience and usability of the device with real intended users, we can develop a more effective and convenient user interface.

Does Few-Shot Learning Suffer from Backdoor Attacks?

The field of few-shot learning (FSL) has shown promising results in scenarios where training data is limited, but its vulnerability to backdoor attacks remains largely unexplored. We first explore this topic by first evaluating the performance of the existing backdoor attack methods on few-shot learning scenarios. Unlike in standard supervised learning, existing backdoor attack methods failed to perform an effective attack in FSL due to two main issues. Firstly, the model tends to overfit to either benign features or trigger features, causing a tough trade-off between attack success rate and benign accuracy. Secondly, due to the small number of training samples, the dirty label or visible trigger in the support set can be easily detected by victims, which reduces the stealthiness of attacks. It seemed that FSL could survive from backdoor attacks. However, in this paper, we propose the Few-shot Learning Backdoor Attack (FLBA) to show that FSL can still be vulnerable to backdoor attacks. Specifically, we first generate a trigger to maximize the gap between poisoned and benign features. It enables the model to learn both benign and trigger features, which solves the problem of overfitting. To make it more stealthy, we hide the trigger by optimizing two types of imperceptible perturbation, namely attractive and repulsive perturbation, instead of attaching the trigger directly. Once we obtain the perturbations, we can poison all samples in the benign support set into a hidden poisoned support set and fine-tune the model on it. Our method demonstrates a high Attack Success Rate (ASR) in FSL tasks with different few-shot learning paradigms while preserving clean accuracy and maintaining stealthiness. This study reveals that few-shot learning still suffers from backdoor attacks, and its security should be given attention.

Learning Sliding Policy of Flat Multi-target Objects in Clutter Scenes

In clutter scenes, one or several targets need to be obtained, which is hard for robot manipulation task. Especially, when the targets are flat objects like book, plates, due to limitation of common robot end-effectors, it will be more challenging. By employing pre-grasp operation like sliding, it becomes feasible to rearrange objects and shift the target towards table edge, enabling the robot to grasp it from a lateral perspective. In this paper, the proposed method transfers the task into a Parameterized Action Markov Decision Process to solve the problem, which is based on deep reinforcement learning. The mask images are taken as one of observations to the network for avoiding the impact of noise of original image. In order to improve data utilization, the policynetwork predicts the parameters for the sliding primitive of each object, which is weight-sharing, and then the Q-network selects the optimal execution target. Meanwhile, extra reward mechanism is adopted for improving the efficiency of task actions to cope with multiple targets. In addition, an adaptive policy scaling algorithm is proposed to improve the speed and adaptability of policy training. In both simulation and real system, our method achieves a higher task success rate and requires fewer actions to accomplish the flat multi-target sliding manipulation task within clutter scene, which verifies the effectiveness of ours.

Energy‐efficient reliability‐aware offloading for delay‐sensitive tasks in collaborative edge computing

SummaryAs a burgeoning paradigm, collaborative mobile edge computing (C‐MEC) can cater to growing computation demand of mobile devices (MDs). However, there are great challenges for joint task offloading and resource allocation. In addition, failures on both MDs and edge servers greatly affect reliable task execution. This paper investigates the joint optimization problem of offloading decision, power allocation, and computation resource allocation in the multi‐user C‐MEC system, where nondivisible tasks may be executed locally, offloaded to nearby collaborative devices, or processed by the edge server. We aim at minimizing the total energy consumption of MDs while satisfying the reliability and delay constraints, and the replication technique is adopted to enhance task execution reliability. To tackle this problem, we first transform it into a bi‐level optimization problem, and then propose an iterative algorithm named JOC. Specifically, the offloading decision is constructed in the upper level on the basis of ant colony system (ACS), and the allocation of working power, transmission power, and computation resources is optimized in the lower level by using the monotonic optimization approach. Simulation experimental results reveal that the proposed algorithm saves more energy and achieves a higher task success rate in comparison with baseline schemes.

ADTO: A Trust Active Detecting-based Task Offloading Scheme in Edge Computing for Internet of Things

In edge computing, Internet of Things devices with weak computing power offload tasks to nearby edge servers for execution, so the task completion time can be reduced and delay-sensitive tasks can be facilitated. However, if the task is offloaded to malicious edge servers, then the system will suffer losses. Therefore, it is significant to identify the trusted edge servers and offload tasks to trusted edge servers, which can improve the performance of edge computing. However, it is still challenging. In this article, a trust Active Detecting-based Task Offloading (ADTO) scheme is proposed to maximize revenue in edge computing. The main innovation points of our work are as follows: (a) The ADTO scheme innovatively proposes a method to actively get trust by trust detection. This method offloads microtasks to edge servers whose trust needs to be identified, and then quickly identifies the trust of edge servers according to the completion of tasks by edge servers. Based on the identification of the trust, tasks can be offloaded to trusted edge servers, to improve the success rate of tasks. (b) Although the trust of edge servers can be identified by our detection, it needs to pay a price. Therefore, to maximize system revenue, searching the most suitable number of trusted edge servers for various conditions is transformed into an optimization problem. Finally, theoretical and experimental analysis shows the effectiveness of the proposed strategy, which can effectively identify the trusted and untrusted edge servers. The task offloading strategy based on trust detection proposed in this article greatly improves the success rate of tasks, compared with the strategy without trust detection, the task success rate is increased by 40.27%, and there is a significant increase in revenue, which fully demonstrates the effectiveness of the strategy.

SSKHOA: Hybrid Metaheuristic Algorithm for Resource Aware Task Scheduling in Cloud-fog Computing

Cloud fog computing is a new paradigm that combines cloud computing and fog computing to boost resource efficiency and distributed system performance. Task scheduling is crucial in cloud fog computing because it decides the way computer resources are divided up across tasks. Our study suggests that the Shark Search Krill Herd Optimization (SSKHOA) method be incorporated into cloud fog computing's task scheduling. To enhance both the global and local search capabilities of the optimization process, the SSKHOA algorithm combines the shark search algorithm and the krill herd algorithm. It quickly explores the solution space and finds near-optimal work schedules by modelling the swarm intelligence of krill herds and the predator-prey behavior of sharks. In order to test the efficacy of the SSKHOA algorithm, we created a synthetic cloud fog environment and performed some tests. Traditional task scheduling techniques like LTRA, DRL, and DAPSO were used to evaluate the findings. The experimental results demonstrate that the SSKHOA outperformed the baseline algorithms in terms of task success rate increased 34%, reduced the execution time by 36%, and reduced makespan time by 54% respectively.

Intelligent Computation Offloading Based on Digital Twin-Enabled 6G Industrial IoT

Digital twin (DT) technology, which can provide larger and more accurate amounts of data, combined with the additional computility brought by virtual environments, can support more complex connected industrial applications. Simultaneously, the development and maturity of 6G technology has driven the development of industrial manufacturing and greatly improved the operational efficiency of the industrial internet of things (IIoT). Nevertheless, massive data, heterogeneous IoT device attributes, and the deterministic and bounded latency for delay sensitive applications are major barriers to improving the quality of services (QoS) in the IIoT. In this article, we first construct a new DT-enabled network architecture and computation offloading delay model in the IIoT. Then, the computation offloading problem is formulated with the goal of minimizing the overall task completion delay and achieving resource allocation. Since the formulation is a joint optimization problem, we use deep reinforcement learning (DRL) to solve the original problem, which can be described by a Markov decision process (MDP). Numerical results show that our proposed scheme is able to improve the task success rate and reduce the task processing end-to-end delay compared to the benchmark schemes.