Rescue Tasks Research Articles

Text-Guided Multi-Class Multi-Object Tracking for Fine-Grained Maritime Rescue

The rapid development of remote sensing technology has provided new sources of data for marine rescue and has made it possible to find and track survivors. Due to the requirement of tracking multiple survivors at the same time, multi-object tracking (MOT) has become the key subtask of marine rescue. However, there exists a significant gap between fine-grained objects in realistic marine rescue remote sensing data and the fine-grained object tracking capability of existing MOT technologies, which mainly focuses on coarse-grained object scenarios and fails to track fine-grained instances. Such a gap limits the practical application of MOT in realistic marine rescue remote sensing data, especially when rescue forces are limited. Given the promising fine-grained classification performance of recent text-guided methods, we delve into leveraging labels and attributes to narrow the gap between MOT and fine-grained maritime rescue. We propose a text-guided multi-class multi-object tracking (TG-MCMOT) method. To handle the problem raised by fine-grained classes, we design a multi-modal encoder by aligning external textual information with visual inputs. We use decoding information at different levels, simultaneously predicting the category, location, and identity embedding features of objects. Meanwhile, to improve the performance of small object detection, we also develop a data augmentation pipeline to generate pseudo-near-infrared images based on RGB images. Extensive experiments demonstrate that our TG-MCMOT not only performs well on typical metrics in the maritime rescue task (SeaDronesSee dataset), but it also effectively tracks open-set categories on the BURST dataset. Specifically, on the SeaDronesSee dataset, the Higher Order Tracking Accuracy (HOTA) reached a score of 58.8, and on the BURST test dataset, the HOTA score for the unknown class improved by 16.07 points.

A Two-Stage Target Search and Tracking Method for UAV Based on Deep Reinforcement Learning

To deal with the complexities of decision-making for unmanned aerial vehicles (UAVs) in denial environments, this paper applies deep reinforcement learning algorithms to search and rescue (SAR) tasks. It proposes a two-stage target search and tracking method for UAVs based on deep reinforcement learning, which divides SAR tasks into search and tracking stages, and the controllers for each stage are trained based on the proposed deep deterministic policy gradient with three critic networks (DDPG-3C) algorithm. Simulation experiments are carried out to evaluate the performance of each stage in a two-dimensional rectangular SAR scenario, including search, tracking, and the integrated whole stage. The experimental results show that the proposed DDPG-3C model can effectively alleviate the overestimation problem, and hence results in a faster convergence and improved performance during both the search and tracking stages. Additionally, the two-stage target search and tracking method outperforms the traditional single-stage approach, leading to a more efficient and effective decision-making ability in SAR tasks.

Evacuation of the Injured Using Unmanned Systems

This article explores the use of unmanned systems for evacuating wounded personnel from the battlefield. It focuses on conclusions drawn from conventional medical evacuation methods, particularly when wounded soldiers are still in the enemy’s operational zone, following the principles of Tactical Combat Casualty Care (TCCC). Then, general conditions of the injured were determined, which would require evacuation to a safe zone. This allowed us to define the critical capabilities needed for unmanned systems to perform these tasks. The ability of the unmanned system to pick up an injured person who is unconscious and unable to cooperate was identified as crucial. In the further part of the work, existing unmanned systems intended for rescue tasks were collected and critically analysed.

White Paper on Adaptive Situational Awareness Enhancing Augmented Reality Interface Design on First Responders in Rescue Tasks

The advance in the development of augmented reality technologies has attracted interest in their applicability in rescue scenarios. The characteristics of the different missions covered by First Responders, as well as the different objectives they can cover in a rescue operation, condition the importance of the additional information they can receive in these rescue processes through technology. This white paper aims to analyze the difficulties encountered when converging on the design of an interface that is adaptable to the professional and contextual circumstances in a rescue task.

Extending boids for safety-critical search and rescue

Robot swarms can accomplish complex tasks, and in this work, we seek to design swarm robotic algorithms for search and rescue that are scalable to large swarms, efficient in terms of computations, safe from collisions, and tunable to mediate the trade-off between exploration and exploitation in the search. We propose extending the Boids algorithm to accomplish this. Without modifying the three Boids rules of alignment, cohesion, and separation, we add target-seeking and general collision avoidance by using ghost boids. Additionally, we use a control barrier function to improve safety at the cost of increased computation. Via simulation in a search and rescue task, we analyze the trade-offs between safety, computational efficiency, and coverage of the environment for our algorithm.

Research on the water-ground multimodal transport emergency scheduling model and decision-making method considering the actual road network inundation situation.

As urbanization continues to accelerate worldwide, urban flooding is becoming increasingly destructive, making it important to improve emergency scheduling capabilities. Compared to other scheduling problems, the urban flood emergency rescue scheduling problem is more complicated. Considering the impact of a disaster on the road network passability, a single type of vehicle cannot complete all rescue tasks. A reasonable combination of multiple vehicle types for cooperative rescue can improve the efficiency of rescue tasks. This study focuses on the urban flood emergency rescue scheduling problem considering the actual road network inundation situation. First, the progress and shortcomings of related research are analyzed. Then, a four-level emergency transportation network based on the collaborative water-ground multimodal transport transshipment mode is established. It is shown that the transshipment points have random locations and quantities according to the actual inundation situation. Subsequently, an interactive model based on hierarchical optimization is constructed considering the travel length, travel time, and waiting time as hierarchical optimization objectives. Next, an improved A* algorithm based on the quantity of specific extension nodes is proposed, and a scheduling scheme decision-making algorithm is proposed based on the improved A* and greedy algorithms. Finally, the proposed decision-making algorithm is applied in a practical example for solving and comparative analysis, and the results show that the improved A* algorithm is faster and more accurate. The results also verify the effectiveness of the scheduling model and decision-making algorithm. Finally, a scheduling scheme with the shortest travel time for the proposed emergency scheduling problem is obtained.

АНАЛИЗ АЭРОМОБИЛЬНЫХ ТЕХНОЛОГИЙ, ПРИМЕНЯЕМЫХ ДЛЯ ДОСТАВКИ ЛИЧНОГО СОСТАВА СПАСАТЕЛЬНЫХ ПОДРАЗДЕЛЕНИЙ И РАЗЛИЧНЫХ ГРУЗОВ В УСЛОВИЯХ АРКТИЧЕСКОЙ ЗОНЫ РОССИЙСКОЙ ФЕДЕРАЦИИ

The relevance of the implementation of airmobile technologies for the purpose of emergency rescue and search and rescue operations, carrying out measures to protect the population in case of natural and man-made emergencies, measures to eliminate the consequences of emergencies in the Arctic zone of the Russian Federation with its huge distances and the focal nature of infrastructure development is due to: an underdeveloped road network; the dispersion of settlements over a vast territory and remoteness small settlements from district centers with vital infrastructure; seasonality of navigation of water transport; the rapidly developing economic activity in the region, the need for search and rescue and rescue support for facilities in the Arctic zone of the Russian Federation. When analyzing the effectiveness of airmobile technologies used to deliver personnel of rescue units and various cargoes when performing tasks for their intended purpose in the Arctic zone of the Russian Federation, the following aspects of the use of aviation during search and rescue tasks by units of EMERCOM of Russia were considered: types, tactical and technical characteristics, capabilities of aircraft equipped with aviation of EMERCOM of Russia; ways of implementing airmobile technologies for the delivery of personnel and cargo by air transport; the specifics of the implementation of airmobile technologies in the Arctic zone of the Russian Federation. In order to ensure the implementation of airmobile technologies, a number of measures are recommended, in particular, to organize an airborne service in the Arctic integrated emergency rescue center, to provide cargo and human parachute systems, landing equipment and property.

AMARL: An Attention-Based Multiagent Reinforcement Learning Approach to the Min-Max Multiple Traveling Salesmen Problem.

In recent years, the multiple traveling salesmen problem (MTSP or multiple TSP) has received increasing research interest and one of its main applications is coordinated multirobot mission planning, such as cooperative search and rescue tasks. However, it is still challenging to solve MTSP with improved inference efficiency as well as solution quality in varying situations, e.g., different city positions, different numbers of cities, or agents. In this article, we propose an attention-based multiagent reinforcement learning (AMARL) approach, which is based on the gated transformer feature representations for min-max multiple TSPs. The state feature extraction network in our proposed approach adopts the gated transformer architecture with reordering layer normalization (LN) and a new gate mechanism. It aggregates fixed-dimensional attention-based state features irrespective of the number of agents and cities. The action space of our proposed approach is designed to decouple the interaction of agents' simultaneous decision-making. At each time step, only one agent is assigned to a non-zero action so that the action selection strategy can be transferred across tasks with different numbers of agents and cities. Extensive experiments on min-max multiple TSPs were conducted to illustrate the effectiveness and advantages of the proposed approach. Compared with six representative algorithms, our proposed approach achieves state-of-the-art performance in solution quality and inference efficiency. In particular, the proposed approach is suitable for tasks with different numbers of agents or cities without extra learning, and experimental results demonstrate that the proposed approach realizes powerful transfer capability across tasks.

HDetect-VS: Tiny Human Object Enhancement and Detection Based on Visual Saliency for Maritime Search and Rescue

Strong sun glint noise is an inevitable obstruction for tiny human object detection in maritime search and rescue (SAR) tasks, which can significantly deteriorate the performance of local contrast method (LCM)-based algorithms and cause high false alarm rates. For SAR tasks in noisy environments, it is more important to find tiny objects than localize them. Hence, considering background clutter and strong glint noise, in this study, a noise suppression methodology for maritime scenarios (HDetect-VS) is established to achieve tiny human object enhancement and detection based on visual saliency. To this end, the pixel intensity value distributions, color characteristics, and spatial distributions are thoroughly analyzed to separate objects from background and glint noise. Using unmanned aerial vehicles (UAVs), visible images with rich details, rather than infrared images, are applied to detect tiny objects in noisy environments. In this study, a grayscale model mapped from the HSV model (HSV-gray) is used to suppress glint noise based on color characteristic analysis, and large-scale Gaussian Convolution is utilized to obtain the pixel intensity surface and suppress background noise based on pixel intensity value distributions. Moreover, based on a thorough analysis of the spatial distribution of objects and noise, two-step clustering is employed to separate objects from noise in a salient point map. Experiments are conducted on the SeaDronesSee dataset; the results illustrate that HDetect-VS has more robust and effective performance in tiny object detection in noisy environments than other pixel-level algorithms. In particular, the performance of existing deep learning-based object detection algorithms can be significantly improved by taking the results of HDetect-VS as input.

Intelligent Ship Scheduling and Path Planning Method for Maritime Emergency Rescue

Intelligent ship navigation scheduling and planning is of great significance for ensuring the safety of maritime production and life and promoting the development of the marine economy. In this paper, an intelligent ship scheduling and path planning method is proposed for a practical application scenario wherein the emergency rescue center receives rescue messages and dispatches emergency rescue ships to the incident area for rescue. Firstly, the large-scale sailing route of the task ship is pre-planned in the voyage planning stage by using the improved A* algorithm. Secondly, the full-coverage path planning algorithm is used to plan the ship’s search route in the regional search stage by updating the ship’s navigation route in real time. In order to verify the effectiveness of the proposed algorithm, comparative experiments were carried out with the conventional algorithm in the two operation stages of rushing to the incident sea area and regional search and rescue. The experimental results show that the proposed algorithm can adapt to emergency search and rescue tasks in the complex setting of the sea area and can effectively improve the efficiency of the operation, ensure the safety of the operation process, and provide a more intelligent and efficient solution for the planning of maritime emergency rescue tasks.

Active robotic search for victims using ensemble deep learning techniques

In recent years, legged quadruped robots have proved to be a valuable support to humans in dealing with search and rescue operations. These robots can move with great ability in complex terrains, unstructured environments or regions with many obstacles. This work employs the quadruped robot A1 Rescue Tasks UPM Robot (ARTU-R) by Unitree, equipped with an RGB-D camera and a lidar, to perform victim searches in post-disaster scenarios. Exploration is done not by following a pre-planned path (as common methods) but by prioritising the areas most likely to harbour victims. To accomplish that task, both indirect search and next best view techniques have been used. When ARTU-R gets inside an unstructured and unknown environment, it selects the next exploration point from a series of candidates. This operation is performed by comparing, for each candidate, the distance to reach it, the unexplored space around it and the probability of a victim being in its vicinity. This probability value is obtained using a Random Forest, which processes the information provided by a convolutional neural network. Unlike other AI techniques, random forests are not black box models; humans can understand their decision-making processes. The system, once integrated, achieves speeds comparable to other state-of-the-art algorithms in terms of exploration, but concerning victim detection, the tests show that the resulting smart exploration generates logical paths—from a human point of view—and that ARTU-R tends to move first to the regions where victims are present.

Collaborative Task Allocation and Optimization Solution for Unmanned Aerial Vehicles in Search and Rescue

Earthquakes pose significant risks to national stability, endangering lives and causing substantial economic damage. This study tackles the urgent need for efficient post-earthquake relief in search and rescue (SAR) scenarios by proposing a multi-UAV cooperative rescue task allocation model. With consideration the unique requirements of post-earthquake rescue missions, the model aims to minimize the number of UAVs deployed, reduce rescue costs, and shorten the duration of rescue operations. We propose an innovative hybrid algorithm combining particle swarm optimization (PSO) and grey wolf optimizer (GWO), called the PSOGWO algorithm, to achieve the objectives of the model. This algorithm is enhanced by various strategies, including interval transformation, nonlinear convergence factor, individual update strategy, and dynamic weighting rules. A practical case study illustrates the use of our model and algorithm in reality and validates its effectiveness by comparing it to PSO and GWO. Moreover, a sensitivity analysis on UAV capacity highlights its impact on the overall rescue time and cost. The research results contribute to the advancement of vehicle-routing problem (VRP) models and algorithms for post-earthquake relief in SAR. Furthermore, it provides optimized relief distribution strategies for rescue decision-makers, thereby improving the efficiency and effectiveness of SAR operations.

Dynamic Modeling of Unmanned Underwater Vehicles with Online Disturbance Compensation Scheme

With the advancement in robotics technology over the recent years, underwater robots’ design and development are gaining interest. Unmanned underwater vehicles (UUVs) have many applications in aquaculture, deep-sea exploration, research, and enhanced rescue tasks. However, various factors must be considered when developing any underwater vehicle system to explore the deep ends of the underwater world. In this paper, we develop the most suitable model for understanding various system parameters. The new mathematical model considers certain constraints and external disturbances exerted on the system. Also, a control strategy is suggested for the UUV’s stability and robustness. The suggested observer and model are simple, allowing for accurate estimations of all system states and the global impacts of unknown limped perturbations with a minimal computational cost.

Research on Multi-UAV Task Assignment Based on a Multi-Objective, Improved Brainstorming Optimization Algorithm

In response to the practice of rescue channel blocking and a shortage of emergency materials in the event of sudden significant disasters, a multi-UAV collaborative distribution scheme was designed based on the demand for rapid and accurate distribution of materials. This paper constructed a multi-UAV collaborative task assignment and routing problem with simultaneous delivery and pick-up and time windows (MVTARPSDPTW), considering the factors of UAV load, energy consumption, cargo quality, and volume to minimize the total cost of UAV distribution and the full penalty of the task, as well as optimizing the balance of UAV efficiency. This paper proposes a multi-objective, improved brainstorming optimization algorithm based on Pareto dominance (MIBSO) to solve the MVTARPSDPTW problem. With DTLZ4, DTLZ5, and DTLZ6 benchmarks, this work tests the algorithm performance according to the characteristic attributes of the model sought, selecting the four indicators of GD, the Spacing metric, HV, and IGD, concerning convergence, solution distribution, and comprehensive performance. Case validation is based on a COVID-19 scenario in Changchun, China, and the results show that the model algorithm designed in this paper has good performance and feasibility in convergence and distribution of reconciliation. Finally, the multi-UAV emergency material distribution solution provides practical, theoretical support for rescue tasks in sudden significant disasters.

Triboelectric in-sensor deep learning for self-powered gesture recognition toward multifunctional rescue tasks

Gesture recognition is an effective method to voice-off communication in daily life, offering an intuitive and immersive means to interactive terminal. Current commercialized solutions to gesture recognition are exposed to limitations of either susceptibility to environmental conditions or the huge cost on fabrication, energy, and computing power. Here, we propose a deep-learning-enabled smart gesture recognition glove system that can capture and recognize the hand gesture in real time and execute the users’ interactive instructions for multifunctional rescue tasks. The self-powered gesture recognition glove system can successfully classify 7 typical gestures with an accuracy as high as 98.57%. As a proof of concept, a rescue vehicle equipped with water pump, dexterous robotic arm, or laser radar is demonstrated to be guided by gestures outside the environment to complete rescue tasks even in environments that are inaccessible to humans. We aim to use the self-powered gesture recognition glove for future intelligent interaction and intelligent manufacturing, especially in the diversified scenarios that require more advanced and dexterous manipulations in the dark or blocked spaces.

A Herd-Foraging-Based Approach to Adaptive Coverage Path Planning in Dual Environments.

Coverage path planning (CPP) is essential for robotic tasks, such as environmental monitoring and terrain surveying, which require covering all surface areas of interest. As the pioneering approach to CPP, inspired by the concept of predation risk in predator-prey relations, the predator-prey CPP (PPCPP) has the benefit of adaptively covering arbitrary bent 2-D manifolds and can handle unexpected changes in an environment, such as the sudden introduction of dynamic obstacles. However, it can only work in bounded environment and cannot handle tasks in unbounded one, e.g., search and rescue tasks where the search boundary is unknown. Sometimes, robots are required to handle both bounded and unbounded environments, i.e., dual environments, such as capturing criminals in a city. Once encountering a building, the robot enters it to cover the bounded environment, then continues to cover the unbounded one when leaving the building. Therefore, the capability of swarm robots for the coverage tasks both in bounded and unbounded environments is important. In nature, herbivores live in groups to find more food and reduce the risk of predation. Especially the juvenile ones prefer to forage near the herd to protect themselves. Inspired by the foraging behavior of animals in a herd, this article proposes an online adaptive CPP approach that enables swarm robots to handle both bounded and unbounded environments without knowing the environmental information in advance, called dual-environmental herd-foraging-based CPP (DH-CPP). It's performance is evaluated in dual environments with stationary and dynamic obstacles of different shapes and quantity, and compared with three state-of-the-art approaches. Simulation results demonstrate that it is highly effective to handle dual environments.

Use of cross‐training in human–robot collaborative rescue

AbstractHuman–robot collaboration has been widely used in postdisaster investigation and rescue. Human–robot team training is a good way to improve the team rescue efficiency and safety; two common training methods, namely, procedural training and cross‐training, are explored in this study. Currently, relatively few studies have explored the impact of cross‐training on human–robot collaboration in rescue tasks. Cross‐training will be novel to most rescuers and as such, an evaluation of cross‐training in comparison with more conventional procedural training is warranted. This study investigated the effects of these two training methods on rescue performance, situation awareness and workload. Forty‐two participants completed a path‐planning and a photo‐taking task in an unfamiliar simulated postdisaster environment. The rescue performance results showed that cross‐training method had significant advantages over procedural training for human–robot collaborative rescue tasks. During the training process, compared with procedural training, participants were more likely to achieve excellent photo‐taking performance after cross‐training; after training, the length of the route planned by the cross‐training group was significantly shorter than that of the procedural‐training group. In addition, procedural‐training marginal significantly increased the emotion demand, which proves that cross‐training can well control the emotions of the operators and make them more involved in the rescue task. The study also found that arousal level increased significantly after the first cross‐training session, and decreased to the same level as procedural training after multiple sessions. These results contribute to the application of cross‐training in human–robot collaborative rescue teams.

Biomechanical characterization of firefighters running under different rescue tasks

The biomechanical characteristics of runs in firefighters with different rescue tasks are unclear. This study aimed to explore the biomechanical characteristics of firefighters running in different rescue tasks and provide theoretical and practical references for firefighter training and occupational injury prevention. Eighteen professional healthy male firefighters were randomly selected as participants and tested running on different rescue tasks: wearing firefighting protective clothing (FPC), FPC+carrying a gas can (20 kg, FPC+ C), and FPC+carrying a mannequin (60 kg, FPC+M). Eight Qualisys infrared cameras and an AMTI 3D force measurement platform were used for the participant's acquisition of lower limb kinematic/kinetic data. The results showed that gait velocity and stride length of the FPC+GC and FPC+ M rescue tasks were significantly decreased compared to the FPC rescue task, while the support time was significantly increased. Compared to the FPC rescue task, the FPC+GC rescue task showed significant decreases in vertical ground reaction force (vGRF), minimum ankle dorsiflexion angle, and the maximum ankle plantarflexion power. In contrast, the FPC+M rescue task demonstrated significant increases in ankle range of motion, maximum hip extension angle, minimum knee flexion angle, maximum ankle dorsiflexion angle, maximum hip extension moment, maximum knee flexion moment, maximum hip flexion power, and hip and knee stiffness while exhibiting significant decreases in minimum ankle dorsiflexion angle. Compared to the FPC+ GC rescue task, the FPC+M rescue task exhibited significant increases in the maximum hip extension angle, minimum knee flexion angle, maximum ankle dorsiflexion angle, maximum hip flexion moment, maximum hip extension moment, maximum knee flexion moment, maximum ankle plantarflexion moment, maximum hip flexion power, maximum ankle dorsiflexion power, hip stiffness, and vGRF. Conversely, it showed significant decreases in the maximum knee flexion power. In conclusion, compared to the FPC rescue task, the FPC+GC and FPC+M rescue tasks altered the firefighter's gait performance, as evidenced by decreased gait velocity and stride length and increased support time. FPC+M rescue task would increase firefighter's risk of hip and knee injuries. Therefore, we suggest firefighters increase their strength training of the trunk, hip, and knee joint muscles as part of their daily training programs under large weight load status (60 kg and above) to reduce injury risk during rescue tasks.

Generating Task Reallocation Suggestions to Handle Contingencies in Human-Supervised Multi-Robot Missions

In a mission with significant uncertainty due to intermittent communications, delayed information flow, and robotic failures, the role of human supervisors is extremely challenging. As and when any new information arrives, humans must infer both the existing and predicted future states, identify potential contingencies, and update task assignments to robots rapidly. We propose methodologies for automated generation of task reallocation suggestions to humans to assist in the decision-making process. Our generated robot retasking plan minimizes a modified makespan of the mission, which incorporates task criticality and penalty for incomplete tasks. The plan considers the effects of potential mission contingencies on the tasks, the robots, and the future performance of the robots operating based on the previous task plans. Our method includes the incorporation of two optional tasks, i.e., relay and robot rescue, for performance improvement. The rescue task has probabilistic outcomes affecting the team size. One or more rescues are incorporated in a way that can minimize the expected value of the modified makespan overall possibilities of rescue outcomes. We have conducted performance evaluation using simulation, demonstrating the value of the optional tasks and performance enhancement using our method of incorporating them. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The work reported in this paper will be useful in applications where a team of agents is deployed to carry on a large-scale mission with communication constraints where the number of functional agents can change probabilistically. Typically, such uncertainty is encountered in applications that are challenging or dangerous in nature. Agents can have non-zero probabilities to fail while doing certain risky tasks and to get recovered by other agents. The proposed centralized multi-agent task reallocation method can help in proactively addressing potential contingencies in surveillance, search and rescue, or disaster relief to support resilient operations while having a supervisor in the higher chain of command.

Perceived and Actual Fire Safety – Case of Hybrid and Electric Vehicle Fires in Finland 2015–2023

As an emerging technology, hybrid and electric vehicles draw media attention and so does their fire safety. For this article, Finnish national rescue task database (Pronto) was reviewed for electric and hybrid vehicle fires for years 2015–2023, and the 44 records found were analyzed for the fire behavior and the successfulness of the rescue operation. Hybrid and electric vehicle fires are both absolutely and relatively rare compared to conventional vehicle fires. The incident rate for plug-in vehicles has been 0.2–0.9 per 10000 vehicles per year in the 2020s, which is significantly smaller than the rate for all passenger vehicles (4.6). Small absolute number is due to the small total number of hybrid and electric vehicles. Small relative number of fires may be due to the age of vehicles. Electrified vehicles can catch fire while plugged in, parked or while driving, with no significant differences in the incidence of the state of the vehicle. In road accidents, only one vehicle caught fire after a crash. All electric vehicle fires, even if the lithium-ion traction battery is involved, successfully extinguished with a branch pipe and traditional firefighting gear, as the rescue staff had been trained for handling incidents involving electric vehicles. There exists a media bias in reporting battery electric vehicle fires: of all 5 fires of BEV origin found, 4 have been reported in the media as an electric vehicle fire.