Rescue Environments Research Articles

Robotic Valve Turning with a Wheeled Mobile Manipulator via Hybrid Passive/Active Compliance.

This paper addresses the problems of valve-turning operation in rescue environments where a wheeled mobile manipulator (WMM) is employed, including the possible occurrence of large internal forces. Rather than attempting to obtain the exact position of the valve, this paper presents a solution to two main problems in robotic valve-turning operations: the radial position deviation between the rotation axes of the tool and the valve handle, which may cause large radial forces, and the possible axial displacement of the valve handle as the valve turns, which may lead to large axial forces. For the former problem, we designed a compliant end-effector with a tolerance of approximately 3.5° (angle) and 9.7 mm (position), and provided a hybrid passive/active compliance method. For the latter problem, a passivity-based force tracking algorithm was employed. Combining the custom-built compliant end-effector and the passivity-based control method can significantly reduce both the radial and the axial forces. Additionally, for valves with different installation types and WMMs with different configurations, we analyzed the minimum required number of actuators for valve turning. Simulation and experimental results are presented to show the effectiveness of the proposed approach.

Optimization of Predefined-Time Agent-Scheduling Strategy Based on PPO

In this paper, we introduce an agent rescue scheduling approach grounded in proximal policy optimization, coupled with a singularity-free predefined-time control strategy. The primary objective of this methodology is to bolster the efficiency and precision of rescue missions. Firstly, we have designed an evaluation function closely related to the average flying distance of agents, which provides a quantitative benchmark for assessing different scheduling schemes and assists in optimizing the allocation of rescue resources. Secondly, we have developed a scheduling strategy optimization method using the Proximal Policy Optimization (PPO) algorithm. This method can automatically learn and adjust scheduling strategies to adapt to complex rescue environments and varying task demands. The evaluation function provides crucial feedback signals for the PPO algorithm, ensuring that the algorithm can precisely adjust the scheduling strategies to achieve optimal results. Thirdly, aiming to attain stability and precision in agent navigation to designated positions, we formulate a singularity-free predefined-time fuzzy adaptive tracking control strategy. This approach dynamically modulates control parameters in reaction to external disturbances and uncertainties, thus ensuring the precise arrival of agents at their destinations within the predefined time. Finally, to substantiate the validity of our proposed approach, we crafted a simulation environment in Python 3.7, engaging in a comparative analysis between the PPO and the other optimization method, Deep Q-network (DQN), utilizing the variation in reward values as the benchmark for evaluation.

Fault-tolerant joint state feedback adaptive control for snake robots that move through random pole environments

ABSTRACT In unstructured terrain, snakes can move more efficiently by actively pushing and avoiding obstacles, a capability believed to be realized through a decentralized mechanism. Moreover, considering harsh and unpredictable rescue environments where snake robots are expected to be deployed, such as post-earthquake scenarios, some joint sensors are prone to failure. We propose a decentralized adaptation mechanism for snake robots to navigate through random pole environments. This mechanism integrates Fault-Tolerant Joint State Feedback Adaptive Control (FT-JSFAC) with Extended Curvature Derivatives control (ECD) and Angular Damping control (AD), leveraging the FT-JSFAC method to maintain operational robustness even when some joint sensors fail. To meet critical goals of adaptability, we utilize real-time angle and electrical current data from the robot's internal servo motors. Using this data, FT-JSFAC determines whether an obstacle is beneficial or detrimental to propulsion and then allows for decentralized torque adjustment to adapt to the environment. FT-JSFAC also employs historical data from functioning joints to compensate for erroneous sensor data, ensuring operational robustness even with joint sensor failure. Simulation experiments demonstrate that the proposed mechanism effectively adapts to environmental obstacles and maintains adaptability when one or two joint sensors fail.

A Review of Electric UAV Visual Detection and Navigation Technologies for Emergency Rescue Missions

Sudden disasters often result in significant losses of human lives and property, and emergency rescue is a necessary response to disasters. In recent years, with the development of electric unmanned aerial vehicles (UAVs) and artificial intelligence technology, the combination of these technologies has been gradually applied to emergency rescue missions. However, in the face of the complex working conditions of emergency rescue missions, the application of electric UAV visual detection still faces great challenges, particularly in relation to a lack of GPS positioning signal in closed emergency rescue environments, as well as unforeseen obstacle avoidance and autonomous planning and searching flights. Although the combination of visual detection and visual navigation technology shows great potential and added value for use in the context of emergency rescue, at present it remains in the research and experimental stages. Consequently, this paper summarizes and discusses the current status and development of visual detection and navigation technologies for electric UAVs, as well as issues related to emergency rescue applications, with a view to accelerating the research and application of visual detection and navigation technologies for electric UAVs in emergency rescue missions. In this study, we first summarize the classification of typical disasters, analyze the application of sample UAV and configurations in typical disasters with a high frequency of occurrence, refine key electric UAV technologies in emergency rescue missions, and propose the value of exploring electric UAV visual detection and navigation technologies. Subsequently, current research on electric UAV visual detection and navigation technology is analyzed and its application in emergency rescue missions is discussed. Finally, this paper presents the problems faced in the application of electric UAV visual detection and navigation technology in urban emergency rescue environments and offers insights into future research directions.

Geometry‐based flipper motion planning for articulated tracked robots traversing rough terrain in real‐time

AbstractTracked robots operating on rough terrain are often equipped with controllable flippers to help themselves overcome large obstacles or gaps. How to automate the control of these auxiliary flippers to achieve autonomous traversal remains an open question, which still necessitates inefficient manual teleoperation in practice. To tackle this problem, this article presents a geometry‐based motion planning method for an articulated tracked robot to self‐control its flippers during autonomous or semiautonomous traversal over rough terrain in urban search and rescue environments. The proposed method is developed by combining dynamic programming with a novel geometry‐based pose prediction method of high computational efficiency, which is applicable for typical challenging rescue terrains, such as stairs, Stepfields, and rails. The efficient pose prediction method allows us to make thousands of predictions about the robot poses at future locations for given flipper configurations within the onboard sensor range. On the basis of such predictions, our method evaluates the entire discretized configuration space and thereby determines the optimal flipper motion online for a smooth traversal over the terrain. The overall planning algorithm is tested with both simulated and real‐world robots and compared with a reinforcement‐learning‐based method using the RoboCup Rescue Robot League standard testing scenarios. The experimental results show that our method enables the robots to automatically control the flippers, successfully go over challenging terrains, and outperform the baseline method in passing smoothness and robustness to different terrains.

Multi-Drone Optimal Mission Assignment and 3D Path Planning for Disaster Rescue

In a three-dimensional (3D) disaster rescue mission environment, multi-drone mission assignments and path planning are challenging. Aiming at this problem, a mission assignment method based on adaptive genetic algorithms (AGA) and a path planning method using sine–cosine particle swarm optimization (SCPSO) are proposed. First, an original 3D digital terrain model is constructed. Second, common threat sources in disaster rescue environments are modeled, including mountains, transmission towers, and severe weather. Third, a cost–revenue function that considers factors such as drone performance, demand for mission points, elevation cost, and threat sources, is formulated to assign missions to multiple drones. Fourth, an AGA is employed to realize the multi-drone mission assignment. To enhance convergence speed and optimize performance in finding the optimal solution, an AGA using both the roulette method and the elite retention method is proposed. Additionally, the parameters of the AGA are adjusted according to the changes in the fitness function. Furthermore, the improved circle algorithm is also used to preprocess the mission sequence for AGA. Finally, based on the sine–cosine function model, a SCPSO is proposed for planning the optimal flight path between adjacent task points. In addition, the inertia and acceleration coefficients of linear weights are designed for SCPSO so as to enhance its performance to escape the local minimum, explore the search space more thoroughly, and achieve the purpose of global optimization. A multitude of simulation experiments have demonstrated the validity of our method.

Performance Evaluation of Standard and Modified OLSR Protocols for Uncoordinated UAV Ad-Hoc Networks in Search and Rescue Environments

Widespread usage of unmanned aerial vehicles (UAVs) in new and emerging applications needs dynamic and adaptive networking. The development of routing protocols for UAV ad hoc networks faces numerous issues because of the unique characteristics of UAVs, such as rapid mobility, frequent changes in network topology, and limited energy consumption. The Optimized Link State Routing (OLSR) protocol seems to be a promising solution as it offers improved delay performance. It is expected that OLSR will satisfy the strict demands of real-time UAV applications such as “search and rescue” (SAR) missions as it involves the most recent update of routing information. The classical OLSR routing protocol and its enhanced versions, D-OLSR, ML-OLSR, and P-OLSR, use different techniques to make an appropriate decision for routing packets. These routing techniques consider the quality of a wireless link, type of antenna, load, and mobility-aware mechanism to select the best UAV to send the message to the destination. This study evaluates and examines the performance of the original and modified OLSR routing protocols in UAV ad hoc networks for three SAR scenarios: (1) increasing mobility, (2) increasing scalability, and (3) increasing the allowed space of UAVs. It analyzes and validates the performance of the four OLSR-based routing protocols. It determines the best OSLR routing protocol by taking into account the packet delivery ratio, latency, energy consumption, and throughput. The four routing protocols and the SAR scenarios were simulated using NS-3.32. Based on the simulation results, ML-OLSR outperforms OLSR, D-OLSR, and P-OLSR in the considered measures.

Research on Lung Ultrasound Image Classification Based on Compressed Sensing.

Pneumothorax is a common injury in disaster rescue, traffic accidents, and war trauma environments and requires early diagnosis and treatment. The commonly used X-ray, CT, and other diagnostic instruments are not suitable for rescue sites due to their large size, heavy weight, and difficulty in transportation. Ultrasound equipment is easy to carry and suitable for rescue environments. However, ultrasound images are noisy, have low resolution, and are difficult to get started, which affects the efficiency of diagnosis. This paper studies the effect of lung ultrasound image recognition and classification based on compressed sensing and BP neural network. We use ultrasound equipment to build a lung simulation model, collect five typical features of lung ultrasound images in M-mode, and build a dataset. Using compressed sensing theory, we design sparse matrix and observation matrix and perform data compression on the image data in the dataset to obtain observation values. We design a BP neural network, input the observations into the network for training, and compare it with the commonly used VGG16 network. The method proposed in this paper has higher recognition accuracy and significantly fewer parameters than VGG16, so it is suitable for use in embedded devices.

Semantic Hazard Labelling and Risk Assessment Mapping During Robot Exploration

This paper proposes an innovative hazard identification and risk assessment mapping model for Urban Search and Rescue (USAR) environments, concentrating on a 3D mapping of the environment and performing grid-level semantic labeling to recognize all hazards types found in the scene and to distinguish their risk severity level. The introduced strategy employs a deep learning model to create semantic segments for hazard objects in 2D images and create semantically annotated point clouds that encapsulate occupancy and semantic annotations such as hazard type and risk severity level. After that, a 3D semantic map that provides situational awareness about the risk in the environment is built using the annotated point cloud. The proposed strategy is evaluated in a realistic simulated indoor environment, and the results show that the system successfully generates a risk assessment map. Further, an open-source package for the proposed approach is provided online for testing and reproducibility.

2D Lidar-Based SLAM and Path Planning for Indoor Rescue Using Mobile Robots

As the basic system of the rescue robot, the SLAM system largely determines whether the rescue robot can complete the rescue mission. Although the current 2D Lidar-based SLAM algorithm, including its application in indoor rescue environment, has achieved much success, the evaluation of SLAM algorithms combined with path planning for indoor rescue has rarely been studied. This paper studies mapping and path planning for mobile robots in an indoor rescue environment. Combined with path planning algorithm, this paper analyzes the applicability of three SLAM algorithms (GMapping algorithm, Hector-SLAM algorithm, and Cartographer algorithm) in indoor rescue environment. Real-time path planning is studied to test the mapping results. To balance path optimality and obstacle avoidance, A ∗ algorithm is used for global path planning, and DWA algorithm is adopted for local path planning. Experimental results validate the SLAM and path planning algorithms in simulated, emulated, and competition rescue environments, respectively. Finally, the results of this paper may facilitate researchers quickly and clearly selecting appropriate algorithms to build SLAM systems according to their own demands.

Using Deep Learning to Find Victims in Unknown Cluttered Urban Search and Rescue Environments

We investigate the first use of deep networks for victim identification in Urban Search and Rescue (USAR). Moreover, we provide the first experimental comparison of single-stage and two-stage networks for body part detection, for cases of partial occlusions and varying illumination, on a RGB-D dataset obtained by a mobile robot navigating cluttered USAR-like environments. We considered the single-stage detectors Single Shot Multi-box Detector, You Only Look Once, and RetinaNet and the two-stage Feature Pyramid Network detector. Experimental results show that RetinaNet has the highest mean average precision (77.66%) and recall (86.98%) for detecting victims with body part occlusions in different lighting conditions. End-to-end deep networks can be used for finding victims in USAR by autonomously extracting RGB-D image features from sensory data. We show that RetinaNet using RGB-D is robust to body part occlusions and low-lighting conditions and outperforms other detectors regardless of the image input type.

Data-Driven Skin Detection in Cluttered Search and Rescue Environments

Locating human victims in cluttered urban search and rescue (USAR) environments is still a challenge. In this paper, we present an approach to generate skin objectness windows to assist human rescuers. We introduce the term skin objectness to denote the task of extracting windows in the scene with a high probability of skin presence for locating victims. Unlike naive skin segmentation approaches, the presented algorithm accounts for both color and spatial information to extract regions of interest and at the same time, rejects the background clutter. We use temporal information of the video sequence to make the skin objectness windows more reliable. To selectively boost skin regions, the RGB skin pixels are transformed to Gabor space to generate a transformation matrix. The matrix is used to generate skin affinity maps removing the unwanted background clutter. Further, the Bayesian inference and temporal cues from previous frames are used for detecting skin objectness windows. It has real-time applications in image retrieval, action classification, virtual reality, etc. The experimental analysis demonstrates quantitative and qualitative results on a disaster victim dataset showing plausibility and efficiency of the proposed method in cluttered environments.

IoT Based Multi-Terrain Rover for Urban Search and Rescue Application

This paper presents the development of a semi-autonomous exploration perspective (approach) for Urban search and rescue environments (USAR). The developed rescue robot consists of a 2- wheel drive with a traction system capable of traversing in various terrain within a 47-degree inclination. A 2-DOF articulated end-effector is attached to robot, which can reach to a height of 45 cm from the ground, hold and lift the object up to 20 Kg. The robot movement is controlled by RFID andWi-Fi for low latency audio and video feed by a mobile unit. The rover has a night mode with less noise capabilities to aid rescue in dark among various sensors for topography mapping. The objective of the robot is to maneuver with in the narrow-spaces where there is zero-visibility and create space by removing the possible obstacles in the path using the arm. The rover was tested in places with strong EM interference and was found to be viable.

A PointNet Application for Semantic Classification of Ramps in Search and Rescue Arenas

Search and rescue environments could be dangerous for humans due to risks for potential structural breaking down and leakage of hazardous materials. Using robots in these environments can be appropriate solution to reduce these risks. The National Institute of Standards and Technology (NIST) proposed reference test areas for measuring autonomous navigation capabilities of mobile robots. In this paper, we present a PointNet application for semantic classification of ramps through point cloud data in reference test arenas. Since the walls and terrain carry important semantic information for robot navigation, they are also considered. The previous studies that address the semantic classification problem mostly used image and/or 2D laser range data. However, the image data may not be suitable for dusty and poorly lightened search and rescue environments and 2D laser range data may not represent 3D geometry of the objects. Since point cloud data have the ability to describe 3D geometry and it is not affected by the negative aspects of these environments, it could be appropriate to classify ramps, walls, and terrain. Eskisehir Osmangazi University (ESOGU) laboratory building is modelled in GAZEBO simulation environment and a Pioneer P3-AT mobile robot with Asus Xtion Pro sensor is launched. Then, the ESOGU RAMPS dataset is generated through navigating the robot via Robot Operation System (ROS). The dataset involves two types of ramp (inclined and flat), terrain and wall classes. The PointNet is applied to train and test the dataset. The metric and visual PointNet results are presented to analyze the classification performance.

Unknown geometrical constraints estimation and trajectory planning for robotic door-opening task with visual teleoperation assists

Purpose The purpose of this paper is to enable autonomous door-opening with unknown geometrical constraints. Door-opening is a common action needed for mobile manipulators to perform rescue operation. However, it remains difficult for them to handle it in real rescue environments. The major difficulties of rescue manipulation involve contradiction between unknown geometrical constraints and limited sensors because of extreme physical constraints. Design/methodology/approach A method for estimating the unknown door geometrical parameters using coordinate transformation of the end-effector with visual teleoperation assists is proposed. A trajectory planning algorithm is developed using geometrical parameters from the proposed method. Findings The relevant experiments are also conducted using a manipulator suited to extreme physical constraints to open a real door with a locked latch and unknown geometrical parameters, which demonstrates the validity and efficiency of the proposed approach. Originality/value This is a novel method for estimating the unknown door geometrical parameters with coordinate transformation of the end-effector through visual teleoperation assists.

Three-Dimensional Reconstruction Based on Visual SLAM of Mobile Robot in Search and Rescue Disaster Scenarios

SummaryConventional simultaneous localization and mapping (SLAM) has concentrated on two-dimensional (2D) map building. To adapt it to urgent search and rescue (SAR) environments, it is necessary to combine the fast and simple global 2D SLAM and three-dimensional (3D) objects of interest (OOIs) local sub-maps. The main novelty of the present work is a method for 3D OOI reconstruction based on a 2D map, thereby retaining the fast performances of the latter. A theory is established that is adapted to a SAR environment, including the object identification, exploration area coverage (AC), and loop closure detection of revisited spots. Proposed for the first is image optical flow calculation with a 2D/3D fusion method and RGB-D (red, green, blue + depth) transformation based on Joblove–Greenberg mathematics and OpenCV processing. The mathematical theories of optical flow calculation and wavelet transformation are used for the first time to solve the robotic SAR SLAM problem. The present contributions indicate two aspects: (i) mobile robots depend on planar distance estimation to build 2D maps quickly and to provide SAR exploration AC; (ii) 3D OOIs are reconstructed using the proposed innovative methods of RGB-D iterative closest points (RGB-ICPs) and 2D/3D principle of wavelet transformation. Different mobile robots are used to conduct indoor and outdoor SAR SLAM. Both the SLAM and the SAR OOIs detection are implemented by simulations and ground-truth experiments, which provide strong evidence for the proposed 2D/3D reconstruction SAR SLAM approaches adapted to post-disaster environments.

Robot SLAM with Ad hoc wireless network adapted to search and rescue environments

An innovative multi-robot simultaneous localization and mapping (SLAM) is proposed based on a mobile Ad hoc local wireless sensor network (Ad-WSN). Multiple followed-robots equipped with the wireless link RS232/485 module act as mobile nodes, with various on-board sensors, Tp-link wireless local area network cards, and Tp-link wireless routers. The master robot with embedded industrial PC and a complete robot control system autonomously performs the SLAM task by exchanging information with multiple followed-robots by using this self-organizing mobile wireless network. The PC on the remote console can monitor multi-robot SLAM on-site and provide direct motion control of the robots. This mobile Ad-WSN complements an environment devoid of usual GPS signals for the robots performing SLAM task in search and rescue environments. In post-disaster areas, the network is usually absent or variable and the site scene is cluttered with obstacles. To adapt to such harsh situations, the proposed self-organizing mobile Ad-WSN enables robots to complete the SLAM process while improving the performances of object of interest identification and exploration area coverage. The information of localization and mapping can communicate freely among multiple robots and remote PC control center via this mobile Ad-WSN. Therefore, the autonomous master robot runs SLAM algorithms while exchanging information with multiple followed-robots and with the remote PC control center via this local WSN environment. Simulations and experiments validate the improved performances of the exploration area coverage, object marked, and loop closure, which are adapted to search and rescue post-disaster cluttered environments.

Investigating Human-Robot Teams for Learning-Based Semi-autonomous Control in Urban Search and Rescue Environments

Teams of semi-autonomous robots can provide valuable assistance in Urban Search and Rescue (USAR) by efficiently exploring cluttered environments and searching for potential victims. Their advantage over solely teleoperated robots is that they can address the task handling and situation awareness limitations of human operators by providing some level of autonomy to the multi-robot team. Our research focuses on developing learning-based semi-autonomous controllers for rescue robot teams. In this paper, we specifically investigate the influence of the operator-to-robot ratio on the performance of our proposed MAXQ hierarchical reinforcement learning based semi-autonomous controller for USAR missions. In particular, we propose a unique learning-based system architecture that allows operator control of larger numbers of rescue robots in a team as well as effective sharing of information between these robots. A rigorous comparative study of our learning-based semi-autonomous controller versus a fully teleoperation-based approach was conducted in a 3D simulation environment. The results, as expected, show that, for both semi-autonomous and teleoperation modes, the total scene exploration time increases as the number of robots utilized increases. However, when using the proposed learning-based semi-autonomous controller, the rate of exploration-time increase and operator-interaction effort are significantly lower, while task performance is significantly higher. Furthermore, an additional case study showed that our learning-based approach can provide more scene coverage during robot exploration when compared to a non-learning based method.

Information-Fusion Methods Based Simultaneous Localization and Mapping for Robot Adapting to Search and Rescue Postdisaster Environments

The first application of utilizing unique information-fusion SLAM (IF-SLAM) methods is developed for mobile robots performing simultaneous localization and mapping (SLAM) adapting to search and rescue (SAR) environments in this paper. Several fusion approaches, parallel measurements filtering, exploration trajectories fusing, and combination sensors’ measurements and mobile robots’ trajectories, are proposed. The novel integration particle filter (IPF) and optimal improved EKF (IEKF) algorithms are derived for information-fusion systems to perform SLAM task in SAR scenarios. The information-fusion architecture consists of multirobots and multisensors (MAM); multiple robots mount on-board laser range finder (LRF) sensors, localization sonars, gyro odometry, Kinect-sensor, RGB-D camera, and other proprioceptive sensors. This information-fusion SLAM (IF-SLAM) is compared with conventional methods, which indicates that fusion trajectory is more consistent with estimated trajectories and real observation trajectories. The simulations and experiments of SLAM process are conducted in both cluttered indoor environment and outdoor collapsed unstructured scenario, and experimental results validate the effectiveness of the proposed information-fusion methods in improving SLAM performances adapting to SAR scenarios.

Goal-oriented training affects decision-making processes in virtual and simulated fire and rescue environments.

Decisions made by operational commanders at emergency incidents have been characterized as involving a period of information gathering followed by courses of action that are often generated without explicit plan formulation. We examined the efficacy of goal-oriented training in engendering explicit planning that would enable better communication at emergency incidents. While standard training mirrored current operational guidance, goal-oriented training incorporated "decision controls" that highlighted the importance of evaluating goals, anticipated consequences, and risk/benefit analyses once a potential course of action has been identified. In Experiment 1, 3 scenarios (a house fire, road traffic collision, and skip fire) were presented in a virtual environment, and in Experiment 2 they were recreated on the fireground. In Experiment 3, the house fire was recreated as a "live burn," and incident commanders and their crews responded to this scenario as an emergency incident. In all experiments, groups given standard training showed the reported tendency to move directly from information gathering to action, whereas those given goal-oriented training were more likely to develop explicit plans and show anticipatory situational awareness. These results indicate that training can be readily modified to promote explicit plan formulation that could facilitate plan sharing between incident commanders and their teams.