Occlusion Avoidance Research Articles

USV-Tracker: A novel USV tracking system for surface investigation with limited resources

This paper introduces USV-Tracker, a novel tracking system for Unmanned Surface Vehicles (USVs) tailored for practical applications such as surface investigation and target tracking. The system tackles three pivotal challenges: perception robustness, tracking concealment, and planning efficiency. The contributions of this work are manifold: (1) A multi-sensor fusion framework utilizing an Extended Kalman Filter (EKF) to enhance target detection and positioning accuracy, integrating data from cameras, LiDAR, GPS, and IMU sensors. (2) A two-stage path planning algorithm that generates occlusion avoidance trajectories and employs a virtual elastic force constraint to maintain appropriate relative positioning. In dense obstacle environments, the algorithm tends to get closer to the target and incorporates FOV orientation constraints to ensure stable perception. (3) A visibility-aware control strategy that ensures continuous target observability through EKF-based trajectory prediction. Simulations in Gazebo and corresponding physical experiments validate the system’s effectiveness and robustness, demonstrating its applicability in real-world scenarios. The computational workload is managed on a constrained on-board computer, underscoring the system’s practicality.

Efficient occlusion avoidance based on active deep sensing for harvesting robots

With the increasing shortage of agricultural labor, the development of harvesting robots is becoming more and more urgent. Most of them require vision to locate the target, however, occlusion is common in agricultural environment, which restricts the accuracy of visual target recognition, and even leads to failure in serious cases. The active perception method is an effective means, but how to efficiently find the best observation position remains difficult to avoid the waste of time caused by repeated invalid motion. Targeting these problems, an active deep sensing method is proposed for harvesting clustered and single fruits. First, the region of interest of the target is extracted by a segmentation network, and then the occlusion status of it is obtained by image processing methods. Taking the current observation position as the starting point, the camera is moved within a matrix to form confidence and occlusion rate distribution maps. After establishing a series of occlusion rate and confidence matrix datasets, a designed deep network has been trained, which is used to predict the maximum confidence/minimum occlusion rate position after the current occlusion status is estimated. To verify the reliability of the method, laboratory and field experiments were carried out for apples and clustered tomatoes. After 1000 times of verification, results show that the successful pick/recognition rate is increased by 38.7 %, and the average successful recognition time is 5.2 s, which is 63.1 % and 46.4 % faster than that of a fixed movement method and a simple heuristic method.

Occlusion Avoidance in Image-Based Control of Multiple Spacecraft

This study develops a novel image-based control scheme to address the on-orbit service mission that multiple spacecraft track a tumbling uncooperative target under a visibility constraint. The chaser spacecraft are commanded to hover above and track stably the feature points attached to the target as well as to avoid the mutual visual occlusion caused by interspacecraft motion. To this end, a new expression of image-based visual servoing dynamics for spacecraft is derived, where the physical meaning of each term is analyzed in detail so that the exact upper bounds of noncooperative terms can be estimated. After that, a novel occlusion description model is established by the projective theory of quadrics. The mathematical elements for describing the imaging elliptic curve and occlusion discrimination function are proposed, whose association with interspacecraft motion is given to guide occlusion avoidance controller design. Combining the artificial potential function with the backstepping control method, a robust control algorithm is designed. The proposed control scheme can achieve the six-degree-of-freedom control objective based on image space, and motion trajectory will not influence others’ visibility. Finally, the validity of the control scheme is verified by simulation for a given multispacecraft observation scenario.

Occlusion Avoidance in a Simulated Environment Using Reinforcement Learning

Neural network-based solutions have revolutionized the field of computer vision by achieving outstanding performance in a number of applications. Yet, while these deep learning models outclass previous methods, they still have significant shortcomings relating to generalization and robustness to input disturbances, such as occlusion. Most existing methods that tackle this latter problem use passive neural network architectures that are unable to act on and, thus, influence the observed scene. In this paper, we argue that an active observer agent may be able to achieve superior performance by changing the parameters of the scene, thus, avoiding occlusion by moving to a different position in the scene. To demonstrate this, a reinforcement learning environment is introduced that implements OpenAI Gym’s interface, and allows the creation of synthetic scenes with realistic occlusion. The environment is implemented using differentiable rendering, allowing us to perform direct gradient-based optimization of the camera position. Moreover, two additional methods are also presented, one utilizing self-supervised learning to predict occlusion segments, and optimal camera positions, while the other learns to avoid occlusion using Reinforcement Learning. We present comparative experiments of the proposed methods to demonstrate their efficiency. It was shown, via Bayesian t-tests, that the neural network-based methods credibly outperformed the gradient-based avoidance strategy by avoiding occlusion with an average of 5.0 fewer steps in multi-object scenes.

Occlusion-free Image-Based Visual Servoing using Probabilistic Control Barrier Certificates

Image based visual servoing (IBVS) is a widely-used approach in robotics that employs visual information to guide robots towards desired positions. However, occlusions in this approach can lead to visual servoing failure and degrade the control performance due to the obstructed vision feature points that are essential for providing visual feedback. In this paper, we propose a control barrier function (CBF) based controller that enables occlusion-free IBVS tasks by automatically adjusting the robot's configuration to keep the feature points in the field of view and away from obstacles. In particular, to account for measurement noise of the feature points, we develop the probabilistic control barrier certificates (PrCBC) using control barrier functions that encode the chance-constrained occlusion avoidance constraints under uncertainty into deterministic admissible control space for the robot, from which the resulting configuration of robot ensures that the feature points stay occlusion free from obstacles with a satisfying predefined probability. By integrating such constraints with a model predictive control (MPC) framework, the sequence of optimized control inputs can be derived to achieve the primary IBVS task while enforcing the occlusion avoidance during robot movements. Simulation results are provided to validate the performance of our proposed method.

Multiple Object Tracking through Background Learning

This paper discusses about the new approach of multiple object tracking relative to background information. The concept of multiple object tracking through background learning is based upon the theory of relativity, that involves a frame of reference in spatial domain to localize and/or track any object. The field of multiple object tracking has seen a lot of research, but researchers have considered the background as redundant. However, in object tracking, the background plays a vital role and leads to definite improvement in the overall process of tracking. In the present work an algorithm is proposed for the multiple object tracking through background learning. The learning framework is based on graph embedding approach for localizing multiple objects. The graph utilizes the inherent capabilities of depth modelling that assist in prior to track occlusion avoidance among multiple objects. The proposed algorithm has been compared with the recent work available in literature on numerous performance evaluation measures. It is observed that our proposed algorithm gives better performance.

Image-Based Multi-UAV Tracking System in a Cluttered Environment

A tracking controller for unmanned aerial vehicles (UAVs) is developed to track moving targets undergoing unknown translational and rotational motions. The main challenges are to control both the relative positions and angles between the target and the UAVs to within desired values, and to guarantee that the generated control inputs to the UAVs are feasible (i.e., within their motion capabilities). Moreover, the UAVs are controlled to ensure that the target always remains within the fields of view of their onboard cameras. To the best of our knowledge, this is the first work to apply multiple UAVs to cooperatively track a dynamic target while ensuring that the UAVs remain connected and that both occlusion and collisions are avoided. To achieve these control objectives, a designed controller solved based on the aforementioned tracking controller using quadratic programming can generate minimally invasive control actions to achieve occlusion avoidance and collision avoidance. Furthermore, control barrier functions (CBFs) with a distributed design are developed in order to reduce the amount of inter-UAV communication. Simulations were performed to assess the efficacy and performance of the developed CBF-based controller for the multi-UAV system in tracking a target.

Directional Sensor Planning for Occlusion Avoidance

Directional sensors, such as video cameras, have become ubiquitous to many autonomous robots applications, such as monitoring and surveillance. The performance of these sensors and processing algorithms, however, may be hindered by the presence of objects that block visibility. This article presents a novel approach for planning the path of a mobile directional sensor deployed to observe multiple targets distributed in an environment populated with multiple obstacles and occlusions. Unlike existing art gallery or watchman’s route methods, the visibility theory and motion planners developed in this article account for both line-of-sight visibility and bounded field-of-view constraints, and can provide obstacle avoidance based on robot geometry and kinodynamic constraints. The computational complexity analysis and experiments on a camera-equipped drone demonstrate that, despite the challenging geometric characteristics of directional C-targets, the approach scales to real-world problems. Furthermore, when compared to algorithms inspired by traveling salesman and target coverage approaches, the directional visibility planners presented in this article are significantly more effective both at guaranteeing complete target visibility and at minimizing distance traveled.

Minimize Tracking Occlusion in Collaborative Pick-and-Place Tasks: An Analytical Approach for Non-Wrist-Partitioned Manipulators.

Several industrial pick-and-place applications, such as collaborative assembly lines, rely on visual tracking of the parts. Recurrent occlusions are caused by the manipulator motion decrease line productivity and can provoke failures. This work provides a complete solution for maintaining the occlusion-free line of sight between a variable-pose camera and the object to be picked by a 6R manipulator that is not wrist-partitioned. We consider potential occlusions by the manipulator as well as the operator working at the assembly station. An actuated camera detects the object goal (part to pick) and keeps track of the operator. The approach consists of using the complete set of solutions obtained from the derivation of the univariate polynomial equation solution to the inverse kinematics (IK). Compared to numerical iterative solving methods, our strategy grants us a set of joint positions (posture) for each root of the equation from which we extract the best (minimizing the risks of occlusion). Our analytical-based method, integrating collision and occlusion avoidance optimizations, can contribute to greatly enhancing the efficiency and safety of collaborative assembly workstations. We validate our approach with simulations as well as with physical deployments on commercial hardware.

Design of an Eye-in-Hand Smart Gripper for Visual and Mechanical Adaptation in Grasping

With the advancement of robotic technologies, more and more tasks in industrial and commercial applications rely on the use of robots to assist or even replace humans. To fulfill the needs of grasping and handling different objects, the development of a universal grasping device acting as an end-effector to a robotic manipulator has been one of the main robotic research and development focuses. Therefore, this study was aimed at the development of a general robotic gripper with three fingers for adaptive actuation and an eye-in-hand vision system. With the adaptive actuation feature, each finger of the robotic gripper contained multiple degrees of freedom that allowed the finger to change its shape to wrap around an object’s geometry adaptively for stable grasping. With the eye-in-hand configuration in the adaptive gripper, it offered advantages including occlusion avoidance, intuitive teleoperation, imaging from different angles, and simple calibration. This study proposed and integrated a plug-and-play gripper module, controller module, and visual calculation module all in the model smart gripper, of which the gripper was further validated by calibrated experiments. The proposed gripper featured mechanical adaptation and visual servoing adaptivity to achieve 100% gripping success rate when gripping a moving target of any shape that was carried by conveyor belt with moving speed less than 70 mm/s. By integrating mechanical and visual adaptivity, the proposed gripper enabled the inclusion of intelligence in robotic applications and can further be used in smart manufacturing and intelligent robotic applications.

Deep Reinforcement Learning-Based Robotic Grasping in Clutter and Occlusion

In robotic manipulation, object grasping is a basic yet challenging task. Dexterous grasping necessitates intelligent visual observation of the target objects by emphasizing the importance of spatial equivariance to learn the grasping policy. In this paper, two significant challenges associated with robotic grasping in both clutter and occlusion scenarios are addressed. The first challenge is the coordination of push and grasp actions, in which the robot may occasionally fail to disrupt the arrangement of the objects in a well-ordered object scenario. On the other hand, when employed in a randomly cluttered object scenario, the pushing behavior may be less efficient, as many objects are more likely to be pushed out of the workspace. The second challenge is the avoidance of occlusion that occurs when the camera itself is entirely or partially occluded during a grasping action. This paper proposes a multi-view change observation-based approach (MV-COBA) to overcome these two problems. The proposed approach is divided into two parts: 1) using multiple cameras to set up multiple views to address the occlusion issue; and 2) using visual change observation on the basis of the pixel depth difference to address the challenge of coordinating push and grasp actions. According to experimental simulation findings, the proposed approach achieved an average grasp success rate of 83.6%, 86.3%, and 97.8% in the cluttered, well-ordered object, and occlusion scenarios, respectively.

Development and Experimental Verification of a Person Tracking System of Mobile Robots Using Sensor Fusion of Inertial Measurement Unit and Laser Range Finder for Occlusion Avoidance

The authors have been developing a mobile robot to assist doctors in hospitals in managing medical tools and patient electronic medical records. The robot tracks behind a mobile medical worker while maintaining a constant distance from the worker. However, it was difficult to detect objects in the sensor’s invisible region, called occlusion. In this study, we propose a sensor fusion method to estimate the position of a robot tracking target indirectly by an inertial measurement unit (IMU) in addition to the direct measurement by an laser range finder (LRF) and develop a human tracking system to avoid occlusion by a mobile robot. Based on this, we perform detailed experimental verification of tracking a specified person to verify the validity of the proposed method.

Occlusion Avoidance Behavior During Gazing at a Rim Drawn by Blue-Yellow Opposite Colors

Occlusion Avoidance Behavior During Gazing at a Rim Drawn by Blue-Yellow Opposite Colors

Obstacle Magnification for 2-D Collision and Occlusion Avoidance of Autonomous Multirotor Aerial Vehicles

Collision or occlusion avoidance is one of the most important functions of autonomous unmanned vehicles when maneuvering or conducting missions, including monitoring or following targets in environments with various obstacles such as trees and buildings. While considering the minimum distance to obstacles or approximating the shape of obstacles is convenient, incorporating the shape of the original obstacles, sometimes, can provide more effective strategies for collision and occlusion avoidance of obstacles with complex shapes or arrangements. This article presents an obstacle magnification algorithm that magnifies but retains the original shape of the detected obstacles. Two simple parameters can adjust the position and magnification of the virtual obstacles. The simulation and outdoor experimental results for the obstacle magnification, implemented on an onboard embedded system, demonstrate the effectiveness of this concept, which should be useful in various applications for small autonomous multirotor aircrafts.

Visibility, Topology, and Inertia: New Methods in Flow Visualization.

In this article, we address three different topics in scientific visualization. The first part introduces optimization strategies that determine the visibility of line and surface geometry, such that a balance between occlusion avoidance and preservation of context is found. The second part proposes new methods for the visualization of time-dependent fluid flows, including the accurate depiction of Lagrangian scalar fields, as well as a new category of vortex identification methods. The third part introduces finite-sized particles as new application area for flow visualization, covering geometry-based methods, particle separation, topology, vortex corelines, and the determination of the origin of finite-sized particles.

Study on Incongruence of Binocular Images for Blue Based on Occlusion Avoidance Behavior When Gazing at the Rim of a Column

Study on Incongruence of Binocular Images for Blue Based on Occlusion Avoidance Behavior When Gazing at the Rim of a Column

Study on Incongruence of Binocular Images for Blue Based on Occlusion Avoidance Behavior When Gazing at the Rim of a Column

Study on Incongruence of Binocular Images for Blue Based on Occlusion Avoidance Behavior When Gazing at the Rim of a Column

Occlusion-Free Visual Servoing for the Shared Autonomy Teleoperation of Dual-Arm Robots

Visual servoing in telerobotics provides information to the operator about the remote location and assists in the task execution to reduce stress on the user. Occlusions in this scenario can, on the one hand, lead to the visual servoing failure, and, on the other hand, degrade the user experience and navigation performance due to the obstructed vision of elements of interest. In this letter, we consider a teleoperation system composed of two robot arms where one is remotely operated, while the other is autonomous and equipped with an eye-in-hand camera sensor. We propose a reactive unified convex optimization based controller that allows the execution of occlusion-free tasks by autonomously adjusting the camera robot, so as to keep the teleoperated one in the field of view. The occlusion avoidance is formulated inside the optimization as a constraint in the image space, it is formally derived from a collision avoidance analogy and made robust against noisy measurements and dynamic environment. A state machine is used to switch the control policy whenever an occlusion might occur. We validate our approach with experiments on a 14 d.o.f. dual-arm ABB YuMi robot equipped with an red, green, blue (RGB) camera and teleoperated by a 3 d.o.f. Novint Falcon device.

Coverage enhancement with occlusion avoidance in networked rotational video sensors for post-disaster management

Coverage enhancement with occlusion avoidance in networked rotational video sensors for post-disaster management

Coverage Enhancement with Occlusion Avoidance in Networked Rotational Video Sensors for Post-Disaster Management

Wireless video sensor networks (WVSNs) have recently emerged as a new class of sensor networks in which large amounts of visual data are sensed. This kind of networks, strengthened with rotation capability for each video sensor, is envisioned to be deployed to monitor a plethora of real-world phenomena. In this work, we propose a model of WVSN for post-disaster management in order to assist search and rescue operations by locating survivors and identifying risky areas. Rotatable video sensors are randomly deployed and used to switch to the best direction with the purpose of getting high coverage while avoiding obstacles. To address fault tolerance problem in WVSN, we build potential cover sets to unsure field of view's coverage of failed nodes. We have conducted a set of comprehensive experiments using OMNeT++ simulator and the obtained results reveal that our proposal gives better performance in terms of coverage enhancement and fault tolerance.