Occlusion Handling Research Articles

PCNet: a human pose compensation network based on incremental learning for sports actions estimation

Human pose estimation has a wide range of applications. Existing methods perform well in conventional domains, but there are certain defects when they are applied to sports activities. The first is lack of estimation of the extremity posture, making it impossible to comprehensively evaluate the movement posture; the second is insufficient occlusion handling. Therefore, we propose a human pose compensation network based on incremental learning, which obtains shared weights to extract detailed features under the premise of limited extremity training data. We propose a higher-order feature compensator (HOF-compensator) to embed the attributes of the extremity into the torso and limbs topology structure, building a complete higher-order feature. In addition, to improve the occlusion handling performance, we propose an occlusion feature enhancement attention mechanism (OFE-attention) that can identify occluded keypoints and enhance attention to occlusion areas. We design comparative experiments on three public datasets and a self-built sports dataset, achieving the highest mean accuracy among all comparative methods. In addition, we design a series of ablation analysis and visualization displays to verify that our method performs best in sports pose estimation.

A dual-channel network based on occlusion feature compensation for human pose estimation

Human pose estimation is an important technique in computer vision. Existing methods perform well in ideal environments, but there is room for improvement in occluded environments. The specific reasons are that the ambiguity of the features in the occlusion area makes the network pay insufficient attention to it, and the inadequate expressive ability of the features in the occlusion part cannot describe the true keypoint features. To address the occlusion issue, we propose a dual-channel network based on occlusion feature compensation. The dual channels are occlusion area enhancement channel based on convolution and occlusion feature compensation channel based on graph convolution, respectively. In the convolution channel, we propose an occlusion handling enhanced attention mechanism (OHE-attention) to improve the attention to the occlusion area. In the graph convolution channel, we propose a node feature compensation module that eliminates the obstacle features and integrates the shared and private attributes of the keypoints to improve the expressive ability of the node features. We conduct experiments on the COCO2017 dataset, COCO-Wholebody dataset, and CrowdPose dataset, achieving accuracy of 78.7%, 66.4%, and 77.9%, respectively. In addition, a series of ablation experiments and visualization demonstrations verify the performance of the dual-channel network in occluded environments.

Occlusion Handling in Depth Estimation of a Scene from a Given Light Field Using a Geodesic Distance and the Principle of Symmetry of the View

Occlusion Handling in Depth Estimation of a Scene from a Given Light Field Using a Geodesic Distance and the Principle of Symmetry of the View

AmazingFT: A Transformer and GAN-Based Framework for Realistic Face Swapping

Current face-swapping methods often suffer from issues of detail blurriness and artifacts in generating high-quality images due to the inherent complexity in detail processing and feature mapping. To overcome these challenges, this paper introduces the Amazing Face Transformer (AmazingFT), an advanced face-swapping model built upon Generative Adversarial Networks (GANs) and Transformers. The model is composed of three key modules: the Face Parsing Module, which segments facial regions and generates semantic masks; the Amazing Face Feature Transformation Module (ATM), which leverages Transformers to extract and transform features from both source and target faces; and the Amazing Face Generation Module (AGM), which utilizes GANs to produce high-quality swapped face images. Experimental results demonstrate that AmazingFT outperforms existing state-of-the-art (SOTA) methods, significantly enhancing detail fidelity and occlusion handling, ultimately achieving movie-grade face-swapping results.

Releaf: An Efficient Method for Real-Time Occlusion Handling by Game Theory.

Receiving uninterrupted videos from a scene with multiple cameras is a challenging task. One of the issues that significantly affects this task is called occlusion. In this paper, we propose an algorithm for occlusion handling in multi-camera systems. The proposed algorithm, which is called Real-time leader finder (Releaf), leverages mechanism design to assign leader and follower roles to each of the cameras in a multi-camera setup. We assign leader and follower roles to the cameras and lead the motion by the camera with the least occluded view using the Stackelberg equilibrium. The proposed approach is evaluated on our previously open-sourced tendon-driven 3D-printed robotic eye that tracks the face of a human subject. Experimental results demonstrate the superiority of the proposed algorithm over the Q-leaning and Deep Q Networks (DQN) baselines, achieving an improvement of 20% and 18% for horizontal errors and an enhancement of 81% for vertical errors, as measured by the root mean squared error metric. Furthermore, Releaf has the superiority of real-time performance, which removes the need for training and makes it a promising approach for occlusion handling in multi-camera systems.

Enhanced human motion detection with hybrid RDA-WOA-based RNN and multiple hypothesis tracking for occlusion handling

Human motion detection in complex scenarios poses challenges due to occlusions. This paper presents an integrated approach for accurate human motion detections by combining Adapted Canny Edge detection as a preprocessing step, backbone-modified Mask R-CNN for precise segmentation, Hybrid RDA-WOA-based RNN as a classification, and a Multiple-hypothesis model for effective occlusion handling. Adapted Canny Edge detection is utilized as an initial preprocessing step to highlight significant edges in the input image. The resulting edge map enhances object boundaries and highlights structural features, simplifying subsequent processing steps. The improved image is then passed through backbone-modified Mask R-CNN for the pixel-level segmentation of humans. Backbone-modified Mask R-CNN along with IoU, Euclidean Distance, and Z-Score recognizes moving objects in complex scenes exactly. After recognizing moving objects, the optimized Hybrid RDA-WOA-based RNN classifies humans. To handle the self-occlusion, Multiple Hypothesis Tracking (MHT) is used. Real-world situations frequently include occlusions where humans can be partially or completely hidden by objects. The proposed approach integrates a Multiple-hypothesis model into the detection pipeline to address this challenge. Moreover, the proposed human motion detection approach includes an optimized Hybrid RDA-WOA-based RNN trained with 2D representations of 3D skeletal motion. The proposed work was evaluated using the IXMAS, KTH, Weizmann, NTU RGB + D, and UCF101 Datasets. It achieved an accuracy of 98% on the IXMAS, KTH, Weizmann, and UCF101 Datasets and 97.1% on the NTU RGB + D Dataset. The simulation results unveil the superiority of the proposed methodology over the existing approaches.

Image-based virtual try-on: Fidelity and simplification

We introduce a novel image-based virtual try-on model designed to replace a candidate’s garment with a desired target item. The proposed model comprises three modules: segmentation, garment warping, and candidate-clothing fusion. Previous methods have shown limitations in cases involving significant differences between the original and target clothing, as well as substantial overlapping of body parts. Our model addresses these limitations by employing two key strategies. Firstly, it utilises a candidate representation based on an RGB skeleton image to enhance spatial relationships among body parts, resulting in robust segmentation and improved occlusion handling. Secondly, truncated U-Net is employed in both the segmentation and warping modules, enhancing segmentation performance and accelerating the try-on process. The warping module leverages an efficient affine transform for ease of training. Comparative evaluations against state-of-the-art models demonstrate the competitive performance of our proposed model across various scenarios, particularly excelling in handling occlusion cases and significant differences in clothing cases. This research presents a promising solution for image-based virtual try-on, advancing the field by overcoming key limitations and achieving superior performance.

Optimized Deep LSTM-Based Multi-Object Tracking with Occlusion Handling Mechanism

The multi-object tracking is a basic computer vision process having a huge class of real-life tools that range from monitoring of medical video to surveillance. The goal of tracking numerous items is to place numerous objects in a scene, handle their identities throughout time, and construct trajectories for analysis. However, this is a complex task, because of certain issues like occlusions, complicated object dynamics, and variations in the appearance of objects. In this research, a new technique named TPRO-based Deep LSTM is developed for tracking multi-object with occlusion handling. Here, the videos are considered as input wherein the extraction of frames is done from each video. Each frame undergoes pre-processing with filtering to eliminate noise from frames. By using a sparse Fuzzy c-Means (FCM) and Local Optimal-Oriented Pattern (LOOP) features, the localization of objects is done. Moreover, the visual and spatial trackings are considered for hybrid tracking. The second derivative model and neighborhood search model are used to perform visual tracking. Then the occlusion handling is performed. Concurrently, with the use of Deep Long Short-Term Memory (Deep LSTM) the spatial tracking is performed and the Taylor Poor Rich Optimization (TPRO) algorithm assigns the weight and bias of the Deep LSTM. The TPRO is obtained by the unification of the Taylor series along with the Poor and Rich Optimization algorithm. By combining visual and spatial tracking, the final tracked output is generated. The devised method achieves a performance with the highest value of 88.9% for Multiple Object Tracking Precision (MOTP), smallest tracking distance (TD) of 4.185, average MOTP of 0.889, average TD of 4.201, and highest tracking number (TN) of 14.

Visual multi-object tracking with re-identification and occlusion handling using labeled random finite sets

This paper proposes an online visual multi-object tracking (MOT) algorithm that resolves object appearance–reappearance and occlusion. Our solution is based on the labeled random finite set (LRFS) filtering approach, which in principle, addresses disappearance, appearance, reappearance, and occlusion via a single Bayesian recursion. However, in practice, existing numerical approximations cause reappearing objects to be initialized as new tracks, especially after long periods of being undetected. In occlusion handling, the filter’s efficacy is dictated by trade-offs between the sophistication of the occlusion model and computational demand. Our contribution is a novel modeling method that exploits object features to address reappearing objects whilst maintaining a linear complexity in the number of detections. Moreover, to improve the filter’s occlusion handling, we propose a fuzzy detection model that takes into consideration the overlapping areas between tracks and their sizes. We also develop a fast version of the filter to further reduce the computational time.

A lightweight tea buds detection model with occlusion handling

A lightweight tea buds detection model with occlusion handling

Track initialization and re-identification for 3D multi-view multi-object tracking

We propose a 3D multi-object tracking (MOT) solution using only 2D detections from monocular cameras, which automatically initiates/terminates tracks as well as resolves track appearance–reappearance and occlusions. Moreover, this approach does not require detector retraining when cameras are reconfigured but only the camera matrices of reconfigured cameras need to be updated. Our approach is based on a Bayesian multi-object formulation that integrates track initiation/termination, re-identification, occlusion handling, and data association into a single Bayes filtering recursion. However, the exact filter that utilizes all these functionalities is numerically intractable due to the exponentially growing number of terms in the (multi-object) filtering density, while existing approximations trade-off some of these functionalities for speed. To this end, we develop a more efficient approximation suitable for online MOT by incorporating object features and kinematics into the measurement model, which improves data association and subsequently reduces the number of terms. Specifically, we exploit the 2D detections and extracted features from multiple cameras to provide a better approximation of the multi-object filtering density to realize the track initiation/termination and re-identification functionalities. Further, incorporating a tractable geometric occlusion model based on 2D projections of 3D objects on the camera planes realizes the occlusion handling functionality of the filter. Evaluation of the proposed solution on challenging datasets demonstrates significant improvements and robustness when camera configurations change on-the-fly, compared to existing multi-view MOT solutions.

IMMERSIVE AUGMENTED REALITY APPLICATIONS IN JAW AND KNEE REPLACEMENT SURGERY: ACCURACY AND PROCESSING TIME RESULTS

In medicine and healthcare, augmented reality (AR) has been used by physicians during surgical procedures. It has provedhelpful in preoperative planning and procedure navigation by allowing them to display in-depth information and visualizedetails in real time during surgery, prioritizing patient safety and healthcare. Due to the critical nature of surgical procedures,extreme accuracy is required when using ar technology to maintain patients' health. A few years ago, ar faced severalchallenges and limitations in surgeries, such as noise in real-time images, cutting errors, navigation errors, wrong implantplacement, overlay errors, navigating narrow areas, geometric accuracy limitations, image alignment, image registration,and occlusion handling. This paper reviews several recently published articles exploring ar technology usage in jaw andknee replacement surgery, focusing on identifying the newest technologies, methods, and solutions for the abovementionedlimitations. Based on data collected from the published papers, the results were compared for each problem solved in eacharticle regarding accuracy and processing time.

Real-Time Vehicle Detection for Traffic Monitoring: A Deep Learning Approach

Vehicle detection is an essential technology for intelligent transportation systems and autonomous vehicles. Reliable real-time detection allows for traffic monitoring, safety enhancements and navigation aids. However, vehicle detection is a challenging computer vision task, especially in complex urban settings. Traditional methods using hand-crafted features like HAAR cascades have limitations. Recent deep learning advances have enabled convolutional neural networks (CNNs) like Faster R-CNN, SSD and YOLO to be applied to vehicle detection with significantly improved accuracy. But each technique has tradeoffs between precision and processing speed. Two-stage detectors like Faster R-CNN are highly accurate but slow at 7 FPS. Single-shot detectors like SSD are faster at 22 FPS but less precise. YOLO is extremely fast at 45 FPS but has lower accuracy. This paper reviews prominent deep learning vehicle detectors. It proposes a new integrated method combining YOLOv3 detection, optical flow tracking and trajectory analysis to enhance both accuracy and speed. Results on highway and urban datasets show improved precision, recall and F1 scores compared to YOLOv3 alone. Optical flow helps filter noise and recover missed detections. Trajectory analysis enables consistent object IDs across frames. Compared to other CNN models, the proposed technique achieves a better balance of real-time performance and accuracy. Occlusion handling and small object detection remain open challenges. In summary, deep learning has enabled major progress but enhancements in model architecture, training data and occlusion handling are needed to realize the full potential for traffic management applications. The integrated method proposed offers improved performance over baseline detectors. We have achieved 99 % accuracy in our project

MSPAN: Multi‐scale pyramid attention network for efficient skin cancer lesion segmentation

AbstractSkin cancer is common and deadly, needs to be detected and treated properly. Deep learning algorithms like UNet have shown potential results in medical imaging. Such approaches still struggle to capture fine‐grained details and scale differences in skin lesions‐based occlusions' appearance, size etc. This research proposes a redesign UNet, the Multi‐Scale Pyramid Attention Network (MSPAN), to improve skin cancer lesion segmentation. The input data is processed at numerous scales with varied receptive fields. This enhances the network's ability to identify lesion locations by capturing local and global context. Attention approaches also help the network to suppress noise by focusing on informative features. We have evaluated MSPAN model on the publicly available ISIC2018 benchmark dataset for skin lesion segmentation. The method surpasses traditional UNet and other current methods in accuracy and effectiveness. The model also has a post‐processing to estimate lesion area for fast inference, making it suitable for extensive screening. Redesigned UNet with the Multi‐Scale Pyramid Attention Network improves skin cancer lesion segmentation. The model's ability to collect fine‐grained information and handle occlusions allows for more accurate skin cancer diagnosis and treatment. The MSPAN design can improve computer‐aided diagnosis systems and help dermatologists make precise clinical decisions.

Robust ensemble person reidentification via orthogonal fusion with occlusion handling

Occlusion remains one of the major challenges in person reidentification (ReID) due to the diversity of poses and the variation of appearances. Developing novel architectures to improve the robustness of occlusion-aware person Re-ID requires new insights, especially on low-resolution edge cameras. We propose a deep ensemble model that harnesses both CNN and Transformer architectures to generate robust feature representations. To achieve robust Re-ID without manually labeling occluded regions, we propose to take an ensemble learning-based approach derived from the analogy between arbitrarily shaped occluded regions and robust feature representation. Using the orthogonality principle, our developed deep CNN model uses masked autoencoder (MAE) and global–local feature fusion for robust person identification. Furthermore, we present a part occlusion-aware transformer capable of learning feature space that is robust to occluded regions. Experimental results are reported on several Re-ID datasets to show the effectiveness of our developed ensemble model named orthogonal fusion with occlusion handling (OFOH). Compared to competing methods, the proposed OFOH approach has achieved competent rank-1 and mAP performance.

Research on adaptive particle swarm optimization particle filter target tracking algorithm in wireless sensor networks

AbstractWith regard to target tracking in wireless sensor networks, we are faced with problems like deficient occlusion handling and tracking failures during rapid movements due to complex and diverse circumstances. In order to effectively improve the accuracy of particle filter tracking caused by particle degradation, we propose an adaptive particle swarm optimization (APSO) particle filter algorithm. This algorithm uses particle filters to predict the target location in a particular area and introduces the particle swarm optimization (PSO) algorithm, of which both the evolutionary speed and the convergence accuracy are further improved by investigating the particle distribution through an entropy analysis, employing three different inertial weighting strategies and dynamic double mutation strategy, and exploiting the capabilities of the adaptive balancing algorithm in global and local searching. The simulation results show that the improved algorithm has a reduced root mean square error, shorter time consumption, faster speed, reduced target tracking error, and higher average success rate, so this algorithm exhibits sound real‐time performance and accuracy in terms of occlusion handling and tracking loss.

BGaitR-Net: An effective neural model for occlusion reconstruction in gait sequences by exploiting the key pose information

Gait recognition in the presence of occlusion is a challenging problem and the solutions proposed to date either lack robustness or depend on several unrealistic constraints. In this work, we propose a Deep Learning framework to detect and reconstruct the occluded frames in a gait sequence. Initially, occlusion detection is done using a VGG-16 network and for each frame the corresponding pose information is represented as a one-hot encoded vector. This vector is next fused with the corresponding spatial information using a Conditional Variational Autoencoder (CVAE) to obtain an effective embedding. Following this, a Bi-directional Long Short Term Memory (Bi-LSTM) is used to predict the occluded frames using the encoded vector sequence. A decoder next transforms these predicted frames back to the image space. Our proposed reconstruction model termed the Bidirectional Gait Reconstruction Network (BGaitR-Net) is formed by stacking the CVAE, Bi-LSTM, and the decoder. The CASIA-B and OU-ISIR LP datasets are used to prepare extensive gallery sets to train each of the above sub-networks and testing is done using synthetically occluded sequences from the CASIA-B data and real-occluded sequences from the TUM-IITKGP data. A thorough evaluation of our work through Dice Score and GEINet-based recognition accuracy for varying degrees of occlusion highlight the effectiveness of our model in generating frames consistent with the temporal gait pattern. Comparative study with other existing gait recognition techniques (with or without occlusion handling mechanism) and with recent Deep Learning-based video frame prediction methods emphasizes the superiority of BGaitR-Net over the others.

Real-Time Multiple Human Height Measurements With Occlusion Handling Using LiDAR and Camera of a Mobile Device

Real-Time Multiple Human Height Measurements With Occlusion Handling Using LiDAR and Camera of a Mobile Device

Real-time tracking and berthing aid system with occlusion handling based on LiDAR

In smart ports, accurate perception of the state of a berthing ship relative to the berth is critical for the safety and efficiency of ships and ports. A real-time tracking and berthing system with occlusion handling based on Light Detection and Ranging (LiDAR) is proposed. First, the data is pre-processed, including fast filtering and clustering, and then a real-time tracking framework is proposed to solve the occlusion problem. This framework adapts to different berthing scenarios through accurate determination of dynamic and static targets and efficient retracking of berthing ships. To verify the effectiveness of the method, an on-site monitoring experiment of the “Yukun” training ship in Dalian Port is conducted, and a simulated berthing scene under occlusion is constructed to test the algorithm. The practical results prove that our method shows feasibility and effectiveness in the application of ship dynamic target tracking and safe ship berthing.

SkyroadAR: An Augmented Reality System for UAVs Low-Altitude Public Air Route Visualization

Augmented Reality (AR) technology visualizes virtual objects in the real environment, offering users an immersive experience that enhances their spatial perception of virtual objects. This makes AR an important tool for visualization in engineering, education, and gaming. The Unmanned Aerial Vehicles’ (UAVs’) low-altitude public air route (Skyroad) is a forward-looking virtual transportation infrastructure flying over complex terrain, presenting challenges for user perception due to its invisibility. In order to achieve a 3D and intuitive visualization of Skyroad, this paper proposes an AR visualization framework based on a physical sandbox. The framework consists of four processes: reconstructing and 3D-printing a sandbox model, producing virtual scenes for UAVs Skyroad, implementing a markerless registration and tracking method, and displaying Skyroad scenes on the sandbox with GPU-based occlusion handling. With the support of the framework, a mobile application called SkyroadAR was developed. System performance tests and user questionnaires were conducted on SkyroadAR; the results showed that our approachs to tracking and occlusion provided an efficient and stable AR effect for Skyroad. This intuitive visualization is recognized by both professional and non-professional users.