Heavy Occlusion Research Articles

IMapper

Next generation smart and augmented reality systems demand a computational understanding of monocular footage that captures humans in physical spaces to reveal plausible object arrangements and human-object interactions. Despite recent advances, both in scene layout and human motion analysis, the above setting remains challenging to analyze due to regular occlusions that occur between objects and human motions. We observe that the interaction between object arrangements and human actions is often strongly correlated, and hence can be used to help recover from these occlusions. We present iMapper, a data-driven method to identify such human-object interactions and utilize them to infer layouts of occluded objects. Starting from a monocular video with detected 2D human joint positions that are potentially noisy and occluded, we first introduce the notion of interaction-saliency as space-time snapshots where informative human-object interactions happen. Then, we propose a global optimization to retrieve and fit interactions from a database to the detected salient interactions in order to best explain the input video. We extensively evaluate the approach, both quantitatively against manually annotated ground truth and through a user study, and demonstrate that iMapper produces plausible scene layouts for scenes with medium to heavy occlusion. Code and data are available on the project page.

Long‐term adaptive tracking via complementary trackers

In recent years, kernelised correlation filter-based trackers have been employed to manage short-term tracking problems and help long-term trackers achieve excellent accuracy and robustness under challenging conditions, such as geometry/photometry changes, heavy occlusion, fast motion, motion blur, and out-of-camera view. Nonetheless, the inherent boundary effects and risky update strategy of correlation filters constrain the performance of short-term tracking, which limits the performance of long-term trackers. Moreover, the complicated redetection module leads to high-computational cost, which results in the long-term trackers to run at a low speed, thereby significantly restricting their applications. In the present work, the authors propose to employ complementary trackers in designing an efficient long-term tracker. Furthermore, a sigmoid penalty coefficient is proposed to update the tracking model with an adaptive learning rate that adjusts the learning rate while the target encounters appearance variation. Finally, they propose a novel redetection method that combines a redetection classifier with a short-term component to redetect the target while satisfying the explicit condition. The long-term tracker proposed in this study is proven to perform real-time speed of more than 65 frames per second and state-of-the-art accuracy by the experimental result on several challenging benchmarks.

Deformation Capture via Soft and Stretchable Sensor Arrays

We propose a hardware and software pipeline to fabricate flexible wearable sensors and use them to capture deformations without line-of-sight. Our first contribution is a low-cost fabrication pipeline to embed multiple aligned conductive layers with complex geometries into silicone compounds. Overlapping conductive areas from separate layers form local capacitors that measure dense area changes. Contrary to existing fabrication methods, the proposed technique only requires hardware that is readily available in modern fablabs. While area measurements alone are not enough to reconstruct the full 3D deformation of a surface, they become sufficient when paired with a data-driven prior. A novel semi-automatic tracking algorithm, based on an elastic surface geometry deformation, allows us to capture ground-truth data with an optical mocap system, even under heavy occlusions or partially unobservable markers. The resulting dataset is used to train a regressor based on deep neural networks, directly mapping the area readings to global positions of surface vertices. We demonstrate the flexibility and accuracy of the proposed hardware and software in a series of controlled experiments and design a prototype of wearable wrist, elbow, and biceps sensors, which do not require line-of-sight and can be worn below regular clothing.

Real-Time Event-Triggered Object Tracking in the Presence of Model Drift and Occlusion

In this paper, we propose a novel event-triggered tracking framework for fast and robust visual tracking in the presence of model drift and occlusion. The resulting tracker not only operates in real time, but also is resilient to tracking failures caused by factors such as fast motion and heavy occlusion. Specifically, the tracker consists of an event-triggered decision model as the core module that coordinates other functional modules, including a short-term tracker, occlusion and drift identification, target redetection, short-term tracker updating, and online discriminative learning for a detector. Each functional module is associated with a defined event that is triggered when the proposed conditions are met. The occlusion and drift identification module is intended to perform online evaluation of the short-term tracking. When a model drift event occurs, the target redetection module is activated by the event-triggered decision model to relocate the target and reinitialize the short-term tracker. The short-term tracker updating is carried out at each frame with a variable learning rate depending on the degree of occlusion. A sampling pool is constructed to store discriminative samples that are used to update the detector model. Extensive experiments on large benchmark datasets demonstrate that the proposed tracking algorithm can effectively detect model drift and restore tracking, and more importantly, it outperforms the state-of-the-art approaches in terms of accuracy, efficiency, and robustness.

Correlation filter-based visual tracking via holistic and reliable local parts

Visual tracking is a challenging task in computer vision due to various appearance changes of the target. Although correlation filter-based trackers have achieved competitive results, they may easily lead to tracking failure because of the high sensitivity of correlation filter to occlusion. Part-based correlation filter trackers can deal with partial occlusion to some extent, but they may easily drift to the background in the case of fast motion or heavy occlusion. To better solve the above-mentioned problems, a kernelized correlation filter-based tracker that processes both holistic and reliable local parts is proposed. For local parts, reliable parts are identified by peak-to-sidelobe ratio. When all parts are unreliable, we propose to apply a sliding window on each part to generate patches, among which a reliable patch is identified, and the part is replaced by the reliable patch. In holistic level, holistic tracking is performed with the rough position voted by reliable local parts, and then the holistic tracking result is used to provide feedback for parts to update its scale and filter. Moreover, we propose to reset unreliable parts when the holistic tracking result is reliable. The experimental results illustrate that the proposed tracker outperforms those of several state-of-the-art trackers.

Pose detection in complex classroom environment based on improved Faster R‐CNN

Pose detection of small targets in poor imaging conditions like heavy occlusion and low resolution is still an open and challenging task in computer vision. For instance, detection of students' poses in classrooms that are even indistinguishable to human eyes remains a rather difficult task. Motivated by the success of convolutional feature merging and locality preserving, the authors propose a pose detection framework combining merged region of interest (ROI) pooling and locality preserving learning. Unlike usual object detection algorithms which use general top-level convolutional features as inputs, their method uses a merged ROI pooling structure to merge semantic feature and high-resolution feature from the last two levels of convolutional feature maps, so that this merged feature is made more expressive than the single-level feature. In addition, the locality feature-preserving learning is used in the last fully-connected layer. Through locality preserving learning, features belonging to the same class would be forced to be closer in the feature space, which enables the model with stronger classification ability. Experimental results show that the proposed method outperforms the state-of-the-art methods.

Behavioral Pedestrian Tracking Using a Camera and LiDAR Sensors on a Moving Vehicle.

In this paper, we present a novel 2D–3D pedestrian tracker designed for applications in autonomous vehicles. The system operates on a tracking by detection principle and can track multiple pedestrians in complex urban traffic situations. By using a behavioral motion model and a non-parametric distribution as state model, we are able to accurately track unpredictable pedestrian motion in the presence of heavy occlusion. Tracking is performed independently, on the image and ground plane, in global, motion compensated coordinates. We employ Camera and LiDAR data fusion to solve the association problem where the optimal solution is found by matching 2D and 3D detections to tracks using a joint log-likelihood observation model. Each 2D–3D particle filter then updates their state from associated observations and a behavioral motion model. Each particle moves independently following the pedestrian motion parameters which we learned offline from an annotated training dataset. Temporal stability of the state variables is achieved by modeling each track as a Markov Decision Process with probabilistic state transition properties. A novel track management system then handles high level actions such as track creation, deletion and interaction. Using a probabilistic track score the track manager can cull false and ambiguous detections while updating tracks with detections from actual pedestrians. Our system is implemented on a GPU and exploits the massively parallelizable nature of particle filters. Due to the Markovian nature of our track representation, the system achieves real-time performance operating with a minimal memory footprint. Exhaustive and independent evaluation of our tracker was performed by the KITTI benchmark server, where it was tested against a wide variety of unknown pedestrian tracking situations. On this realistic benchmark, we outperform all published pedestrian trackers in a multitude of tracking metrics.

An Optimal Long-Term Aerial Infrared Object Tracking Algorithm With Re-Detection

In the field of automatic target recognition and tracking, long-term tracking for aerial infrared target has been recently seen with great interest. Although deep trackers and correlation filtering trackers offer competitive results on performance, the problems of deformation, abrupt motion, heavy occlusion, and out of view still remain unsolved. In addition, since this paper focus on infrared images, it is also important to consider that infrared images have a significant drawbacks, such as low resolution, low contrast, and lack of textures. In this paper, we adopt correlation filtering trackers and deep learning detection method to achieve accurate tracking results. Our tracking system composed of three parts: the DTB correlation filtering tracker (DTB-CF), a better regression model to discriminate the target from the background with adjustable Gaussian window functions; the UTA correlation filtering tracker (UTA-CF), an optimum regression model to update the target appearance with simultaneously optimal in position, scale, and integration of multi-feature fusion; and the YOLOv3 re-detector, which ensures re-location of the correct position of the target when the tracking fails. In addition, we introduce the ratio between average peak-to-correlation energy (APCE) of the current frame and average APCE of former frames as a criterion to update the UTA-CF tracker to maintain the target model stability. And we combine the nearest neighbor maximum value method with APCE as criterion together to initialize the YOLOv3 re-detector. We evaluate our algorithm on real aerial infrared target thermal image sequences in terms of precision plot, success plot, and speed. The experimental results show that our method has a significant improvement than the state-of-the-art methods for long-term tracking both in accuracy and robustness for aerial infrared object tracking.

Improving Faster R‐CNN Framework for Fast Vehicle Detection

Vision‐based vehicle detection plays an important role in intelligent transportation systems. With the fast development of deep convolutional neural networks (CNNs), vision‐based vehicle detection approaches have achieved significant improvements compared to traditional approaches. However, due to large vehicle scale variation, heavy occlusion, or truncation of the vehicle in an image, recent deep CNN‐based object detectors still showed a limited performance. This paper proposes an improved framework based on Faster R‐CNN for fast vehicle detection. Firstly, MobileNet architecture is adopted to build the base convolution layer in Faster R‐CNN. Then, NMS algorithm after the region proposal network in the original Faster R‐CNN is replaced by the soft‐NMS algorithm to solve the issue of duplicate proposals. Next, context‐aware RoI pooling layer is adopted to adjust the proposals to the specified size without sacrificing important contextual information. Finally, the structure of depthwise separable convolution in MobileNet architecture is adopted to build the classifier at the final stage of the Faster R‐CNN framework to classify proposals and adjust the bounding box for each of the detected vehicle. Experimental results on the KITTI vehicle dataset and LSVH dataset show that the proposed approach achieved better performance compared to original Faster R‐CNN in both detection accuracy and inference time. More specific, the performance of the proposed method is improved comparing with the original Faster R‐CNN framework by 4% on the KITTI test set and 24.5% on the LSVH test set.

Learning Pixel-Level and Instance-Level Context-Aware Features for Pedestrian Detection in Crowds

Pedestrian detection in crowded scenes is an intractable problem in computer vision, in which occlusion often presents a great challenge. In this paper, we propose a novel context-aware feature learning method for detecting pedestrians in crowds, with the purpose of making better use of context information for dealing with occlusion. Unlike most current pedestrian detectors that only extract context information from a single and fixed region, a new pixel-level context embedding module is developed to integrate multi-cue context into a deep CNN feature hierarchy, which enables access to the context of various regions by multi-branch convolution layers with different receptive fields. In addition, to utilize the distinctive visual characteristics formed by pedestrians that appear in groups and occlude each other, we propose a novel instance-level context prediction module which is actually implemented by a two-person detector, to improve the one-person detection performance. Applying with these strategies, we achieve an efficient and lightweight detector that can be trained in an end-to-end fashion. We evaluate the proposed approach on two popular pedestrian detection datasets, i.e., Caltech and CityPersons. The extensive experimental results demonstrate the effectiveness of the proposed method, especially under heavy occlusion cases.

A Study on the Detection of Visible Parts of Cordons Using Deep Learning Networks for Automated Green Shoot Thinning in Vineyards

Green shoot thinning operation in vineyards helps to reduce the crop load in favor of optimal quality wines. Mechanical green shoot thinning exists, but it causes cluster removal efficiencies to vary widely between 10-85 % because of difficulty in controlling the thinning end-effector position precisely to cordon trajectories. Automatically positioning the thinning end-effector to cordon trajectories will help to precisely remove the green shoots and to increase the efficiency and performance of the mechanical green shoot thinning operation. However, heavy occlusion of cordons due to shoots/leaves during thinning season makes it challenging to accurately determine the trajectories of cordons. Successfully detecting the visible parts of the cordons during the thinning season will help to estimate the trajectories of cordons for automated/robotics operation. In this study, a total of 390 wine grape vines were selected, and color images of these wine grapes were captured from a fixed distance and height for three weeks during the thinning season in real-time field conditions. Faster R-CNN (Faster regions-convolutional neural network) was deployed through transfer learning and fine tuning using the pre-trained networks (AlexNet, VGG16, VGG19 and ResNet18) to detect the visible parts of the cordons. Results showed that, Faster-RCNN model trained with ResNet18 networks provides higher accuracy in detecting visible parts of cordons compared to other tested networks with faster detection speed. Moreover, the detection accuracy with week 2 dataset was higher compared to that with week 3 and week 4 datasets because of the higher visibility of cordons. These results show the potential of Faster-RCNN model in detecting the visible parts of cordons, which will be used in the future to estimate the trajectories of the cordons for the automated green shoot thinning operation in vineyards.

An Efficient Crossing-Line Crowd Counting Algorithm with Two-Stage Detection

Crowd counting is a challenging task in crowded scenes due to heavy occlusions, appearance variations and perspective distortions. Current crowd counting methods typically operate on an image patch level with overlaps, then s... | Find, read and cite all the research you need on Tech Science Press

Context-aware long-term correlation tracking with hierarchical convolutional features

Visual tracking is of great significance in computer vision. Utilization of efficient correlation filters and discriminative convolutional features has gained much attention. In this paper, we propose a long–term tracking method (LHCAT) to deal with appearance variation caused by occlusion, deformation, out–of–view, etc. Firstly, we integrate context features into correlation filters by sampling surrounding patches around the target. Secondly, we reformulate the HCFT tracker with modified correlation filters. Then a HOG based scale correlation filter is applied to search target pyramids cropped around target position for optimal scale. In case of tracking failure and heavy occlusion, we train a SVM based module to re–detect the target. Additionally, for computational efficiency, proper strategy is proposed to determine when to activate the re–detector and update correlation filters. Experimental results on OTB–50 and OTB–100 show that our algorithm is of effectiveness and robustness in case of heavy appearance changes.

Real-Time Thermal Infrared Tracking Based on Collaborative Online and Offline Method

Most tracking-by-detection based trackers employ the online model update scheme based on the spatiotemporal consistency of visual cues. In presence of self-deformation, abrupt motion and heavy occlusion, these trackers suffer from different attributes and are prone to drifting. The model based on offline training, namely Siamese networks is invariant when suffering from the attributes. While the tracking speed of the offline method can be slow which is not enough for real-time tracking. In this paper, a novel collaborative tracker which decomposes the tracking task into online and offline modes is proposed. Our tracker switches between the online and offline modes automatically based on the tracker status inferred from the present failure tracking detection method which is based on the dispersal measure of the response map. The present Real-Time Thermal Infrared Collaborative Online and Offline Tracker (TCOOT) achieves state-of-the-art tracking performance while maintaining real-time speed at the same time. Experiments are carried out on the VOT-TIR-2015 benchmark dataset and our tracker achieves superior performance against Staple and Siam FC trackers by 3.3% and 3.6% on precision criterion and 3.8% and 5% on success criterion, respectively. The present method is real-time tracker as well.

Motion-Aware Correlation Filters for Online Visual Tracking

The discriminative correlation filters-based methods struggle deal with the problem of fast motion and heavy occlusion, the problem can severely degrade the performance of trackers, ultimately leading to tracking failures. In this paper, a novel Motion-Aware Correlation Filters (MACF) framework is proposed for online visual object tracking, where a motion-aware strategy based on joint instantaneous motion estimation Kalman filters is integrated into the Discriminative Correlation Filters (DCFs). The proposed motion-aware strategy is used to predict the possible region and scale of the target in the current frame by utilizing the previous estimated 3D motion information. Obviously, this strategy can prevent model drift caused by fast motion. On the base of the predicted region and scale, the MACF detects the position and scale of the target by using the DCFs-based method in the current frame. Furthermore, an adaptive model updating strategy is proposed to address the problem of corrupted models caused by occlusions, where the learning rate is determined by the confidence of the response map. The extensive experiments on popular Object Tracking Benchmark OTB-100, OTB-50 and unmanned aerial vehicles (UAV) video have demonstrated that the proposed MACF tracker performs better than most of the state-of-the-art trackers and achieves a high real-time performance. In addition, the proposed approach can be integrated easily and flexibly into other visual tracking algorithms.

Visibility Constrained Generative Model for Depth-Based 3D Facial Pose Tracking.

In this paper, we propose a generative framework that unifies depth-based 3D facial pose tracking and face model adaptation on-the-fly, in the unconstrained scenarios with heavy occlusions and arbitrary facial expression variations. Specifically, we introduce a statistical 3D morphable model that flexibly describes the distribution of points on the surface of the face model, with an efficient switchable online adaptation that gradually captures the identity of the tracked subject and rapidly constructs a suitable face model when the subject changes. Moreover, unlike prior art that employed ICP-based facial pose estimation, to improve robustness to occlusions, we propose a ray visibility constraint that regularizes the pose based on the face model's visibility with respect to the input point cloud. Ablation studies and experimental results on Biwi and ICT-3DHP datasets demonstrate that the proposed framework is effective and outperforms completing state-of-the-art depth-based methods.

Robust UAV-based Tracking Using Hybrid Classifiers.

Visual object tracking is a key factor for unmanned aerial vehicles (UAVs). In this paper, we propose a robust and effective visual object tracking method with an appearance model based on the locally adaptive regression kernel (LARK). The proposed appearance model encodes the geometric structure of the target. The tracking problem is formulated as two binary classifiers via two support vector machines (SVMs) with online model update. The backward tracking which tracks the target in reverse of time is utilized to evaluate the accuracy and robustness of the two SVMs. The final locations are adaptively fused based on the results of the forward tracking and backward tracking validation. Several state-of-the-art tracking algorithms are evaluated on large scale benchmark datasets which include challenging factors such as heavy occlusion, pose variation, illumination variation and motion blur. Experimental results demonstrate that our method achieves appealing performance.

Hierarchical convolutional features for visual tracking via two combined color spaces with SVM classifier

As the state-of-the-art object trackers majority, hierarchical convolutional features (HCF) cannot recover tracking processes from problems of drifting caused by several challenges, especially by heavy occlusion, scale variation, and illumination variation. In this paper, we present a new effective method with the aim of treating these challenges robustly based on two principal tasks. First, we infer the target location using multichannel correlation maps, resulting from the combination of five learned correlation filters with convolutional features. In order to handle the illumination variation and get more rich features, we exploit an HSV energy condition to control the use of two color spaces, RGB and HSV. Second, we use the histogram of gradient features to learn another correlation filter in order to estimate the scale variation. Furthermore, we exploit an online training SVM classifier for target re-detecting in failure cases. The extensive experiments on a commonly used tracking benchmark dataset justify that our tracker significantly improves HCF and outperforms the state-of-the-art methods.

A Direct 3D Object Tracking Method Based on Dynamic Textured Model Rendering and Extended Dense Feature Fields

We propose a novel method for robust 6-DOF pose tracking of rigid objects from monocular images. In our method, 3D object tracking is achieved by directly aligning video frames to dynamic templates rendered from a textured 3D object model. Unlike previous methods, which usually utilize a small number of discrete templates to align with video frames, we employ an online textured model, rendering to create dynamic templates in continuous pose space according to the previously estimated object pose. In this way, a pose estimator could be easily converged to the optimal state. Besides, the rendered template also helps to detect the occlusion area by comparing it with the current frame, making our method highly robust to partial occlusions. The performance of our method is further improved by introducing a generic representation of dense images features, which we call extended dense feature fields (EDFF). Different kinds of pixel-level image features can be added to the EDFF and be optimized simultaneously in a unified Gauss-Newton optimization scheme. Attributing to dynamic templates from the textured model rendering and complementary features in EDFF, our method is able to deal with poor-textured and specular objects, as well as lighting variation and heavy occlusions. While our method is quite simple and straightforward, it achieves competitive or even superior results compared with the state of the art on challenging data sets.

Robust long-term correlation tracking using convolutional features and detection proposals

Abstract Correlation filter based trackers have achieved appealing performance and high efficiency in recent years. However, for long-term tracking where target objects undergo dramatic appearance variation due to heavy occlusion or out-of-view, conventional correlation filter based tracking algorithms would be distracted by irrelevant objects. Once the trained tracker loses its way, it is impossible to recover the information for the following frames as the model has drifted. In this paper, we decompose the long-term tracking task into tracking and detection. Tracker learns separate correlation filters for explicit translation and scale estimation. Specifically, in order to improve tracking accuracy, the convolutional features for translation filter are extracted, and the scale filter is learned using the target appearance sampled at different scales. Detector trains an online long-term filter and applies it to the entire frame to generate detection proposals. By exploiting these detection proposals, it helps the tracker to recover from problems such as temporary or persistent occlusions. In this way, the proposed approach could handle the model drifting problem effectively for long-term tracking with more accurate estimation of object scale and location. Extensive experimental results on large-scale benchmark sequences have shown the robustness of the proposed method.