Presence Of Occlusion Research Articles

Surgical Robot with Environment Reconstruction and Force Feedback.

We present a new surgical robot hardware-in-the-loop simulator, with 3D surgical field reconstruction in RGB-D sensor range, which allows tool-tissue interactions to be presented as haptic feedback and thus provides the situation awareness of unwanted collision. First, the point cloud of the complete surgical environment is constructed from multiple frames of sensor data to avoid the occlusion issue. Then the user selects a region of interest where the robot's tool must avoid (also called forbidden region). The real-time haptic force rendering algorithm computes the interaction force which is then communicated to a haptic device at 1 kHz, to assist the surgeon to perform safe actions. The robot used is a RAVEN II system, RGB-D sensor is used to scan the environment, and two Omni haptic devices provide the 3-DoF haptic force. A registration pipeline is presented to complete the surgical environment point cloud mapping in preoperative surgery planning phase, which improves quality of haptic rendering in the presence of occlusion. Furthermore, we propose a feasible and fast algorithm which extends the existing work on the proxy-based method for haptic rendering between a Haptic Interaction Point (HIP) and a point cloud. The proposed methodology has the potential of improving the safety of surgical robots.

Forestry crane posture estimation with a two‐dimensional laser scanner

AbstractCrane posture estimation is the stepping stone to forest machine automation. Here, we introduce a robust minimal perception solution, that is, one that uses minimal constraints for maximal benefits. Specifically, we introduce a robust particle‐filter‐based method to estimate and track the posture of a flexible hydraulic crane by using only low‐cost equipment, namely, a two‐dimensional (2D) laser scanner, two short magnetically attached metal tubes as targets, and an angle sensor. An important feature of our method is that it incorporates control signals for hydraulic actuators. In contrast to the previous works employing laser scanners, we do not use the full shape of the crane to estimate the crane posture, but, instead, we use only two small targets in the field of view of the laser scanner. Thus, a large share of the range data is useful for other purposes, for example, to map the surrounding environment. We test the proposed method in a challenging forest environment and show that the particle filter is able to estimate the posture of the hydraulic crane efficiently and reliably in the presence of occlusions and obstructions. During our comprehensive testing, the tip position was measured with average errors smaller than 4.3 cm whereas the absolute maximum error was 15 cm.

Occlusion detection and restoration techniques for 3D face recognition: a literature review

Methodologies for 3D face recognition which work in the presence of occlusions are core for the current needs in the field of identification of suspects, as criminals try to take advantage of the weaknesses among the implemented security systems by camouflaging themselves and occluding their face with eyeglasses, hair, hands, or covering their face with scarves and hats. Recent occlusion detection and restoration strategies for recognition purposes of 3D partially occluded faces with unforeseen objects are here presented in a literature review. The research community has worked on face recognition systems under controlled environments, but uncontrolled conditions have been investigated in a lesser extent. The paper details the experiments and databases used to handle the problem of occlusion and the results obtained by different authors. Lastly, a comparison of various techniques is presented and some conclusions are drawn referring to the best outcomes.

A two‐stage sampling for robust feature matching

AbstractFor any visual feature‐based SLAM (simultaneous localization and mapping) solutions, to estimate the relative camera motion between two images, it is necessary to find “correct” correspondence between features extracted from those images. Given a set of feature correspondents, one can use a n‐point algorithm with robust estimation method, to produce the best estimate to the relative camera pose. The accuracy of a motion estimate is heavily dependent on the accuracy of the feature correspondence. Such a dependency is even more significant when features are extracted from the images of the scenes with drastic changes in viewpoints and illuminations and presence of occlusions. To make a feature matching robust to such challenging scenes, we propose a new feature matching method that incrementally chooses a five pairs of matched features for a full DoF (degree of freedom) camera motion estimation. In particular, at the first stage, we use our 2‐point algorithm to estimate a camera motion and, at the second stage, use this estimated motion to choose three more matched features. In addition, we use, instead of the epipolar constraint, a planar constraint for more accurate outlier rejection. With this set of five matching features, we estimate a full DoF camera motion with scale ambiguity. Through the experiments with three, real‐world data sets, our method demonstrates its effectiveness and robustness by successfully matching features (1) from the images of a night market where presence of frequent occlusions and varying illuminations, (2) from the images of a night market taken by a handheld camera and by the Google street view, and (3) from the images of a same location taken daytime and nighttime.

Association of FGB, LPL, GPIIIA and TGFB gene polymorphisms with variants of atherosclerosis severity

Aim. To estimate the effect of transforming growth factor в, lipoprotein lipase, fibrinogen в and glycoprotein 3б gene polymorphisms on different variants of atherosclerosis severity. 
 Methods. The study included 319 patients with angiografically verified atherosclerosis. In all patients FGB (rs1800788), LPL (rs328), GpIIIa (rs5918) and TGFВ (rs1800469) polymorphism genotyping was performed. In all patients we evaluated the influence of the studied gene polymorphisms on parameters of atherosclerosis severity such as occlusion and presence and severity of stenosis. Presence of occlusion, occlusion and/or two and more stenoses over 70%, occlusion and multiple critical stenoses were considered the severity criteria. 
 Results. The studied gene polymorphisms were shown to be associated with severe course of atherosclerosis. GPIIIa polymorphism was commonly determined in patients with more severe course regardless of occlusion and number and severity of stenoses. FGB gene influenced the rate of occlusion development, TGF gene was expressed only in the group with occlusion and/or two and more stenoses over 70%, and LPL gene proved to be significant for the development of occlusion associated with critical stenosis. 
 Conclusion. The frequency of rare alleles of polymorphic regions is significantly associated with different variants of severe atherosclerosis course; this could suggest unequal influence of gene polymorphisms as well as their proteins on different stages of atherogenesis and requires further investigation.

Robust Hidden Markov Model based intelligent blood vessel detection of fundus images

In this paper, we consider the challenging problem of detecting retinal vessel networks. Precise detection of retinal vessel networks is vital for accurate eye disease diagnosis. Most of the blood vessel tracking techniques may not properly track vessels in presence of vessels’ occlusion. Owing to problem in sensor resolution or acquisition of fundus images, it is possible that some part of vessel may occlude. In this scenario, it becomes a challenging task to accurately trace these vital vessels. For this purpose, we have proposed a new robust and intelligent retinal vessel detection technique on Hidden Markov Model. The proposed model is able to successfully track vessels in the presence of occlusion. The effectiveness of the proposed technique is evaluated on publically available standard DRIVE dataset of the fundus images. The experiments show that the proposed technique not only outperforms the other state of the art methodologies of retinal blood vessels segmentation, but it is also capable of accurate occlusion handling in retinal vessel networks. The proposed technique offers better average classification accuracy, sensitivity, specificity, and area under the curve (AUC) of 95.7%, 81.0%, 97.0%, and 90.0% respectively, which shows the usefulness of the proposed technique.

Segmentation of Airborne Point Cloud Data for Automatic Building Roof Extraction

Roof plane segmentation is a complex task since point cloud data carry no connection information and do not provide any semantic characteristics of the underlying scanned surfaces. Point cloud density, complex roof profiles, and occlusion add another layer of complexity which often encounter in practice. In this article, we present a new technique that provides a better interpolation of roof regions where multiple surfaces intersect creating non-manifold points. As a result, these geometric features are preserved to achieve automated identification and segmentation of the roof planes from unstructured laser data. The proposed technique has been tested using the International Society for Photogrammetry and Remote Sensing benchmark and three Australian datasets, which differ in terrain, point density, building sizes, and vegetation. The qualitative and quantitative results show the robustness of the methodology and indicate that the proposed technique can eliminate vegetation and extract buildings as well as their non-occluding parts from the complex scenes at a high success rate for building detection (between 83.9% and 100% per-object completeness) and roof plane extraction (between 73.9% and 96% per-object completeness). The proposed method works more robustly than some existing methods in the presence of occlusion and low point sampling as indicated by the correctness of above 95% for all the datasets.

Spatiotemporal Colorization of Video Using 3D Steerable Pyramids

We propose a new technique for video colorization based on spatiotemporal color propagation in the 3D video volume, utilizing the dominant orientation response obtained from the steerable pyramid decomposition of the video. The volumetric color diffusion from the sources that are marked by scribbles occurs across spatiotemporally smooth regions, and the prevention of leakage is facilitated by the spatiotemporal discontinuities in the output of steerable filters, representing the object boundaries and motion boundaries. Unlike most existing methods, our approach dispenses with the need of motion vectors for interframe color transfer and provides a general framework for image and video colorization. The experimental results establish the effectiveness of the proposed approach in colorizing videos having different types of motion and visual content even in the presence of occlusion, in terms of accuracy and computational requirements.

3D geometry-based automatic landmark localization in presence of facial occlusions

This study proposes a novel automatic method for facial landmark localization relying on geometrical properties of 3D facial surface working both on complete faces displaying different emotions and in presence of occlusions. In particular, 12 descriptors coming from Differential Geometry including the coefficients of the fundamental forms, Gaussian, mean, principal curvatures, shape index and curvedness are extracted as facial features and their local geometric properties are exploited to localize 13 soft-tissue landmarks from eye and nose areas. The method is deterministic and is backboned by a thresholding technique designed by studying the behaviour of each geometrical descriptor in correspondence to the locus of each landmark. Occlusions are managed by a detection algorithm based on geometrical properties which allows to proceed with the landmark localization avoiding the covered areas. Experimentations were carried out on 3132 faces of the Bosphorus database and of a 230-sized internal database, including expressive and occluded ones (mouth, eye, and eyeglasses occlusions), obtaining 4.75 mm mean localization error.

Learning Deep NBNN Representations for Robust Place Categorization

This paper presents an approach for semantic place categorization using data obtained from RGB cameras. Previous studies on visual place recognition and classification have shown that, by considering features derived from pre-trained Convolutional Neural Networks (CNNs) in combination with part-based classification models, high recognition accuracy can be achieved, even in presence of occlusions and severe viewpoint changes. Inspired by these works, we propose to exploit local deep representations, representing images as set of regions applying a Na\"{i}ve Bayes Nearest Neighbor (NBNN) model for image classification. As opposed to previous methods where CNNs are merely used as feature extractors, our approach seamlessly integrates the NBNN model into a fully-convolutional neural network. Experimental results show that the proposed algorithm outperforms previous methods based on pre-trained CNN models and that, when employed in challenging robot place recognition tasks, it is robust to occlusions, environmental and sensor changes.

Navigation Path Detection for Cotton Field Operator Robot Based on Horizontal Spline Segmentation

Visual navigation is one of the fundamental techniques of intelligent cotton-picking robot. Cotton field composition is complex and the presence of occlusion and illumination makes it hard to accurately identify furrows so as to extract the navigation line. In this paper, a new field navigation path extraction method based on horizontal spline segmentation is presented. Firstly, the color image in RGB color space is pre-processed by the OTSU threshold algorithm to segment the binary image of the furrow. The cotton field image components are divided into four categories: furrow (ingredients include land, wilted leaves, etc.), cotton fiber, other organs of cotton and the outside area or obstructions. By using the significant differences in hue and value of the HSV model, the authors segment the threshold by two steps. Firstly, they segment cotton wool in the S channel, and then segment the furrow in the V channel in the area outside the cotton wool area. In addition, morphological processing is needed to filter out small noise area. Secondly, the horizontal spline is used to segment the binary image. The authors detect the connected domains in the horizontal splines, and merger the isolate small areas caused by the cotton wool or light spots in the nearby big connected domains so as to get connected domain of the furrow. Thirdly, they make the center of the bottom of the image as the starting point, and successively select the candidate point from the midpoint of the connected domain, according to the principle that the distance between adjacent navigation line candidate is smaller. Finally, the authors count the number of the connected domains and calculate the change of parameters of boundary line of the connected domain to make sure whether the robot reaches the outside of the field or encounters obstacles. If there is no anomaly, the navigation path is fitted by the navigation points using the least squares method. Experiments prove that this method is accurate and effective, which is suitable for visual navigation in the complex environment of a cotton field in different phases.

A robust face recognition approach through symbolic modeling of Polar FFT features

Face recognition is a challenging and one of the most active area of research in computer vision. Variations in face image due to change in illumination, expression and presence of occlusions along with aging present many challenges to face recognition system. In this paper a new approach to robust face recognition using Polar FFT features modeled as symbolic data is proposed. Initially the face segment is cropped from the image using Viola-Jones algorithm and is converted into gray scale image of size 120 × 120 pixels. 2D-DFT is performed on pre-processed image. The dominant magnitude of 2D-DFT coefficients are computed using polar fourier transform technique and are represented as 1D P-FFT. The magnitude that represents maximum value in 1D P-FFT is considered as a feature value. The extracted feature value is used to construct a symbolic object to represent a face image. Further, a new symbolic similarity measure is devised and employed for assigning test symbolic object to a face class. The performance of the proposed method is evaluated with AR database and an average accuracy of 96.25% is achieved. 100% and 97% verification rate is achieved on the ORL and LFW face databases respectively. The various experiments conducted on AR, ORL and LFW databases using the proposed symbolic data modeling approach shows that this method has a very high degree of recognition and is better compared to some of the recently reported face recognition techniques.

FOREGROUND DETECTION ON DEPTH MAPS USING SKELETAL REPRESENTATION OF OBJECT SILHOUETTES

Abstract. This article considers the problem of foreground detection on depth maps. The problem of finding objects of interest on images appears in many object detection, recognition and tracking applications as one of the first steps. However, this problem becomes too complicated for RGB images with multicolored or constantly changing background and in presence of occlusions. Depth maps provide valuable information about distance to the camera for each point of the scene, making it possible to explore object detection methods, based on depth features. We define foreground as a set of objects silhouettes, nearest to the camera relative to the local background. We propose a method of foreground detection on depth maps based on medial representation of objects silhouettes which does not require any machine learning procedures and is able to detect foreground in near real-time in complex scenes with occlusions, using a single depth map. Proposed method is implemented to depth maps, obtained from Kinect sensor.

Moving objects tracking based on improved particle filter algorithm by elimination of unimportant particles

Object tracking is an important subject in machine vision. Various methods have been devised for object tracking, and among these methods particle filter (PF) has been found to be of particular significance. This method is based on random sampling of a probability density function, and estimating the desired variable based on samples weight. One advantage of this method is its ability to track the targets even in presence of occlusion; an ability which is due to the inclusion of unlikely areas in distribution of particles. The main difficulty with PF is however its slow performance, which makes it unfit for real-time applications. This paper presents an approach to improve the performance and increase the speed of PF for real-time object tracking. For this purpose, the classical PF method based on color histogram is exploited to develop an algorithm with reduced computational cost and increased tracking speed. In the proposed method, in each stage unimportant particles were removed using a binary mask. This mask is generated through consecutive frames difference or using Gaussian mixture model. Applying the proposed algorithms on benchmark databases gives promising results. The comparison of obtained results with those of classical PF demonstrates that the proposed methods not only improve the accuracy of tracking but also increase its speed by 39%.

Augmented Reality Tool for the Situational Awareness Improvement of UAV Operators.

Unmanned Aerial Vehicles (UAVs) are being extensively used nowadays. Therefore, pilots of traditional aerial platforms should adapt their skills to operate them from a Ground Control Station (GCS). Common GCSs provide information in separate screens: one presents the video stream while the other displays information about the mission plan and information coming from other sensors. To avoid the burden of fusing information displayed in the two screens, an Augmented Reality (AR) tool is proposed in this paper. The AR system has two functionalities for Medium-Altitude Long-Endurance (MALE) UAVs: route orientation and target identification. Route orientation allows the operator to identify the upcoming waypoints and the path that the UAV is going to follow. Target identification allows a fast target localization, even in the presence of occlusions. The AR tool is implemented following the North Atlantic Treaty Organization (NATO) standards so that it can be used in different GCSs. The experiments show how the AR tool improves significantly the situational awareness of the UAV operators.

K‐means based multiple objects tracking with long‐term occlusion handling

This study presents a novel multiple objects tracking (MOT) approach that models object's appearance based on K‐means, while introducing a new statistical measure for association of objects after occlusion. The proposed method is tested on several standard datasets dealing complex situations in both indoor and outdoor environments. The experimental results show that the proposed model successfully tracks multiple objects in the presence of occlusion with high accuracy. Moreover, the presented work has the capability to deal long term and complete occlusion without any prior training of the shape and motion model of the objects. Accuracy of the proposed method is comparable with that of the existing state‐of‐the‐art techniques as it successfully deals with all MOT cases in the standard datasets. Most importantly, the proposed method is cost effective in terms of memory and/or computation as compared with that of the existing state‐of‐the‐art techniques. These traits make the proposed system very useful for real‐time embedded video surveillance platforms especially those that have low memory/compute resources.

Data Association and Prediction for Tracking Multiple Objects

Objectives: Tracking moving objects is essential for high level computer vision analysis, like object behavior interpretation or gait recognition etc. In this paper, a new method to track multiple moving objects in the surveillance video sequence is proposed. Methods/Statistical Analysis: Object tracking is done by extracting color moments feature from the segmented foreground object and associating individual objects in the successive frames using nearest neighbor classifier and Chi- Square dissimilarity measure. Further, object tracking during occlusion has been addressed by predicting the missing state of the occluded object by applying Lagrange’s polynomial extrapolation using the apriori knowledge. Findings: The proposed method is evaluated using several challenging sequences of the benchmark IEEE PETS, IEEE CHANGE DETECTION, and EPFL dataset. Further, comparative evaluation with contemporary methods has been carried out to corroborate the efficacy of the proposed method. Application/Improvements: This work focuses on devising a new method to track multiple moving objects using color moments and Lagrange’s extrapolation framework which works for a complex environment and in the presence of occlusions. Further, the shadow elimination task after the motion segmentation could be considered as an improvement step in this work which will aid in better tracking results.

PANORAMA IMAGE SETS FOR TERRESTRIAL PHOTOGRAMMETRIC SURVEYS

Abstract. High resolution 3D models produced from photographs acquired with consumer-grade cameras are becoming increasingly common in the fields of geosciences. However, the quality of an image-based 3D model depends on the planning of the photogrammetric surveys. This means that the geometric configuration of the multi-view camera network and the control data have to be designed in accordance with the required accuracy, resolution and completeness. From a practical application point of view, a proper planning (of both photos and control data) of the photogrammetric survey especially for terrestrial acquisition, is not always ensured due to limited accessibility of the target object and the presence of occlusions. To solve these problems, we propose a different image acquisition strategy and we test different geo-referencing scenarios to deal with the practical issues of a terrestrial photogrammetric survey. The proposed photogrammetric survey procedure is based on the acquisition of a sequence of images in panorama mode by rotating the camera on a standard tripod. The offset of the pivot point from the projection center prevents the stitching of these images into a panorama. We demonstrate how to still take advantage of this capturing mode. The geo-referencing investigation consists of testing the use of directly observed coordinates of the camera positions, different ground control point (GCP) configurations, and GCPs with different accuracies, i.e. artificial targets vs. natural features. Images of the test field in a low-slope hill were acquired from the ground using an SLR camera. To validate the photogrammetric results a terrestrial laser scanner survey is used as benchmark.

PANORAMA IMAGE SETS FOR TERRESTRIAL PHOTOGRAMMETRIC SURVEYS

High resolution 3D models produced from photographs acquired with consumer-grade cameras are becoming increasingly common in the fields of geosciences. However, the quality of an image-based 3D model depends on the planning of the photogrammetric surveys. This means that the geometric configuration of the multi-view camera network and the control data have to be designed in accordance with the required accuracy, resolution and completeness. From a practical application point of view, a proper planning (of both photos and control data) of the photogrammetric survey especially for terrestrial acquisition, is not always ensured due to limited accessibility of the target object and the presence of occlusions. To solve these problems, we propose a different image acquisition strategy and we test different geo-referencing scenarios to deal with the practical issues of a terrestrial photogrammetric survey. The proposed photogrammetric survey procedure is based on the acquisition of a sequence of images in panorama mode by rotating the camera on a standard tripod. The offset of the pivot point from the projection center prevents the stitching of these images into a panorama. We demonstrate how to still take advantage of this capturing mode. The geo-referencing investigation consists of testing the use of directly observed coordinates of the camera positions, different ground control point (GCP) configurations, and GCPs with different accuracies, i.e. artificial targets vs. natural features. Images of the test field in a low-slope hill were acquired from the ground using an SLR camera. To validate the photogrammetric results a terrestrial laser scanner survey is used as benchmark.

Efficient multi-target tracking via discovering dense subgraphs

In this paper, we cast multi-target tracking as a dense subgraph discovering problem on the undirected relation graph of all given target hypotheses. We aim to extract multiple clusters (dense subgraphs), in which each cluster contains a set of hypotheses of one particular target. In the presence of occlusion or similar moving targets or when there is no reliable evidence for the target’s presence, each target trajectory is expected to be fragmented into multiple tracklets. The proposed tracking framework can efficiently link such fragmented target trajectories to build a longer trajectory specifying the true states of the target. In particular, a discriminative scheme is devised via learning the targets’ appearance models. Moreover, the smoothness characteristic of the target trajectory is utilised by suggesting a smoothness tracklet affinity model to increase the power of the proposed tracker to produce persistent target trajectories revealing different targets’ moving paths. The performance of the proposed approach has been extensively evaluated on challenging public datasets and also in the context of team sports (e.g. soccer, AFL), where team players tend to exhibit quick and unpredictable movements. Systematic experimental results conducted on a large set of sequences show that the proposed approach performs better than the state-of-the-art trackers, in particular, when dealing with occlusion and fragmented target trajectory.