Occluding Contours Research Articles

What do we see behind an occluder? Amodal completion of statistical properties in complex objects.

When a spiky object is occluded, we expect its spiky features tocontinue behind the occluder. Although many real-world objects contain complex features, it is unclear how more complex features are amodally completed and whether this process is automatic. To investigate this issue, we created pairs of displays with identical contour edges up to the point of occlusion, but with occluded portions exchanged. We then asked participants to search for oddball targets among distractors and asked whether relations between searches involving occluded displays would match better with relations between searches involving completions that are either globally consistent or inconsistent with the visible portions of these displays. Across two experiments involving simple and complex shapes, search times involving occluded displays matched better with those involving globally consistent compared with inconsistent displays. Analogous analyses on deep networks pretrained for object categorization revealed a similar pattern of results for simple but not complex shapes. Thus, deep networks seem to extrapolate simple occluded contours but not more complex contours. Taken together, our results show that amodal completion in humans is sophisticated and can be based on extrapolating global statistical properties.

An objective comparison of methods for augmented reality in laparoscopic liver resection by preoperative-to-intraoperative image fusion from the MICCAI2022 challenge

Augmented reality for laparoscopic liver resection is a visualisation mode that allows a surgeon to localise tumours and vessels embedded within the liver by projecting them on top of a laparoscopic image. Preoperative 3D models extracted from Computed Tomography (CT) or Magnetic Resonance (MR) imaging data are registered to the intraoperative laparoscopic images during this process. Regarding 3D-2D fusion, most algorithms use anatomical landmarks to guide registration, such as the liver’s inferior ridge, the falciform ligament, and the occluding contours. These are usually marked by hand in both the laparoscopic image and the 3D model, which is time-consuming and prone to error. Therefore, there is a need to automate this process so that augmented reality can be used effectively in the operating room. We present the Preoperative-to-Intraoperative Laparoscopic Fusion challenge (P2ILF), held during the Medical Image Computing and Computer Assisted Intervention (MICCAI 2022) conference, which investigates the possibilities of detecting these landmarks automatically and using them in registration. The challenge was divided into two tasks: (1) A 2D and 3D landmark segmentation task and (2) a 3D-2D registration task. The teams were provided with training data consisting of 167 laparoscopic images and 9 preoperative 3D models from 9 patients, with the corresponding 2D and 3D landmark annotations. A total of 6 teams from 4 countries participated in the challenge, whose results were assessed for each task independently. All the teams proposed deep learning-based methods for the 2D and 3D landmark segmentation tasks and differentiable rendering-based methods for the registration task. The proposed methods were evaluated on 16 test images and 2 preoperative 3D models from 2 patients. In Task 1, the teams were able to segment most of the 2D landmarks, while the 3D landmarks showed to be more challenging to segment. In Task 2, only one team obtained acceptable qualitative and quantitative registration results. Based on the experimental outcomes, we propose three key hypotheses that determine current limitations and future directions for research in this domain.

New Insights in Smooth Occluding Contours for Nonphotorealistic Rendering.

Computing occluding contours is often a crucial step in stroke-based artistic 3-D stylization for movies, video games, and visualizations. However, many existing applications use only simple curve stylization techniques, such as thin black lines or hand-animated strokes. This is because sophisticated procedural stylization requires accurate curve topology, which has long been an unsolved research problem. This article describes a recent theoretical breakthrough in the topology problem. Specifically, the new theory points out that existing contour algorithms often generate curves that cannot have any valid visibility, and new algorithms show how to correct the problem. This article surveys classes of algorithms that can compute contours accurately and identifies new research opportunities.

ConTesse: Accurate Occluding Contours for Subdivision Surfaces

This article proposes a method for computing the visible occluding contours of subdivision surfaces. The article first introduces new theory for contour visibility of smooth surfaces. Necessary and sufficient conditions are introduced for when a sampled occluding contour is valid, that is, when it may be assigned consistent visibility. Previous methods do not guarantee these conditions, which helps explain why smooth contour visibility has been such a challenging problem in the past. The article then proposes an algorithm that, given a subdivision surface, finds sampled contours satisfying these conditions, and then generates a new triangle mesh matching the given occluding contours. The contours of the output triangle mesh may then be rendered with standard non-photorealistic rendering algorithms, using the mesh for visibility computation. The method can be applied to any triangle mesh, by treating it as the base mesh of a subdivision surface.

Liouville Integrability in a Four-Dimensional Model of the Visual Cortex.

We consider a natural extension of the Petitot–Citti–Sarti model of the primary visual cortex. In the extended model, the curvature of contours is taken into account. The occluded contours are completed via sub-Riemannian geodesics in the four-dimensional space M of positions, orientations, and curvatures. Here, models the configuration space of neurons of the visual cortex. We study the problem of sub-Riemannian geodesics on M via methods of geometric control theory. We prove complete controllability of the system and the existence of optimal controls. By application of the Pontryagin maximum principle, we derive a Hamiltonian system that describes the geodesics. We obtain the explicit parametrization of abnormal extremals. In the normal case, we provide three functionally independent first integrals. Numerical simulations indicate the existence of one more first integral that results in Liouville integrability of the system.

Detecting the occluding contours of the uterus to automatise augmented laparoscopy: score, loss, dataset, evaluation and user study.

The registration of a preoperative 3D model, reconstructed, for example, from MRI, to intraoperative laparoscopy 2D images, is the main challenge to achieve augmented reality in laparoscopy. The current systems have a major limitation: they require that the surgeon manually marks the occluding contours during surgery. This requires the surgeon to fully comprehend the non-trivial concept of occluding contours and surgeon time, directly impacting acceptance and usability. To overcome this limitation, we propose a complete framework for object-class occluding contour detection (OC2D), with application to uterus surgery. Our first contribution is a new distance-based evaluation score complying with all the relevant performance criteria. Our second contribution is a loss function combining cross-entropy and two new penalties designed to boost 1-pixel thickness responses. This allows us to train a U-Net end to end, outperforming all competing methods, which tends to produce thick responses. Our third contribution is a dataset of 3818 carefully labelled laparoscopy images of the uterus, which was used to train and evaluate our detector. Evaluation shows that the proposed detector has a similar false false-negative rate to existing methods but substantially reduces both false-positive rate and response thickness. Finally, we ran a user study to evaluate the impact of OC2D against manually marked occluding contours in augmented laparoscopy. We used 10 recorded gynecologic laparoscopies and involved 5 surgeons. Using OC2D led to a reduction of 3 min and 53 s in surgeon time without sacrificing registration accuracy. We provide a new set of criteria and a distance-based measure to evaluate an OC2D method. We propose an OC2D method which outperforms the state-of-the-art methods. The results obtained from the user study indicate that fully automatic augmented laparoscopy is feasible.

Camera Calibration Using Projective Invariants of Sphere Images

Two novel linear camera calibration algorithms were investigated in this study. In general, an occluding contour circle of a sphere and the optical centre of a camera form a right circular cone, and the generalised eigenvectors of two right circular cones encode an infinity point that lies on two support planes containing the contour circle. In the image plane, a vanishing line can be determined by connecting two vanishing points. Furthermore, the direction of the diameter passing through the infinity point is orthogonal to the direction of its conjugate diameter; thus, a set of orthogonal vanishing points can be computed. The imaged circular points or orthogonal vanishing points provide constraints that enable the intrinsic parameters to be recovered completely. The results of simulations and comparison data obtained from actual experiments confirm the effectiveness and feasibility of the proposed algorithms.

Irrational contour synthesis

The mechanisms responsible for generating illusory contours are thought to fulfil an adaptive role in providing estimates of missing contour fragments generated by partial camouflage. One striking apparent counter-example to this view was described in Current Biology 21 (2011) 492–496, which showed that illusory contours could arise in motion displays depicting visible occluding discs occluding and disoccluding thin contours. These motion sequences generate illusory contours even though they play no necessary role in accounting for occlusion and disocclusion of the thin contours. The present work sought to more precisely characterize the quantitative dependence of these ‘irrational’ contours on the relative contrasts in the image. We show that the perceived strength of the illusory contours generated by these displays depends monotonically on the relative contrast of the occluding and occluded contours and that previous attempts to measure their strength with a method of adjustment appears to be contaminated by response bias. We further show that these illusory contours also arise when the occluding disks are rendered transparent and exhibit similar forms of contrast dependencies. These findings reveal a general methodological problem that can arise using methods of adjustment and provide quantitative data that may be used to identify the neural mechanisms responsible for IC genesis and their perceived strength.

Plant Species Identification from Occluded Leaf Images

We present an approach to identify the plant species from the contour information from occluded leaf image using a database of full plant leaves. Although contour based 2D shape matching has been studied extensively in the last couple of decades, matching occluded leaves with full leaf databases is an open and little worked on problem. Classifying occluded plant leaves is even more challenging than full leaf matching because of large variations and complexity of leaf structures. Matching an occluded contour with all the full contours in a database is an NP-hard problem, so our algorithm is necessarily suboptimal. First, we represent the 2D contour points as a β-Spline curve. Then, we extract interest points on these curves via the Discrete Contour Evolution (DCE) algorithm. We use subgraph matching using the DCE points as graph nodes, which produces a number of open curves for each closed leaf contour. Next, we compute the similarity transformation parameters (translation, rotation, and uniform scaling) for each open curve. We then "overlay" each open curve with the inverse similarity transformed occluded curve and use the Fréchet distance metric to measure the quality of the match, retaining the best η matched curves. Since the Fréchet metric is cheap to compute but not perfectly correlated with the quality of the match, we formulate an energy functional that is well correlated with the quality of the match, but is considerably more expensive to compute. The functional uses local and global curvature, Shape Context descriptors, and String Cut features. We minimize this energy functional using a convex-concave relaxation framework. The curve among these best η curves, that has the minimum energy, is considered to be the best overall match with the occluded leaf. Experiments on three publicly available leaf image database shows that our method is both effective and efficient, outperforming other current state-of-the-art methods. Occlusion is measured as the percentage of the overall contour (and not leaf area) that is missing. We show that our algorithm can, even for leaves with a high amounts of occlusion (say 50 percent occlusion), still identify the best full leaf match from the databases.

Figure and ground: how the visual cortex integrates local cues for global organization.

Inferring figure-ground organization in two-dimensional images may require different complementary strategies. For isolated objects, it has been shown that mechanisms in visual cortex exploit the overall distribution of contours, but in images of cluttered scenes where the grouping of contours is not obvious, that strategy would fail. However, natural scenes contain local features, specifically contour junctions, that may contribute to the definition of object regions. To study the role of local features in the assignment of border ownership, we recorded single-cell activity from visual cortex in awake behaving Macaca mulatta. We tested configurations perceived as two overlapping figures in which T- and L-junctions depend on the direction of overlap, whereas the overall distribution of contours provides no valid information. While recording responses to the occluding contour, we varied direction of overlap and variably masked some of the critical contour features to determine their influences and their interactions. On average, most features influenced the responses consistently, producing either enhancement or suppression depending on border ownership. Different feature types could have opposite effects even at the same location. Features far from the receptive field produced effects as strong as near features and with the same short latency. Summation was highly nonlinear: any single feature produced more than two-thirds of the effect of all features together. These findings reveal fast and highly specific organization mechanisms, supporting a previously proposed model in which "grouping cells" integrate widely distributed edge signals with specific end-stopped signals to modulate the original edge signals by feedback. NEW & NOTEWORTHY Seeing objects seems effortless, but defining objects in a scene requires sophisticated neural mechanisms. For isolated objects, the visual cortex groups contours based on overall distribution, but this strategy does not work for cluttered scenes. Here, we demonstrate mechanisms that integrate local contour features like T- and L-junctions to resolve clutter. The process is fast, evaluates widely distributed features, and gives any single feature a decisive influence on figure-ground representation.

When Is Accreting/Deleting Texture Seen as In Front? Interpretation of Depth From Texture Motion.

Standard accounts of accretion/deletion of texture treat it as a definite cue to depth ordering: The accreting/deleting surface is interpreted as being behind the adjoining surface. Froyen, Feldman, and Singh showed that accretion/deletion can also, under certain circumstances, be perceived as a 3D column rotating in front, with the accretion/deletion explained by self-occlusion. These displays differ from traditional accretion/deletion displays in a number of factors, including the presence of figure/ground cues, accretion/deletion on both sides of boundaries, and in the number of distinct regions. In a series of experiments, we systematically manipulated each of these factors in order to determine what factors are actually instrumental in creating the rotating column (accretion/deletion in front) interpretation. In Experiment 1, the width of each region was kept fixed while manipulating the number of regions, and in Experiment 2, the width of the overall display was kept fixed. Observers indicated which set of regions they perceived to be in front. In both experiments, accreting/deleting regions were most likely to be seen in front when geometric figural cues favored a figural interpretation and when textural motion was introduced in all regions (rather than on just one side of each boundary). The number of regions had a relatively small effect (although this effect was larger in Experiment 2). These findings indicate that the geometry of the occluding contour is a critical factor in the interpretation of accretion/deleting, and future models of 3D interpretation involving accretion/deletion must include contour geometry as a key component.

Projectively inconsistent occluding contours have surprisingly little influence on SFM percepts

Projectively inconsistent occluding contours have surprisingly little influence on SFM percepts

Differentiating global and local contour completion using a dot localization paradigm.

Competing theories of partially occluded object perception (amodal completion) emphasize either relatively local contour relationships or global factors such as symmetry. These disparate theories may reflect 2 separate processes: a low-level contour interpolation process and a higher-order global recognition process. The 2 could be distinguished experimentally if only the former produces precise representations of occluded object boundaries. Using a dot localization paradigm, we measured the precision and accuracy of perceived object boundaries for participants instructed to complete occluded objects with divergent local and global interpretations. On each trial, a small red dot was flashed on top of an occluder. Participants reported whether the dot fell inside or outside the occluded object's boundaries. Interleaved, 2-up, 1-down staircases estimated points on the psychometric function where the probability was .707 that the dot would be seen as either outside or inside the occluded object's boundaries. The results reveal that local contour interpolation produces precise and accurate representations of occluded contours, and consistency across observers, but completion according to global symmetry does not. These results support a distinction between local, automatic contour interpolation processes and global processes based on recognition from partial information. (PsycINFO Database Record

Statistical 3D face shape estimation from occluding contours

This paper addresses the problem of 3D face shape approximation from occluding contours, i.e., the boundaries between the facial region and the background. To this end, a linear regression process that models the relationship between a set of 2D occluding contours and a set of 3D vertices is applied onto the corresponding training sets using Partial Least Squares. The result of this step is a regression matrix which is capable of estimating new 3D face point clouds from the out-of-training 2D Cartesian pixel positions of the selected contours. Our approach benefits from the highly correlated spaces spanned by the 3D vertices around the occluding boundaries of a face and their corresponding 2D pixel projections. As a result, the proposed method resembles dense surface shape recovery from missing data. Our technique is evaluated over four scenarios designed to investigate both the influence of the contours included in the training set and the considered number of contours. Qualitative and quantitative experiments demonstrate that using contours outperform the state of the art on the database used in this article. Even using a limited number of contours provides a useful approximation to the 3D face surface.

A discriminative approach to perspective shape from shading in uncalibrated illumination

Estimating surface normals from a single image alone is a challenging problem. Previous work made various simplifications and focused on special cases, such as having directional lighting, known reflectance maps, etc. This is problematic, however, as shape from shading becomes impractical outside the lab. We argue that addressing more realistic settings requires multiple shading cues to be combined as well as generalized to natural illumination. However, this requires coping with an increased complexity of the approach and more parameters to be adjusted. Starting from a novel large-scale dataset for training and analysis, we pursue a discriminative learning approach to shape from shading. Regression forests enable efficient pixel-independent prediction and fast learning. The regression trees are adapted to predicting surface normals by using von Mises–Fisher distributions in the leaves. Spatial regularity of the normals is achieved through a combination of spatial features, including texton as well as novel silhouette features. The proposed silhouette features leverage the occluding contours of the surface and yield scale-invariant context. Their benefits include computational efficiency and good generalization to unseen data. Importantly, they allow estimating the reflectance map robustly, thus addressing the uncalibrated setting. Our method can also be extended to handle perspective projection. Experiments show that our discriminative approach outperforms the state of the art on various synthetic and real-world datasets.

Spatially-global integration of closed, fragmented contours by finding the shortest-path in a log-polar representation

Finding the occluding contours of objects in real 2D retinal images of natural 3D scenes is done by determining, which contour fragments are relevant, and the order in which they should be connected. We developed a model that finds the closed contour represented in the image by solving a shortest path problem that uses a log-polar representation of the image; the kind of representation known to exist in area V1 of the primate cortex. The shortest path in a log-polar representation favors the smooth, convex and closed contours in the retinal image that have the smallest number of gaps. This approach is practical because finding a globally-optimal solution to a shortest path problem is computationally easy. Our model was tested in four psychophysical experiments. In the first two experiments, the subject was presented with a fragmented convex or concave polygon target among a large number of unrelated pieces of contour (distracters). The density of these pieces of contour was uniform all over the screen to minimize spatially-local cues. The orientation of each target contour fragment was randomly perturbed by varying the levels of jitter. Subjects drew a closed contour that represented the target’s contour on a screen. The subjects’ performance was nearly perfect when the jitter-level was low. Their performance deteriorated as jitter-levels were increased. The performance of our model was very similar to our subjects’. In two subsequent experiments, the subject was asked to discriminate a briefly-presented egg-shaped object while maintaining fixation at several different positions relative to the closed contour of the shape. The subject’s discrimination performance was affected by the fixation position in much the same way as the model’s.

3-D object modeling from 2-D occluding contour correspondences by opti-acoustic stereo imaging

Utilizing in situ measurements to build 3-D volumetric object models under variety of turbidity conditions is highly desirable for marine sciences. To address the ineffectiveness of feature-based structure from motion and stereo methods under poor visibility, we explore a multi-modal stereo imaging technique that utilizes coincident optical and forward-scan sonar cameras, a so-called opti-acoustic stereo imaging system. The challenges of establishing dense feature correspondences in either opti-acoustic or low-contrast optical stereo images are avoided, by employing 2-D occluding contour correspondences, namely, the images of 3-D object occluding rims. Collecting opti-acoustic stereo pairs while circling an object, matching 2-D apparent contours in optical and sonar views to construct the 3-D occluding rim, and computing the stereo rig trajectory by opti-acoustic bundle adjustment, we generate registered samples of 3-D surface in a reference coordinate system. A surface interpolation gives the 3-D object model.In addition to the key advantage of utilizing range measurements from sonar, the proposed paradigm requires no assumption about local surface curvature as traditionally made in 3-D shape reconstruction from occluding contours. The reconstruction accuracy is improved by computing both the 3-D positions and local surface normals of sampled contours. We also present (1) a simple calibration method to estimate and correct for small discrepancy from the desired relative stereo pose; (2) an analytical analysis of the degenerate configuration that enables special treatment in mapping (tall) elongated objects with dominantly vertical edges. We demonstrate the performance of our method based on the 3-D surface rendering of certain objects, imaged by an underwater opti-acoustic stereo system.

What causes the facing-the-viewer bias in biological motion?

Orthographically projected biological motion point-light displays are generally ambiguous with respect to their orientation in depth, yet observers consistently prefer the facing-the-viewer interpretation. There has been discussion as to whether this bias can be attributed to the social relevance of biological motion stimuli or relates to local, low-level stimulus properties. In the present study we address this question. In Experiment 1, we compared the facing-the-viewer bias produced by a series of four stick figures and three human silhouettes that differed in posture, gender, and the presence versus absence of walking motion. Using a paradigm in which we asked observers to indicate the spinning direction of these figures, we found no bias when participants observed silhouettes, whereas a pronounced degree of bias was elicited by most stick figures. We hypothesized that the ambiguous surface normals on the lines and dots that comprise stick figures are prone to a visual bias that assumes surfaces to be convex. The local surface orientations of the occluding contours of silhouettes are unambiguous, and as such the convexity bias does not apply. In Experiment 2, we tested the role of local features in ambiguous surface perception by adding dots to the elbows and knees of silhouettes. We found biases consistent with the facing directions implied by a convex body surface. The results unify a number of findings regarding the facing-the-viewer bias. We conclude that the facing-the-viewer bias is established at the level of surface reconstruction from local image features rather than on a semantic level.

Extracting curve-skeletons from digital shapes using occluding contours

Curve-skeletons are compact and semantically relevant shape descriptors, able to summarise both topology and pose of a wide range of digital objects. Most of the state-of-the-art algorithms for their computation rely on the type of geometric primitives used and sampling frequency. In this paper, we introduce a formally sound and intuitive definition of a curve-skeleton, then we propose a novel method for skeleton extraction that relies on the visual appearance of the shapes. To achieve this result, we inspect the properties of occluding contours, showing how information about the symmetry axes of a 3D shape can be inferred from a small set of its planar projections. The proposed method is fast, insensitive to noise and resolution, capable of working with different shape representations, and easy to implement.

Artificial vision by multi-layered neural networks: Neocognitron and its advances

The neocognitron is a neural network model proposed by Fukushima (1980). Its architecture was suggested by neurophysiological findings on the visual systems of mammals. It is a hierarchical multi-layered network. It acquires the ability to robustly recognize visual patterns through learning. Although the neocognitron has a long history, modifications of the network to improve its performance are still going on. For example, a recent neocognitron uses a new learning rule, named add-if-silent, which makes the learning process much simpler and more stable. Nevertheless, a high recognition rate can be kept with a smaller scale of the network. Referring to the history of the neocognitron, this paper discusses recent advances in the neocognitron. We also show that various new functions can be realized by, for example, introducing top-down connections to the neocognitron: mechanism of selective attention, recognition and completion of partly occluded patterns, restoring occluded contours, and so on.