Model-based Computer Vision Research Articles

Learning Fluid Flow Visualizations From In-Flight Images With Tufts

To better understand fluid flows around aerial systems, strips of wire or rope, widely known as tufts, are often used to visualize the local flow direction. This paper presents a computer vision system that automatically extracts the shape of tufts from images, which have been collected during real flights of a helicopter and an unmanned aerial vehicle (UAV). As images from these aerial systems present challenges to both the model-based computer vision and the end-to-end supervised deep learning techniques, we propose a semantic segmentation pipeline that consists of three uncertainty-based modules namely, (a) active learning for object detection, (b) label propagation for object classification, and (c) weakly supervised instance segmentation. Overall, these probabilistic approaches facilitate the learning process without requiring any manual annotations of semantic segmentation masks. Empirically, we motivate our design choices through comparative assessments and provide real-world demonstrations of the proposed concept, for the first time to our knowledge. The project website can be accessed via the link: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://sites.google.com/view/tuftrecognition/</uri> .

Learning human shape model from multiple databases with correspondence considering kinematic consensus

In various applications of computer graphics and model-based computer vision, a human shape model cannot only model the kinematic properties of a subject to drive the mesh into various postures, but it can also be utilized to parameterize the shape variations across individuals. It is of great benefit to improve the diversity of the training databases by learning the model from multiple databases, once the correspondences among scans of these databases can be achieved. To accomplish this goal, we proposed a framework to match the scans from multiple databases, using the assistance of kinematic properties, to compute the correspondences. The resulting correspondence is accurate, robust, capable of handling scan incompletion, and is homogeneous across shapes and postures. In our approach, we start with evaluating how a correspondence, which is achieved via minimizing the deformation energy, agrees with the kinematic properties, and then, we jointly fit the source scans to the target scans to derive the correspondences between the databases. The extensive results show that our approach can generate a faithful correspondence even in extreme cases, without carefully selecting the deformation factors and markers. We also developed a method, with which a commendable and predictable result can be synthesized, to control the rendered shape in an intuitive way.

A system for automated disassembly of snap-fit covers

This paper reports on a prototype automated system for the disassembly of batteries from a family of electronic devices whose plastic, snap-fit covers house AA or AAA batteries, such as remote controls and calculators. Included in the development of the prototype system was the design of a disassembly tool that uses three force-sensing resistors to provide force feedback information. A pneumatically actuated vacuum gripper and electromagnet system was also developed for recovering the snap-fit cover and batteries once they were released by the disassembly tool. The disassembly module was mounted on the tool head of a three-axis translational motion robot, and a Visual Basic application was developed to interface and control the robot with a Galil digital motion controller. A model-based computer vision application was also developed in Visual C++ using a Kinect sensor and the Open Source Computer Vision library to identify and localize the electronic device placed on the disassembly robot. Using the information gathered by the model-based computer vision application, the robot was able to use the disassembly tool module to perform the necessary disassembly operations to remove the device’s snap-fit cover and batteries. The design of such a disassembly system could aid the future development of fully autonomous disassembly systems that can handle a broader range of electronic products.

Modelling the 3D pose of a human arm and the shoulder complex utilising only two parameters

In model-based computer vision it is necessary to have a geometric model of the object the pose of which is being estimated. In this paper a very compact model of the human shoulder complex and arm is presented. First an investigation of the anatomy

Modelling the 3D pose of a human arm and the shoulder complex utilising only two parameters

In model-based computer vision it is necessary to have a geometric model of the object the pose of which is being estimated. In this paper a very compact model of the human shoulder complex and arm...

Convergent matching for model-based computer vision

We consider matching in model-based computer vision as a converging discrete iteration and give a basis for examining the convergence as the movement of the working point in a lattice. Because the matching is non-deterministic we discuss convergence in terms of completing sub-problems within a time slot. This form of low-level scheduling avoids effectively unlimited trials of sub-graphs, a phenomenon that we call the NP-trap. We define high-level scheduling as the need to test each reference class at least once and thereafter focus attention on the most promising candidates. Examples show the bounding of matching time with a time slot and focusing of attention guided by a figure of merit.

Decision Trees for Geometric Models

A fundamental problem in model-based computer vision is that of identifying which of a given set of geometric models is present in an image. Considering a "probe" to be an oracle that tells us whether or not a model is present at a given point, we study the problem of computing efficient strategies ("decision trees") for probing an image, with the goal to minimize the number of probes necessary (in the worst case) to determine which single model is present. We show that a ⌈l g k⌉ height binary decision tree always exists for k polygonal models (in fixed position), provided (1) they are non-degenerate (do not share boundaries) and (2) they share a common point of intersection. Further, we give an efficient algorithm for constructing such decision tress when the models are given as a set of polygons in the plane. We show that constructing a minimum height tree is NP-complete if either of the two assumptions is omitted. We provide an efficient greedy heuristic strategy and show that, in the general case, it yields a decision tree whose height is at most ⌈l g k⌉ times that of an optimal tree. Finally, we discuss some restricted cases whose special structure allows for improved results.

Similarity Hashing: A Computer Vision Solution to the Inverse Problem of Linear Fractals

The difficult task of finding a fractal representation of an input shape is called the inverse, problem of fractal geometry. Previous attempts at solving this problem have applied techniques from numerical minimization, heuristic search and image compression. The most appropriate domain from which to attack this problem is not numerical analysis nor signal processing, but model-based computer vision. Self-similar objects cause an existing computer vision algorithm called geometric hashing to malfunction. Similarity hashing capitalizes on this observation to not only detect a shape's morphological self-similarity but also find the parameters of its self-transformations.

Geometric pattern matching under Euclidean motion

Given two planar sets A and B, we examine the problem of determining the smallest ϵ such that there is a Euclidean motion (rotation and translation) of A that brings each member of A within distance ϵ of some member of B. We establish upper bounds on the combinatorial complexity of this subproblem in model-based computer vision, when the sets A and B contain points, line segments, or (filled-in) polygons. We also show how to use our methods to substantially improve on existing algorithms for finding the minimum Hausdorff distance under Euclidean motion.

Parameterized families of polynomials for bounded algebraic curve and surface fitting

Interest in algebraic curves and surfaces of high degree as geometric models or shape descriptors for different model-based computer vision tasks has increased in recent years, and although their properties make them a natural choice for object recognition and positioning applications, algebraic curve and surface fitting algorithms often suffer from instability problems. One of the main reasons for these problems is that, while the data sets are always bounded, the resulting algebraic curves or surfaces are, in most cases, unbounded. In this paper, the authors propose to constrain the polynomials to a family with bounded zero sets, and use only members of this family in the fitting process. For every even number d the authors introduce a new parameterized family of polynomials of degree d whose level sets are always bounded, in particular, its zero sets. This family has the same number of degrees of freedom as a general polynomial of the same degree. Three methods for fitting members of this polynomial family to measured data points are introduced. Experimental results of fitting curves to sets of points in R/sup 2/ and surfaces to sets of points in R/sup 3/ are presented.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Joint solution of low, intermediate, and high-level vision tasks by evolutionary optimization: Application to computer vision at low SNR

Methods for conducting model-based computer vision from low-SNR (=/<1 dB) image data are presented. Conventional algorithms break down in this regime due to a cascading of noise artifacts, and inconsistencies arising from the lack of optimal interaction between high- and low-level processing. These problems are addressed by solving low-level problems such as intensity estimation, segmentation, and boundary estimation jointly (synergistically) with intermediate-level problems such as the estimation of position, magnification, and orientation, and high-level problems such as object identification and scene interpretation. This is achieved by formulating a single objective function that incorporates all the data and object models, and a hierarchy of constraints in a Bayesian framework. All image-processing operations, including those that exploit the low and high-level variables to satisfy multi-level pattern constraints, result directly from a parallel multi-trajectory global optimization algorithm. Experiments with simulated low-count (7-9 photons/pixel) 2-D Poisson images demonstrate that compared to non-joint methods, a joint solution not only results in more reliable scene interpretation, but also a superior estimation of low-level imaging variables. Typically, most object parameters are estimated to within a 5% accuracy even with overlap and partial occlusion.

Extracting Geometric Primitives

Extracting geometric primitives is an important task in model-based computer vision. The Hough transform is the most common method of extracting geometric primitives. Recently, methods derived from the field of robust statistics have been used for this purpose. We show that extracting a single geometric primitive is equivalent to finding the optimum value of a cost function which has potentially many local minima. Besides providing a unifying way of understanding different primitive extraction algorithms, this model also shows that for efficient extraction the true global minimum must be found with as few evaluations of the cost function as possible. In order to extract a single geometric primitive we choose a number of minimal subsets randomly from the geometric data. The cost function is evaluated for each of these, and the primitive defined by the subset with the best value of the cost function is extracted from the geometric data. To extract multiple primitives, this process is repeated on the geometric data that do not belong to the primitive. The resulting extraction algorithm can be used with a wide variety of geometric primitives and geometric data. It is easily parallelized, and we describe some possible implementations on a variety of parallel architectures. We make a detailed comparison with the Hough transform and show that it has a number of advantages over this classic technique.

Employing sensor repositioning to refine spatial reasoning in an industrial robotic environment

Industrial and space applications present environments in which it is possible, and in fact desirable to solve robotic problems using a model-based approach. From a sensory standpoint, the reasons for employing knowledge about objects to be manipulated are twofold. First, such knowledge permits high-level expectation driven reasoning as opposed to low-level data driven searches for primitive features. This is advantageous since purely data driven feature extraction is typically undirected and the search space is unconstrained. The second reason is that expectation driven reasoning can exploit knowledge derived from features that have already been found, thus expediting subsequent searches. Conversely, however, there is a rigid requirement to specify the geometry and kinematics for object models about which reasoning is to occur. This paper describes a model-based computer vision system that has been coupled with a robot arm for the purpose of accurately reasoning about entities on a reconfigurable task panel. The final goal of the integrated system is to be able to manipulate substructures such as hinged doors and laterally translatable drawers using computer vision as the primary sensory input. This overall objective is accomplished by first locating the camera at a position where it can view the entire panel such that an initial worksite registration can be computed. Next, an approximation for each substructure's spatial configuration is determined by employing kinematic and geometric knowledge using a generate and test paradigm. This step is followed by repositioning the robotically mounted camera to a location and orientation that is preferable for further, more accurate spatial inferences. The camera is automatically recalibrated at the new location and a final move is made to grasp and open or close the specified substructure. The primary advantage of the approach is that final moves can be achieved within a few millimeters of ideal target locations, even when target objects are initially viewed from locations which initially produce poor pose estimation results, since object pose estimations are successively refined as the result of information obtained at new viewpoints. In addition to describing the mechanisms and algorthms that are utilized in the research, a comparison of the accuracy of the results obtained from both non-repositionable and repositionable sensor based spatial reasoning systems is presented.

Tristage Hough transform for multiple ellipse extraction

The importance of rapid and accurate extraction of ellipses in images for model-based computer vision systems has long been recognised. The Hough transform technique has established itself as a robust method for shape extraction, however, its application to ellipses has been restricted by excessive computational and storage requirements. Direct extension of the Hough technique to ellipses is particularly demanding since a five-dimensional accumulator array is required. This paper presents a novel tristage technique for ellipse extraction based on decomposing the problem into sequentially executed stages. Candidate ellipse centres are determined using an improved centre-finding procedure and novel Hough-based procedures extract the remaining parameters in two further stages. Although multistage parameter extraction is not new, many of the problems that exist in previous schemes are resolved. In the new technique, the dimensionality of the accumulator arrays used does not exceed two, while their ranges are predefined. Memory savings are therefore significant. Concentric ellipses of the same orientation are shown to be extracted with little extra computational demand. Two extensions of the tristage technique to multiple ellipse extraction are presented and compared. When applied to real images, the new technique extracted multiple, concentric ellipses reliably and with reduced complexity.

A model-based computer vision system for recognizing handwritten ZIP codes

This paper describes a recognition system for handwritten ZIP Codes currently under development at the Environmental Research Institute of Michigan (ERIM). Included within this system are techniques for preprocessing address block images, locating ZIP Codes, splitting touching characters, and identifying handwritten numerals. These techniques rely on mathematical morphology-based image processing and on hierarchical matching of object models to symbolic image representations. The image processing uses adaptive filtering, thresholding, and skeletonizing to create binary and state-labeled images. The matching process uses these images and extensively developed handwritten digit models to identify ZIP Codes. The end-to-end system has been tested on 500 randomly selected address block images. The system correctly recognized a large portion of the ZIP Codes in the test images (45.0%), and incorrectly classified a very low percentage of isolated handwritten digits (0.9%). Overall performance continues to be improved through incremental digit model refinement.

HANDX: a model-based system for automatic segmentation of bones from digital hand radiographs

The authors detail the design and implementation of HANDX, a model-based computer vision system used in the domain of medical image processing. Given a digitized hand radiograph, HANDX segments out specific bones and measures particular parameters of the bones, without requiring specific characterization of noise variations in background contrast and anatomical differences which arise from patient variation. Observer variability is reduced by the system, and the resulting measurement may be useful for detecting short-term skeletal growth abnormalities in children and may additional clinical applications. The overall system is modularized into three stages: preprocessing, segmentation, and measurement. In the preprocessing state model-based histogram modification is used to normalize the radiograph. The histogram model is based on the physics of the imaging process. The segmentation stage finds and outlines specific bones using domain-dependent and domain-independent knowledge of hand anatomy and physiology and image edges. The measurement stage obtains clinically useful quantitative parameters from the segmented image.

Geometric reasoning for computer vision

Abstract Some general principles are formulated about geometric reasoning in the context of model-based computer vision. Such reasoning tries to draw inferences about the spatial relationships between objects in a scene based on the fragmentary and uncertain geometric evidence provided by an image. The paper discusses the tasks the reasoner is to perform for the vision program, the basic competences it requires and the various methods of implementation. In the section on basic competences, some specifications of the data types and operations needed in any geometric reasoner are given.