Disparity Space Research Articles

Dense Stereo Matching Method Based on Local Affine Model

A new method for constructing an accurate disparity space image and performing an efficient cost aggregation in stereo matching based on local affine model is proposed in this paper. The key algorithm includes a new self-adapting dissimilarity measurement used for calculating the matching cost and a local affine model used in cost aggregation stage. Different from the traditional region-based methods, which try to change the matching window size or to calculate an adaptive weight to do the aggregation, the proposed method focuses on obtaining the efficient and accurate local affine model to aggregate the cost volume while preserving the disparity discontinuity. Moreover, the local affine model can be extended to the color space. Experimental results demonstrate that the proposed method is able to provide subpixel precision disparity maps compared with some state-of-the-art stereo matching methods.

1A2-I06 全方位ステレオ視の距離誤差を考慮したICPによるロボットの相対位置姿勢推定(移動ロボットの自己位置推定と地図構築(1))

Range-based ego-motion estimation method, which is so-called "scan-matching", has an important role in robotics. However, iterative closest points (ICP), which is basic scan-matching method, assumes that the error of obtained range data is constant. This degrades estimation results when the robot uses sensors whose error is not constant, i.e.stereo vision. In this paper, we propose a ICP-based ego-motion estimation method considering range ambiguity of the stereo vision. The key idea is comparing obtained 3D points in not Euclidean space but disparity space. We implement the method to the robot mounting omnidirectional stereo vision. Simulation experiments are conducted to show the effectiveness of our method.

Collaboration in the Archive: Finding, Shaping, and Disseminating Stories from a Missionary Writer's Network

Sarah Ruffing Robbins and Ann Ellis Pullen's Nellie Arnott Writings on Angola, 19051913: Missionary Narratives Linking Africa and America recently earned Honorable Mention for SSAWW Edition Award, which was established to honor excellence recovery of American women writers. A familiar portrait of archival research depicts a lonely task performed quiet, musty spaces, as solitary researcher seeks to commune with past while interpreting materials cataloged long ago by some unnamed, unseen collector. For instance, Jill Lepore's romantic and compelling account of Historians Who Love Too Much, an opening image positions scholar-author sitting alone in crisply air-conditioned Special Collections reading room at Amherst College Library, stroking Noah Webster's hair, with sole relationship highlighted that between a single scholar and her long-dead subject (129). In fact, however, as our own shared experiences attest, most archival scholarship is actually quite collaborative. Accordingly, this essay we revisit multiple dimensions of collaboration involved our critical edition of Nellie Arnott's Writings on Angola, 1905-1913 and still supporting our ongoing work on Arnott's authorship within context of larger cultural movements. Undergirding this analysis are contributions from a growing chorus of voices describing social nature of archival studies, including several essay collections and earlier accounts published Legacy. (1) By joining such reports, we hope to encourage others who are beginning archival projects to embrace collaboration as a powerful strategy for recovering women writers of past. Taking a self-consciously feminist collaborative approach has been particularly appropriate for studying Arnott, since she was herself embedded a highly gendered cooperative enterprise through service as a Congregational missionary Angola (then Portuguese West Africa) for American Board of Commissioners for Foreign Missions (AscFm). Arnott was part of a network of women, at home and abroad, participating a worldwide movement. Thus, her personal archive, scattered across disparate spaces and contained a variety of textual forms, tells both a personal and a social story. Like research subjects recommended Legacy's 2002 Archive Survival Guide, Arnott would not be classified today as a famous figure, even though she was well known to niche readership of Congregational mission magazines. In line with pattern described by Jennifer Bernhardt Steadman and her coauthors Elizabeth Engelhardt, Frances Smith Foster, and Laura Micham, Arnott's story is more of interrelations and than of pioneering (232, 231), so recovering her writing affirms their premise that feminist scholars should resist research's traditional emphasis on the 'only' or 'first' their fields as too often reinscribing women's exclusion (231). Arnott's experiences follow that pattern, for to extent they are retrievable they offer a window into larger trends associated with whole groups of women. She was, a word, typical. Yet, ironically, very features that make Arnott an ideal representative writer, operating a broader social network, sometimes made it difficult for us to move forward with our first publishing project about her. Fortunately, we were able to counter those occasionally discouraging forces through collaboration, as has recently been urged by archival researchers such as Lisa Mas-trangelo and Barbara L'Eplattenier. (2) Our collaboration took different forms at progressive stages our research. Initially, finding our subject linked our own long-standing research partnership to a more serendipitous yet crucial connection with an experienced archival librarian. Then, digging deeper into our subject's personal past and its connections with institutional histories involved us as collaborators with other scholars, students, and members of Arnott's family. …

A Improved Stereo Matching Fast Algorithm Based on Dynamic Programming

Compared with the local algorithm in stereo matching, the high quality disparity space image is calculated by the global algorithm, which is difficult to use in practical application for its long computation time. The dynamic programming is one of the global algorithms with a fast matching speed, but it has strip blemish in matching result. In this paper, a new dynamic programming based method is proposed to accelerate the matching speed and improve the matching quality. Firstly, the color feature of two images are calculated using the Laplacians of Gaussian pyramid algorithm, and the color feature of the image pair obtained are matched. Secondly, the matching points are taken as the ground control points of the scan line, which is cut into several short line segments. Finally, all line segments are matched to obtain the disparity of the scan line. The experimental results show that the matching speed is accelerated greatly with improved disparity image quality

Multiresolution energy minimisation framework for stereo matching

Global optimisation algorithms for stereo dense depth map estimation have demonstrated how to outperform other stereo algorithms such as local methods or dynamic programming. The energy minimisation framework, using Markov random fields model and solved using graph cuts or belief propagation, has especially obtained good results. The main drawback of these methods is that, although they achieve accurate reconstruction, they are not suited for real-time applications. Subsampling the input images does not reduce the complexity of the problem because it also reduces the resolution of the output in the disparity space. Nonetheless, some real-time applications such as navigation would tolerate the reduction of the depth map resolutions (width and height) while maintaining the resolution in the disparity space (number of labels). In this study a new multiresolution energy minimisation framework for real-time robotics applications is proposed where a global optimisation algorithm is applied. A reduction by a factor R of the final depth map's resolution is considered and a speed of up to 50 times has been achieved. Using high-resolution stereo pair input images guarantees that a high resolution on the disparity dimension is preserved. The proposed framework has shown how to obtain real-time performance while keeping accurate results in the Middlebury test data set.

A Visibility-Based Approach for Occupancy Grid Computation in Disparity Space

Occupancy grids are a very convenient tool for environment representation in robotics. This paper will detail a novel approach for computing occupancy grids from stereo vision and show its application to intelligent vehicles. In the proposed approach, occupancy is initially computed directly in the stereoscopic sensor's disparity space. The calculation formally accounts for the detection of obstacles and road pixels in disparity space, as well as partial occlusions in the scene. In the second stage, this disparity-space occupancy grid is transformed into a Cartesian space occupancy grid to be used by subsequent applications. This transformation includes spatial and temporal filtering. The proposed method is designed to easily be processed in parallel. Consequently, we chose to implement it on a graphics processing unit, which allows real-time processing for demanding applications. In this paper, we present this method, and we propose an application to the problem of perception in a road environment. Results are presented with real road data, qualitatively comparing this approach with other methods.

Accurate real-time neural disparity MAP estimation with FPGA

We propose in this paper a new method for real-time dense disparity map computing using a stereo pair of rectified images. Based on the neural network and Disparity Space Image (DSI) data structure, the disparity map computing consists of two main steps: initial disparity map estimation by combining the neuronal network and the DSI structure, and its refinement. Four improvements are introduced so that an accurate and fast result will be reached. The first one concerns the proposition of a new strategy in order to optimize the computation time of the initial disparity map. In the second one, a specific treatment is proposed in order to obtain more accurate disparity for the neighboring pixels to boundaries. The third one, it concerns the pixel similarity measure for matching score computation and it consists of using in addition to the traditional pixel intensities, the magnitude and orientation of the gradients providing more accuracy. Finally, the processing time of the method has been decreased consequently to our implementation of some critical steps on FPGAs. Experimental results on real datasets are conducted and a comparative evaluation of the obtained results relative to the state-of-art methods is presented.

Improving sub-pixel accuracy for long range stereo

Dense stereo algorithms are able to estimate disparities at all pixels including untextured regions. Typically these disparities are evaluated at integer disparity steps. A subsequent sub-pixel interpolation often fails to propagate smoothness constraints on a sub-pixel level. We propose to increase the sub-pixel accuracy in low-textured regions in four possible ways: First, we present an analysis that shows the benefit of evaluating the disparity space at fractional disparities. Second, we introduce a new disparity smoothing algorithm that preserves depth discontinuities and enforces smoothness on a sub-pixel level. Third, we present a novel stereo constraint (gravitational constraint) that assumes sorted disparity values in vertical direction and guides global algorithms to reduce false matches, especially in low-textured regions. Finally, we show how image sequence analysis improves stereo accuracy without explicitly performing tracking. Our goal in this work is to obtain an accurate 3D reconstruction. Large-scale 3D reconstruction will benefit heavily from these sub-pixel refinements. Results based on semi-global matching, obtained with the above mentioned algorithmic extensions are shown for the Middlebury stereo ground truth data sets. The presented improvements, called ImproveSubPix, turn out to be one of the top-performing algorithms when evaluating the set on a sub-pixel level while being computationally efficient. Additional results are presented for urban scenes. The four improvements are independent of the underlying type of stereo algorithm.

Accurate Real-Time Disparity Map Computation Based on Variable Support Window

We propose in this paper a real-time dense stereo matching algorithm using variable support window for disparity map computing. Basic real-time local algorithms relying on a fixed and rectangular correlation window suffer from the difficulty for window-based methods lies in determining the best window shape and size for each pixel. This work proposes a novel local approach using a combination of the DSI (Disparity Space Image) structure and gradient information. Two improvements are introduced so that an accurate and fast result will be reached. The first one concerns the proposition of a new strategy in order to optimize the computation time of the initial disparity map. The second one, concerns the pixel similarity measure for matching score computation and it consists to use in addition to the traditional pixel intensities, the magnitude and orientation of the gradients providing more accuracy. Experimental results on real data sets are conducted and a comparative evaluation of the obtained results relatively to the state-of-art methods is presented.

Design and hardware implementation of a stereo-matching system based on dynamic programming

A new real-time stereo system is presented based on a hardware implementation of an efficient Dynamic Programming algorithm. A simple state-machine calculates the cost-matrix along the diagonal of the 2-D disparity space for each epipolar pair of image scan-lines. Minimum transition costs are stored in embedded RAM and are used to backtrack disparities at clock rate. All calculations are within a pre-determined slice of the cost plane, representing the useful disparity range. The system is designed as a VHDL library component and is implemented as a SoC in a medium-capacity Field Programmable Gate Array chip. It can process stereo-pairs in full VGA resolution at a rate of 25 Mpixels/s and produces 8-bit dense disparity maps within a range of disparities up to 65 pixels. The design is evaluated comparing to ground truth and in terms of resource usage. It is also compared to a software implementation of the Dynamic Programming algorithm and to other FPGA-based stereo systems.

Region growing stereo matching method for 3D building reconstruction

This paper presents a new stereo matching algorithm based on region growing algorithm. To avoid visiting the entire disparity space, an algorithm has been proposed that greedily grow the corresponding patches from a given set of reliable seed correspondences. The proposed algorithm is using regions as matching primitives and defines the corresponding region energy functional for matching by utilising matching technique. Initially, a pre-matching technique i.e., Harris corner detector is used to obtain the initial matching points called, seed points. Secondly, a local window-based matching method is used to determine the disparity estimate from an initial set of seeds. Finally mode filter technique fills the gap of insufficiency of the sparse disparity for whole surface reconstruction. The results show that the proposed scheme is reliable, accurate and robust to high resolution aerial images. The proposed approach is very useful in 3D city model as buildings are the most important objects in producing a 3D city model.

An optimal time–space algorithm for dense stereo matching

The increasing demand for higher resolution images and higher frame rate videos will always pose a challenge to computational power when performance is required to solve the stereo-matching problem in 3D reconstruction applications. Therefore, the use of asymptotic analysis is necessary to measure the time and space performance of stereo-matching algorithms regardless of the size of the input and of the computational power available. In this paper, we survey several classic stereo-matching algorithms with regard to time---space complexity. We also report running time experiments for several algorithms that are consistent with our complexity analysis. We present a new dense stereo-matching algorithm based on a greedy heuristic path computation in disparity space. A procedure which improves disparity maps in depth discontinuity regions is introduced. This procedure works as a post-processing step for any technique that solves the dense stereo-matching problem. We prove that our algorithm and post-processing procedure have optimal O(n) time---space complexity, where n is the size of a stereo image. Our algorithm performs only a constant number of computations per pixel since it avoids a brute force search over the disparity range. Hence, our algorithm is faster than real-time techniques while producing comparable results when evaluated with ground-truth benchmarks. The correctness of our algorithm is demonstrated with experiments in real and synthetic data.

Dense stereo matching by region growing based on disparity space sampling

Through the research of the existing region growing method,the authors presented an improved algorithm,which was unlike the existing region growing algorithm that depended on the accuracy of the seeds.The algorithm can continue growing on the smooth region and repetitive patterns region.Firstly,obtain the initial set of seeds in the disparity space by matching the Harris points.Then build the newly growing method in the disparity space.The results demonstrate it can trace out the disparity space.In the growing process,it can rectify the wrong matching automatically,and guarantee matching accuracy and correctness in the smooth and repetitive pattern region.

A portable stereo vision system for whole body surface imaging

This paper presents a whole body surface imaging system based on stereo vision technology. We have adopted a compact and economical configuration which involves only four stereo units to image the frontal and rear sides of the body. The success of the system depends on a stereo matching process that can effectively segment the body from the background in addition to recovering sufficient geometric details. For this purpose, we have developed a novel sub-pixel, dense stereo matching algorithm which includes two major phases. In the first phase, the foreground is accurately segmented with the help of a predefined virtual interface in the disparity space image, and a coarse disparity map is generated with block matching. In the second phase, local least squares matching is performed in combination with global optimization within a regularization framework, so as to ensure both accuracy and reliability. Our experimental results show that the system can realistically capture smooth and complete whole body shapes with high accuracy.

"In a time before nomenclature was and each was all": Blood Meridian 's Neomythic West and the Heterotopian Zone

Blood Meridian exists in a world, or worlds, apart from any we know, present or past, a world(s) we envision as emerging forth from the mists of a recent current of revisionist histories. One of these worlds creates a new twentieth-century history for an old nineteenth-century world, in which the unimaginable violence of our time might be reconciled with that of the past; another world or zone of Blood Meridian acts as a retelling of western expansion under the ostensibly swaying flag of a colonial manifest destiny. McCarthy's novel is history, but it is also myth, both narratively and stylistically, although the myth he creates runs concurrent and, importantly, through the realities of a revisionist West. Beyond the violence, the most troubling and unsettling feature of Blood Meridian is the creation of these multiple fictional zones, reproduced simultaneously and unbearably; McCarthy's text creates a postmodern heterotopian zone in which multiple disparate spaces come to exist impossibly together, leaving him to dispatch a gang of historical figures into a neomythic, postapocalyptic, and postmodern unreality that is and is not Texas, that is and is not the West.

Hierarchical adaptive stereo matching algorithm for obstacle detection with dynamic programming

An adaptive weighted stereo matching algorithm with multilevel and bidirectional dynamic programming based on ground control points (GCPs) is presented. To decrease time complexity without losing matching precision, using a multilevel search scheme, the coarse matching is processed in typical disparity space image, while the fine matching is processed in disparity-offset space image. In the upper level, GCPs are obtained by enhanced volumetric iterative algorithm enforcing the mutual constraint and the threshold constraint. Under the supervision of the highly reliable GCPs, bidirectional dynamic programming framework is employed to solve the inconsistency in the optimization path. In the lower level, to reduce running time, disparity-offset space is proposed to efficiently achieve the dense disparity image. In addition, an adaptive dual support-weight strategy is presented to aggregate matching cost, which considers photometric and geometric information. Further, post-processing algorithm can ameliorate disparity results in areas with depth discontinuities and related by occlusions using dual threshold algorithm, where missing stereo information is substituted from surrounding regions. To demonstrate the effectiveness of the algorithm, we present the two groups of experimental results for four widely used standard stereo data sets, including discussion on performance and comparison with other methods, which show that the algorithm has not only a fast speed, but also significantly improves the efficiency of holistic optimization.

Music before the nation, music after nationalism

There are few ritual events in which music magnifies the spectacle of nationalism more grandly than a parade. For spectators and marchers alike, a parade unfolds like a panorama in real time, exposing the nation and its symbols for all to see and hear, to review and remember. Past, present, and future are coeval, the sound of one band or float or collective of marchers fading in and fading out, enacting the very transience of history and its movable stages. I love parades, having spent my youth in the rural American Midwest marching up and down the streets of the United States and Canada, holding forth on an instrument rather badly suited for my local marching band, the French horn.2 Parades may well be one of the most universal sites of what Michael Herzfeld calls ‘cultural intimacy’ (Herzfeld 1997), symbols of cultural practices that in themselves may reveal parochially devalued cultural sameness, but that together create a collective sense of cultural value. As performance, parades, above all because of their reliance on sounding their passage through disparate public spaces to mark important moments, similarly embody the global moment of sounding selfness that Benedict Anderson calls ‘unisonance’, an occasion, that is, for ‘unisonality, for the echoed physical realization of the imagined community’ of the nation-state (Anderson 2006, 145).

Multistage Branch-and-Bound Merging for Planar Surface Segmentation in Disparity Space

An iterative split-and-merge framework for the segmentation of planar surfaces in the disparity space is presented. Disparity of a scene is modeled by approximating various surfaces in the scene to be planar. In the split phase, the number of planar surfaces along with the underlying plane parameters is assumed to be known from the initialization or from the previous merge phase. Based on these parameters, planar surfaces in the disparity image are labeled to minimize the residuals between the actual disparity and the modeled disparity. The labeled planar surfaces are separated into spatially continuous regions which are treated as candidates for the merging that follows. The regions are merged together under a maximum variance constraint while maximizing the merged area. A multistage branch-and-bound algorithm is proposed to carry out this optimization efficiently. Each stage of the branch-and-bound algorithm separates a planar surface from the set of spatially continuous regions. The multistage merging estimates the number of planar surfaces and their labeling. The splitting and the multistage merging is repeated till convergence is reached or satisfactory results are achieved. Experimental results are presented for variety of stereo image data.

Real-time joint disparity and disparity flow estimation on programmable graphics hardware

Disparity flow depicts the 3D motion of a scene in the disparity space of a given view and can be considered as view-dependent scene flow. A novel algorithm is presented to compute disparity maps and disparity flow maps in an integrated process. Consequently, the disparity flow maps obtained helps to enforce the temporal consistency between disparity maps of adjacent frames. The disparity maps found also provides the spatial correspondence information that can be used to cross-validate disparity flow maps of different views. Two different optimization approaches are integrated in the presented algorithm for searching optimal disparity values and disparity flows. The local winner-take-all approach runs faster, whereas the global dynamic programming based approach produces better results. All major computations are performed in the image space of the given view, leading to an efficient implementation on programmable graphics hardware. Experimental results on captured stereo sequences demonstrate the algorithm’s capability of estimating both 3D depth and 3D motion in real-time. Quantitative performance evaluation using synthetic data with ground truth is also provided.

Hardware principles for the design of a stereo-matching state machine based on dynamic programming

This paper presents basic design principles for hardware implementation of a two-pass stereo-matching algorithm based on dynamic programming. For the first-pass a state-machine is proposed for the recursive calculation of the cost-function. The state-machine works along the diagonal of a 2-D disparity space for each epipolar pair of image scan-lines. On-chip local RAM stores tags that denote the minimum transition cost to every point in the disparity space among possible costs from all three neighboring points. All calculations are within a pre-determined useful disparity range. For the second pass, hardware rules are presented that produce the correct disparity per pixel, by backtracking stored cost values. Hard- ware stages are structured along a fully parallel pipeline, that outputs disparities in step with the input serial pixel stream at clock rate. Abstract This paper presents basic design principles for hardware implementation of a two-pass stereo-matching algorithm based on dynamic programming. For the first-pass a state-machine is proposed for the recursive calculation of the cost-function. The state-machine works along the diagonal of a 2-D disparity space for each epipolar pair of image scan-lines. On-chip local RAM stores tags that denote the minimum transition cost to every point in the disparity space among possible costs from all three neighboring points. All calculations are within a pre-determined useful disparity range. For the second pass, hardware rules are presented that produce the correct disparity per pixel, by backtracking stored cost values. Hard- ware stages are structured along a fully parallel pipeline, that outputs disparities in step with the input serial pixel stream at clock rate. Abstract This paper presents basic design principles for hardware implementation of a two-pass stereo-matching algorithm based on dynamic programming. For the first-pass a state-machine is proposed for the recursive calculation of the cost-function. The state-machine works along the diagonal of a 2-D disparity space for each epipolar pair of image scan-lines. On-chip local RAM stores tags that denote the minimum transition cost to every point in the disparity space among possible costs from all three neighboring points. All calculations are within a pre-determined useful disparity range. For the second pass, hardware rules are presented that produce the correct disparity per pixel, by backtracking stored cost values. Hard- ware stages are structured along a fully parallel pipeline, that outputs disparities in step with the input serial pixel stream at clock rate.