Computational theories of structure-from-motion ( Ullman, The Interpretation of Visual Motion, MIT Press, 1979 ) and stereo vision ( Marr and Poggio, Proc. R. Soc. London Ser. B 204, 1979, 301–328 ) only specify the computation of three-dimensional surface information at particular points in the image. Yet, the visual perception is clearly of complete surfaces. To account for this, a computational theory of the interpolation of surfaces from visual information was presented in Grimson. ( From Images to Surfaces: A Computational Study of the Human Early Visual System, MIT Press, 1981 ; A Computational Theory of Visual Surface Interpolation, MIT Artificial Intelligence Lab Memo, No. 613, 1981 ; and Philos. Trans. R. Soc. London Ser. B 298, 1982, 395–427 ). The problem is constrained by the fact that the surface must agree with the information from stereo or motion correspondence, and not vary radically between these points. Using the image irradiance equation ( Horn, MIT Project MAC Tech. Rep. MACTR-79, 1970; The Psychology of Computer Vision, McGraw-Hill, 1975 ; and Artif. Intell. 8, 1977, 201–231 ), an explicit form of this surface consistency constraint can be derived ( Grimson, MIT Artificial Intelligence Lab Memo, No. 646, 1981). To determine which of two possible surfaces is more consistent with the surface consistency constraint, one must be able to compare the two surfaces. To do this, a functional from the space of possible functions to the real numbers is required. In this way, the surface most consistent with the visual information will be that which minimizes the functional. In Grimson, a set of conditions was derived which ensures that the functional has a unique minimal surface. Based on these conditions, a number of possible functionals were proposed. In Brady and Horn (MIT Artificial Intelligence Lab Memo, No. 654, 1981), it was shown that this set of possible functionals forms a vector space, spanned by the functional of quadratic variation and the functional of the square Laplacian. Analytic arguments were given in Grimson to support the choice of the quadratic variation as the functional whose minimal surface is the “best” interpolation of the known points. In this paper, algorithms for computing the minimal surface are derived. Using this implementation of the computational theory derived in Grimson the differences between minimal surfaces computed using quadratic variation and those computed using the square Laplacian are illustrated. These examples provide additional support for the choice of the quadratic variation. The performance of the algorithm in interpolating both random dot and natural stereograms which have been processed by the Marr-Poggio stereo algorithm ( Grimson, Computing Shape Using a Theory of Human Stereo Vision, Ph.D. Thesis, MIT, Cambridge, 1980 and Philos. Trans. R. Soc. London Ser. B 292, 1981, 217–253 ) is also illustrated.