We present weak lensing mass reconstructions for the 20 high-redshift clusters in the ESO Distant Cluster Survey. The weak lensing analysis was performed on deep, 3-color optical images taken with VLT/FORS2, using a composite galaxy catalog with separate shape estimators measured in each passband. We find that the EDisCS sample is composed primarily of clusters that are less massive than those in current X-ray selected samples at similar redshifts, but that all of the fields are likely to contain massive clusters rather than superpositions of low mass groups. We find that 7 of the 20 fields have additional massive structures which are not associated with the clusters and which can affect the weak lensing mass determination. We compare the mass measurements of the remaining 13 clusters with luminosity measurements from cluster galaxies selected using photometric redshifts and find evidence of a dependence of the cluster mass-to-light ratio with redshift. Finally we determine the noise level in the shear measurements for the fields as a function of exposure time and seeing and demonstrate that future ground-based surveys which plan to perform deep optical imaging for use in weak lensing measurements must achieve point-spread functions smaller than a median of FWHM.