This study quantitatively reviews the hedonic literature examining surface water quality to assess how attributes of the commodity, housing market, and methodological choices lead to variation in the significance and expected sign of the estimated property value effects (i.e., elasticities). We conduct a meta-analysis of 29 studies with 290 unique estimates, published or released between 1985 and 2017, and find evidence based on probit meta-regression models that some of the definitions and decisions made in primary studies do influence the estimated relationship between water quality and home prices. Our most robust evidence suggests that methodological choices (e.g., accounting for spatial dependence, or if the water quality measure was based on something other than in situ measurement) have a critical role in determining the likelihood of finding a significant and theoretically expected result; and perhaps most importantly, it is not always selections that reflect best practices that lead to this finding. This study can help identify potential concerns with data and modeling choices in the collective hedonic literature focused on water quality.