In many developing countries, water supply sources are not well protected. However, many people depend on such water sources for their daily consumption. Hence, this study assessed the physicochemical and heavy metal constituents of drinking water and evaluated the quality of the potable water supply in Haramaya Woreda. Water samples were taken from five sampling sites. Physicochemical and heavy metal parameters were analyzed according to the procedures outlined in the APHA (Standard methods for the examination of water and wastewater, 21st edn. Washington DC, 2008). The results showed that the acidity ranged from 32.67 ± 1.15 to 45.33 ± 0.58 mg/L, alkalinity from 276.67 ± 1.15 to 391.00 ± 1.73 mg/L, total hardness from 271.67 ± 0.57 to 410.67 ± 0.58 mg/L, chloride from 32.90 ± 1.32 to 85.77 ± 0.25 mg/L, sulfate from 194.33 ± 0.58 to 250.67 ± 0.58 mg/L, nitrate from 1.93 ± 0.15 to 13.33 ± 1.53 mg/L, phosphate from 0.015 ± 0.001 to 0.053 ± 0.002 mg/L, COD from 9.33 ± 0.702 to 24 ± 1.000 mg/L, Ca from 61.13 ± 1.06 to 103.17 ± 1.26 mg/L, Cd not detected to 0.069 ± 0.004 mg/L, Cr not detected to 0.133 ± 0.002 mg/L, Fe from 0.123 ± 0.015 to 0.530 ± 0.008 mg/L, Pb not detected to 0.217 ± 0.040 mg/L, Zn from 0.135 ± 0.002 to 0.170 ± 0.003 mg/L, and Cu from 0.221 ± 0.002 to 0.288 ± 0.002 mg/L. Statistically significant differences (p < 0.05) among the same parameter from different sites were observed. Except Pb, Cr, Cd, and water hardness, the majority of physicochemical and heavy metal values were within the maximum permissible limits of WHO and Ethiopian drinking water quality standards. The constituents outside the limits of the quality standards might cause health problems in the local communities. Therefore, strict monitoring and water treatment is needed to ensure quality sources before the water supply is used for human consumption.