In this paper, we report field-induced ferroelectricity in ${\mathrm{BaHoFeO}}_{4}$ in a narrow range of applied magnetic fields ($1.5\phantom{\rule{4.pt}{0ex}}\text{T}\ensuremath{\le}H\ensuremath{\le}5\phantom{\rule{4.pt}{0ex}}\text{T}$). Magnetization measurements reveal three magnetic anomalies, the first one at ${T}_{N}^{\text{Fe}}\ensuremath{\sim}52.8$ K, the second one at ${T}_{{N}_{2}}\ensuremath{\sim}36.4$ K, and the third one at ${T}_{N}^{\text{Ho}}\ensuremath{\sim}6.8$ K, corresponding to the antiferromagnetic ordering of ${\mathrm{Fe}}^{3+}$, a possible spin structure change of ${\mathrm{Fe}}^{3+}$, and the antiferromagnetic ordering of ${\mathrm{Ho}}^{3+}$ sublattices, respectively. Surprisingly, the heat capacity data did not show any anomaly at ${T}_{N}^{\text{Fe}}$ but exhibited a broad maximum at ${T}_{N}^{\text{Ho}}$. Below 40 K, isothermal magnetization measurements indicate a metamagnetic transition at 1.5 T. The temperature-dependent dielectric measurements performed below 1 T reveal two anomalies, around 6 and 36 K, corresponding to ${\mathrm{Ho}}^{3+}$ ordering and a possible change in the magnetic phase of the Fe sublattice. These anomalies disappear at higher magnetic fields, and a different dielectric anomaly appears above the metamagnetic transition with a broad asymmetric peak around 13 K (${T}_{P}$), which shifts to higher temperatures, reaches a maximum under 2 T, and then is suppressed further on increasing the magnetic field up to 9 T. A switchable electric polarization appears below 25 K and above the metamagnetic transition field ($H\ensuremath{\ge}1.5$ T). Intriguingly, the polarization disappears above 5 T. The presence of electric polarization in intermediate magnetic fields and its strong magnetic field dependence are interesting and make this material unique.