We consider the problem of fermions interacting with gapless long-wavelength collective bosonic modes. The theory describes, among other cases, a ferromagnetic quantum-critical point (QCP) and a QCP towards nematic ordering. We construct a controllable expansion at the QCP in two steps: we first create a new, non Fermi-liquid ``zero-order'' Eliashberg-type theory, and then demonstrate that the residual interaction effects are small. We prove that this approach is justified under two conditions: the interaction should be smaller than the fermionic bandwidth, and either the band mass $m_B$ should be much smaller than $m = p_F/v_F$, or the number of fermionic flavors $N$ should be large. For an SU(2) symmetric ferromagnetic QCP, we find that the Eliashberg theory itself includes a set of singular renormalizations which can be understood as a consequence of an effective long-range dynamic interaction between quasi-particles, generated by the Landau damping term. These singular renormalizations give rise to a negative non-analytic $q^{3/2}$ correction to the static spin susceptibility, and destroy a ferromagnetic QCP. We demonstrate that this effect can be understood in the framework of the $\phi^4$ theory of quantum criticality. We also show that the non-analytic $q^{3/2}$ correction to the bosonic propagator is specific to the SU(2) symmetric case. For systems with a scalar order parameter, the $q^{3/2}$ contributions from individual diagrams cancel out in the full expression of the susceptibility, and the QCP remains stable.