We consider the spatially homogeneous Boltzmann equation for {\em inelastic hard spheres}, in the framework of so-called {\em constant normal restitution coefficients} $\alpha \in [0,1]$. In the physical regime of a small inelasticity (that is $\alpha \in [\alpha_*,1)$ for some constructive $\alpha_*>0$) we prove uniqueness of the self-similar profile for given values of the restitution coefficient $\alpha \in [\alpha_*,1)$, the mass and the momentum; therefore we deduce the uniqueness of the self-similar solution (up to a time translation). Moreover, if the initial datum lies in $L^1_3$, and under some smallness condition on $(1-\alpha_*)$ depending on the mass, energy and $L^1_3$ norm of this initial datum, we prove time asymptotic convergence (with polynomial rate) of the solution towards the self-similar solution (the so-called {\em homogeneous cooling state}). These uniqueness, stability and convergence results are expressed in the self-similar variables and then translate into corresponding results for the original Boltzmann equation. The proofs are based on the identification of a suitable elastic limit rescaling, and the construction of a smooth path of self-similar profiles connecting to a particular Maxwellian equilibrium in the elastic limit, together with tools from perturbative theory of linear operators. Some universal quantities, such as the "quasi-elastic self-similar temperature" and the rate of convergence towards self-similarity at first order in terms of $(1-\alpha)$, are obtained from our study. These results provide a positive answer and a mathematical proof of the Ernst-Brito conjecture [16] in the case of inelastic hard spheres with small inelasticity.