We consider a nonlinear system describing a juvenile-adult population undergoing small mutations. We analyze two aspects: from a mathematical point of view, we use an entropy method to prove that the population neither goes extinct nor blows-up; from an adaptive evolution point of view, we consider small mutations on a long time scale and study how a monomorphic or a dimorphic initial population evolves towards an Evolutionarily Stable State. Our method relies on an asymptotic analysis based on a constrained Hamilton–Jacobi equation. It allows to recover earlier predictions in Calsina and Cuadrado [À. Calsina, S. Cuadrado, Small mutation rate and evolutionarily stable strategies in infinite dimensional adaptive dynamics, J. Math. Biol. 48 (2004) 135; À. Calsina, S. Cuadrado, Stationary solutions of a selection mutation model: the pure mutation case, Math. Mod. Meth. Appl. Sci. 15(7) (2005) 1091.] that we also assert by direct numerical simulation. One of the interests here is to show that the Hamilton–Jacobi approach initiated in Diekmann et al. [O. Diekmann, P.-E. Jabin, S. Mischler, B. Perthame, The dynamics of adaptation: an illuminating example and a Hamilton–Jacobi approach, Theor. Popul. Biol. 67(4) (2005) 257.] extends to populations described by systems.