Suspended carbon nanotubes display at cryogenic temperatures a distinct interaction between the quantized longitudinal vibration of the macromolecule and its embedded quantum dot, visible via Franck-Condon conductance side bands in transport spectroscopy. We present data on such side bands at known absolute number N=1 and N=2 of conduction band electrons and consequently well-defined electronic ground and excited states in a clean nanotube device. The interaction evolves only at a finite axial magnetic field and displays a distinct magnetic field dependence of the Franck-Condon coupling, different for different electronic base states and indicating a valley-dependent electron-vibron coupling. A tentative cause of these effects, reshaping of the electronic wavefunction by the magnetic field, is discussed and demonstrated in a model.