A combination of electron and bremsstrahlung diagnostics was used to study the properties of the highly charged plasma of an electron cyclotron resonance ion source (ECRIS). The bremsstrahlung radiation in both axial and radial directions was correlated with the energy distribution of the electrons escaping axially from the confined plasma, i.e. the lost electron energy distribution (LEED), to achieve a more comprehensive view of the plasma. The evolution of the bremsstrahlung spectra and the LEED were determined as a function of the main operating parameters of the ion source. Also, the effects of the transition from a stable to an unstable plasma regime were determined for the bremsstrahlung and LEED changes in the shape of the measured LEED were found to correlate with changing bremsstrahlung spectral temperature. It was also found that the magnetic field has a strong impact on the axial and radial bremsstrahlung spectra and the LEED affecting the bremsstrahlung intensity, spectral temperatures and the LEED high energy local maximum. A comprehensive discussion is provided to explain the observed different behaviour of axial and radial bremsstrahlung emissions with varying magnetic field based on the directionality of plasma bremsstrahlung in ECRIS specific magnetic confinement scheme. Furthermore, it was observed that the onset of kinetic plasma instabilities has a clear impact on the shape of the LEED