We investigated the electronic structure of the enigmatic CeRh$_3$B$_2$ using resonant inelastic scattering and x-ray absorption spectroscopy in combination with $ab$ $initio$ density functional calculations. We find that the Rh 4$d$ states are irrelevant for the high-temperature ferromagnetism and the Kondo effect. We also find that the Ce 4$f$ crystal-field strength is too small to explain the strong reduction of the Ce magnetic moment. The data reveal instead the presence of two different active Ce 4$f$ orbitals, with each coupling selectively to different bands in CeRh$_3$B$_2$. The inter-site hybridization of the |J=5/2,Jz=+/-1/2> crystal-field state and Ce 5$d$ band combined with the intra-site Ce 4$f$-5$d$ exchange creates the strong ferromagnetism, while hybridization between the |J=5/2,Jz=+/-5/2> and the B $sp$ in the $ab$-plane contributes to the Kondo interaction which causes the moment reduction. This orbital selective coupling explains the unique and seemingly contradictory properties of CeRh$_3$B$_2$.