B-added low carbon steels exhibit excellent hardenability. The reason has been frequently attributed to B segregation at prior austenite grain boundaries, which prevents the austenite to ferrite transformation and favors the formation of martensite. The segregation behavior of B at prior austenite grain boundaries is strongly influenced by processing conditions such as austenitization temperatures and cooling rates and by alloying elements such as Mo, Cr, and Nb. Here an local electrode atom probe was employed to investigate the segregation behavior of B and other alloying elements (C, Mn, Si, and Cr) in a Cr-added Mo-free martensitic steel. Similar to our previous results on a Mo-added steel, we found that in both steels B is segregated at prior austenite grain boundaries with similar excess values, whereas B is neither detected in the martensitic matrix nor at martensite–martensite boundaries at the given cooling rate of 30K/s. These results are in agreement with the literature reporting that Cr has the same effect on hardenability of steels as Mo in the case of high cooling rates. The absence of B at martensite–martensite boundaries suggests that B segregates to prior austenite grain boundaries via a non-equilibrium mechanism. Segregation of C at all boundaries such as prior austenite grain boundaries and martensite–martensite boundaries may occur by an equilibrium mechanism.