Boron-rich superhard materials have complex structural and chemical variations, making it challenging to determine their atomic structures precisely from experiments. Here, we employ first-principles calculation to examine Raman spectra of four boron-rich icosahedral crystals: boron carbide (B4C), boron-rich boron carbide (B13C2), boron suboxide (B6O), and boron subphosphide (B12P2). We find qualitative agreements in Raman peak locations between the calculated Raman spectra and the experimental measurements. Furthermore, vibrations of the icosahedra or chains, occurring in all the spectra, are interpreted by performing vibration mode analyses. Our results indicate that the vibrations of the two-atom chain are responsible for the low-frequency region of B6O and B12P2. In contrast, both low-frequency and high-frequency regions are partially related to the vibration of chain atoms for B4C and B13C2. Besides, the icosahedral vibration modes are attributed to the high-frequency region of all these four crystals. More importantly, the peak positions of Raman spectra can be used to identify the various icosahedral structure with different B/C ratios and various chain structures, making Raman spectroscopy an important tool to monitor the processing of newly developed icosahedral materials with enhanced physical properties. This study not only provides a deeper understanding of the atomic structures of boron-rich superhard materials from the perspective of Raman spectroscopy but also helps to interpret the experimental Raman spectra.