The molecular composition of the organic and inorganic matrices of bone undergoes alterations during maturation. The aim of this study was to compare Fourier transform infrared (FT-IR) and near-infrared (NIR) Raman microspectroscopy techniques for characterization of the composition of growing and developing bone from young to skeletally mature rabbits. Moreover, the specificity and differences of the techniques for determining bone composition were clarified. The humeri of female New Zealand White rabbits, with age range from young to skeletally mature animals (four age groups, n = 7 per group), were studied. Spectral peak areas, intensities, and ratios related to organic and inorganic matrices of bone were analyzed and compared between the age groups and between FT-IR and Raman microspectroscopic techniques. Specifically, the degree of mineralization, type-B carbonate substitution, crystallinity of hydroxyapatite (HA), mineral content, and collagen maturity were examined. Significant changes during maturation were observed in various compositional parameters with one or both techniques. Overall, the compositional parameters calculated from the Raman spectra correlated with analogous parameters calculated from the IR spectra. Collagen cross-linking (XLR), as determined through peak fitting and directly from the IR spectra, were highly correlated. The mineral/matrix ratio in the Raman spectra was evaluated with multiple different peaks representing the organic matrix. The results showed high correlation with each other. After comparison with the bone mineral density (BMD) values from micro-computed tomography (micro-CT) imaging measurements and crystal size from XRD measurements, it is suggested that Raman microspectroscopy is more sensitive than FT-IR microspectroscopy for the inorganic matrix of the bone. In the literature, similar spectroscopic parameters obtained with FT-IR and NIR Raman microspectroscopic techniques are often compared. According to the present results, however, caution is required when performing this kind of comparison.