A comparison of different standard samples—artificial glass SRM-612 (Standard Reference Material) and standard samples of natural composition SG-1A, SG-3, ST-1A, and SGD-1A—used for external calibration in the elemental analysis of silicate rocks in the form of fused glasses by inductively coupled plasma mass spectrometry and laser ablation (LA-ICP-MS) is carried out with a goal of selecting the most suitable samples for plotting the calibration dependence upon determination of the major and trace elements when using LA-ICP-MS for routine analysis. The results showed that the error of determination for both major and trace elements is lower (compared to SRM-612) when external calibration is carried out using the reference materials of natural composition with Si and Fe contents close to those in the analyzed samples. The use of internal standards in both cases decreases the systematic error attributed to the drift of LA parameters and different ablation yields. The correctness of the determination of 28 elements is proved in comparison of the results of the analysis of four standard samples with the reference values. The obtained results are used to select calibration standards for LA-ICP-MS analysis of the samples of unknown composition. The results of analysis of six samples of unknown composition match within the error limit the results of X-ray fluorescence analysis (major elements) and ICP-MS (trace elements in solutions). Refining of the content of trace elements in the previously certified reference materials allows them to be used for calibration in routine analysis of geological rocks. The developed LA-ICP-MS technique is a rapid method for determination of a wide range of elements, in particular, rare earth elements, in silicate rocks and can be used for routine analysis without additional sample preparation after X-ray fluorescence analysis.