The article discusses the experimental and theoretical results of studying the optical properties for planar quartz surfaces modified with gold nanostars using Surface-Enhanced Raman Spectroscopy (SERS). Surfaces functionalized with nanoparticles, prototypes of optical sensors, have been obtained. It is shown that the obtained experimental Raman signal enhancement from the dye rhodamine 6G (R6G) can be at least order 104. The paper presents the results of calculation, synthesis and study of the optical and morphological properties of such structures. A complex method for creating and evaluating the optical properties of planar nanostructures and evaluating their optical properties is proposed. The method includes the approaches based on mathematical modeling using finite differences in the time domain (FDTD - Finite-Difference Time-Domain) evaluating electromagnetic field strengths near the surfaces of star-shaped gold nanoparticles in colloidal solutions and on a silicon surface. During the simulation, we studied such parameters as the particle size, the wavelength of the exciting radiation, and the dependence of the effective amplification of the electromagnetic field on the morphological parameters of the star. The theoretical approach to the calculation of such structures is shown to be promising for the subsequent chemical synthesis described in the article and the study of optical properties. In this paper, a theoretical calculation of the parameters of the electric field and optical properties is carried out near star-shaped nanoparticles of varying sizes. These parameters were calculated by the finite difference method in the time domain. The task included finding the optimal sizes of nanoparticles as well as varying the thickness of the surface layers in order to obtain the maximum scattering values and further use of similar structures in SERS experiments. Also, within the framework of this work, syntheses of star-shaped nanoparticles was carried out, their optical and morphological properties were studied, quartz surfaces were functionalized with nanoparticles, and sensor prototypes were created in order to further assess the Raman signal amplification. Based on the results of mathematical modeling, the optimal sizes for the synthesis of star-shaped NPs were determined. Synthesis of NPs was carried out by a chemical method using seeds. To carry out the experimental part, SERS spectra were obtained using Centaur U spectrometer (OOO NanoScanTechnology, Russia). The spectrometer was equipped with three different laser sources: 632.8 nm He-Ne laser (17 mW), 532 nm and 473 nm DPSS lasers (45 mW). During the experiment, a He-Ne laser with λ = 632 nm and DPSS laser with λ = 532 nm were used to match the excitation of the plasmon maximum for the nanostars. The optical scheme of the spectrometer included an Olympus BX41 microscope (Olympus, Japan) with a 100× objective (NA 0.9) for positioning the beam and collecting scattered photons. The monochromator of the spectrometer had a focal length of 800 mm. A holographic diffraction grating of 300 gr/mm, and was equipped with a 1024×256 pixel thermoelectric cooled CCD detector (Andor Tech., UK). As a result of the paper, surfaces functionalized by nanoparticles were obtained prototypes of optical sensors. It is shown that the experimental gain of the Raman signal can be no less than 104 times. The presented data will serve as the basis for obtaining the required sizes of nanostars in the methods of the controlled chemical synthesis of colloidal nanoparticles. The results presented in the article can be applied to the development of biocompatible and highly sensitive optical sensors based on the effect of Raman scattering of light.