High-energy physics detectors with internal charge multiplication, like Low Gain Avalanche Detectors (LGADs), that will be used for fast timing in the High Luminosity LHC experiments, have to exhibit a significant radiation tolerance. In this context, the impact of radiation on the highly boron-doped gain layer is of particular interest, since due to the so-called Acceptor Removal Effect (ARE) a radiation-induced deactivation of active boron dopants takes place, that is causing a progressive loss in the gain with increasing irradiation level. In this paper we present defect-spectroscopy measurements (Deep-Level Transient Spectroscopy and Thermally Stimulated Current technique) on neutron, proton and electron irradiated p-type silicon pad diodes of different resistivity as well as LGADs neutron irradiated at fluences up to 1×1015 neq/cm2. We show that compared to silicon pad diodes the determination of LGAD defect introduction rates is less straightforward as they are strongly influenced by the impact of the gain layer. The measured gain layer capacitance has a strong frequency and temperature dependence which makes DLTS measurements challenging to perform with results difficult to interpret. With the TSC technique the defects formed in the LGADs are nicely observed and can be compared to the defects formed in the silicon pad diodes. However, the exact assignment of defects to the gain layer or bulk region remains challenging and the charge amplification effect of the LGADs impacts the exact determination of defect concentrations. We also demonstrate that, depending on the TSC measurement conditions, defect induced internal electric fields are built up in the irradiated LGADs which impact the signal current.