SUMMARY Backazimuthal variations in the shear wave splitting of core-refracted shear waves (SKS, SKKS and PKS phases, jointly referred to as XKS) at the Black Forest Observatory (BFO, Southwest Germany) indicate small-scale lateral and partly vertical variations of the seismic anisotropy. However, existing anisotropy studies and models for the nearby Upper Rhine Graben (URG) area in the northern Alpine foreland are mostly based on short-term recordings and by this suffer from a limited backazimuthal coverage and averaging over a wide or the whole backazimuth range. To identify and delimit laterally confined anisotropy regimes in this region, we carry out XKS splitting measurements at six neighbouring (semi-)permanent broad-band seismological recording stations (interstation distance 10–80 km). We manually analyse long-term (partly > 20 yr) recordings to achieve a sufficient backazimuthal coverage to resolve complex anisotropy. The splitting parameters (fast polarization direction $\phi $, delay time $\delta t$) are determined in a single- and multi-event analysis. We test structural anisotropy models with one layer with horizontal or tilted symmetry axis and with two layers with horizontal symmetry axes (transverse isotropy). To account for lateral variations around a single recording site, modelling is compared for the whole and for limited backazimuth ranges. Based on this, we provide a 3-D block model with spatial variation of anisotropic properties. Based on delay times > 0.3 s and missing discrepancies between SKS and SKKS phases, which do not support lower mantle anisotropy, the found anisotropy is placed in the lithosphere and asthenosphere. The spatial distribution as well as the lateral and backazimuthal variations of the splitting parameters confirm lateral and partly vertical variations in anisotropy. On the east side of the URG, we suggest two anisotropic layers in the Moldanubian Zone (south) and one anisotropic layer in the Saxothuringian Zone (north). In the Moldanubian Zone, a change of the fast polarization directions is observed between the east and the west side of the URG, indicating different textures. At the boundary between the two terranes, an inclined anisotropy is modelled which may be related with deformation during Variscan subduction. Regarding the observation of numerous null measurements and inconsistent splitting parameters, especially (southwest of BFO) in the southern URG, different hypothesis are tested: scattering of the seismic wavefield due to small-scale lateral heterogeneities, a vertical a-axis due to a vertical mantle flow related to the Kaiserstuhl Volcanic Complex, as well as a different preferred orientation of the olivine crystals (not A-type, but C-type) due to specific ambient conditions (high temperature, water content).