We prove that if a surface group embeds as a normal subgroup in a K\"ahler group and the conjugation action of the K\"ahler group on the surface group preserves the conjugacy class of a non-trivial element, then the K\"ahler group is virtually given by a direct product, where one factor is a surface group. Moreover we prove that if a one-ended hyperbolic group with infinite outer automorphism group embeds as a normal subgroup in a K\"ahler group then it is virtually a surface group. More generally we give restrictions on normal subgroups of K\"ahler groups which are amalgamated products or HNN extensions.