The evolution of the metallic state in lead clusters and its structural implications are subject to ongoing discussions. Here we present molecular beam electric deflection studies of neutral PbN (N = 19-25, 31, 36, 54) clusters. Many of them exhibit dipole moments or anomalies of the polarizability indicating a non-metallic state. In order to resolve their structures, the configurational space is searched using the Pool Birmingham Cluster Genetic algorithm based on density functional theory. Spin-orbit effects on the geometries and dipole moments are taken into account by further relaxing them with two-component density functional theory. Geometries and dielectric properties from quantum chemical calculations are then used to simulate beam deflection profiles. Structures are assigned by the comparison of measured and simulated beam profiles. Energy gaps are calculated using time-dependent density functional theory. They are compared to Kubo gaps, which are an indicator of the metallicity in finite particles. Both, experimental and theoretical data suggest that lead clusters are not metallic up to at least 36 atoms.