We compute the full isometry group of any left-invariant metric on a simply connected, non-unimodular Lie group of dimension three. As an application, we determine the index of symmetry of such metrics and prove that the singularities of the moduli space of left-invariant metrics, up to isometric automorphism are contained in the subspace of classes of metrics with maximal index of symmetry.