Search for lepton-flavour violating and lepton-number violating decays of the $D^0$ meson and observation of $D^0\to K^-\pi^+e^+e^-$

Abstract

We report the observation of the rare charm decay $D^0\rightarrow K^-\pi^+e^+e^-$ and a search for nine lepton-number-violating (LNV) and three lepton-flavor-violating (LFV) neutral charm decays of the type $D^0\rightarrow h^{\prime-}h^{-}l^{\prime+}l^+$ and $D^0\rightarrow h^{\prime-}h^+ l^{\prime\pm}l^{\mp}$, where $h$ and $h^{\prime}$ represent a $K$ or $\pi$ meson and $l$ and $l^{\prime}$ an electron or muon. The analysis is based on $468$ fb$^{-1}$ of $e^+e^-$ annihilation data collected at or close to the $\Upsilon(4S)$ resonance with the BaBar detector at the SLAC National Accelerator Laboratory. We find the $D^0\rightarrow K^-\pi^+e^+e^-$ branching fraction in the invariant mass range $0.675< m(e^+e^-) < 0.875$ GeV/c$^{2}$ of the electron-positron pair to be $\mathcal{B}(D^0\rightarrow K^-\pi^+e^+e^-) = (4.0 \pm 0.5 \pm 0.2 \pm 0.1) \times 10^{-6}$, where the first uncertainty is statistical, the second systematic, and the third due to the uncertainty in the branching fraction of the decay $D^0\rightarrow K^-\pi^+\pi^+\pi^-$ used as a normalization mode. In a set of regions of $m(e^+e^-)$ where long-distance effects are potentially small, we determine a 90\% confidence level (C.L.) upper limit on the branching fraction $\mathcal{B}(D^0\rightarrow K^-\pi^+\pi^+\pi^-) < 3.1\times 10^{-6}$. No significant signal is observed for any of the LFV and LNV modes, and we establish 90\% C.L. upper limits on the branching fractions in the range $(1.0 - 30.6)\times 10^{-7}$. The limits are between one and three orders of magnitude more stringent than previous measurements.