Coming decade of h→τμ and τ→μγ interplay in τ flavor violation search

Abstract

We explore the interplay between $h(125) \to \tau\mu$ search at the LHC and $\tau \to \mu\gamma$ at the up and coming Belle~II experiment, in context of the general two Higgs doublet model with extra Yukawa couplings such as $\rho_{\tau\mu}$. The search for $h \to \tau\mu$ constrains $\rho_{\tau\mu} \cos\gamma$, where $\cos\gamma$ is the $h$--$H$ mixing angle of $h$ with the exotic $CP$-even scalar $H$. For $\tau \to \mu\gamma$, we define the "BSM-benchmark" by setting the extra top Yukawa coupling $\rho_{tt} = 1\ (\cong \lambda_t)$ in two-loop diagrams, and $\cos\gamma = 0$ to decouple $h$. We show that this leading effect due to $H$ and $CP$-odd scalar $A$ can be readily probed by Belle~II, even for the conservative value of $\rho_{\tau\mu} = 0.7\lambda_\tau$. We define the subleading "$h$-benchmark" by setting $\rho_{tt} = 0$ in two-loop diagrams, and take the conservative maximal value of $\cos\gamma = 0.2$. We show that it falls beyond Belle~II reach, but can interfere with the BSM-benchmark effect, which in principle probes the phase of $\rho_{tt}$. We further show that the one-loop $H$, $A$ effect, proportional to $\rho_{\tau\mu}\rho_{\tau\tau}$ in amplitude, is beyond the sensitivity of Belle~II to probe, even for $\rho_{\tau\tau}$ as large as $3\lambda_\tau$. With the working assumption that $\rho_{32}^f, \rho_{33}^f = {\cal O}(\lambda_3^f)$ for all charged fermions $f$, we find good discovery potential for both $\tau$ lepton flavor violation searches in the coming decade.