Di-Higgs couplings to fermions of the form h2f‾f are absent in the Standard Model, however, they are present in several physics Beyond Standard Model (BSM) extensions, including those with vector-like fermions. In Effective Field Theories (EFTs), such as the Standard Model Effective Field Theory (SMEFT) and the Higgs Effective Field Theory (HEFT), these couplings appear at dimension 6 and can in general, be flavour-violating (FV). In the present work, we employ a bottom-up approach to investigate the FV in the lepton and quarks sectors through the di-Higgs effective couplings. We assume that all FV arises from this type of couplings and assume that the Yukawa couplings Yij are given by their SM values, i.e. Yij=2miδij/v. In the lepton sector, we set upper limits on the Wilson coefficients Cll′ from l→3l′ decays, l→lγ decays, muonium oscillations, the (g−2)μ anomaly, LEP searches, muon conversion in nuclei, FV Higgs decays, and Z decays. We also make projections on some of these coefficients from Belle II, the Mu2e experiment and the LHC's High Luminosity (HL) run. In the quark sector, we set upper limits on the Wilson coefficients Cqq′ from meson oscillations and from B-physics searches. A key takeaway from this study is that current and future experiments should set out to measure the effective di-Higgs couplings Cff′, whether these couplings are FV or flavour-conserving. We also present a matching between our formalism and the SMEFT operators and show the bounds in both bases.