Context. In recent years, a number of Lyman continuum (LyC) leaker candidates have been found at intermediate redshifts, providing insight into how the Universe was reionised at early cosmic times. There are now around 100 known LyC leakers at all redshifts, which enables us to analyse their properties statistically. Aims. Here, we identify new LyC leaker candidates at z ≈ 3 − 4.5 and compare them to objects from the literature to get an overview of the different observed escape fractions and their relation to the properties of the Lyman α (Lyα) emission line. The aim of this work is to test the indicators (or proxies) for LyC leakage suggested in the literature and to improve our understanding of the kinds of galaxies from which LyC radiation can escape. Methods. We used data from the Hubble Deep Ultraviolet (HDUV) legacy survey to search for LyC emission based on a sample of ≈2000 Lyα emitters (LAEs) detected previously in two surveys with the Multi-Unit Spectroscopic Explorer (MUSE), namely MUSE-Deep and MUSE-Wide. Based on the redshifts and positions of the LAEs, we look for potential LyC leakage in the WFC3/UVIS F336W band of the HDUV survey. The escape fractions are measured and compared in different ways, including spectral energy distribution (SED) fitting performed using the CIGALE software. Results. We add 12 objects to the sample of known LyC leaker candidates (5 highly likely leakers and 7 potential ones), 1 of which was previously known, and compare their Lyα properties to their escape fractions. We find escape fractions of between ∼20% and ∼90%, assuming a high transmission in the intergalactic medium (IGM). We present a method whereby the number of LyC leaker candidates we find is used to infer the underlying average escape fraction of galaxies, which is ≈12%. Conclusion. Based on their Lyα properties, we conclude that LyC leakers are not very different from other high-z LAEs and suggest that most LAEs could be leaking LyC even if this cannot always be detected because of the direction of emission and the transmission properties of the IGM.