We compute photometric redshifts based on the template-fitting method in the fourth public release of the Canada-France-Hawaii Telescope Legacy Survey. This unique multi-colour catalogue comprises u*,g',r',i',z' photometry in four deep fields of 1 deg2 each and 35 deg2 distributed over three Wide fields. Our photometric redshifts are calibrated with and compared to 16,983 high-quality spectroscopic redshifts from several surveys. We find a dispersion of 0.028 and an outlier rate of 3.5% in the Deep field at i'AB < 24 and a dispersion of 0.036 and an outlier rate of 2.8% in the Wide field at i'AB < 22.5. Beyond i'AB = 22.5 in the Wide field the number of outliers rises from 5% to 10% at i'AB<23 and i'AB<24 respectively. For the Wide sample, we find the systematic redshift bias keeps below 1% to i'AB < 22.5, whereas we find no significant bias in the Deep field. We investigated the effect of tile-to-tile photometric variations and demonstrate that the accuracy of our photometric redshifts is reduced by at most 21%. We separate stars from galaxies using both the size and colour information, reducing the contamination by stars in our catalogues from 50% to 8% at i'AB < 22.5 in fields with the highest stellar density while keeping a complete galaxy sample. Our CFHTLS T0004 photometric redshifts are distributed to the community. Our release include 592,891 (i'AB < 22.5) and 244,701 (i'AB < 24) reliable galaxy photometric redshifts in the Wide and Deep fields, respectively.