Second-year traits of growth, stem form, terminal flushing, and survival were assessed in 1770 ramets from 295 clones of 16 full-sib families of Castanea spp. Additive, dominance, and epistatic genetic variances were estimated in a clonally replicated incomplete 5 × 4 factorial test. Parents of the mating design were selected mainly on their phenotypes for wood quality (Castanea sativa traditional varieties) and their proven resistance to Phytophthora spp. (Asiatic species and Castanea crenata × C. sativa hybrids). Additive genetic variances were estimated to be 1.7–9 times greater than the dominance components. Inferred epistatic variance components showed a significant role in controlling growth traits and branch length. Narrow- and broad-sense heritability estimates showed that terminal flushing date was the most heritable trait, followed by height. The high estimates of half-sib, full-sib, and clonal mean heritabilities for almost all traits suggest that different strategies of backwards and forwards selection could be proposed. The ranking of the breeding values of parents allow us to select the best parents for new crosses and extend the mating design. Favorable genetic correlations were found between growth traits and straightness, so multi-trait selection looks promising. Our results provide the first information on the partitioning of genetic variance in Castanea spp. and a starting point for devising new selection strategies.