The goal of this study was to investigate whether the inclusion of genomic information and epistatic (additive by additive) genetic effects would increase the accuracy of predicting phenotypes adjusted for known environmental effects, reduce prediction bias and minimize the confounding between additive and additive by additive epistatic effects on fertility and calving traits in Holstein cattle. Phenotypic and genotypic records were available for 6090 cows. Eight cow traits were assessed including 56-day nonreturn rate (NRR), number of services (NS), calving to first insemination (CTFS), first insemination to conception (FSTC), gestation length (GL), calving ease (CE), stillbirth (SB) and calf size (CZ). Four scenarios were assessed for their ability to predict adjusted phenotypes, which included: (1) traditional pedigree-based Best Linear Unbiased Prediction (P-BLUP) for additive genetic effects (PA); (2) P-BLUP for additive and epistatic (additive by additive) genetic effects (PAE); (3) genomic BLUP (G-BLUP) for additive genetic effects (GA); and (4) G-BLUP for additive and epistatic genetic effects (GAEn, where n = 1-3 depending on the alternative ways to construct the epistatic genomic matrix used). Constructing epistatic relationship matrix as the Hadamard product of the additive genomic relationship matrix (GAE1), which is the usual method and implicitly assumes a model that fits all pairwise interactions between markers twice and includes the interactions of the markers with themselves (dominance). Two additional constructions of the epistatic genomic relationship matrix were compared to test whether removing the double counting of interactions and the interaction of the markers with themselves (GAE2), and removing double counting of interactions between markers, but including the interaction of the markers with themselves (GAE3) would had an impact on the prediction and estimation error correlation (i.e. confounding) between additive and epistatic genetic effects. Fitting epistatic genetic effects explained up to 5.7% of the variance for NRR (GAE3), 7.7% for NS (GAE1), 11.9% for CTFS (GAE3), 11.1% for FSTC (GAE2), 25.7% for GL (GAE1), 2.3% for CE (GAE1), 14.3% for SB (GAE3) and 15.2% for CZ (GAE1). Despite a substantial proportion of variance being explained by epistatic effects for some traits, the prediction accuracies were similar or lower for GAE models compared with pedigree models and genomic models without epistatic effects. Although the prediction accuracy of direct genomic values did not change significantly between the three variations of the epistatic genetic relationship matrix used, removing the interaction of the markers with themselves reduced the confounding between additive and additive by additive epistatic effects. These results suggest that epistatic genetic effects contribute to the variance of some fertility and calving traits in Holstein cattle. However, the inclusion of epistatic genetic effects in the genomic prediction of these traits is complex and warrant further investigation.