The BXD family has become one of the preeminent genetic reference populations to understand the genetic and environmental control of phenotypic variation. Here we evaluate the responses to different levels of fat in the diet using both chow diet (CD, 13–18% fat) and a high-fat diet (HFD, 45–60% fat). We studied cohorts of BXD strains, both inbred parents C57BL/6J and DBA/2J (commonly known as B6 and D2, respectively), as well as B6D2 and D2B6 reciprocal F1 hybrids. The comparative impact of genetic and dietary factors was analyzed by profiling a range of phenotypes, most prominently their cecum bacterial composition. The parents of the BXDs and F1 hybrids express limited differences in terms of weight and body fat gain on CD. In contrast, the strain differences on HFD are substantial for percent body fat, with DBA/2J accumulating 12.5% more fat than C57BL/6J (P < 0.0001). The F1 hybrids born to DBA/2J dams (D2B6F1) have 10.6% more body fat (P < 0.001) than those born to C57BL/6J dams. Sequence analysis of the cecum microbiota reveals important differences in bacterial composition among BXD family members with a substantial shift in composition caused by HFD. Relative to CD, the HFD induces a decline in diversity at the phylum level with a substantial increase in Firmicutes (+13.8%) and a reduction in Actinobacteria (-7.9%). In the majority of BXD strains, the HFD also increases cecal sIgA (P < 0.0001)—an important component of the adaptive immunity response against microbial pathogens. Host genetics modulates variation in cecum bacterial composition at the genus level in CD, with significant quantitative trait loci (QTLs) for Oscillibacter mapped to Chr 3 (18.7–19.2 Mb, LRS = 21.4) and for Bifidobacterium mapped to Chr 6 (89.21–89.37 Mb, LRS = 19.4). Introduction of HFD served as an environmental suppressor of these QTLs due to a reduction in the contribution of both genera (P < 0.001). Relations among liver metabolites and cecum bacterial composition were predominant in CD cohort, but these correlations do not persist following the shift to HFD. Overall, these findings demonstrate the important impact of environmental/dietary manipulation on the relationships between host genetics, gastrointestinal bacterial composition, immunological parameters, and metabolites—knowledge that will help in the understanding of the causal sources of metabolic disorders.