Nonalcoholic fatty liver disease (NAFLD), circulating insulin-like growth factor-1 (IGF-1), and IGF-1/IGF-binding protein-3 (IGFBP-3) concentrations are associated with adiposity and insulin resistance. We aimed to determine whether serum IGF-1, IGFBP-3, and IGF-1/IGFBP-3 are associated with presence or severity of NAFLD independent of potential confounding. We performed a cross-sectional analysis of data from the Third National Health and Nutrition Examination Survey, 1988–1994, a representative sample of the United States adult population. Among participants who had a fasting blood draw and ultrasound examination, we excluded those with missing data, viral hepatitis, iron overload, excessive alcohol intake, pregnancy, or taking glucose-lowering therapy, yielding 4172 adults for this analysis. In logistic regression analyses adjusted for age, gender, and race/ethnicity, higher IGF-1 and IGF-1/IGFBP-3 quartiles were associated with lower likelihood of NAFLD and lower grade steatosis. These associations became non-significant when further adjusted for adiposity (body mass index, waist circumference) with the exception of the association between IGF-1/IGFBP-3 and severity of NAFLD which remained significant after adjustment for homeostasis model assessment for insulin resistance (HOMA-IR) (odds ratio [95% CI]: Q3: 0.71 [0.53–0.96], Q4: 0.62 [0.43–0.89]) and adiposity (Q4: 0.67 [0.47–0.96]). Full adjustment (age, gender, race/ethnicity, adiposity, HOMA-IR, A1C%) further attenuated associations between IGF-1 or IGF-1/IGFBP-3 and liver fat such that they were no longer significant. Adiposity explains much of the observed association between IGF-1 or IGF-1/IGFBP-3 and liver fat. These findings do not support a direct role for the growth hormone-IGF-1/IGFBP-3 axis in the pathophysiology of NAFLD.