The prevalence of nonalcoholic fatty liver disease is rapidly rising. We aimed to investigate associations of diet quality and dietary patterns with nonalcoholic fatty liver disease (NAFLD) in Black and White adults. We included 1726 participants who attended the Year 20 Exam of the Coronary Artery Risk Development in Young Adults (CARDIA) study and had their liver attenuation (LA) measured using computed tomography at Year 25 (2010–2011). NAFLD was defined as an LA of ≤51 Hounsfield units after the exclusion of other causes of liver fat. The a priori diet-quality score (APDQS) was used to assess diet quality, and dietary patterns were derived from principal components analysis. Univariate and multivariable logistic regression models were used to evaluate the association between the APDQS, dietary patterns, and NAFLD, and were adjusted for Year 20 covariates. NAFLD prevalence at Year 25 was 23.6%. In a model adjusted for age, race, sex, education, alcohol use, physical activity, smoking, and center at Year 25, the APDQS was inversely associated (p = 0.004) and meat dietary pattern was positively associated (p < 0.0001) with NAFLD, while the fruit-vegetable dietary pattern was not significantly associated (p = 0.40). These associations remained significant when additionally adjusting for comorbidities (type 2 diabetes mellitus, dyslipidemia, hypertension), however, significant associations were diminished after additionally adjusting for body mass index (BMI). Overall, this study finds that the APDQS and meat dietary patterns are associated with prevalent NAFLD in mid-life. The associations appear to be partially mediated through higher BMI.