We examined sociodemographic factors, lifestyle behaviors, and pregnancy-related determinants associated with adherence to the Mediterranean diet (MD) during pregnancy. A total of 152 Caucasian pregnant women were included in this cross-sectional study. Dietary habits and MD adherence were assessed with a food frequency questionnaire. Physical activity (PA) levels and physical fitness (PF) components (cardiorespiratory fitness, relative muscle strength, and flexibility) were objectively measured. A clustered overall PF index was calculated. Participants with a high MD adherence were older, had a lower body mass index (BMI), spent more time in moderate–vigorous PA, had a greater overall PF, cardiorespiratory fitness, and relative muscle strength compared to participants with low MD adherence (all, p < 0.05). When we explored factors associated with improved MD adherence with logistic regression analysis, we found that the following factors: lower pre-pregnancy BMI (OR = 2.337; p = 0.026), meeting PA recommendations (OR = 2.377; p = 0.045), higher relative muscle strength (OR = 2.265; p = 0.016), and higher overall PF (OR = 5.202; p = 0.004) increased the chances to adhere to the MD. Older age, lower BMI, greater PF, and meeting PA recommendations were associated with higher MD adherence. These factors should be considered for a better design of educational programs and guidelines focused on improving materno–fetal health status during pregnancy.