Legumes are a rich source of dietary fibre, plant protein, and low-Glycaemic Index (GI) carbohydrate. Evidence suggests a positive effect on glycaemic control following a single meal; however, the effects of habitual consumption are less clear. This review aimed to investigate whether medium-to-long-term legume consumption had an effect on markers of glycaemic control in individuals with diabetes mellitus, without diabetes mellitus, or with prediabetes. As per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol, the online databases MEDLINE, Embase, CENTRAL, and CINAHL were searched from inception through to 31 March 2020. Randomised controlled trials (RCTs) ≥6 weeks in duration, reporting ≥1 of the following: fasting blood glucose (FBG), fasting blood insulin (FBI), glycosylated haemoglobin (HbA1c), homeostatic model assessment-insulin resistance (HOMA-IR), or 2-h postprandial glucose (2-h PPG), were deemed eligible. The overall quality of evidence was determined using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) assessment. A total of 18 RCTs were included, of which, 5 focused on individuals with diabetes mellitus, 12 on individuals without diabetes mellitus, and one on individuals with prediabetes. Only studies of those with type 2 diabetes mellitus (n = 5) reported significant effects for legume interventions, three of which consistently reported reductions in FBG, two reported reductions in HbA1c, one reported a reduction in FBI, and another a reduction in 2-h PPG (p < 0.05); however, the overall quality of evidence was very low. The findings of this review support the dietary inclusion of legumes; however, the need for further high-quality RCTs to be conducted is also highlighted, particularly among individuals with prediabetes, gestational diabetes mellitus and type 1 diabetes mellitus.