Several prehospital scales have been designed to aid paramedics in identifying stroke patients in the ambulance setting. However, external validation and comparison of these scales are largely lacking. To compare all published prehospital stroke detection scales in a large cohort of unselected stroke code patients. We conducted a systematic literature search to identify all stroke detection scales. Scales were reconstructed with prehospital acquired data from two observational cohort studies: the Leiden Prehospital Stroke Study (LPSS) and PREhospital triage of patients with suspected STrOke (PRESTO) study. These included stroke code patients from four ambulance regions in the Netherlands, including 15 hospitals and serving four million people. For each scale, we calculated the accuracy, sensitivity and specificity for a diagnosis of stroke (ischemic, hemorrhagic or TIA). Moreover, we assessed the proportion of stroke patients who received reperfusion treatment with intravenous thrombolysis or endovascular thrombectomy that would have been missed by each scale. We identified 14 scales, of which seven (CPSS, FAST, LAPSS, MASS, MedPACS, OPSS, and sNIHSS-EMS) could be reconstructed. Of 3317 included stroke code patients, 2240 (67.5%) had a stroke (1528 ischemic, 242 hemorrhagic, 470 TIA) and 1077 (32.5%) a stroke mimic. Of ischemic stroke patients, 715 (46.8%) received reperfusion treatment. Accuracies ranged from 0.60 (LAPSS) to 0.66 (MedPACS, OPSS and sNIHSS-EMS), sensitivities from 66% (LAPSS) to 84% (MedPACS and sNIHSS-EMS), and specificities from 28% (sNIHSS-EMS) to 49% (LAPSS). MedPACS, OPSS and sNIHSS-EMS missed the fewest reperfusion-treated patients (10.3-11.2%), whereas LAPSS missed the most (25.5%). Prehospital stroke detection scales generally exhibited high sensitivity but low specificity. While LAPSS performed poorest, MedPACS, sNIHSS-EMS and OPSS demonstrated the highest accuracy and missed the fewest reperfusion-treated stroke patients. Use of the most accurate scale could reduce unnecessary stroke code activations for patients with a stroke mimic by almost a third, but at the cost of missing 16% of strokes and 10% of patients who received reperfusion treatment.