Introduction Acute stroke is a medical emergency. Identifying patients suffering a stroke is crucial if paramedics are to maximise delivery of appropriate management. One suggested screening tool to identify stroke is ‘FAST’ (Face, Arms, Speech, Time), but the accuracy of identifying pre-hospital stroke is unknown. Objectives We aimed to: a) examine how well stroke is identified by paramedics using the emergency department (ED) discharge diagnosis as the comparator; and b) assess compliance with ambulance clinical practice guidelines. Methods A retrospective cohort study was conducted in the Perth metropolitan area in Western Australia between July 2012 and June 2014 using linked data from ambulance and ED databases. Patients aged 45 years and over, transported to the ED by road ambulance and assigned the ambulance problem code or ED discharge diagnosis of stroke, were selected. Positive predictive value (PPV), negative predictive value (NPV), specificity and sensitivity were calculated. Text fields were examined for documentation that patients met FAST criteria. Results There were 2,217 patients were identified as stroke by paramedics. Of 1834 patients diagnosed as stroke in ED, 876 patients were not identified as stroke by paramedics. Sensitivity for identification of stroke was 958/1834 (52.2%). Of 2,096 patients who were identified as stroke by paramedics and had an ED record, 958 patients were identified as stroke by paramedics and in ED. PPV was 958/2096 (45.7%), NPV 99.5% and specificity 99.4%. Paramedics recorded 2 or 3 stroke signs and symptoms in 1,137 (51%) patients. Conclusion A systematic approach is needed to better identify patients with stroke in the prehospital setting.