Introduction In patients suspected of stroke or transient ischemic attack (TIA), rapid triaging is imperative to improve clinical outcomes. For this purpose, balance-eye-face-arm-speech-time (BEFAST) items are used in out-of-hours primary care (OHS-PC). We explored the risk of stroke and TIA among BEFAST positive patients calling to the OHS-PC, and assessed whether additional predictors could improve risk stratification. Methods This is a cross-sectional study of retrospectively gathered routine care data from telephone triage tape-recordings of patients calling the OHS-PC with neurological deficit symptoms, classified as BEFAST positive. Four models–with the predictors age, sex, a history of cardiovascular or cerebrovascular disease, and cardiovascular risk factors–were fitted using logistic regression to predict the outcome stroke or TIA. Likelihood ratio testing was used to select the best model, which was subsequently internally validated. Results The risk of stroke or TIA diagnosis was 52% among 1,289 BEFAST positive patients, median age 72 years, 56% female sex. Of patients with the outcome stroke/TIA, 24% received a low urgency allocation, while 92% had signs or symptoms when calling. Only the addition of age and sex improved predicting stroke or TIA (internally validated c-statistic 0.72, 95%CI 0.69–0.75). The predicted risk of stroke or TIA remained below 20% in those aged below 40. Females aged 70 or over and males aged 55 or over, had a predicted risk above 50%. Discussion Urgency allocation appears to be suboptimal in BEFAST positive patients calling the OHS-PC. Risk stratification could be improved in this setting by adding age and sex.