Oral non-neoplastic and neoplastic lesions have similar clinical manifestations, increasing the risk of inaccurate screening decisions that adversely affect oral cancer (OC) outcomes. Tobacco-use-related changes in the oral soft tissues may affect the accuracy of “smart” oral screening modalities. Because smoking is such a strong predictor of OC risk, it may overwhelm the impact of other variables on algorithm performance. The objective was to evaluate the screening accuracy in tobacco users vs. non-users of a previously developed prototype smartphone and machine-learning algorithm-based oral health screening modality. 318 subjects with healthy mucosa or oral lesions were allocated into either a “tobacco smoker” group or a “tobacco non-smoker” group. Next, intraoral autofluorescence (AFI) and polarized white light images (pWLI), risk factors as well as clinical signs and symptoms were recorded using the prototype screening platform. OC risk status as determined by the algorithm was compared with OC risk evaluation by an oral medicine specialist (gold standard). The screening platform achieved 80.0% sensitivity, 87.5% specificity, 83.67% agreement with specialist screening outcome in tobacco smokers, and 62.1% sensitivity, 82.9% specificity, 73.1% agreement with specialist screening outcome in non-smokers. Tobacco use should be carefully weighted as a variable in the architecture of any imaging-based screening algorithm for OC risk.