No previous study has analyzed the impact of a low caffeine dose ingested before an evening training session on sleep and recovery-stress state. Nine highly trained judo athletes underwent a randomized, double-blind, placebo-controlled crossover experiment in which each athlete acted as their own control. Each athlete performed two identical trials after the ingestion of (i) a placebo and (ii) 3 mg of caffeine per kg of body mass, administered 60 min before an evening randori training session. Sleep was assessed using actigraphy and a Karolinska Sleep Diary (KSD), while the recovery-stress state was assessed using a short recovery and stress scale the morning following the trial. No significant differences were observed in any actigraphy sleep measures between conditions, or in the recovery-stress state (p > 0.05 for all). However, sleep quality assessed using the KSD was worse following caffeine ingestion compared with the placebo (3.0 ± 1.0 vs. 3.9 ± 0.6, respectively; p = 0.03, ES: 1.09). The ingestion of 3 mg/kg of caffeine before an evening training session has no impact on actigraphy-derived sleep measures or recovery-stress state. However, it leads to a substantial decrease in self-reported sleep quality.