SAX J2224.9+5421 is an extraordinary neutron star low-mass X-ray binary. Albeit discovered when it exhibited a ~10-s long thermonuclear X-ray burst, it had faded to a 0.5-10 keV luminosity of Lx<8E32 (D/7.1 kpc)^2 erg/s only ~8 hr later. It is generally assumed that neutron stars are quiescent (i.e., not accreting) at such an intensity, raising questions about the trigger conditions of the X-ray burst and the origin of the faint persistent emission. We report on a ~51 ks XMM-Newton observation aimed to find clues explaining the unusual behavior of SAX J2224.9+5421. We identify a likely counterpart that is detected at Lx~5E31 (D/7.1 kpc)^2 erg/s (0.5-10 keV) and has a soft X-ray spectrum that can be described by a neutron star atmosphere model with a temperature of ~50 eV. This would suggest that SAX J2224.9+5421 is a transient source that was in quiescence during our XMM-Newton observation and experienced a very faint (ceasing) accretion outburst at the time of the X-ray burst detection. We consider one other potential counterpart that is detected at Lx~5E32 (D/7.1 kpc)^2 erg/s and displays an X-ray spectrum that is best described by power law with a photon index of ~1.7. Similarly hard X-ray spectra are seen for a few quiescent neutron stars and may be indicative of a relatively strong magnetic field or the occurrence of low-level accretion.