The development of high-performance memory devices has played a key role in the innovation of modern electronics. Non-volatile memory devices have manifested high capacity and mechanical reliability as a mainstream technology; however, their performance has been hampered by low extinction ratio and slow operational speed. Despite substantial efforts to improve these characteristics, typical write times of hundreds of micro- or milliseconds remain a few orders of magnitude longer than that of their volatile counterparts. Here we demonstrate non-volatile, floating-gate memory devices based on van der Waals heterostructures with atomically sharp interfaces between different functional elements, achieving ultrahigh-speed programming/erasing operations in the range of nanoseconds with extinction ratio up to 1010. This enhanced performance enables new device capabilities such as multi-bit storage, thus opening up applications in the realm of modern nanoelectronics and offering future fabrication guidelines for device scale up.