TinyVers: A Tiny Versatile System-on-Chip With State-Retentive eMRAM for ML Inference at the Extreme Edge

Abstract

Extreme edge devices or Internet-of-Things (IoT) nodes require both ultra-low power (ULP) always-on (AON) processing as well as the ability to do on-demand sampling and processing. Moreover, support for IoT applications, such as voice recognition, machine monitoring, and so on, requires the ability to execute a wide range of machine learning (ML) workloads. This brings challenges in hardware (HW) design to build flexible processors operating in ULP regime. This article presents TinyVers, a tiny versatile ULP ML system-on-chip (SoC) to enable enhanced intelligence at the extreme edge. TinyVers exploits dataflow reconfiguration to enable multi-modal support and aggressive on-chip power management for duty cycling to enable smart sensing applications. The SoC combines an reduced instruction set computer-V (RISC-V) host processor, a 17-tera operations per second per watt (TOPS/W) dataflow reconfigurable ML accelerator, a 1.7- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> W deep sleep wake-up controller (WuC), and an embedded magnetoresistive random access memory (eMRAM) for boot code and ML parameter retention. The SoC can perform up to 17.6 giga operations per second (GOPS) while achieving a power consumption range from 1.7 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> W to 20 mW. Multiple ML workloads aimed for diverse applications are mapped on the SoC to showcase its flexibility and efficiency. All the models achieve 1–2 TOPS/W of energy efficiency with a power consumption below 230 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> W in continuous operation. In a duty-cycling use case for machine monitoring, this power is reduced to below 10 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> W.