TinyOOD: Effective Out-of-Distribution Detection for TinyML

Tiny machine learning (TinyML) has emerged recently for resource constrained Internet of Things (IoT) devices. However, the deployed TinyML model cannot handle out-of-distribution (OOD) inputs appropriately. While many high-accuracy OOD detection methods have emerged, they often ignore the limitations of the deployment environment. In this paper, we propose a novel effective out-of-distribution detection method for TinyML (TinyOOD), which exploits cascading early exit and channel-attention-based neural mean discrepancy (CA-NMD) for dynamic and efficient OOD detection on microcontroller units (MCUs). To demonstrate its effectiveness, we extensively evaluate TinyOOD using four public datasets, one as the in-distribution (ID) dataset and the others as the OOD datasets. Experiments demonstrate that TinyOOD significantly reduces the computations by up to 38.23% in inference while maintaining the performance of OOD detection.