Development of a facial recognition model using optimized convolution neural network with Tiny-ML and fingerprint authentication for safe lock entry

Abstract

Safe boxes have been used for centuries to protect valuable items, and their design and construction have evolved over time to improve security. In this study, we carried out the design and implementation of a facial recognition technology using optimized convolutional neural networks (CNNs) with TinyML and fingerprint. The facial recognition system is designed using pre-trained CNN architectures then optimized with TinyML to make it fit into resource constrained edge device, the fingerprint recognition is designed using ridge matching biometric algorithm which fits in perfectly into the microcontroller-based system. The safe box was fabricated using sheet metal and insulated using heat resistive material (fiber glass), components like OV2640 camera module, ESP32-S3, R305 Fingerprint sensor, 4.3 inches touch screen TFT display and a charging module to manage and charge the systems battery were used. The system was implemented to be a level two security and two-way verification system using secured Wi-Fi to access the system before the facial and fingerprint recognition functionality is activated, the sequence of operation can either be facial recognition before fingerprint recognition or fingerprint recognition before facial recognition. Evaluating the system’s facial recognition performance with confusion matrix, of all the optimized CNN architectures like VGGNet, SENet, RestNet, MobileNet-V2, EfficientNet-B0, SENet Architecture performed best outputting an accuracy of 98%, precision of 92%, also SENet unoptimized model performed same, MobileNet-V2 performed optimally in the inference speed test with a speed of 15ms/images with the fingerprint also evaluated using confusion matrix and perform with 98% accuracy. It was noticed that the inference speed of the unoptimized model is faster than that of the TinyML optimized model which is justifiable based on the processing clock speed, memory usage, GPU and architecture of the microcontroller/microprocessor. From the result obtained TinyML proves to be suitable mean for edge AI (artificial intelligence) and is recommended for future research and development.