Object classification and visualization with edge artificial intelligence for a customized camera trap platform

Abstract

The camera traps have revolutionized the image and video capture in ecology and are often used to monitor and record animal presence. With miniaturization of low power electronic devices, better battery technologies, and software advancements, it has become possible to use the edge devices, such as Raspberry Pi as camera traps that can not only capture images and videos, but can also enable sophisticated image processing, and off-site communications. These developments can help to provide near real-time insights and reduce the manual processing of images. The on-board image classification and visualization is facilitated by the advancements in the Deep Neural Networks (DNN), transfer learning approaches, and software libraries.This paper provides an investigation of image classification with transfer learning approaches using pre-trained DNN models, and visualizations with Explainable Artificial Intelligence (XAI) techniques on Raspberry Pi Zero (RPi-Z) edge device. The MobileNetV2 model was used for image classification on the Florida-Part1 dataset obtaining the results for precision, recall, and F1-score as 0.95, 0.96, and 0.95 respectively. We also compared the model performance of MobileNetV2, EfficientNetV2B0, and MobileViT models for classification on the Extinction dataset with the best results for precision, recall, and F1-score as 0.97, 0.96, and 0.96 respectively, obtained with the EfficientNetV2B0 model. Two XAI techniques, Gradient-weighted-Class Activation Mapping (Grad-CAM) and Occlusion Sensitivity were used for visualization through heatmaps, to highlight the relative importance of the image areas contributing to the DNN model's prediction, that can also help to understand the model's performance and bias. The results provide practical use case scenarios for utilizing the transfer learning approaches, model optimization and deployment to edge devices, and model visualizations in ecological research.