Profile Recognition for Accessibility and Inclusivity in Smart Cities Using a Thermal Imaging Sensor in an Embedded System

Abstract

With the modernization of smart cities and the technological advancement of the Internet of Things, we are now reaching a point where technology can be weaved into the fabric of our cities. In this article, we tackle the challenge of using machine learning to recognize the profile of pedestrians based on their gait and silhouette with the use a thermal camera (FLIR Lepton) connected to a Raspberry Pi. We present the additional challenges faced by collecting gait data in a noisy environment, and by taking human identification a level of abstraction higher and recognizing categories of people. The far-reaching implications of such a system in terms of accessibility, inclusivity, and social participation of semi-autonomous populations are discussed. The hardware, software, and cost of the handmade prototypes used for data collection are described. In an effort to take a step toward sustainable smart cities, the possibility of powering this system using solar panels is investigated. This article aims to share the lessons learned throughout the creation and deployment of the system and to share the promising first results obtained by the team. We have reached an accuracy of 74.63% for binary age classification (adult, elder), 93.98% for mobility recognition (mobile subject, subject with reduced mobility), 85.77% for group size estimation (one subject, two subjects, or more), and 77.29% for observed gender recognition (male, female) using a 2-layer CNN without any preprocessing on very low-resolution thermal images.