Food Image Recognition via Superpixel Based Low-Level and Mid-Level Distance Coding for Smart Home Applications

Jiannan Zheng,Chunsheng Zhu,Z Wang

doi:10.3390/su9050856

Abstract

Food image recognition is a key enabler for many smart home applications such as smart kitchen and smart personal nutrition log. In order to improve living experience and life quality, smart home systems collect valuable insights of users’ preferences, nutrition intake and health conditions via accurate and robust food image recognition. In addition, efficiency is also a major concern since many smart home applications are deployed on mobile devices where high-end GPUs are not available. In this paper, we investigate compact and efficient food image recognition methods, namely low-level and mid-level approaches. Considering the real application scenario where only limited and noisy data are available, we first proposed a superpixel based Linear Distance Coding (LDC) framework where distinctive low-level food image features are extracted to improve performance. On a challenging small food image dataset where only 12 training images are available per category, our framework has shown superior performance in both accuracy and robustness. In addition, to better model deformable food part distribution, we extend LDC’s feature-to-class distance idea and propose a mid-level superpixel food parts-to-class distance mining framework. The proposed framework show superior performance on a benchmark food image datasets compared to other low-level and mid-level approaches in the literature.

Highlights

Food image recognition has been attracting increasing attention as a key component in many smart home applications
Since many food image recognition applications are based on mobile devices, which generally are not suitable for direct employment of deep learning approaches, in this work, we focus on cost efficient low-level and mid-level approaches
We evaluate the performances of the proposed superpixel based Linear Distance Coding (LDC) for low-level food image feature extraction framework and mid-level food parts-to-class distance mining framework

Summary

Introduction

Food image recognition has been attracting increasing attention as a key component in many smart home applications. Low-level local features such as SIFT with common coding frameworks, such as BoW, Soft Assignment BoW and LLC, will introduce information loss due to the feature encoding, which limits the image classification performance. To address this issue, Naive Bayes Nearest Neighbor (NBNN) is introduced to employ an image-to-class distance representation for image classification. LDC is recently introduced in [30] to incorporate NBNN’s distance feature between local feature and class as a more discriminative feature to avoid local feature information loss in the feature encoding process and to improve image classification performance.

Methods

Results

Conclusion