Sequential transfer learning based on hierarchical clustering for improved performance in deep learning based food segmentation

Mia S. N. Siemon,Gitte Ravn-Haren,A. S. M. Shihavuddin

doi:10.1038/s41598-020-79677-1

Mia S. N. Siemon, Gitte Ravn-Haren + Show 1 more

Open Access

https://doi.org/10.1038/s41598-020-79677-1

Copy DOI

Journal: Scientific Reports	Publication Date: Jan 12, 2021
Citations: 11	License type: open-access

Affiliation: Technical University of Denmark

Abstract

Accurately segmenting foods from optical images is a challenging task, yet becoming possible with the help of recent advances in Deep Learning based solutions. Automated identification of food items opens up possibilities of useful applications like nutrition intake monitoring. Given large variations in food choices, Deep Learning based solutions still struggle to generate human level accuracy. In this work, we propose a novel Sequential Transfer Learning method using Hierarchical Clustering. This novel approach simulates a step by step problem solving framework based on clustering of similar types of foods. The proposed approach provides up to 6% gain in accuracy compared to traditional network training and generated a robust model performing better in challenging unseen cases. This approach is also tested for segmenting foods in Danish school children meals for dietary intake monitoring as an application.

Highlights

Segmenting foods from optical images is a challenging task, yet becoming possible with the help of recent advances in Deep Learning based solutions
We focus on the segmentation part of this process and propose a novel Sequential Transfer Learning technique that provides significant gain in performance compared to the traditional training approach of Deep Learning networks
Extensive analysis of these features is further conducted based on Hierarchical Clustering (HC) providing the essential foundation for Sequential Transfer Learning (STL)

Summary

Introduction

Segmenting foods from optical images is a challenging task, yet becoming possible with the help of recent advances in Deep Learning based solutions. The proposed approach provides up to 6% gain in accuracy compared to traditional network training and generated a robust model performing better in challenging unseen cases. This approach is tested for segmenting foods in Danish school children meals for dietary intake monitoring as an application. From weights it is converted into the total nutrient intake which is the primitive requirement for further dietary assessment in future applications. We focus on the segmentation part of this process and propose a novel Sequential Transfer Learning technique that provides significant gain in performance compared to the traditional training approach of Deep Learning networks. Particular attention shall be put on the U-Net[7] and D eepLab[8] network models

Objectives

Methods

Results

Conclusion