FoodChangeLens: CNN-Based Food Transformation on HoloLens

Abstract

In this demonstration, we implemented food category transformation in mixed reality using both image generation and HoloLens. Our system overlays transformed food images to food objects in the AR space, so that it is possible to convert in consideration of real shape. This system has the potential to make meals more enjoyable. In this work, we use the Conditional CycleGAN trained with a large-scale food image data collected from the Twitter Stream for food category transformation which can transform among ten kinds of foods mutually keeping the shape of a given food. We show the virtual meal experience which is food category transformation among ten kinds of typical Japanese foods: ramen noodle, curry rice, fried rice, beef rice bowl, chilled noodle, spaghetti with meat source, white rice, eel bowl, and fried noodle. Note that additional results including demo videos can be see at https://negi111111.github.io/FoodChangeLensProjectHP/