Illumination-Insensitive Skin Depth Estimation from a Light-Field Camera Based on CGANs toward Haptic Palpation

Myeongseob Ko,Mingi Kim,Kwangtaek Kim,Donghyun Kim

doi:10.3390/electronics7110336

Myeongseob Ko, Mingi Kim + Show 2 more

Open Access

https://doi.org/10.3390/electronics7110336

Copy DOI

Journal: Electronics	Publication Date: Nov 20, 2018
Citations: 5	License type: CC BY 4.0

Affiliation: Incheon National University, Korea University

Abstract

A depth estimation has been widely studied with the emergence of a Lytro camera. However, skin depth estimation using a Lytro camera is too sensitive to the influence of illumination due to its low image quality, and thus, when three-dimensional reconstruction is attempted, there are limitations in that either the skin texture information is not properly expressed or considerable numbers of errors occur in the reconstructed shape. To address these issues, we propose a method that enhances the texture information and generates robust images unsusceptible to illumination using a deep learning method, conditional generative adversarial networks (CGANs), in order to estimate the depth of the skin surface more accurately. Because it is difficult to estimate the depth of wrinkles with very few characteristics, we have built two cost volumes using the difference of the pixel intensity and gradient, in two ways. Furthermore, we demonstrated that our method could generate a skin depth map more precisely by preserving the skin texture effectively, as well as by reducing the noise of the final depth map through the final depth-refinement step (CGAN guidance image filtering) to converge into a haptic interface that is sensitive to the small surface noise.

Highlights

Because the monitoring of skin surface information, such as color, shape, texture, roughness, and temperature, can be utilized importantly for medical diagnoses of skin lesions and tumors [1,2], skin aging [3,4,5], and the development of cosmetics [6,7,8], its methods have been studied constantly in the field of biomedical research [9]
For light-field images that were taken under diverse illumination conditions, our method outperformed state-of-the-art methods in terms of depth estimation and showed the skin texture more clearly
This study proposes a method to reconstruct accurately the 3D skin surface details by recovering skin surface details weakened by illumination through a deep learning technique, conditional generative adversarial networks (CGANs), in order to improve the quality of light sensitive skin images of the low-resolution light field, which has never been studied before

Summary

Introduction

Because the monitoring of skin surface information, such as color, shape, texture, roughness, and temperature, can be utilized importantly for medical diagnoses of skin lesions and tumors [1,2], skin aging [3,4,5], and the development of cosmetics [6,7,8], its methods have been studied constantly in the field of biomedical research [9]. Methods to acquire various information of a skin surface using cameras have been highlighted as a way to prevent the secondary infection of lesions, reliance on only visual information based on acquired images is limited in the provision of sufficient information to dermatologists and cosmetic professionals. Many studies have been conducted on acquiring 3D information of the skin surface, as in the stereo system [14,16] and multiple-view system [17], limitations have continued to exist in terms of cost and computational complexity in the Electronics 2018, 7, 336; doi:10.3390/electronics7110336 www.mdpi.com/journal/electronics. Most previous studies were aimed at depth estimation of synthetic data or large and rigid objects or scenes, and subsequently proposed a matching-based method by modifying stereo-image-based depth-computation methods to match micro-lens array images [18,19,20,21,22,23]

Methods

Results

Conclusion