Universal Framework for Joint Image Restoration and 3D Body Reconstruction

Jonathan Samuel Lumentut,Matthew Marchellus,Joshua Santoso,Tae Hyun Kim,In Kyu Park,Ju Yong Chang

doi:10.1109/access.2021.3132148

Jonathan Samuel Lumentut, Matthew Marchellus + Show 4 more

Open Access

https://doi.org/10.1109/access.2021.3132148

Copy DOI

Abstract

Universal Framework for Joint Image Restoration and 3D Body Reconstruction

Highlights

The task of 3D body reconstruction is gaining popularity in recent times
We introduce a modular-based plug-and-play universal framework (PPUF) encapsulated within a self-adaptive algorithm capable of receiving any deep-learning-based image restoration methods while jointly carrying the 3D body reconstruction task
We presented a universal framework in the algorithm forms capable of utilizing various prior state-of-the-art restoration modules, in solving denoising, deblurring, and SR, while jointly tasked with 3D body reconstruction modules

Summary

INTRODUCTION

The task of 3D body reconstruction is gaining popularity in recent times. The 3D version of the human body can be parsed directly using only a single image as input. Degraded image case affects negatively unto the body reconstruction output, and most of the state-of-the-art works exclude this constraint One may solve this issue straightforwardly by synthesizing a large corrupted image dataset and utilizing it in a finetuned or newly re-created body network. The pseudo-data, generated directly from the test input image, is split into pseudoclean and pseudo-corrupted information Their elaborations, along with our algorithms, are discussed in the Method section. The scope of our study in the restoration case involves 3 major degradation-solver, namely: denoising, deblurring, and super-resolution (SR) Each of these scenarios is worked jointly with the 3D body reconstruction under the universal framework. We introduce a modular-based plug-and-play universal framework (PPUF) encapsulated within a self-adaptive algorithm capable of receiving any deep-learning-based image restoration methods while jointly carrying the 3D body reconstruction task. We show that using the proposed algorithm and pseudodata alone within the joint framework, both restoration and 3D body reconstruction modules work simultaneously while producing significant quantitative scores and visual quality improvements

RELATED WORKS

PERFORMANCE ANALYSIS a Preliminary

Findings

CONCLUSION