Senescence Reversion in Plant Images Using Perception and Unpaired Data

Abstract

Recent work on using herbarium images for automatic plant identification, and in particular to do domain adaptation to field images, has been promising. A potential way to address such domain adaptation problem is generative: Hallucinate how a herbarium image would have looked like when it was in the field, and use such synthetic, fresh and green image for plant identification purposes. Such generative task, called senescence reversion, has been poorly explored. To our knowledge, it has been studied only in terms of paired data, meaning, with pairs of dry and fresh images of the same specimen. Such paired data is hard to produce, curate and find. In this work we explore herbarium senescence reversion via unpaired data. The lack of pairs at specimen level presents its own challenges, as capturing the intricacies of each species depends on images of different specimens from different domains. We explore learning a mapping from a herbarium image to leaf plant images and vice-versa, aligning two models, one for each herbarium and leaf domain. We experiment against the state-of-the-art paired baseline Pix2Pix which yields a SSIM of 0.8986, compared to our unpaired approach that yields 0.8865, showing very similar results on reconstruction metrics regardless of our approach not having the luxury of pairs by specimen. Additionally, we apply perceptual loss in order to improve the natural look of the synthetic images. The balance of how much perception is good to avoid reconstruction problems is also studied. Lastly, using a new unpaired dataset built by ourselves, our results show that using a low lambda value from 0.025 to 0.05 for perceptual loss, helps getting lower Frachet Inception Distances and higher Inception Scores. To our knowledge, no other work has focused on reverting senescence of herbarium sheet images based on unpaired data.