A Comparative Approach between Different Computer Vision Tools, Including Commercial and Open-source, for Improving Cultural Image Access and Analysis

Abstract

Digital cultural heritage objects can benefit greatly from the application of Artificial Intelligence such as computer vision based tools to automatically extract valuable information from them. Novel methods and technologies have been used in the last few years to perform image classification, object detection, caption generation, and other techniques on different types of digital objects from different disciplines. In this pilot study, carried out in the context of the Digital Humanities project ChIA, we present an approach for testing different commercial (Clarifai, IBM Watson, Microsoft Cognitive Services, Google Cloud Vision) and open-source (YOLO) computer vision (CV) tools on a set of selected cultural food images from the Europeana collection with regard to producing relevant concepts. The project generally aims at improving access to implicit cultural knowledge contained in images, and increase analysis possibilities for scientific research as well as for content providers and educational purposes. Preliminary results showed that not only quantitative output results are important, but also the quality of concepts generated. Types of digital objects can pose a challenge to CV solutions.