KGTN-ens: few-shot image classification with knowledge graph ensembles

Abstract

We propose KGTN-ens, a framework extending the recent Knowledge Graph Transfer Network (KGTN) in order to incorporate multiple knowledge graph embeddings at a small cost. There are many real-world scenarios in which the amount of data is severely limited (e.g. health industry, rare anomalies). Prior knowledge can be used to tackle this task. In KGTN, one can use a single knowledge source at once. The purpose of this study is to investigate the possibility of combining multiple knowledge sources. We evaluate it with different embeddings in a few-shot image classification task. Our model is partially trained on k∈{1,2,5,10}\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$k \\in \\{ 1, 2, 5, 10\\}$$\\end{document} samples. We also construct a new knowledge source – Wikidata embeddings – and evaluate it with KGTN and KGTN-ens. With ResNet50, our approach outperforms KGTN in terms of the top-5 accuracy on the ImageNet-FS dataset for the majority of tested settings. For k∈{1,2,5,10}\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$k \\in \\{ 1, 2, 5, 10\\}$$\\end{document} respectively, we obtained +0.63/+0.58/+0.43/+0.26 pp. (novel classes) and +0.26/+0.25/+0.32/–0.04 pp. (all classes).