Evaluation Grade Prediction Method with Limited Information from Experts

Abstract

With the development of artificial intelligence, using machine learning methods to build evaluation models has attracted more and more attention. However, training anevaluation model often needs a lot of labeling samples annotated by experts. It is very difficult to get enough labeling data with a limited group of experts. This paper proposes a method to learn the evaluation model with limited information from experts. This method has two stages. In the first stage, we build a large training set with ordinary people by comparing every two samples. After that, we train a Siamese Network with the paired comparison data set to get a score for each sample. In the second stage, we map the scores to evaluation grades with the help of experts. In the experiments, we use the UCI wine quality data set to evaluate our method. Experimental results demonstrate that we get a basically equivalent accuracy (0.5% decrease) with only1.45% samples labeled byexperts.