Adapting relational logic models using unlabeled data

Abstract

We have developed a framework for tackling the problem of automatically adapting relational logic models, in particular, Markov Logic Network (MLN), from a source domain to a target domains solving the same task using only unlabeled data in the target domain. One characteristic of the problem is that since the data distributions of the two domains are different, there should be different tailor-made relational logic model for each domain. On the other hand, the relational logic models should share certain amount of similarities due to the same goal and similar nature of the data. Unlike ordinary MLN learning methods, which only consider the distribution of the labeled training examples in learning, we also consider the similarities and differences between the labeled examples from the source domain and the unlabeled examples from the target domain. One major idea of our framework is that we aim at maximizing the likelihood of the observations in the unlabeled data in the target domain, and at the same time, minimizing the difference in the model probability distributions between the source and target domains. As a result, the adapted MLN is tailored to the target domain, but it does not deviate far from the source MLN. We have conducted extensive experiments on two different text mining tasks, namely, pronoun resolution and citation matching, showing consistent improvements in the performance of our adapted model.