Lifelong Semi-supervised Learning for Information Extraction

Abstract

This chapter focuses on lifelong semi-supervised learning. It mainly covers one system, called NELL, which stands for Never-Ending Language Learner [Carlson et al., 2010a, Mitchell et al., 2015]. It is the only lifelong semi-supervised learning system that we are aware of. NELL is also a good example of the systems approach to lifelong machine learning (LML). It is perhaps the only live LML system that has been reading the Web to extract certain types of information (or knowledge) 24 hours a day and 7 days a week since January 2010. Although several efforts have been made by other researchers to read the Web to extract various types of knowledge to build large knowledge bases, e.g., WebKB [Craven et al., 1998], KnowItAll [Etzioni et al., 2004], and YAGO [Suchanek et al., 2007], they are not lifelong learning systems, except ALICE [Banko and Etzioni, 2007]. ALICE works in an LML setting and is unsupervised. Its goal is to extract information to build a domain theory of concepts and their relationships. The extraction in ALICE is done using a set of hand-crafted lexico-syntactic patterns (e.g., "< ? grains > such as buckwheat" and "buckwheat is a < ? food >"). ALICE also has some ability to produce general propositions by abstraction, which deduces a general proposition from a set of extracted fact instances. ALICE's LML feature is realized by updating the current domain theory with new extractions and by using the output of each learning cycle to suggest the focus of subsequent learning tasks, i.e., the process is guided by earlier learned knowledge.