A deeply entrenched axiom in the theory of learning states that the more one learns the easier it is to learn. In other words, the more proficient one becomes in performing familiar tasks, the easier it is to learn new tasks. This phenomenon, long recognized by psychologists and educators, has also been demonstrated in machine learning, especially in self-taught classification tasks, where unlabeled samples are used during training (Thrun, 1996; Ando and Zhang 2005; Caruana 1997). It has been shown, for example, that the performance of an image classifier can be improved by initially giving it unlimited access to unlabeled, randomly chosen images downloaded from the Internet (Raina etal 2007). Evidently, at this initial stage of the process the classifier learns generic relationships, applicable to all visual patterns and, subsequently, when the classifier is trained with limited samples from a specific target domain (say, to distinguish elephants from rhinos), the classifier makes use of its previously learned knowledge to perform the target task more efficiently.
Read full abstract