On batch learning in a binary weight setting

Abstract

Considers the problem of inferring a finite binary sequence w*/spl isin/{-1,1}/sup n/ from a random sequence of half-space data {u/sup (t)//spl isin/{-1,1}/sup n/: /spl ges/0,t/spl ges/1}. In this context, we show that a previously proposed randomised on-line learning algorithm dubbed directed drift [Venkatesh, 1993] has minimal space complexity but an expected mistake bound exponential in n. We show that batch incarnations of the algorithm allow of massive improvements in running time. In particular, using a batch of 1/2 /spl pi/n log n examples at each update epoch reduces the expected mistake bound to /spl Oscr/(n) in a single bit update mode, while using a batch of /spl pi/n log n examples at each update epoch in a multiple bit update mode leads to convergence to w* with a constant (independent of n) expected mistake bound.