Training MEMM with PSO: A Tool for Part-of-Speech Tagging

Abstract

Maximum Entropy Markov Models (MEMM) can avoid the assumption of independence in traditional Hidden Markov Models (HMM), and thus take advantage of context information in most text mining tasks. Because the convergence rate of the classic generalized iterative scaling (GIS) algorithm is too low to be tolerated, researchers proposed a lot of improved methods such as IIS, SCGIS and LBFGS for parameters training in MEMM. However these methods sometimes do not satisfy task requirements in efficiency and robustness. This article modifies the traditional Particle Swarm Optimization (PSO) algorithm by using dynamic global mutation probability (DGMP) to solve the local optimum and infinite loops problems and use the modified PSO in MEMM for estimating the parameters. We introduce the MEMM trained by modified PSO into Chinese Part-of-Speech (POS) tagging, analysis the experimental results and find it has higher convergence rate and accuracy than traditional MEMM.