In this work, a new linearized alternating direction implicit (ADI) compact difference method (CDM) is proposed for solving nonlinear two-dimensional (2D) and three-dimensional (3D) partial integrodifferential equation (PIDE) with a weakly singular kernel (WSK). The time derivative is treated by Crank-Nicolson (CN) method and the Riemann-Liouville (R-L) integral by product integration (PI) rule on graded meshes. The linear interpolation combining with Taylor formula is applied in time to deal with nonlinear term v∇v in interval (t0,t1), and linear interpolation concerning two previous time points is employed to deal with v∇v in intervals (tn−1,tn),n≥2. A linearized semi-discrete scheme is obtained, which can achieve second-order convergence in time. Then via introducing two kinds of compact difference operators to discretize the spatial derivatives. To improve the computing efficiency, we construct a ADI compact difference method. It is the first time that the ADI compact difference method is applied for the nonlinear 2D and 3D PIDE with a WSK. The advantage of our proposed scheme is that it not only has second-order accuracy in time and fourth-order accuracy in space, but also fast computational speed, just by solving the linear coupled equations for tridiagonal matrices. In addition, we prove the existence, uniqueness and convergence of 2D scheme. Four numerical examples in 2D and 3D are present to demonstrate our proposed method.