ABSTRACTThis paper considers a special boundary element method for Fredholm integral equations of the second kind with singular and highly oscillatory kernels. To accelerate the resolution of the linear system and the matrix-vector multiplication in each iteration, the fast multipole method (FMM) is applied, which reduces the complexity from to . The oscillatory integrals are calculated by the steepest decent method, whose accuracy becomes more accurate as the frequency increases. We study the role of the high-frequency w in the FMM, showing that the discretization system is more well conditioned as high-frequency w increase. Moreover, the larger w may reduce rank expressions from the kernel function, and decrease the absolute errors. At last, the optimal convergence rate of truncation is also represented in this paper. Numerical experiments and applications support the claims and further illustrate the performance of the method.