A novel fast multipole boundary element method (BEM) is developed to analyze the stress fields of plane structures with multiple V-notches. To accurately evaluate the singular stresses in the vicinity of the V-notch tip, a novel singular element with multi-order asymptotic terms (SEMAT) is first proposed. The tractions and displacements on the SEMAT are expressed as the asymptotic expansions, where both the first singular term and other higher-order terms are considered. The SEMAT is then introduced into the fast multipole BEM, and several singularity problems of boundary integrals on the SEMAT are treated with caution. Finally, the fast multipole BEM with the SEMAT is used to obtain the whole stress and displacement fields for the plane V-notch/crack structures. Numerical examples show that the present method is accurate for analyzing the V-notch with opening angle of arbitrary size in both single and bonded materials. It is not necessary to use a dense mesh in modeling, and the computational efficiency of the present method is high for large-scale problems.