AbstractIn this paper acceleration and computer memory reduction of an algorithm for the simulation of laminar viscous flows and heat transfer is presented. The algorithm solves the velocity–vorticity formulation of the incompressible Navier–Stokes equations in 3D. It is based on a combination of a subdomain boundary element method (BEM) and single domain BEM. The CPU time and storage requirements of the single domain BEM are reduced by implementing a fast multipole expansion method. The Laplace fundamental solution, which is used as a special weighting function in BEM, is expanded in terms of spherical harmonics. The computational domain and its boundary are recursively cut up forming a tree of clusters of boundary elements and domain cells. Data sparse representation is used in parts of the matrix, which correspond to boundary‐domain clusters pairs that are admissible for expansion. Significant reduction of the complexity is achieved.The paper presents results of testing of the multipole expansion algorithm by exploring its effect on the accuracy of the solution and its influence on the non‐linear convergence properties of the solver. Two 3D benchmark numerical examples are used: the lid‐driven cavity and the onset of natural convection in a differentially heated enclosure. Copyright © 2008 John Wiley & Sons, Ltd.