The nearly singular integral, arising in simulating thin coatings or close-boundary physical quantities, are not adequately dealt with in the isogeometric boundary element method (IGABEM), especially in 3D problems. In this paper, we propose a semi-analytical approach for the nearly singular integrals of 3D potential problems. We first expand all the kernel items by Taylor series up to second order accuracy. In order to employ the semi-analytical formulae when integrating in parametric space, coordinate (ξ,η) is further transformed to polar coordinate (ρ,θ). We then use the subtraction technique to separate the integrals to near-singular parts and regular parts. For the near-singular parts, a semi-analytical treatment is performed where the integrations with respect to ρ are expressed by analytical formulae recursively, while the ones related to θ are computed by Gaussian quadrature. The remaining regular integrals are treated numerically by the sinh transformation method. By adding them together, we could efficiently handle the nearly singular integrals and therefore obtain accurate close-boundary potentials and flux densities in 3D potential problems. The accuracy of the presented method for nearly singular integrals to a curved element with different orders of singularities, namely the nearly weakly, strongly and highly singular integrals, are first tested. We then further consider potential problems of three typical 3D structures. All the presented results are compared with the recently proposed improved sinh transformation method and analytical solutions. The above numerical examples fully show the efficiency and competitiveness of the presented semi-analytical schemes.