This paper is concerned with an efficient numerical method for a class of parabolic integro-differential equations with weakly singular kernels. Due to the presence of the weakly singular kernel, the exact solution has singularity near the initial time t=0. A generalized Crank-Nicolson-type scheme for the time discretization is proposed by designing a product integration rule for the integral term, and a compact difference approximation is used for the space discretization. The proposed method is constructed on the uniform time mesh, but it can still achieve the second-order convergence in time for weakly singular solutions. The unconditional stability and convergence of the method is proved and the optimal error estimate in the discrete L2-norm is obtained. The error estimate shows that the method has the second-order convergence in time and the fourth-order convergence in space. The extension of the method to two-dimensional problems is also discussed. A simple comparison is made with several existing methods. Numerical results confirm the theoretical analysis result and show the effectiveness of the proposed method.