This paper investigates the energy efficiency (EE) maximization problem with secrecy constraints for a multi-user underlay cognitive radio network (CRN), where each single-antenna cognitive user is surrounded by a multi-antenna eavesdropper user. Specifically, we aim to maximize the EE of the cognitive base station (CBS), which is defined as the ratio of the achievable rate to total power consumption, under the quality-of-service requirement of primary user, the secrecy rate (SR) constraint of CRN and the transmit power limitation of CBS. The formulated optimization problem is nonconvex and nontrivial due to the fractional form of EE and the subtractive form of SR. To solve this complex non-convex problem, we first convert the original fractional form of the EE into an equivalent subtractive form, and then adopt the difference of two-convex functions (D.C.) method to approximate the subtractive form of SR into a convex one. Finally, a convergent iterative algorithm is proposed to obtain the optimal solution of the considered EE maximization problem. Numerical results showing the effectiveness of the proposed algorithm are given.