Device-to-device (D2D) communication based on cognitive radio (CR) technology can significantly improve the coverage and spectral efficiency. Existing research on D2D communications mainly focus on optimizing the network quality of service in single-tier networks. However, the exponential growth in data traffic has inspired the move from traditional single-tier cellular networks toward heterogeneous cellular networks. Hence, in this paper, we consider a CR-based HetNet coexisting with cognitive D2D pairs and cellular users, where the cellular users are primary users and D2D pairs are secondary users. Considering quality of experience (QoE) is an important metric to quantify and measure quality of experience from the user perspective, we focus on the QoE optimization of the D2D pairs via the BS association, the discrete power control, and the resource block (RB) assignment. To do so, we first formulate the cross-layer optimization problem to maximize the average QoE of the D2D pairs while satisfying the QoE requirements of cellular users. We then propose the centralized resource allocation, namely the genetic algorithm (GA), and semi-distributed resource allocation method, namely Stackelberg game based algorithm, to solve the non-convex optimization problem. The GA is proposed to ensure the maximum achievable QoE with known channel state information (CSI), whereas the Stackelberg game based algorithm is proposed to cope with the strong needs for distributed D2D solutions with only local CSI of each D2D link. Our proposed algorithms can achieve substantial improvement of QoE performance for D2D pairs via increasing the number of RBs.