Based on the Crank-Nicolson alternating direction implicit finite volume (CN-ADI-FV) discretization of two dimensional Riesz space-fractional diffusion equations (2 D RSFDEs) proposed in [1], a fast direct solver is proposed to obtain the numerical solutions. Linear systems result from the CN-ADI-FV discretization can be treated as two matrix equations with symmetric positive definite (SPD) Toeplitz coefficient matrices. Then numerical solutions, expressed as multiple matrix-vector multiplications involving the inverse of SPD Toeplitz matrices, can be effectively obtained by using Gohberg-Semencul formula (GSF), where the preconditioned conjugate gradient (PCG) method is applied as an acceleration technique. Moreover, under certain assumptions, the spectral of the preconditioned matrices of the linear system associated with the GSF are theoretically proved to be clustered around 1. Then the PCG method is verified to converge superlinearly and can be implemented with O(NlogN) operations, where N is the number of spatial unknowns in both x- and y- directions. Besides, the PCG method is used only twice in the whole solution process, and the complete solution process can be finished with O(MN2logN) operations and O(N) memory, where M is the number of temporal unknowns. Numerical experiments are given to demonstrate the efficiency of our approach.