We propose a general model to investigate the joint impact of viral diffusion and cell-to-cell transmission on viral dynamics. The mathematical challenge lies in the fact that the model system is partially degenerate and the solution map is not compact. While the simpler cases with only indirect transmission mode or weak cell-to-cell transmission mode have been extensively studied in the literature, it remains an open problem to understand the local and global dynamics of fully coupled viral infection model with partial degeneracy. In this paper, we identify the basic reproduction number as the spectral radius of the sum of two linear operators corresponding to direct and indirect transmission modes. It is well-known that viral mobility may induce infection in low-risk regions. However, as diffusion coefficient increases, we prove that the basic reproduction number actually decreases, which indicates that faster viral movements may result in a lower level of viral infection. By an innovative construction of Lyapunov functionals, we further demonstrate that the basic reproduction number is the threshold parameter which determines global picture of viral dynamics. In addition to the traditional dichonomy results of extinction and persistence as obtained in earlier works for many simpler models, we are able to prove global asymptotic stability of infection-free steady state and global attractiveness (as well as uniqueness) of chronic-infection steady state, depending on whether the basic reproduction number is smaller or greater than one. Numerical simulation supports our theoretical results and suggests an interesting phenomenon: boundary layer and internal layer may occur when the diffusion parameter tends to zero.