Antimony selenide (Sb<sub>2</sub>Se<sub>3</sub>) is an element-rich, cost-effective, and non-toxic material used as an absorber layer in solar cells. The performance of solar cells is significantly influenced by the transport characteristics of charge carriers. However, these characteristics in Sb<sub>2</sub>Se<sub>3</sub> have not been well understood. In this work, through density functional theory and deformation potential theory, we investigate the hole transport properties of pure Sb<sub>2</sub>Se<sub>3</sub> and As-, Bi-doped Sb<sub>2</sub>Se<sub>3</sub>. The incorporation of as element and Bi element does not introduce additional impurity levels within the band gap. However, the band gaps are reduced in both As-Sb<sub>2</sub>Se3 and Bi-Sb<sub>2</sub>Se<sub>3</sub> due to the band shifts of energy levels. This phenomenon is primarily attributed to the interactions between the unoccupied 4p and 6p states of the doping atoms and the unoccupied 4p states of Se atoms, as well as the unoccupied 5p states of Sb atoms. In this study, we calculate and analyze three key parameters affecting mobility: effective mass, deformation potential, and elastic constants. The results indicate that effective mass has the greatest influence on mobility, with Bi-Sb<sub>2</sub>Se<sub>3</sub> exhibiting the highest average mobility. The average effective mass is highest in As-Sb<sub>2</sub>Se<sub>3</sub> and lowest in Bi-Sb<sub>2</sub>Se<sub>3</sub>. The elastic constants of the As- and Bi-doped Sb<sub>2</sub>Se<sub>3</sub> structures show minimal differences compared with that of the intrinsic Sb<sub>2</sub>Se<sub>3</sub> structure. By comparing the intrinsic, As-doped, and Bi-doped Sb<sub>2</sub>Se<sub>3</sub>, it is evident that doping has a minor influence on deformation potential energy along various directions. The study reveals that the hole mobility in Sb<sub>2</sub>Se<sub>3</sub> displays significant anisotropy, with higher mobilities observed in the <i>x</i>-direction and the <i>y</i>-direction than in the <i>z</i>-direction. This discrepancy is attributed to stronger covalent bonding primarily in the <i>x</i>- and <i>y</i>-direction, while in the <i>z</i>-direction weaker van der Waals forces is dominant. The directions with enhanced charge carrier transport capability contribute to efficient transfer and collection of photo-generated charge carriers. Therefore, our research theoretically underscores the significance of controlling the growth of antimony selenide along specific directions.