As the largest non-gravitational force, solar radiation pressure (SRP) causes significant errors in precise orbit determination (POD) of the BeiDou global navigation satellite system (BDS-3) medium Earth orbit (MEO) satellite. This is mainly due to the imperfect modeling of the satellite’s cuboid body. Since the BDS-3’s inter-satellite link (ISL) can enhance the orbit estimation of BDS-3 satellites, the aim of this study is to establish an a priori SRP model for the satellite body using 281-day ISL observations to reduce the systematic errors in the final orbits. The adjustable box wind (ABW) model is employed to refine the optical parameters for the satellite buses. The self-shadow effect caused by the search and rescue (SAR) antenna is considered. Satellite laser ranging (SLR), day-boundary discontinuity (DBD), and overlapping Allan deviation (OADEV) are utilized as indicators to assess the performance of the a priori model. With the a priori model developed by both ISL and ground observation, the slopes of SLR residual for the China Academy of Space Technology (CAST) and Shanghai Engineering Center for Microsatellites (SECM) satellites decrease from −0.097 cm/deg and 0.067 cm/deg to −0.004 cm/deg and −0.009 cm/deg, respectively. The standard deviation decreased by 21.8% and 26.6%, respectively. There are slight enhancements in the average values of DBD and OADEV, and a reduced β-dependent variation is observed in the OADEV of the corresponding clock offset. We also found that considering the SAR antenna only slightly improves the orbit accuracy. These results demonstrate that an a priori model established for the BDS-3 MEO satellite body can reduce the systematic errors in orbits, and the parameters estimated using both ISL and ground observation are superior to those estimated using only ground observation.