In irradiating the prostate and pelvic lymph node regions, registration based on bony structures matches the pelvic lymph node regions but not necessarily the prostate position, and it is important to identify factors that influence prostate displacement. Therefore, we investigated factors influencing prostate displacement during volumetric modulated arc therapy after single-fraction high-dose-rate brachytherapy (HDR-BT) for prostate cancer and the trends in displacement for each fraction. Seventy patients who underwent pelvic volumetric modulated arc therapy of 46 Gy in the prone position 15 days after 13 Gy HDR-BT were included. Prostate displacement relative to bony structures was calculated using cone beam computed tomography. Systematic error (SE) and random error (RE) were evaluated in the right-left (RL), craniocaudal (CC), and anteroposterior (AP) directions. The association with clinical and anatomic factors on the planning computed tomography or magneticresonanceimaging was analyzed. Prostate volume change (PVC) was defined as the volume change at 2 days after HDR-BT. Displacement trends were individually examined from the first to 23rd fractions. The mean SE in the RL, CC, and AP directions was -0.01 mm, -2.34 mm, and -0.47 mm, respectively. The root mean square of the RE in the RL, CC, and AP directions was 0.44 mm, 1.14 mm, and 1.10 mm, respectively. SE in the CC direction was independently associated with bladder volume (P=.021, t statistic=2.352) and PVC (P < .001, t statistic=-8.526). SE in the AP direction was independently associated with bladder volume (P=.013, t statistic=-2.553), PVC (P < .001, t statistic=5.477), and rectal mean area (P=.008, t statistic=2.743). RE in the CC direction was independently associated with smoking (P=.035). RE in the AP direction was associated with PVC (P=.043). Gradual displacement caudally and posteriorly occurred during the irradiation period. Anatomic characteristics of the bladder, rectum, and prostate predict SE. Smoking and PVC predict RE. In particular, whether PVC is ≥140% affects setting internal margins.