Cardio-ankle vascular index (CAVI) has been increasingly used as an index for arterial stiffness. However, the degree of longitudinal changes and factors affecting these changes in CAVI (ΔCAVI) are largely unknown. The aim of this study was to descriptively clarify factors associated with longitudinal ΔCAVI for the precise use of this index in the assessment of arterial stiffness. Study participants comprised 9501 community residents (53.1 ± 13.3 years), with 8004 patients included in the longitudinal analysis. CAVI was calculated by the transient time interval of arterial waveform measured at four extremities. Factors showing a strong association with baseline CAVI (7.38 ± 1.10) included age (β = 0.586), SBP (β = 0.175), and male sex (β = 0.142), whereas BMI showed an inverse association (β = -0.208) (all P < 0.001). During the 1815 ± 135 days follow-up period, CAVI increased by 0.47 ± 0.68. Multivariate analysis identified age (β = 0.508), male sex (β = 0.089), Brinkman index (β = 0.074), and SBP (β = 0.037) as positive determinants, whereas BMI (β = -0.079) and baseline CAVI (β = -0.590) as inverse determinants for ΔCAVI (all P < 0.001). Further data accumulation is needed to determine a reference value of CAVI due to its large population differences. Several factors were commonly associated with baseline and longitudinal ΔCAVI. These factors should be carefully considered when CAVI is used for the assessment of arterial stiffness.