Sedentary behavior (SB) may affect arterial stiffness, preceding the development of cardiovascular disease. We investigated the association of objectively measured SB with arterial stiffness. We also investigated factors that affected this association. We recruited adult volunteers and measured SB with thigh-worn accelerometery for 24 hrs/day for eight consecutive days. Central (carotid-femoral pulse wave velocity, cfPWV, gold standard) and local carotid arterial stiffness (stiffness index Beta and pressure-strain elasticity EP) were measured with ultrasound. Linear regression was used and adjusted for demographics, cardiometabolic factors, and moderate-to-vigorous physical activity (MVPA) volume. Effect modification was studied with interaction terms. Participants (N = 664, 64 (standard deviation: 11, range: 23-89) years, 397 (59.8%) male) demonstrated 9.1 (1.6) hrs/day of SB, and arterial stiffness was 8.6 (3.0) m/s for cfPWV, 6.4 (2.9) for Beta, and 87 (43) kPa for EP. SB was not associated with cfPWV (β = 0.04 95% CI (-0.11, 0.18), p = 0.60). The association of SB with local arterial stiffness was modified by systolic blood pressure (SBP) and MVPA volume. Stratified analyses revealed positive associations of SB with Beta (β = 0.29 (0.05, 0.53), p = 0.016) and EP (β = 4.83 (1.39, 8.27), p = 0.006) in participants with SBP > 134 mmHg or > 103 min/day of MVPA (β = 0.23 (0.03, 0.42), p = 0.024 and β = 3.55 (0.82, 6.29), p = 0.011, respectively). We found no association of objectively measured SB with central arterial stiffness. However, SB was positively associated with local carotid stiffness in participants with higher SBP or MVPA levels. In certain subgroups, SB may affect carotid arterial stiffening, reinforcing the relation between SB and cardiovascular disease.