The response of suction caisson (bucket) foundations subjected to horizontal head displacement, or head rotation, is parametrically studied by means of three-dimensional finite-element analyses. The undrained clay stratum is either homogeneous or linearly inhomogeneous (Gibson soil). The effect of the nature of the interface between foundation and surrounding soil is explored with two different scenarios: an ideal case of a fully bonded contact (FBC), and a possibly more realistic tensionless and sliding interface of limited shear resistance (TSI). The latter scenario results in a significant reduction of failure loads. Closed-form expressions are developed for the maximum horizontal and moment capacity for a range of bucket aspect ratios in both soil types, and with both interface conditions. They are validated through comparisons with available results from the literature. The contribution of the suction caisson’s cylindrical sidewall shell to the overall response of the system is investigated. It is shown that for an embedment depth exceeding the caisson’s radius the shell provides practically the same lateral bearing capacity as the whole suction caisson, in contrast with the vertical (axial) loading of a caisson with sliding interface in which the role of the shell is much less significant.