We investigate the asymptotic structure of the free Rarita-Schwinger theory in four spacetime dimensions at spatial infinity in the Hamiltonian formalism. We impose boundary conditions for the spin-3/2 field that are invariant under an infinite-dimensional (abelian) algebra of non-trivial asymptotic fermionic symmetries. The compatibility of this set of boundary conditions with the invariance of the theory under Lorentz boosts requires the introduction of boundary degrees of freedom in the Hamiltonian action, along the lines of electromagnetism. These boundary degrees of freedom modify the symplectic structure by a surface contribution appearing in addition to the standard bulk piece. The Poincaré transformations have then well-defined (integrable, finite) canonical generators. Moreover, improper fermionic gauge symmetries, which are also well-defined canonical transformations, are further enlarged and turn out to be parametrized by two independent angle-dependent spinor functions at infinity, which lead to an infinite-dimensional fermionic algebra endowed with a central charge. We extend next the analysis to the supersymmetric spin-(1, 3/2) and spin-(2, 3/2) multiplets. First, we present the canonical realization of the super-Poincaré algebra on the spin-(1, 3/2) multiplet, which is shown to be consistently enhanced by the infinite-dimensional abelian algebra of angle-dependent bosonic and fermionic improper gauge symmetries associated with the electromagnetic and the Rarita-Schwinger fields, respectively. A similar analysis of the spin-(2, 3/2) multiplet is then carried out to obtain the canonical realization of the super-Poincaré algebra, consistently enhanced by the abelian improper bosonic gauge transformations of the spin-2 field (BMS supertranslations) and the abelian improper fermionic gauge transformations of the spin-3/2 field.