Due to the characteristics of good concealment ability and strong mobility, multiple, small spherical underwater robot formations play an important role in near coast defense missions, such as cruising, reconnaissance, surveillance, and sensitive target capturing. Referring to the formation problem for underwater small spherical robots with limited energy, perception, and computation abilities, a trajectory tracking-based formation strategy that transforms the complex formation tracking problem into a simple trajectory tracking problem of a single robot is provided. Two layers are designed in the formation tracking strategy. The upper layer is a virtual structure-based formation algorithm. The bottom layer is a tracking controller based on model predictive control (MPC). The formation algorithm is in charge of calculating reference trajectory for each robot in the formation according to the global formation path. The MPC-based dynamic controller for each robot is designed to track the self reference trajectory. Compared with the model predictive control method used for the traditional trajectory tracking problem of a single robot, this paper additionally considers the formation constraints and the internal collision avoidance. In addition, the extended state observer (ESO) is utilized to estimate the lumped disturbance composed of environment disturbance and the inaccurate dynamic model of a small spherical robot. Not only are the numerical simulations based on MATLAB v.2015a, but physical simulations based on self-building multi-spherical robot formation platform are also carried out. Furthermore, through using two small spherical robots, a formation tracing experiment is conducted. All of the results prove that the proposed formation method is feasible and practical for small spherical robots.