The actuation and control of miniature soft robots are challenging problems due to their limited onboard space and flexible bodies. Smart magnetic materials are promising candidates to address these challenges since they can be powered and guided remotely by magnetic field for functionalities, such as swimming, grasping, and pumping. In this study, we program an undulatory swimming gait into a small rectangular sheet that is made of a flexible magnetic homogeneous composite. The sheet bears a sinusoidal magnetization profile throughout its body and deforms into undulatory shapes in a rotating uniform magnetic field that aligns with its length. The traveling wave-like deformation of the sheet interacts with the surrounding liquid and propels the sheet in a bidirectional nonholonomic swimming gait. Previous studies on this sheet were not able to model the deformation accurately or characterize the swimming systematically due to a lack of understanding of the underlying physical principles involved. For the first time, we develop a model from underlying physical principles to explain and predict the sheet deformation, which enables it to swim at air-water interfaces and generate propulsive forces under water with an additional stiff frame. The swimming capability and maneuverability of the millimeter-scale sheet are demonstrated in experiments, and its swimming performances in various scenarios are characterized quantitatively. The soft swimming sheet can potentially be used for microrobotic tasks, such as delivering cargo or transporting individual cells in poorly accessible workspaces.