In this work, we establish and compare the stochastic and deterministic robustness properties achieved by nominal model predictive control (MPC), stochastic MPC (SMPC), and a proposed constraint-tightened MPC (CMPC) formulation, which represents an idealized version of tube-based MPC. We consider three definitions of robustness for nonlinear systems and bounded disturbances: robust asymptotic stability (RAS), robust asymptotic stability in expectation (RASiE), and RASiE w.r.t. the stage cost <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell (\cdot)$</tex-math></inline-formula> used in these MPC formulations ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell$</tex-math></inline-formula> -RASiE). Via input-to-state stability (ISS) and stochastic ISS (SISS) Lyapunov functions, we establish that MPC, subject to sufficiently small disturbances, and CMPC ensure all three definitions of robustness without a stochastic objective function. While SMPC also ensures RASiE and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell$</tex-math></inline-formula> -RASiE, SMPC does not guarantee RAS for nonlinear systems. Through a few simple examples, we illustrate the implications of these results and demonstrate that, depending on the definition of robustness considered, SMPC is not necessarily more robust than nominal MPC even if the disturbance model is exact.