This study investigates the use of a tendon-driven continuum robot to deform a soft object, whereas the robot body is deformed into an arbitrary shape to adapt to a constrained environment. A dynamic estimator (DE) is developed to approximate the Jacobian matrix that associates the actuator input with the deformed output of the soft object. This helps solve the singularity problem and reduce the effects of noise. Then a visual predictive controller (VPC) with a reference trajectory is developed to ensure a smooth operation. A linear extended-state observer (ESO) is further designed to measure the robot states, such that the controller can compensate for the estimation error. Simulations and experiments are performed to verify the proposed control approach. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this article is to solve the problem of automatic deformation control of soft objects in restricted environments. The existing soft object deformation control is achieved using rigid robots in an open environment, but rigid robots are difficult to use in specific applications where the environment is restricted (e.g., natural orifice surgery). Flexible continuum robots with mechanical compliance can manipulate soft objects in narrow spaces. However, due to environmental constraints, the robot body may be deformed into any shape regardless of the input of the actuator. To solve the problem, this research provides a new visual servo control strategy that deforms soft objects using a continuum robot in a restricted environment. The proposed method can control a flexible robot to manipulate soft objects while taking into account the change in the robot configuration in a restricted environment.