We describe an adaptive control framework for changing-contact robot manipulation tasks that require the robot to make and break contacts with objects and surfaces. The piecewise continuous interaction dynamics of such tasks make it difficult to construct and use a single dynamics model or control strategy. Also, the nonlinear dynamics during contact changes can damage the robot or the domain objects. Our framework enables the robot to incrementally improve its prediction of contact changes in such tasks, efficiently learn models for the piecewise continuous interaction dynamics, and to provide smooth and accurate trajectory tracking based on a task-space variable impedance controller. We experimentally compare the performance of our framework against that of representative control methods to establish that the adaptive control, prediction, and incremental learning capabilities of our framework are essential to achieve the desired smooth control of changing-contact robot manipulation tasks.