This article focuses on the introduction of control, authoring, and composition in human-computer music improvisation through the description of a guided music generation model and a reactive architecture, both implemented in the software ImproteK. This interactive music system is used with expert improvisers in work sessions and performances of idiomatic and pulsed music and more broadly in situations of structured or composed improvisation. The article deals with the integration of temporal specifications in the music generation process by means of a fixed or dynamic “scenario” and addresses the issue of the dialectic between reactivity and planning in interactive music improvisation. It covers the different levels involved in machine improvisation: the integration of anticipation relative to a predefined structure in a guided generation process at a symbolic level, an architecture combining this anticipation with reactivity using mixed static/dynamic scheduling techniques, and an audio rendering module performing live re-injection of captured material in synchrony with a non-metronomic beat. Finally, it sketches a framework to compose improvisation sessions at the scenario level, extending the initial musical scope of the system. All of these points are illustrated by videos of performances or work sessions with musicians.