Most future quantum devices, including quantum computers, require control that is broadband, meaning that the rate of change of the time-dependent Hamiltonian is as fast or faster than the dynamics it generates. In many areas of quantum physics, including quantum technology, one must include dissipation and decoherence induced by the environment. While Markovian master equations provide the only really efficient way to model these effects, these master equations are derived for constant Hamiltonians (or those with a discrete set of well-defined frequencies). In 2006, Alicky, Lidar, and Zanardi [Phys. Rev. A 73, 052311 (2006)] provided detailed qualitative arguments that Markovian master equations could not describe systems under broadband control. Despite apparently broad acceptance of these arguments, such master equations are routinely used to model precisely these systems. This odd state of affairs is likely due to a lack of quantitative results. Here we perform exact simulations of two- and three-level systems coupled to an oscillator bath to obtain quantitative results. Although we confirm that in general Markovian master equations cannot predict the effects of damping under broadband control, we find that there is a widely applicable regime in which they can. Master equations are accurate for weak damping if both the Rabi frequencies and bandwidth of the control are significantly smaller than the system's transition frequencies. They also remain accurate if the bandwidth of control is as large as the frequency of the driven transition so long as this bandwidth does not overlap other transitions. Master equations are thus able to provide accurate descriptions of many quantum information processing protocols for atomic systems.