Driven quantum systems coupled to an environment typically exhibit effectively thermal behavior with relaxational dynamics near criticality. However, a different qualitative behavior might be expected in the weakly dissipative limit due to the competition between coherent dynamics and weak dissipation. In this work, we investigate a driven-dissipative infinite-range Ising model in the presence of individual atomic dissipation, a model that emerges from the paradigmatic open Dicke model in the large-detuning limit. We show that the system undergoes a dynamical crossover from relaxational dynamics, with a characteristic dynamical exponent , to underdamped critical dynamics governed by the exponent in the weakly dissipative regime; a behavior that is markedly distinct from that of equilibrium. Finally, utilizing an exact diagrammatic representation, we demonstrate that the dynamical crossover to underdamped criticality is not an artifact of the mean-field nature of the model and persists even in the presence of short-range perturbations.