Active systems evade the rules of equilibrium thermodynamics by constantly dissipating energy at the level of their microscopic components. This energy flux stems from the conversion of a fuel, present in the environment, into sustained individual motion. It can lead to collective effects without any equilibrium equivalent, some of which can be rationalized by using equilibrium tools to recapitulate nonequilibrium transitions. An important challenge is then to delineate systematically to what extent the character of these active transitions is genuinely distinct from equilibrium analogs. We review recent works that use stochastic thermodynamics tools to identify, for active systems, a measure of irreversibility comprising a coarse-grained or informatic entropy production. We describe how this relates to the underlying energy dissipation or thermodynamic entropy production, and how it is influenced by collective behavior. Then, we review the possibility of constructing thermodynamic ensembles out of equilibrium, where trajectories are biased toward atypical values of nonequilibrium observables. We show that this is a generic route to discovering unexpected phase transitions in active matter systems, which can also inform their design.