We propose a new reionization probe that uses cosmic microwave background (CMB) observations; the cross-correlation between fluctuations in the CMB optical depth which probes the integrated electron density, $\delta\tau$, and the Compton $y$-map which probes the integrated electron pressure. This cross-correlation is much less contaminated than the $y$-map power spectrum by late-time cluster contributions. In addition, this cross-correlation can constrain the temperature of ionized bubbles while the optical-depth fluctuations and kinetic SZ effect can not. We measure this new observable using a Planck $y$-map as well as a map of optical-depth fluctuations that we reconstruct from Planck CMB temperature data. We use our measurements to derive a first CMB-only upper limit on the temperature inside ionized bubbles, $T_{\rm b}\lesssim 7.0\times10^5\,$K ($2\,\sigma$). We also present future forecasts, assuming a fiducial model with characteristic reionization bubble size $R_{\rm b}=5\,$Mpc and $T_{\rm b}=5\times10^4\,$K. The signal-to-noise ratio of the fiducial cross-correlation using a signal dominated PICO-like $y$-map becomes $\simeq7$ with CMB-S4 $\delta\tau$ and $\simeq13$ with CMB-HD $\delta\tau$. For the fiducial model, we predict that the CMB-HD $-$ PICO cross-correlation should achieve an accurate measurement of the reionization parameters; $T_{\rm b}\simeq 49800^{+4500}_{-5100}\,$K and $R_{\rm b}\simeq 5.09^{+0.66}_{-0.79}\,$Mpc. Since the power spectrum of the electron density fluctuations is constrained by the $\delta\tau$ auto spectrum, the temperature constraints should be only weakly model-dependent on the details of the electron distributions and should be statistically representative of the temperature in ionized bubbles during reionization. This cross-correlation could, therefore, become an important observable for future CMB experiments.