We consider the problem of the probabilistic quantification of dynamical systems that have heavy-tailed characteristics. These heavy-tailed features are associated with rare transient responses due to the occurrence of internal instabilities. Systems with these properties can be found in a variety of areas including mechanics, fluids, and waves. Here we develop a computational method, a probabilistic decomposition-synthesis technique, that takes into account the nature of internal instabilities to inexpensively determine the non-Gaussian probability density function for any arbitrary quantity of interest. Our approach relies on the decomposition of the statistics into a ‘non-extreme core’, typically Gaussian, and a heavy-tailed component. This decomposition is in full correspondence with a partition of the phase space into a ‘stable’ region where we have no internal instabilities, and a region where non-linear instabilities lead to rare transitions with high probability. We quantify the statistics in the stable region using a Gaussian approximation approach, while the non-Gaussian distribution associated with the intermittently unstable regions of phase space is inexpensively computed through order-reduction methods that take into account the strongly nonlinear character of the dynamics. The probabilistic information in the two domains is analytically synthesized through a total probability argument. The proposed approach allows for the accurate quantification of non-Gaussian tails at more than 10 standard deviations, at a fraction of the cost associated with the direct Monte-Carlo simulations. We demonstrate the probabilistic decomposition-synthesis method for rare events for two dynamical systems exhibiting extreme events: a two-degree-of-freedom system of nonlinearly coupled oscillators, and in a nonlinear envelope equation characterizing the propagation of unidirectional water waves.