The paper explores the stability of a tower obtained by stacking identical rectangular blocks on top of each other with an inherent randomness due to slight positional offsets between successive blocks. With probabilistic modeling techniques, the diffusive behavior of the stacking process is studied and the collapse is seen as a first passage time problem. In the considered model, alignment errors are idealized as independent Gaussian random variables with zero mean and given standard deviation. We derive expressions for the joint probability density functions of block positions, and analyze their correlation. The study extends to the stochastic stability of a stack of given height, by exploring the statistical characteristics of the center of gravity of the part of tower above each block. Eventually, the probabilistic analysis of collapse is developed to quantify the statistics of the number of blocks that can be heaped up before the tower topples. Although this problem may initially appear playful, it offers an illustrated introduction to first passage problems on a non homogenous process. From a practical standpoint, this analysis offers a simple understanding of the influence of alignment errors on the overall stability of a stack, which may find several fields of application.