We obtain a non-trivial bound for cancellations between the Kloosterman sums modulo a large prime power with a prime argument running over very short intervals, which in turn is based on a new estimate on bilinear sums of Kloosterman sums. These results are analogues of those obtained by various authors for Kloosterman sums modulo a prime. However, the underlying technique is different and allows us to obtain non-trivial results starting from much shorter ranges.