On the memory complexity of the forward–backward algorithm

Abstract

The Forward–backward (FB) algorithm forms the basis for estimation of Hidden Markov Model (HMM) parameters using the Baum–Welch technique. It is however, known to be prohibitively costly when estimation is performed from long observation sequences. Several alternatives have been proposed in literature to reduce the memory complexity of FB at the expense of increased time complexity. In this paper, a novel variation of the FB algorithm – called the Efficient Forward Filtering Backward Smoothing (EFFBS) – is proposed to reduce the memory complexity without the computational overhead. Given an HMM with N states and an observation sequence of length T, both FB and EFFBS algorithms have the same time complexity, O ( N 2 T ) . Nevertheless, FB has a memory complexity of O ( NT ) , while EFFBS has a memory complexity that is independent of T, O ( N ) . EFFBS requires fewer resources than FB, yet provides the same results.