The explicit structure of the inverse of block tridiagonal matrices is presented in terms of blocks defined by linear recurrence relations. Parallel algorithms are shown which solve block second order linear recurrences without using commutativity. Moreover we investigate the parallel solution of the associated block tridiagonal linear system. Using this theoretical background, the implementation of the algorithms is analyzed both on a small number of processors and on a hypercube. The resulting complexity is given in terms of parallel steps, each consisting of block operations, and the cost due to interprocessor communications is taken into account, too.