Beurling's theorem characterizes the forward shift invariant subspaces in the Hardy space $H^2$ on the open unit disk $\bold D$. The description is in terms of an inner function, that is, a function in $H^2$ whose nontangential boundary values have modulus $1$ almost everywhere. If $S$ stands for the forward shift $Sf(z)=zf(z)$, then the adjoint $L=S^*$ is the backward shift, $Lf(z)=\break (f(z)-f(0))/z$. The annihilator of a forward shift invariant subspace is then backward shift invariant, and Beurling's theorem leads to a description also of the backward shift invariant subspaces, as noted by R. G. Douglas, H. S. Shapiro and A. L. Shields [Ann. Inst. Fourier (Grenoble) 20 (1970), fasc. 1, 37--76; MR0270196 (42 #5088)]. Whereas the forward invariant subspaces are described primarily in terms of zeros, the backward invariant subspaces are characterized in terms of pseudocontinuations. To be concrete, take $I$ to be the forward invariant subspace of all functions in $H^2$ that vanish along a given finite sequence $A$ of distinct points in $\bold D$. Its annihilator $I^\perp$ is finite-dimensional, and consists of all rational functions with simple poles along the sequence $A^*$ obtained by reflecting $A$ in the unit circle. Then, if we let the finite sequence $A$ ``grow'' to become in the limit a Blaschke sequence plus a negative singular mass on the circle, the annihilator will increase as well, but there will remain a ``connection'' between the behavior inside $\bold D$ and the behavior outside in the exterior disk ${\bold D}_{\rm e}$, the complement of the closed unit disk on the Riemann sphere. The connection is furnished by the pseudocontinuation across the circle: we have a holomorphic Nevanlinna class function on the inside, and a meromorphic Nevanlinna class function on the outside, and they have the same nontangential boundary values almost everywhere on the unit circle. The issue at hand is whether the Hardy space situation is typical of backward invariant subspaces in Banach spaces $\scr B$ of analytic functions on the disk. A dichotomy appears: if $\scr B$ is bigger than the corresponding Hardy space, then the backward invariant subspaces possess pseudocontinuations across the unit circle, whereas if $\scr B$ is smaller, this is no longer generally the case. What happens is best understood in terms of forward invariant subspaces. With the standard Cauchy duality (the extension of the $H^2$-self-duality), we can think of the dual ${\scr B}^*$ of $\scr B$ as a space of holomorphic functions on $\bold D$, and study the forward shift invariant subspaces on ${\scr B}^*$. Let us concentrate on the case when $\scr B$ is a Hilbert space, of Dirichlet or Bergman type; then ${\scr B}^*$ falls into the same category, too. Every forward invariant subspace $\scr M$ of Dirichlet type has index $1$, which means that $S\scr M$ has codimension $1$ in $\scr M$; this is analogous to the $H^2$ case. Apparently, this means that the annihilator $\scr M^\perp$ (which is a backward invariant subspace of a Bergman space) consists of pseudocontinuable functions. However, there are plenty of forward invariant subspaces of a Bergman space which have index bigger than $1$ [see, e.g., H. Hedenmalm, J. Reine Angew. Math. 443 (1993), 1--9; MR1241125 (94k:30092)]. The annihilator of such a forward invariant subspace is a backward invariant subspace of a Dirichlet space, and some playing around with the formulas for pseudocontinuations suggests that in this case, it should not be unique (and hence not exist as a pseudocontinuation). This is then worked out rigorously in the paper.