In this paper we prove a result on lower semicontinuity of pullback attractors for dynamical systems given by semilinear differential equations in a Banach space. The situation considered is such that the perturbed dynamical system is non-autonomous whereas the limiting dynamical system is autonomous and has an attractor given as union of unstable manifold of hyperbolic equilibrium points. Starting with a semilinear autonomous equation with a hyperbolic equilibrium solution and introducing a very small non-autonomous perturbation we prove the existence of a hyperbolic global solution for the perturbed equation near this equilibrium. Then we prove that the local unstable and stable manifolds associated to them are given as graphs (roughness of dichotomy plays a fundamental role here). Moreover, we prove the continuity of this local unstable and stable manifolds with respect to the perturbation. With that result we conclude the lower semicontinuity of pullback attractors.