Local diffusivity of a protein depends crucially on the conformation, and the conformational fluctuations are often non-Markovian. Here, we investigate the Langevin equation with non-Markovian fluctuating diffusivity, where the fluctuating diffusivity is modeled by a generalized Langevin equation under a double-well potential. We find that non-Markovian fluctuating diffusivity affects the global diffusivity, i.e., the diffusion coefficient obtained by the long-time trajectories when the memory kernel in the generalized Langevin equation is a power-law form. On the other hand, the diffusion coefficient does not change when the memory kernel is exponential. More precisely, the global diffusivity obtained by a trajectory whose length is longer than the longest relaxation time in the memory kernel is not affected by the non-Markovian fluctuating diffusivity. We show that these non-Markovian effects are the consequences of an everlasting effect of the initial condition on the stationary distribution in the generalized Langevin equation under a double-well potential due to long-term memory.