In a recent work (Harbola et al 2014 Phys. Rev. E 90 022136), we introduced a minimal random walk model which gives rise to diffusion, sub- and super-diffusion dynamics with a single sweep of a parameter that determines how well the walker follows the past history. Here we investigate the effects of the ‘memory loss’ within the minimal model. To incorporate such a memory-loss we study the dynamics of a random walker which switches states stochastically between two internal states: in one of the states, the walker performs memory driven random walk, while in the other state its dynamics is not affected by the history of previous steps. The model is analytically tractable and the derived exact expressions for the moments give rise to interesting asymptotic dynamic behavior. Our results reveal that even for an infinitesimal possibility for the ‘memory loss’ the subdiffusive behavior is completely lost and its dynamics is characterized only by diffusive and superdiffusive behavior.