Due to labor market reforms around 2003–4, Japan has a growing group of ‘non-regular’ workers who are easy to fire, and have poor carrier perspectives. This marks a break with the traditional Japanese model of life-time employment that allowed for intensive in-company training and commitment of personnel. Drawing from a national wage structure survey, we find indications that employment of non-regular workers has a negative impact on productivity (proxied by wages), this negative impact being largest under innovation regimes that require a high cumulativeness of knowledge. Our findings are consistent with neo-Schumpeterian research in Europe which concluded that certain labor market rigidities, while being undesirable from a neoclassical perspective, can be useful to innovation. Our paper confirms the impression from earlier research that structural reforms of labor markets along supply-side lines are likely to be one of the reasons for a substantial decline of productivity growth in major OECD countries since about 2004/05.