Inspired by recent molecular dynamic simulations of nanocrystalline solids, a generalized self-consistent polycrystal model is proposed to study the transition of yield strength of polycrystalline metals as the grain size decreases from the traditional coarse grain to the nanometer scale. These atomic simulations revealed that a significant portion of atoms resides in the grain boundaries and the plastic flow of the grain-boundary region is responsible for the unique characteristics displayed by such materials. The proposed model takes each oriented grain and its immediate grain boundary to form a pair, which in turn is embedded in the infinite effective medium with a property representing the orientational average of all these pairs. We make use of the linear comparison composite to determine the nonlinear behavior of the nanocrystalline polycrystal through the concept of secant moduli. To this end an auxiliary problem of Christensen and Lo (J. Mech. Phys. Solids 27 (1979) 315) superimposed on the eigenstrain field of Luo and Weng (Mech. Mater. 6 (1987) 347) is first considered, and then the nonlinear elastoplastic polycrystal problem is addressed. The plastic flow of each grain is calculated from its crystallographic slips, but the plastic behavior of the grain-boundary phase is modeled as that of an amorphous material. The calculated yield stress for Cu is found to follow the classic Hall–Petch relation initially, but as the gain size decreases it begins to depart from it. The yield strength eventually attains a maximum at a critical grain size and then the Hall–Petch slope turns negative in the nano-range. It is also found that, when the Hall–Petch relation is observed, the plastic behavior of the polycrystal is governed by crystallographic slips in the grains, but when the slope is negative it is governed by the grain boundaries. During the transition both grains and grain boundaries contribute competitively.