A star with main-sequence mass greater than 25-30 M⊙ may collapse to a black hole of about 10 M⊙ at the final stage of the evolution. About an order of 1 per cent of stellar mass is likely to be in the form of such black holes in galaxies. The presence of even such a small number of 10-M⊙ black holes in a spherical stellar system mainly composed of low-mass main-sequence stars (i.e. M≲1 M⊙) can have significant effects on the dynamical evolution. We have examined the dynamics of two-component stellar systems composed of 0.7-M⊙ main-sequence stars, representing the old population of stars whose main-sequence lifetimes are longer than the Hubble time, and a small fraction of 10-M⊙ black holes. The dynamical friction leads to the segregation of black holes to the core, and the core collapse takes place among the black holes on a time-scale much shorter than that required for a single component cluster. Various physical processes occur as the density of the central cluster of black holes becomes higher: formation of binaries by three-body processes, heating by these binaries, gravitational radiation capture of black hole binary formation and subsequent merger, and tidal capture or disruption of stars by black holes. The ultimate evolution of the two-component stellar system depends on the role of three-body binaries formed among the black holes. In low-velocity-dispersion systems (i.e. ν≲100 km s-1, where v is the three-dimensional velocity dispersion), three-body binaries are eventually ejected from the stellar system after being hardened to a hardness of a few hundred. This picture has to be altered as the velocity dispersion changes. For a system with ν≳100 km s-1, binaries merge by gravitational radiation at some hardness instead of being ejected. The critical hardness, at which the collision time and the merger time become comparable, determines the efficiency of the binary as a heat source. The efficiency is found to be inversely proportional to the velocity dispersion. For clusters without a serious reduction in heating efficiency (i.e. velocity dispersion well below 500 km s-1), heating by three-body binaries has the effect of stopping the core-collapse. The cluster expands, but at a rate set by the half-mass relaxation time of the whole system, which is very long. Thus one obtains a nearly static two-component configuration: a central cluster of black holes surrounded by low-mass clusters. However, such a state would not last longer than the Hubble time if v≳50 km s-1, because most of the black holes would experience binary formation and subsequent mergers. Thus a seed black hole can easily form in the central parts of galaxies with even moderate initial conditions (i.e. νc≳100 km s-1).