• Softening and strengthening in extremely fine nanograined metals are governed by grain boundary (GB) stability. • Atomic sliding in GB layer induces the softening while dislocation activity in grain interior plays the role of coordinator to fit the deformation dominated by GBs. • The solid solution exerts the negative strengthening (the softening) effect to the nanoscale metals. • The GB segregation gives rise to a notable improvement in strength, and this improvement can be further enhanced by optimizing solute concentration and GB excess. The strength of polycrystalline metals increases with decreasing grain size, following the classical Hall-Petch relationship. However, this relationship fails when softening occurs at very small grain sizes (typically less than 10 to 20 nm), which limits the development of ultrahigh-strength materials. In this work, using columnar-grained nanocrystalline Cu-Ag ‘samples’, molecular dynamics simulations were performed to investigate the softening mechanism and explore the strengthening strategies (e.g., formation of solid solution or grain boundary (GB) segregation) in extremely fine nanograined metals. Accordingly, the softening of pure metals is induced by atomic sliding in the GB layer, rather than dislocation activities in the grain interior, although both occur during deformation. The solid solution lowers the stacking fault energy and increases the GB energy, which leads to the softening of NC metals. GB segregation stabilizes GB structures, which causes a notable improvement in strength, and this improvement can be further enhanced by optimizing the solute concentration and GB excess. This work deepens the understanding of the softening mechanism due to atomic sliding in the GB layer and the strengthening mechanism arising from tailoring the GB stability of immiscible alloys and provides insights into the design of ultrahigh-strength materials.