In the era of the digital economy, the cross-boundary innovation activities of many SMEs across industries and fields have become the focus of academic attention. However, existing studies have neglected to explore the evolutionary and development of cross-boundary innovation models from a process perspective. In particular, the induction and summary of the enterprise digital cross-boundary innovation path will provide guidance for the innovation and development of SEMs, which has become the core problem to be solved in this study. This study embeds dynamic capabilities into the evolution process of enterprise ecological niche. Through a qualitative study on the three cross-boundary processes of the digital publishing projects of traditional publishing enterprises under the background of digital transformation, it reveals the growth mechanism and evolution path of dynamic capability affecting enterprise niche change in the process of cross-boundary innovation of SMEs. The results show that:(1) Under the background of digital transformation, the niche structure of SMEs is influenced by technology factor, resource factor, demand factor, and network factor; (2) In the process of cross-boundary innovation, the coupling of enterprise multi-dimensional dynamic capability and niche factors drive the migration of enterprise niche; (3) The development path of cross-boundary innovation of SMEs is the process of niche dynamic adjustment, such as niche expansion, niche dislocation and niche construction. This research conclusion reveals the development law of cross-boundary innovation of SMEs, further enriches the research system of cross-boundary innovation and provides theoretical guidance for the digital transformation of SMEs and the implementation of cross-boundary development strategy.