Small hydropower (SHP) possesses significant economic, technical, and environmental advantages, and accounts for a large proportion of hydropower development in China. However, the concentrated, cascaded, and diversion-type development of SHP has resulted in long-distance dewatering of river sections, and inter-basin water transfers have led to severe exploitation of water resources and damage to river ecosystems. In this paper, the Datong River Basin, a secondary sub-basin of the Yellow River Basin in China, was selected as the illustrative case, which includes 22 hydropower projects (HPPs) and three inter-basin water diversion projects (WDPs). A nexus system model was established that used weighted multi-objective programming to consider three main objectives: the water resources utilization (local water withdrawal and inter-basin water transfer), energy production (by cascaded HPPs), and riverine environmental conservation. The Tennant method was used to estimate the environmental flows (e-flows) at the cross-sections immediately downstream of the dam/sluice gate and immediately downstream of the hydropower plant of diversion-type HPPs. The decreased percentage of regulated flow in comparison with runoff and the guaranteed rate of e-flow at the control cross-section were introduced to assess the degree of environmental impact to the river. Using a historical series of runoff data during 1956–2016 as the model input (i.e., implicit stochastic method), the Multi-start solver of nonlinear programming of LINGO software was used to conduct optimizations and analyses for multiple scenarios (with/without e-flow, with consideration of various levels of e-flow, and with/without water resources utilization). The sectoral linkages relating to the water-energy-ecosystem (WEE) nexus were quantitatively identified. The possible influences of different boundary conditions (i.e., initial/final reservoir storage, inter-basin water diversion capacity, and climate change) on the WEE nexus were further explored. The present study aims to provide an exemplar for the optimal operation and scientific management of a complicated water resources system in a regulated river with cascaded SHP and inter-basin WDPs.