Nanoengineering of interfaces has become an effective way to tune the thermal boundary conductance (TBC) of heterostructures. However, the same nanostructure design can have opposite impacts on TBCs for different systems. To provide a clue toward a unified explanation, in this work, we directly and explicitly reveal the impacts of nanostructures on mode-dependent phonon TBC contributions. We study four representative types of nanostructures, i.e., (1) an intermediate layer, (2) interfacial interlaced teeth, (3) interfacial atomic mixing, and (4) interfacial atomic defects on two example heterostructures: 28Si/Ge and 6Si/Ge, which have moderate and large phonon frequency mismatches, respectively. We find that most of these nanostructures reduce the TBC of 28Si/Ge while increasing the TBC of 6Si/Ge. Each nanostructure is found to have two competing impacts on an interface—one tends to increase TBC while the other tends to decrease TBC. For example, adding an intermediate layer provides a phonon bridging effect, which tends to increase both elastic and inelastic phonon transmission, but it adds one more interface and, thus, more phonon reflection. As a result, an interlayer decreases the TBC of the 28Si/Ge interface by decreasing the inelastic transmission while increasing both elastic and inelastic transmissions of the 6Si/Ge interface. Other nanostructures with atomic disorder can increase transmission by increasing the contact area but can also decrease transmission by phonon-disorder backscattering. This work unveils the fundamental thermal transport physics across interfaces with nanostructures and sheds light on future interface nanoengineering for electronic devices such as high-power transistors, photodiodes, and supercomputing architectures.