Interfacial interaction between matrix polymer and nanofiller greatly influences the mechanical properties of polymer nanocomposites. Herein, two kinds of functionalized polybutadienes containing carboxyl or hydroxyl groups, PB-COOH and PB-OH, were synthesized via thiol-ene reaction and utilized as matrices to investigate the reinforcing effect of CNCs. Homogeneous PB-COOH/CNC and PB-OH/CNC nanocomposites containing up to 30 wt% CNCs were successfully fabricated. The incorporation of CNCs into the polymers led to remarkably enhanced mechanical properties, while the reinforcing effect of CNCs was more pronounced for PB-COOH than that toward PB-OH. Transmission electron microscopy and theoretical modeling studies revealed that the mechanical enhancement is attributed to the formation of a CNC percolation network within the polymer matrix and the interfacial interaction between matrix and CNCs. Differential scanning calorimetry analysis demonstrated that the polymer-filler interaction in the PB-COOH nanocomposites was stronger than that in the PB-OH systems, which results in a larger amount of constrained polymers in the former system and consequently contributes an increase in storage modulus. Thus, the CNC-CNC and polymer-CNC reactions are both responsible to the mechanical enhancement of the resulting nanocomposites.