This study employs first-principles calculations to investigate how introducing Yb into aluminum nitride (AlN) leads to a large enhancement in the material’s piezoelectric response (d33). The maximum d33 is calculated to be over 100 pC/N, which is 20 times higher than that of AlN. One reason for such a significant improvement in d33 is the elastic-softening effect, which is indicated by a decrease in the elastic constant, C33. The strain sensitivity (du/dε) of the internal parameter, u, is also an important factor for improving the piezoelectric stress constant, e33. On the basis of mixing enthalpy calculations, YbxAl1−xN is predicted to be more stable as a wurtzite phase than as a rock salt phase at composition up to x ≈ 0.7. These results suggest that Yb can be doped into AlN at high concentrations. It was also observed that the dielectric constant, , generally increases with increasing Yb concentrations. However, the electromechanical coupling coefficient, , only increases up to x = 0.778, which is likely because of the relatively lower values of within this range.