The actual composition is known to be highly responsible for the physical and chemical properties of a fuel. To understand the foundational physical properties of an aviation kerosene substitute mixture for hypersonic aircraft, n-undecane, butylcyclohexane, and 1-pentanol were used to construct a ternary system. The values of density (ρ) and viscosity (η) for the ternary system and three corresponding binaries were measured at temperatures T = (293.15 to 333.15) K and pressure p = 0.1 MPa. The Redlich–Kister equation was used to fit the excess molar volumes (VmE) and viscosity deviations (Δη) of the binary systems, while those of the ternary system were correlated with four semi-empirical formulas (Cibulka, Singh, Redlich–Kister, and Nagata–Tamura equations). The experimental results show that the VmE values of the three binary mixtures have a positive relationship with the mole fraction of nonpolar components, while Δη values have a negative relationship. The non-ideal behavior of mixtures is discussed from the perspective of molecular interactions and structural effects. This work provides data support and guidance for fuel compatibility research.