The current research investigates the potential advantages of optimally combining wide bandgap Al2O3 with high dielectric constant (high-κ) Ta2O5 for gate dielectric applications. Various compositions of 10 nm AlxTayO oxide films are grown on the Al0.3Ga0.7N/GaN heterostructure by co-sputtering Al and Ta metals, followed by thermal oxidation at 500 °C. The average root-mean-square roughness of the grown oxide films is ∼1.2–1.4 nm compared to 0.4 nm for as-deposited metal. X-ray photoelectron spectroscopy and transmission electron microscopyconfirm the formation and thickness of the grown oxide films. The bandgap (Eg) of the oxide films calculated from O1s electron energy loss spectra show a linear increase from 4.85 eV for pure Ta2O5 to 6.4 eV for pure Al2O3. The dielectric constant (εox) calculated from capacitance–voltage (CG−VG) measurements decreases linearly from 25.7 for Ta2O5 to 7.9 for Al2O3. The interface trap density (Dit) is estimated from the frequency dispersion of capacitance–voltage (CG−VG) characteristics. The DC and radio frequency (RF) characteristics of AlxTayO/Al0.3Ga0.7N/GaN high electron mobility transistor (HEMT) devices are measured and compared with the Schottky HEMT devices (without any gate oxide). Compared to Schottky HEMT devices, AlxTayO/Al0.3Ga0.7N/GaN HEMT devices show superior DC characteristics, which helps us achieve maximum RF output power. Furthermore, the OFF-state measurements show that the AlxTayO/Al0.3Ga0.7N/GaN HEMT devices can sustain higher source-to-drain voltages, from a minimum of 88 V on pure Ta2O5 metal-oxide-semiconductor (MOS)-HEMTs to a maximum of 138 V on pure Al2O3 MOS-HEMTs before the dielectric breakdown happens, compared to a 57 V breakdown voltage on Schottky HEMT devices. An oxide variation of AlxTayO, with Al composition ratio of 0.34, shows an exceptional ION/IOFF ratio of 4 × 1011, a gate leakage current of 8 × 10−12 A/mm, a near-ideal subthreshold slope of 63.8 mV/dec, and the unity current gain frequency (fT) of 25.6 GHz.