Quantum two-level systems (TLSs) are present in the materials of qubits and are considered defects because they limit qubit coherence. For superconducting qubits, the quintessential Josephson junction barrier is made of amorphous alumina, which hosts TLSs. However, TLSs are not understood generally -- either structurally or in atomic composition. In this study, we greatly extend the quantitative data available on TLSs by reporting on the physical dipole moment in two alumina types: polycrystalline $\mathrm{\mathrm{\gamma-Al}_{2}\mathrm{O}_{3}}$ and amorphous $\mathrm{a-Al}\mathrm{O_{x}}$. To obtain the dipole moments $p_z$, rather from the less-structural coupling parameter g, we tune individual TLSs with an external electric field to extract the $p_z$ of the TLSs in a cavity QED system. We find a clear difference in the dipole moment distribution from the film types, indicating a difference in TLS structures. A large sample of approximately 400 individual TLSs are analyzed from the polycrystalline film type. Their dipoles along the growth direction $p_z$ have a mean value of 2.6$\pm$0.3 Debye (D) and standard deviation $\sigma$ = 1.6$\pm$0.2 D . The material distribution fits well to a single Gaussian function. Approximately 200 individual TLSs are analyzed from amorphous films. Both the mean $p_z$ =4.6$\pm$0.5 D and $\sigma$ =2.5$\pm$0.3 D are larger. Amorphous alumina also has some very large $p_z$, > 8.6 D, in contrast to polycrystalline which has none of this moment. These large moments agree only with oxygen-based TLS models. Based on data and the candidate models (delocalized O and hydrogen-based TLSs), we find polycrystalline alumina has smaller ratio of O-based to H-based TLS than amorphous alumina.