Spin qubits defined by valence band hole states are attractive for quantum information processing due to their inherent coupling to electric fields, enabling fast and scalable qubit control. Heavy holes in germanium are particularly promising, with recent demonstrations of fast and high-fidelity qubit operations. However, the mechanisms and anisotropies that underlie qubit driving and decoherence remain mostly unclear. Here we report the highly anisotropic heavy-hole g-tensor and its dependence on electric fields, revealing how qubit driving and decoherence originate from electric modulations of the g-tensor. Furthermore, we confirm the predicted Ising-type hyperfine interaction and show that qubit coherence is ultimately limited by 1/f charge noise, where f is the frequency. Finally, operating the qubit at low magnetic field, we measure a dephasing time of T2*\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${T}_{2}^{* }$$\\end{document} = 17.6 μs, maintaining single-qubit gate fidelities well above 99% even at elevated temperatures of T > 1 K. This understanding of qubit driving and decoherence mechanisms is key towards realizing scalable and highly coherent hole qubit arrays.