The distinction between gauge independence and gauge covariance in electrodynamics is considered for quantities of physical interest. The gauge independence of directly observable quantities, such as the spectra of operators and energy conserving transition rates, is discussed and contrasted with the gauge covariance of other quantities such as scalar products. The quasigauge-invariance of the classical Lagrangian and the invariance of the Euler-Lagrange equations under the addition of a total time derivative to the Lagrangian are discussed. The consequent inherent nonuniqueness of the Lagrangian and Hamiltonian formulations of classical mechanics is pointed out. A physical interpretation of the explicit gauge dependence of classical canonical momenta and of the expectation values of the corresponding quantum mechanical operators is presented. The gauge independence of energy conserving transition rates calculated in the conventional finite-order time dependent perturbation theory is discussed and illustrated. The gauge dependence of the conventional time dependent transition amplitudes in the presence of electromagnetic fields is then discussed, and gauge independent transition amplitudes are constructed. An alternative formulation of the quantum mechanics of charged particles is obtained in terms of a new gauge independent Hamiltonian, which is, as might be expected, unique only within an arbitrary canonical transformation.