Using the solution phase space method, we investigate the thermodynamics of black holes in Einstein-aether-Maxwell theory, for which the traditional Wald method (covariant phase space method) fails. We show the first laws of thermodynamics and definitive entropy expressions at both Killing and universal horizons for some examples of exact black hole solutions, including three-dimensional static charged quasi-BTZ black hole, two four-dimensional static charged black holes and three-dimensional rotating solution. At Killing horizons the entropies are exactly one quarter of the horizon area, but at universal horizons of three-dimensional black holes, the entropies have a corrected term in addition to the one proportional to the horizon area.