We formulate a finite-temperature scheme of the variational cluster approximation (VCA) particularly suitable for an exact-diagonalization cluster solver. Based on the analytical properties of the single-particle Green's function matrices, we explicitly show the branch-cut structure of logarithm of the complex determinant functions appearing in the self-energy-functional theory (SFT) and whereby construct an efficient scheme for the finite-temperature VCA. We first apply the method to explore the antiferromagnetic order in a half-filled Hubbard model by calculating the entropy, specific heat, and single-particle excitation spectrum for different values of on-site Coulomb repulsion $U$ and temperature $T$. We also calculate the $T$ dependence of the single-particle excitation spectrum in the strong coupling region, and discuss the overall similarities to and the fine differences from the spectrum obtained by the spin-density-wave mean-field theory at low temperatures and the Hubbard-I approximation at high temperatures. On the basis of the thermodynamic properties, we obtain a crossover diagram in the $(U,T)$-plane which separates a Slater-type insulator and a Mott-type insulator. Next, we demonstrate the finite-temperature scheme in the cluster-dynamical-impurity approximation (CDIA) and study the paramagnetic Mott metal-insulator transition in the half-filled Hubbard model. Formulating the finite-temperature CDIA, we first address a subtle issue regarding the treatment of the artificially introduced bath degrees of freedom which are absent in the originally considered Hubbard model. We then apply the finite-temperature CDIA to calculate the finite-temperature phase diagram in the $(U,T)$-plane. We find that the Mott transition at low temperatures is discontinuous, and the coexistence region of the metallic and insulating states persists down to zero temperature.