We compute the average characteristic polynomial of the hermitised product of $M$ real or complex Wigner matrices of size $N\times N$ and the average of the characteristic polynomial of a product of $M$ such Wigner matrices times the characteristic polynomial of the conjugate matrix. Surprisingly, the results agree with that of the product of $M$ real or complex Ginibre matrices at finite-$N$, which have i.i.d. Gaussian entries. For the latter the average characteristic polynomial yields the orthogonal polynomial for the singular values of the product matrix, whereas the product of the two characteristic polynomials involves the kernel of complex eigenvalues. This extends the result of Forrester and Gamburd for one characteristic polynomial of a single random matrix and only depends on the first two moments. In the limit $M\to\infty$ at fixed $N$ we determine the locations of the zeros of a single characteristic polynomial, rescaled as Lyapunov exponents by taking the logarithm of the $M$th root. The position of the $j$th zero agrees asymptotically for large-$j$ with the position of the $j$th Lyapunov exponent for products of Gaussian random matrices, hinting at the universality of the latter.