We present a different symplectic point of view in the definition of weighted modulation spaces Mmp,q(Rd) and weighted Wiener amalgam spaces W(FLm1p,Lm2q)(Rd). All the classical time-frequency representations, such as the short-time Fourier transform (STFT), the τ-Wigner distributions and the ambiguity function, can be written as metaplectic Wigner distributions μ(A)(f⊗g¯), where μ(A) is the metaplectic operator and A is the associated symplectic matrix. Namely, time-frequency representations can be represented as images of metaplectic operators, which become the real protagonists of time-frequency analysis. In [13], the authors suggest that any metaplectic Wigner distribution that satisfies the so-called shift-invertibility condition can replace the STFT in the definition of modulation spaces. In this work, we prove that shift-invertibility alone is not sufficient, but it has to be complemented by an upper-triangularity condition for this characterization to hold, whereas a lower-triangularity property comes into play for Wiener amalgam spaces. The shift-invertibility property is necessary: Rihaczek and conjugate Rihaczek distributions are not shift-invertible and they fail the characterization of the above spaces. We also exhibit examples of shift-invertible distributions without upper-triangularity condition which do not define modulation spaces. Finally, we provide new families of time-frequency representations that characterize modulation spaces, with the purpose of replacing the time-frequency shifts with other atoms that allow to decompose signals differently, with possible new outcomes in applications.