Joint transmitter and receiver modulation (JSM) is a promising spatial-domain modulation scheme, which is capable of enhancing the spatial diversity, as well as the spatial multiplexing gain for the modulated multiple-input multiple-output (MIMO) systems. In this piece of paper, in order to generalize the JSM for arbitrary size MIMOs, we propose and investigate a novel joint transmit and receive spatial pattern grouping based JSM (SPG-JSM), which is enabled by RF mirrors configured around transmit antennas. To provide the proposed SPG-JSM with further enhanced spatial diversity, a SPG associated alternative null-space beamforming (ANB) strategy is introduced for accomplishing the transmitter preprocessing. In addition to the classical maximum-likelihood detection (MLD), we design a ratio sorting assisted low-complexity detection (RS-LCD) scheme to decrease the implementation cost at the receiver. Furthermore, we analyze the capacity, error performance, and the spatial diversity achievable by the proposed SPG-JSM, provided that the MLD is exploited at the receiver. Simulation outcomes are characterized to demonstrate the advantages of our novel SPG-JSM, and to validate the analytical performance derivations.