Massive multiple input multiple output (massive MIMO) is a key technology in fifth-generation (5G) and beyond fifth-generation (B5G) networks. It improves performance metrics such as gain, energy efficiency, spectral efficiency, and bit error rate (BER). Because of the large number of users and antennas, sophisticated processing is required to detect the transmitted message signal. One of the challenges in massive MIMO systems is transmitted message signal detection. To respond to these challenges, several detection algorithms have been developed, including minimum mean squared error (MMSE), zero forcing (ZF), matched filter (MF), conjugate-gradient (CG), gauss-seidel (GS), and optimized coordinate descent (OCD). Although the ZF and MMSE algorithms perform well, their computational complexity is high due to direct matrix inversion. When the number of users is much lower than the number of antennas, the MF algorithm performs well. However, as the number of users increases, the performance of the MF algorithm degrades. Although the OCD, CG, and GS algorithms have less computational complexity than the MMSE algorithm, they perform poorly in comparison. To address and resolve the shortcomings of existing methods, an efficient iterative algorithm has been proposed in this manuscript, which is a hybrid method possessing the combination of MMSE with the alternating direction method of multipliers (ADMM) technique and Gauss-Seidel method. The initial vector has a large influence on the performance, complexity, and convergence rate of such iterative algorithms. The proposed detector’s initial solution is determined using the diagonal matrix and MMSE with the ADMM technique. The proposed algorithm’s performance and complexity are compared with existing algorithms based on BER and the real number of multiplications, respectively. The numerical results revealed that the proposed algorithm achieves the desired performance with a small number of iterations and a significant reduction in computational complexity. At 8QAM, SNR = 20 dB, 80 × 120 massive MIMO antenna configuration, and n = 2, the percentage performance improvement of the proposed detector from the GS detector is 99.82%. At 32QAM, SNR = 25 dB, 120 × 180 antenna configuration, and n = 5, performance improvement of the proposed detector is 99.89%. At 64QAM, SNR = 28 dB, 80 × 120 antenna configuration, and n = 3, performance improvement of the proposed detector is 99.93%.