This reseach simulates and analyzes paramaters bit error rate (BER) of 16-QAM and 64-QAM modulation on Additive White Gaussian Noise and Rayleigh Fading channels. This research aims to determine 5G modulation with the level of data quality after the transmission process is carried out. The modulation simulation results obtained will be compared with the theoretical bit error rate (BER). The simulation results obtained from the two channel scenarios used are 16-QAM modulation reaching BER 10-4, AWGN channel only requires 15 dB Eb/N0 and for Rayleigh Fading channel it requires 38 dB Eb/N0. The BER theoretical results obtained for the 16-QAM modulation of the AWGN channel have a difference of 3 dB with the simulation results, while for the Rayleigh Fading channel it is 5 dB. Then, the simulation results of 64-QAM modulation AWGN channel to achieve BER 10-4 requires Eb/N0 of 24.6 dB, Rayleigh Fading of 47 dB. The theoretical results of BER obtained for the 64-QAM modulation of the AWGN channel have a difference of 1 dB with the simulation results, while for the Rayleigh Fading channel it is 0.5 dB. In this study, between 16-QAM and 64-QAM 5G modulation is more suitable to use 16-QAM modulation because it requires less power to achieve the desired BER 10-4.