Tropical countries such as Malaysia experience a high amount of rainfall that can lead to a common type of distress on the roads known as moisture damage. Using waste materials as additives in the asphalt mixture is an innovative measure to improve the quality and strength of the mixture. In this study, 5% of crumb rubber (CR) by the total weight of binder was added to the asphalt mixture in three different sizes of 0.075 mm, 0.15 mm, and 0.3 mm. The wet method was applied to blend the CR with virgin asphalt binder grade 60/70. The aggregate gradation applied in this study was 9.5 Nominal maximum aggregate size. The samples were produced with the Superpave gyratory compactor. Moisture sensitivity (ASSTHO T-283) and indirect tensile strength (ITS) tests were performed to evaluate the impact of the CR sizes on the asphalt mixture. The findings showed that the CR addition improved the strength of the asphalt mixture. Furthermore, there was an increment in the ITS with an increasing size of CR. The addition of 5% CR resulted in a minor reduction of the moisture resistance. Thus, a slight improvement in the moisture resistance was detected with a reduction in CR size. Overall, all asphalt mixtures in this study had TSR of more than 80%, thus passing the moisture damage test requirements.