Recently, machine learning-based prediction models have gained considerable popularity in predicting the compressive strength of concrete using results from Non-Destructive Testing (NDT) methods. This research considers NDT results from Rebound Hammer (RH) and Ultrasonic Pulse Velocity (UPV) tests for concrete samples with different concrete grades. 270 test samples were considered for the analysis using different machine learning techniques such as Multi-Layer Perceptron (MLP), Long Short-Term Memory network (LSTM), and Convolutional Neural Network (CNN). Concrete compressive strength results are obtained from Destructive Test (DT) results of the same for the purpose of labelling. Each model's performance is evaluated through the computation of the Coefficient of Determination (R²), Mean Square Error (MSE), and Mean Absolute Percentage Error (MAPE). The findings of this study underscore the potential of machine learning techniques in predicting concrete compressive strength based on Non-Destructive Test (NDT) results. The analysis findings suggested that employing UPV and RH data in conjunction with a trained MLP model to be an effective approach for accurately estimating the compressive strength of concrete specimens. The outcomes of the research serve as a valuable reference for researchers and industry practitioners alike when assessing in-situ concrete compressive strength through Non-Destructive Testing (NDT) methods.