This paper investigates the impact of bounded and unknown delay of a J-type thermocouple sensor on controlling a Shape Memory Alloy (SMA) actuated rotary actuator. To compensate for the delay, a nonlinear adaptive predictor is utilized, which predicts the exact temperature of the present moment using the output of the sensor. A robust sliding mode control is then designed to track the reference input, which is the rotation angle of the pulley. In addition to temperature estimation, the inner system identification algorithm of the predictor identifies the heat transfer coefficients of the experimental setup. Experimental tests demonstrate the precise control of the proposed algorithm. The tracking error of the adaptive predictor with sliding mode control is compared to that of a regular sliding mode controller without temperature delay compensation consideration, showing the superior performance of the proposed control algorithm.