In this paper, we present an adaptive control method for a conically shaped dielectric elastomer actuator (CSDEA) with different loads to achieve its tracking control objective. Firstly, a dynamic model of the CSDEA is constructed to describe its asymmetrical, rate-dependent hysteresis behavior and creep behavior simultaneously. Then, an offline parameter identification method based on the nonlinear-least-squares algorithm is presented to obtain the nominal values of the model parameters of the CSDEA corresponding to the load of 200 (g). The obtained values are regarded as the initial values of the adaptive online parameter identification. Next, an adaptive online parameter identification method (AOPIM) based on the least-mean-square algorithm is proposed, which can dynamically calculate and update the model parameter values to cope with the load changing of the CSDEA. Lastly, based on two kinds of analytical inverses of the dynamic model and the proposed AOPIM, two adaptive inverse compensators are respectively designed to realize the tracking control of the CSDEA. The control experiments with different desired trajectories and different loads are implemented to demonstrate the effectiveness of the presented adaptive control method. Since the root-mean-square errors of the results of all control experiments are lower than 2.7%, the presented method is remarkable from the perspective of the practical application. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The dielectric elastomer material has beneficial properties of high energy density, low mass density, large deformation, fast response and good biological compatibility. Thus, the soft actuator based on the dielectric elastomer material (SADE) has shown great potentials for being used in soft robots. Nevertheless, the SADE has the complex nonlinear behaviors, which brings a big challenge for its precision control. To deal with this issue, some approaches have been presented in previous literatures. However, the load of the SADE is usually fixed in these studies. This paper focuses on the tracking control of the SADE with different loads. To this end, a dynamic model of the CSDEA is constructed, whose parameter values are dynamically calculated and updated to cope with the load changing of the CSDEA. Through calculating the analytical inverse of the dynamic model, two adaptive inverse compensators are developed. The control experimental results demonstrate that the presented adaptive control method is effective. Considering that the load of the SADE is usually diverse in practical applications, this study pays the way for the application of the SADE.