As a result of new technological developments, drone usage has become common in last-mile logistics activities. Owing to the limited flight range of drones, drones may service customers by using trucks as hubs. Although the truck routing problem with drones (TRP-D) has been studied frequently, studies that have addressed this problem in a multiobjective manner are rare. In this study, multiobjective TRP-D is discussed. In this problem, multiple trucks are considered. Each truck is equipped with multiple heterogeneous drones. A multiobjective mixed-integer linear programming (MILP) model is proposed. Since the problem is NP-hard, a reinforcement learning (RL)-based multiobjective heuristic algorithm is proposed to obtain solutions in large-scale cases. A three-phase heuristic feasible-solution-generation algorithm is proposed. The variable neighbourhood descent (VND) algorithm with RL is proposed as a local search algorithm for the multiobjective optimization problem. The results of the proposed heuristic algorithm are compared with the Pareto optimal solutions for small-sized test problems via the augmented ε-constraint (AUGMECON) method and the CPLEX solver. The proposed algorithm is also compared with state-of-the-art multiobjective heuristic algorithms for large-scale problems. According to the results, the proposed algorithm is an efficient multiobjective heuristic algorithm for the addressed problem.