This work analyzed the spatial and temporal variations of the glaciers in the Ebi Lake basin during the period 1964 to 2019, based on the 1st and 2nd Chinese Glacier Inventories (CGI) and remote sensing data; this is believed to be the first long-term comprehensive remote sensing investigation on the glacier change in this area, and it also diagnosed the response of the glaciers to the warming climate by analyzing digital elevation modeling and meteorology. The results show that there are 988 glaciers in total in the basin, with a total area of 560 km2 and average area of 0.57 km2 for a single glacier. The area and number of the glaciers oriented north and northeast are 205 km2 (327 glaciers) and 180 km2 (265 glaciers), respectively. The glaciers are categorized into eight classes as per their area, which are less than 0.1, 0.1–0.5, 0.5–1.0, 1.0–2.0, 2.0–5.0, 5.0–10.0, 10.0–20.0, and greater than 20.0 km2, respectively. The smaller glaciers between 0.1 km2 and 10.0 km2 account for 509 km2 or 91% in total area, and, in particular, the glaciers smaller than 0.5 km2 account for 74% in the total number. The glacial area is concentrated at 3500–4000 m in altitude (512 km2 or 91.4% in total). The number of glaciers in the basin decreased by 10.5% or 116, and their area decreased by 263.29 km2 (−4.79 km2 a−1) or 32% (−0.58% a−1) from 1964 to 2019; the glaciers with an area between 2.0 km2 and 5.0 km2 decreased by the largest, −82.60 km2 or −40.67% in the total area at −1.50 km2 a−1 or −0.74% a−1), and the largest decrease in number (i.e., 126 glaciers) occurs between 0.1 km2 and 0.5 km2. The total ice storage in the basin decreased by 97.84–153.22 km3 from 1964 to 2019, equivalent to 88.06–137.90 km3 water (taking 0.9 g cm−3 as ice mass density). The temperature increase rate in the basin was +0.37 °C decade−1, while the precipitation was +13.61 mm decade−1 during the last fifty-five years. This analysis shows that the increase in precipitation in the basin was not sufficient to compensate the mass loss of glaciers caused by the warming during the same period. The increase in temperature was the dominant factor exceeding precipitation mass supply for ruling the retreat of the glaciers in the entire basin.