The European Commission has adopted the European Green Deal strategy, which aims to achieve climate neutrality in the EU by 2050. To achieve this goal, it is necessary to shift the economy toward the use of green and renewable energy. Critical raw materials (CRMs), Li, Co, REE, Te, Sc and others, are used in renewable energy sources (RES) production. The EU lacks its own CRM deposits, and additionally, the access to already identified deposits is limited, which is making the EU countries search for alternative CRM sources. One such source of CRMs may be mining waste generated on the Indonesian island of Bangka as a result of processing cassiterite-bearing sands. Studies of the mineral composition of the waste using the XRD method reveal rich contents of xenotime (0.79–17.55 wt%), monazite (1.55–21.23 wt%), zircon (1.87–64.35 wt%) and other minerals, carriers of valuable metals, such as Sn, Ti, Nb, Ta. The point mineral chemistry analyses were performed using EPMA. Xenotime is the main carrier of heavy rare earth elements (HREE), especially the “most critical” HREEs: Gd2O3 (1.42–7.16 wt%), Dy2O3 (2.28–11.21 wt%), Er2O3 (2.44–7.85 wt%), and Yb2O3 (1.71–7.10 wt%). Xenotime is characterized by a complex internal structure resulting from metasomatic processes occurring during their formation. In SEM-BSE imaging, they show zonation of internal structure, which is the effect of an HREE, Y, Si and U substitution in the crystal structure. On the other hand, thorite ThSiO4 and uranothorite (Th,U)SiO4 inclusions are present in xenotimes. The ICP-MS/ES studies of tailings reveal very high contents of HREE + Y (up to 7.58 wt%), U (up to 0.11), Th (up to 0.75 wt%) and Sc (132 ppm). A CRM source diversification is part of the strategy to ensure the security of raw materials for countries of the European Union and the green transformation of the continent. Bilateral EU–Indonesia cooperation in the geological exploration and development of primary and secondary sources may contribute to an increase in the supply of HREEs to the global market.