The epidemic of infectious diseases has a significant impact on society, the economy, and people’s lives. Social media, with its high user participation and rapid information dissemination, plays a crucial role in shaping public opinion. Fine-grained sentiment analysis of public opinion on infectious diseases can provide valuable insights for improving the quality of public services. However, there are few relevant studies on Chinese data due to language complexity and low resources. Moreover, most of the existing approaches utilize the Graph Neural Network (GCN) method by syntactic dependency trees to construct graphs of text, which ignore the potential link relationships between aspects and words. Therefore, to address this limitation, in this article, we propose a new method based on GCN using aspect-specific heterogeneous graphs, named ASHGCN, which combines BiLSTM, heterogeneous graphs, GCN, the mask and the attention mechanism. We mine social media posts related to COVID-19 for aspect-based sentiment analysis task (ABSA) for ten aspect entity types in both Chinese and English data. The heterogeneous graph is designed with two node types (aspect nodes and non-aspect nodes) and four edge connection types, including various relationships between aspect entities, and between aspect entities and non-aspect entities. In addition, we release a Chinese dataset and an English dataset that include medical and named entities, along with corresponding sentiment labels. Experiments on our datasets, as well as two public datasets, demonstrate that our method greatly improves performance in the ABSA task. Ablation experiments and case studies further support the effectiveness of the proposed approach.