Accurate segmentation of tumors is significant for radiation therapy treatment planning and clinical decision-making. While deep convolutional neural network-based methods have found valuable applications in automatic medical segmentation, tumor segmentation, especially small tumor segmentation, remains challenging due to deficiencies of current deep learning in convolutional and pooling operations, which often results in the loss of small object information. This research proposes a global hierarchical attention-based method for accurate and automated segmentation of small tumors by exploiting the associations between small tumors and the feature maps of large tumors. This study included 131 patients with liver cancer. The in-plane resolution of the patients' CTs is from 0.55 mm to 1.0 mm and slice spacing from 0.45 mm to 6.0 mm. We randomly selected 100 CT scans as the training set and others as the testing set. Each CT slice of the testing set was separated into groups according to tumor size as follows: 0.1-2.0, 2.1-5.0, 5.1-10.0, and 10.1-20.0 cm. The CT slice without tumor or tumor size > 20 cm were excluded. This work presents a tumor template-based hierarchical attention method to quantify the relation between small and large tumors by computing their feature maps. The relation of small-large tumors can compensate for the information loss of small tumors during the convolutional and pooling operations and improve the performance of small tumor segmentation. Among 20,693 CT slices of the 31 testing patients, 3.0% CT slices with tumors ≤2 cm, 6.7% ≤5 cm, 10.6% ≤10 cm, and 13.4%≤20 cm. We compared our method with six widely used segmentation models. The results show our model outperforms other methods on all sizes of liver tumors, especially for small size tumors: For the 0.1-2.0 cm liver tumor, it achieved 8.4%, 10.0%, 11.3%, 9.1%, 10.9%, and 9.6% improvement compared to Unet, PAN, DeepLabV3, FPN, LinkNet, and PSPNet, respectively. We found that the small-large tumors relation can significantly improve small tumor segmentation, which is valuable for treatment planning, and clinical decision-making. Our experimental results show that our method can significantly improve the accuracy of segmenting small liver tumors compared to existing deep-learning-based models. The method is quite general and can be extended to other types of tumor detection and segmentation. We discovered that the relationship between small and large tumors can significantly enhance the segmentation of small tumors, which has significant value for treatment planning and clinical decision-making. Our experiments demonstrate that our approach significantly improves the accuracy of small liver tumor segmentation compared to existing deep learning-based models. Our method is quite versatile and can be extended to other types of tumor detection and segmentation.