Morphological features of glands serve as essential diagnostic criteria for colon cancer, and physicians must accurately segment glands in histopathological sections to grade the cancer. However, owing to the diverse gland morphologies and blurred edges of malignant glands, the contextual representations of edges have received limited attention in current segmentation methods, leading to less-than-ideal segmentation results, particularly in malignant cases. We present an edge-based multi-scale enhanced network (BFMSE-Net) for gland segmentation to address these issues. This network is based on U-net and fuses an edge fusion module (BFM) and a multi-scale enhancement module (MSEM). The BFM extracts detailed edge features, fuses edge information, and enhances edge pixel values, which effectively facilitates the segmentation of glands with blurred edges and adhesions. MSEM can handle complex scenarios where gland size and shape significantly change because it adaptively gathers information from various receptive fields in the region of interest. The method applied to evaluate this proposed network was to compare it to current methods through experiments, applying the gland segmentation (GlaS) and colorectal adenocarcinoma gland (CRAG) datasets. The experimental results demonstrate that our proposed method achieves clear gland segmentation, with shape similarity indices of 72.617 and 69.792 on the GlaS and CRAG datasets, respectively. These results show an improvement of 20.627 and 76.978, respectively, over current state-of-the-art methods. In conclusion, BFMSE-Net, which clearly focuses on edge representation, can accurately segment various types of glands.