Hypoparathyroidism is one of the major complications after thyroid cancer surgery, which seriously affects patients’ quality of life. How to effectively protect parathyroid function and reduce the incidence of hypoparathyroidism has been a key research area in thyroid surgery. Before the operation, the identification and localization of the parathyroid glands by ultrasound images can help surgeons effectively protect parathyroid glands during thyroidectomy. Computer-aided diagnosis (CAD) systems are often considered useful tools to aid in medical image analysis examinations, which provide new solutions for preoperative identification and localization of the parathyroid glands in ultrasound images with high accuracy. To reduce the incidence of hypoparathyroidism and capture both local and global feature dependencies, we propose a dual-branch contextual-aware network (DCA-Net) with Transformer. Firstly, the Transformer as the long-range feature extractor branch is used as the input image patches as the input sequence for extracting the global context. Meanwhile, we build the residual bottleneck block and Transformer bottleneck on the feature encoder branch to locally aggregate the sampled features on different semantic scales. Finally, we use the channel and spatial fusion module (CSFM) to aggregate information from these two branches. The proposed DCA-Net can effectively mitigate the loss of details due to information recession and establish global and local feature dependencies by utilizing these complementing components. Experiments using our dataset of Ultrasound images of Parathyroid Glands reveal that our method outperforms state-of-the-art baselines consistently.