Improve Segmentation Accuracy Research Articles

Bone tumor examination based on FCNN-4s and CRF fine segmentation fusion algorithm

Background and objectiveBone tumor is a kind of harmful orthopedic disease, there are benign and malignant points. Aiming at the problem that the accuracy of the existing machine learning algorithm for bone tumor image segmentation is not high, a bone tumor image segmentation algorithm based on improved full convolutional neural network which consists fully convolutional neural network (FCNN-4s) and conditional random field (CRF). MethodologyThe improved fully convolutional neural network (FCNN-4s) was used to perform coarse segmentation on preprocessed images. Batch normalization layers were added after each convolutional layer to accelerate the convergence speed of network training and improve the accuracy of the trained model. Then, a fully connected conditional random field (CRF) was fused to refine the bone tumor boundary in the coarse segmentation results, achieving the fine segmentation effect. ResultsThe experimental results show that compared with the traditional convolutional neural network bone tumor image segmentation algorithm, the algorithm has a great improvement in segmentation accuracy and stability, the average Dice can reach 91.56%, the real-time performance is better. ConclusionCompared with the traditional convolutional neural network segmentation algorithm, the algorithm in this paper has a more refined structure, which can effectively solve the problem of over-segmentation and under-segmentation of bone tumors. The segmentation prediction has better real-time performance, strong stability, and can achieve higher segmentation accuracy.

Target Soybean Leaf Segmentation Model Based on Leaf Localization and Guided Segmentation

The phenotypic characteristics of soybean leaves are of great significance for studying the growth status, physiological traits, and response to the environment of soybeans. The segmentation model for soybean leaves plays a crucial role in morphological analysis. However, current baseline segmentation models are unable to accurately segment leaves in soybean leaf images due to issues like leaf overlap. In this paper, we propose a target leaf segmentation model based on leaf localization and guided segmentation. The segmentation model adopts a two-stage segmentation framework. The first stage involves leaf detection and target leaf localization. Based on the idea that a target leaf is close to the center of the image and has a relatively large area, we propose a target leaf localization algorithm. We also design an experimental scheme to provide optimal localization parameters to ensure precise target leaf localization. The second stage utilizes the target leaf localization information obtained from the first stage to guide the segmentation of the target leaf. To reduce the dependency of the segmentation results on the localization information, we propose a solution called guidance offset strategy to improve segmentation accuracy. We design multiple guided model experiments and select the one with the highest segmentation accuracy. Experimental results demonstrate that the proposed model exhibits strong segmentation capabilities, with the highest average precision (AP) and average recall (AR) reaching 0.976 and 0.981, respectively. We also compare our segmentation results with current baseline segmentation models, and multiple quantitative indicators and qualitative analysis indicate that our segmentation results are better.

A deep learning approach for automatic tumor delineation in stereotactic radiotherapy for non-small cell lung cancer using diagnostic PET-CT and planning CT.

Accurate delineation of tumor targets is crucial for stereotactic body radiation therapy (SBRT) for non-small cell lung cancer (NSCLC). This study aims to develop a deep learning-based segmentation approach to accurately and efficiently delineate NSCLC targets using diagnostic PET-CT and SBRT planning CT (pCT). The diagnostic PET was registered to pCT using the transform matrix from registering diagnostic CT to the pCT. We proposed a 3D-UNet-based segmentation method to segment NSCLC tumor targets on dual-modality PET-pCT images. This network contained squeeze-and-excitation and Residual blocks in each convolutional block to perform dynamic channel-wise feature recalibration. Furthermore, up-sampling paths were added to supplement low-resolution features to the model and also to compute the overall loss function. The dice similarity coefficient (DSC), precision, recall, and the average symmetric surface distances were used to assess the performance of the proposed approach on 86 pairs of diagnostic PET and pCT images. The proposed model using dual-modality images was compared with both conventional 3D-UNet architecture and single-modality image input. The average DSC of the proposed model with both PET and pCT images was 0.844, compared to 0.795 and 0.827, when using 3D-UNet and nnUnet. It also outperformed using either pCT or PET alone with the same network, which had DSC of 0.823 and 0.732, respectively. Therefore, our proposed segmentation approach is able to outperform the current 3D-UNet network with diagnostic PET and pCT images. The integration of two image modalities helps improve segmentation accuracy.

InconSeg: Residual-Guided Fusion With Inconsistent Multi-Modal Data for Negative and Positive Road Obstacles Segmentation

Segmentation of road obstacles, including negative and positive obstacles, is critical to the safe navigation of autonomous vehicles. Recent methods have witnessed an increasing interest in using multi-modal data fusion (e.g., RGB and depth/disparity images). Although improved segmentation accuracy has been achieved by these methods, we still find that their performance could be easily degraded if the two modalities have inconsistent information, for example, distant obstacles that can be viewed in RGB images but cannot be viewed in depth/disparity images. To address this issue, we propose a novel two-encoder-two-decoder RGB-depth/disparity multi-modal network with Residual-Guided Fusion modules. Different from most existing networks that fuse feature maps in encoders, we fuse feature maps in decoder. We also release a large-scale RGB-depth/disparity dataset recorded in both urban and rural environments with manually-labeled ground truth for both negative- and positive-obstacles segmentation. Extensive experimental results demonstrate that our network achieves state-of-the-art performance compared with other networks.

Deep fusion of multi-modal features for brain tumor image segmentation

Accurate segmentation of pathological regions in brain magnetic resonance images (MRI) is essential for the diagnosis and treatment of brain tumors. Multi-modality MRIs, which offer diverse feature information, are commonly utilized in brain tumor image segmentation. Deep neural networks have become prevalent in this field; however, many approaches simply concatenate different modalities and input them directly into the neural network for segmentation, disregarding the unique characteristics and complementarity of each modality. In this study, we propose a brain tumor image segmentation method that leverages deep residual learning with multi-modality image feature fusion. Our approach involves extracting and fusing distinct and complementary features from various modalities, fully exploiting the multi-modality information within a deep convolutional neural network to enhance the performance of brain tumor image segmentation. We evaluate the effectiveness of our proposed method using the BraTS2021 dataset and demonstrate that deep residual learning with multi-modality image feature fusion significantly improves segmentation accuracy. Our method achieves competitive segmentation results, with Dice values of 83.3, 89.07, and 91.44 for enhanced tumor, tumor core, and whole tumor, respectively. These findings highlight the potential of our method in improving brain tumor diagnosis and treatment through accurate segmentation of pathological regions in brain MRIs.

Adversarial Learning of Object-Aware Activation Map for Weakly-Supervised Semantic Segmentation

Recent years have witnessed impressive advances in the area of weakly-supervised semantic segmentation (WSSS). However, most of existing approaches are based on class activation maps (CAMs), which suffer from the under-segmentation problem ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., objects of interest are segmented partially). Although a number of literature works have been proposed to tackle this under-segmentation problem, we argue that these solutions built on CAMs may not be optimal for the WSSS task. Instead, in this paper we propose a network based on the object-aware activation map (OAM). The proposed network, termed OAM-Net, consists of four loss functions (foreground loss, background loss, average pixel and consistency loss) which ensure exactness, completeness, compactness and consistency of segmented objects via adversarial training. Compared to conventional CAM-based methods, our OAM-Net overcomes the under-segmentation drawback and significantly improves segmentation accuracy with negligible computational cost. A thorough comparison between OAM-Net and CAM-based approaches is carried out on the PASCAL VOC2012 dataset, and experimental results show that our network outperforms state-of-the-art approaches by a large margin. The code will be available soon.

Goblet cells segmentation from confocal laser endomicroscopy with an improved U-Net

Gastric intestinal metaplasia (GIM) is regarded as a remarkable precursor for the development of intestinal-type stomach cancer. Goblet cell (GC) segmentation is the crucial step for assessing the degree of GIM by confocal laser endomicroscopy (CLE). However, GC segmentation by hand is difficult, unreliable, and time-consuming. Meanwhile, due to the high resolution and noise interference of CLE images, existing segmentation approaches perform poorly on this task. To tackle those issues, we collected 343 confocal laser endomicroscopy images of 62 patients from a Grade-A tertiary hospital. Each CLE image is manually annotated and then verified three times by skilled medical specialists. Then, U-Net is improved by incorporating the pixel gradient attention mechanism, which focuses on color gradient information around GC and captures color gradient features to direct feature maps in the skip connection layer. At last, the model output vector is used to calculate the possibility map and generate the final segmentation area. Compared with mainstream models, our proposed GC segmentation method from CLE with an improved U-Net (GCSCLE) performs the better segmentation result when tested on our CLE dataset and achieved an IOU of 87.95% and a DICE of 86.64%. Our result shows, the performance of the GCSCLE can be compared with the manual CLE image processing in clinical settings, and it can improve segmentation accuracy and save time and costs.

Deep learning based automatic Debakey classification of aortic dissection using CT plain scan

ObjectiveThis paper presents an automatic segmentation and classification of aortic dissection by utilizing neural network based on CT scanning. MethodologyFirst we present a way of establishing models of indivisible aorta parametrization, designed multiple seed points as the initial parameter and based on the three-dimensional area growth algorithm, finally add morphological processing to realize the segmentation of inner membrane, and then introduced the semantic segmentation framework, and analyzed the characteristics and function of each part, improve the Deeplabv3 + network. This paper analyzes the results of the different experiments with two-dimensional lattices and presents the advantages of being sensitive to differences in shape and position and of not being able to use three-dimensional spatial continuity. Our research establishs the aorta reconstruction, which is simpler for learning the transformation of both the shape and position of, according to a last for three-dimensional CT images interfloor of the entire helix network aorta 3D MPRNet.A new module of multi-scale receptive field information fusion is designed, which realizes the function of acquiring different receptive field features, taking into account details and global features, so that the model obtains the unique ability more suitable for segmentation problems. In the decoder stage, the cross-layer connection mode similar to U-Net is still adopted to realize the fusion of semantic information at high and low levels, which ensures that the network can learn enough features. ResultsThe full convolutional network and traditional algorithms based on domain knowledge of aortic dissection images achieve a robust, automated and high precision CT image aortic dissection segmentation scheme. ConclusionThe large number of decoder parameters is increased, which increases the difficulty of network training and reduces the reasoning speed of the model. At the same time, not all features are useful information, while the high frequency and low frequency, the feature graph only adopts cross-layer connection to connect shallow features to high-level information may bring the redundancy of information and interference with useful information. Therefore, designing an efficient decoder is another research idea to accelerate model inference and improve segmentation accuracy.

Spatial Attention Mechanism and Cascade Feature Extraction in a U-Net Model for Enhancing Breast Tumor Segmentation

In the field of medical imaging, the accurate segmentation of breast tumors is a critical task for the diagnosis and treatment of breast cancer. To address the challenges posed by fuzzy boundaries, vague tumor shapes, variation in tumor size, and illumination variation, we propose a new approach that combines a U-Net model with a spatial attention mechanism. Our method utilizes a cascade feature extraction technique to enhance the subtle features of breast tumors, thereby improving segmentation accuracy. In addition, our model incorporates a spatial attention mechanism to enable the network to focus on important regions of the image while suppressing irrelevant areas. This combination of techniques leads to significant improvements in segmentation accuracy, particularly in challenging cases where tumors have fuzzy boundaries or vague shapes. We evaluate our suggested technique on the Mini-MIAS dataset and demonstrate state-of-the-art performance, surpassing existing methods in terms of accuracy, sensitivity, and specificity. Specifically, our method achieves an overall accuracy of 91%, a sensitivity of 91%, and a specificity of 93%, demonstrating its effectiveness in accurately identifying breast tumors.

Accurate segmentation of infrared images for circuit board diagnosis using an improved Deeplabv3+ network

An effective infrared image segmentation algorithm is essential for non-contact fault diagnosis of circuit boards. However, the uneven grayscale of the infrared images, multiple target regions, and large radiation noise pose challenges to achieving accurate segmentation and efficient data extraction for the interested regions. In this paper, we propose a segmentation algorithm based on the Deeplabv3+ network, using the lightweight MobileNetV2 as a replacement for the original Xception backbone network to improve computational efficiency and reduce overfitting. We also employ a composite loss function and cosine annealing learning rate to balance foreground-background segmentation and avoid local optima. Furthermore, we integrate the Convolutional Block Attention Module (CBAM) to extract and combine important spatial and channel features, allowing the algorithm to focus on identifying elements of the circuit board instead of background pixels, thereby improving segmentation accuracy. Experimental results demonstrate that our proposed algorithm achieves state-of-the-art performance in terms of both segmentation accuracy and computational efficiency on our self-built infrared circuit board dataset, with a MIoU of 90.34%, MPA of 95.26%, and processing speed of 25.19 fps. Overall, our proposed segmentation algorithm can effectively identify the key regions of interest in infrared images of circuit boards, providing technical support for non-contact diagnosis.

Self-Guided Optimization Semi-Supervised Method for Joint Segmentation of Macular Hole and Cystoid Macular Edema in Retinal OCT Images.

Macular hole(MH) and cystoid macular edema(CME) are two common retinal pathologies that cause vision loss. Accurate segmentation of MH and CME in retinal OCT images can greatly aid ophthalmologists to evaluate the relevant diseases. However, it is still challenging as the complicated pathological features of MH and CME in retinal OCT images, such as the diversity of morphologies, low imaging contrast, and blurred boundaries. In addition, the lack of pixel-level annotation data is one of the important factors that hinders the further improvement of segmentation accuracy. Focusing on these challenges, we propose a novel self-guided optimization semi-supervised method termed Semi-SGO for joint segmentation of MH and CME in retinal OCT images. Aiming to improve the model's ability to learn the complicated pathological features of MH and CME, while alleviating the feature learning tendency problem that may be caused by the introduction of skip-connection in U-shaped segmentation architecture, we develop a novel dual decoder dual-task fully convolutional neural network(D3T-FCN). Meanwhile, based on our proposed D3T-FCN, we introduce a knowledge distillation technique to further design a novel semi-supervised segmentation method called Semi-SGO, which can leverage unlabeled data to further improve the segmentation accuracy. Comprehensive experimental results show that our proposed Semi-SGO outperforms other state-of-the-art segmentation networks. Furthermore, we also develop an automatic method for measuring the clinical indicators of MH and CME to validate the clinical significance of our proposed Semi-SGO. The code will be released on Github.

Weakly-Interactive-Mixed Learning: Less Labelling Cost for Better Medical Image Segmentation

Common medical image segmentation tasks require large training datasets with pixel-level annotations which are very expensive and time-consuming to prepare. To overcome such limitation and achieve the desired segmentation accuracy, a novel Weakly-Interactive-Mixed Learning (WIML) framework is proposed by efficiently using weak labels. On one hand, utilize weak labels to reduce annotation time for high-quality strong labels by designing a Weakly-Interactive Annotation (WIA) part of the WIML which prudently introduces interactive learning into the weakly-supervised segmentation strategy. On the other hand, utilize weak labels and very few strong labels to achieve desired segmentation accuracy by designing a Mixed-Supervised Learning (MSL) part of the WIML which can boost the segmentation accuracy by providing strong prior knowledge during training. Besides, a multi-task Full-Parameter-Sharing Network (FPSNet) is proposed to better implement this framework. Specifically, to further reduce annotation time, attention modules (scSE) are integrated into FPSNet to improve the class activation map (CAM) performance for the first time. To further improve segmentation accuracy, a Full-Parameter-Sharing (FPS) strategy is designed in FPSNet to alleviate the overfitting of the segmentation task supervised by very few strong labels. The proposed method is validated on the BraTS 2019 and LiTS 2017 datasets, and experiments demonstrate that the proposed method WIML-FPSNet outperforms several state-of-the-art segmentation methods with minimal annotation efforts. Our code is made publicly available at: https://github.com/NieXiuping/WIML.

Remote sensing image segmentation using feature based fusion on FCM clustering algorithm

AbstractImage segmentation of heterogeneous comparable objects lying beneath the earth’s surface is a fundamental but challenging research area in remote sensing. Learning approaches are used in remote sensing image segmentation to improve segmentation accuracy at the expense of time and a large amount of data, but their performance need to be finely classified due to information diversity constraints. In this work, we proposed an novel feature based fuzzy C-means-extreme learning machine (FBFCM-ELM) algorithm for remote sensing image segmentation in which the classification based on entropy, intensity, and edge features is performed in such a way that it updates the intensity value to preserve the most local characteristics in the image while still being able to clearly distinguish the image’s boundaries by assigning the pixel values of each cluster to the peak value of the cluster’s sub-histogram. Using FBFCM, features are extracted and used as reliable samples for ELM training. Undetermined segmented pixels are obtained using the trained ELM classifier. Experiments performed over number of images that confirmed the proposed method yields a better segmented RGB image, as evidenced by observable details, edges, and improved appearance that resembles the ground truth image and outperforms state-of-the-art algorithms.

Image segmentation technology based on transformer in medical decision‐making system

AbstractDue to the improvement in computing power and the development of computer technology, deep learning has pene‐trated into various fields of the medical industry. Segmenting lesion areas in medical scans can help clinicians make accurate diagnoses. In particular, convolutional neural networks (CNNs) are a dominant tool in computer vision tasks. They can accurately locate and classify lesion areas. However, due to their inherent inductive bias, CNNs may lack an understanding of long‐term dependencies in medical images, leading to less accurate grasping of details in the images. To address this problem, we explored a Transformer‐based solution and studied its feasibility in medical imaging tasks (OstT). First, we performed super‐resolution reconstruction on the original MRI image of osteosarcoma and improved the texture features of the tissue structure to reduce the error caused by the unclear tissue structure in the image during model training. Then, we propose a Transformer‐based method for medical image segmentation. A gated axial attention model is used, which augments existing architectures by introducing an additional control mechanism in the self‐attention module to improve segmentation accuracy. Experiments on real datasets show that our method outper‐forms existing models such as Unet. It can effectively assist doctors in imaging examinations.

Multiscale Feature Fusion for Hyperspectral Marine Oil Spill Image Segmentation

Oil spills have always been a threat to the marine ecological environment; thus, it is important to identify and divide oil spill areas on the ocean surface into segments after an oil spill accident occurs to protect the marine ecological environment. However, oil spill area segmentation using ordinary optical images is greatly interfered with by the absorption of light by the deep sea and the distribution of algal organisms on the ocean surface, and it is difficult to improve segmentation accuracy. To address the above problems, a hyperspectral ocean oil spill image segmentation model with multiscale feature fusion (MFFHOSS-Net) is proposed. Specifically, the oil spill segmentation dataset was created using hyperspectral image data from NASA for the Gulf of Mexico oil spill, small-size images after the waveband filtering of the hyperspectral images were generated and the oil spill images were annotated. The model makes full use of having different layers with different characteristics by fusing feature maps of different scales. In addition, an attention mechanism was used to effectively fuse these features to improve the oil spill region segmentation accuracy. A case study, ablation experiments and model evaluation were also carried out in this work. Compared with other models, our proposed method achieved good results according to various evaluation metrics.

Contouring in the optic plane improves the accuracy of computed tomography-based segmentation of the optic pathway.

Canine optic pathway structures are often contoured on CT images, despite the difficulty of visualizing the optic pathway with CT using standard planes. The purpose of this prospective, analytical, diagnostic accuracy study was to examine the accuracy of optic pathway contouring by veterinary radiation oncologists (ROs) before and after training on optic plane contouring. Optic pathway contours used as the gold standard for comparison were created based on expert consensus from registered CT and MRI for eight dogs. Twenty-one ROs contoured the optic pathway on CT using their preferred method, and again following atlas and video training demonstrating contouring on the optic plane. The Dice similarity coefficient (DSC) was used to assess contour accuracy. A multilevel mixed model with random effects to account for repeated measures was used to examine DSC differences. The median DSC (5th and 95th percentile) before and after training was 0.31 (0.06, 0.48) and 0.41 (0.18, 0.53), respectively. The mean DSC was significantly higher after training compared with before training (mean difference=0.10; 95% CI, 0.08-0.12; P<0.001) across all observers and patients. DSC values were comparable to those reported (0.4-0.5) for segmentation of the optic chiasm and nerves in human patients. Contour accuracy improved after training but remained low, potentially due to the small optic pathway volumes. When registered CT-MRI images are not available, our study supports routine addition of an optic plane with specific window settings to improve segmentation accuracy in mesaticephalic dogs ≥11kg.

LcmUNet: A Lightweight Network Combining CNN and MLP for Real-Time Medical Image Segmentation.

In recent years, UNet and its improved variants have become the main methods for medical image segmentation. Although these models have achieved excellent results in segmentation accuracy, their large number of network parameters and high computational complexity make it difficult to achieve medical image segmentation in real-time therapy and diagnosis rapidly. To address this problem, we introduce a lightweight medical image segmentation network (LcmUNet) based on CNN and MLP. We designed LcmUNet's structure in terms of model performance, parameters, and computational complexity. The first three layers are convolutional layers, and the last two layers are MLP layers. In the convolution part, we propose an LDA module that combines asymmetric convolution, depth-wise separable convolution, and an attention mechanism to reduce the number of network parameters while maintaining a strong feature-extraction capability. In the MLP part, we propose an LMLP module that helps enhance contextual information while focusing on local information and improves segmentation accuracy while maintaining high inference speed. This network also covers skip connections between the encoder and decoder at various levels. Our network achieves real-time segmentation results accurately in extensive experiments. With only 1.49 million model parameters and without pre-training, LcmUNet demonstrated impressive performance on different datasets. On the ISIC2018 dataset, it achieved an IoU of 85.19%, 92.07% recall, and 92.99% precision. On the BUSI dataset, it achieved an IoU of 63.99%, 79.96% recall, and 76.69% precision. Lastly, on the Kvasir-SEG dataset, LcmUNet achieved an IoU of 81.89%, 88.93% recall, and 91.79% precision.

Segmentation of tumor region in respiratory disease by extended algorithm

Introduction: The expansion of pulmonary tumors and their alterations take place in a dynamic manner, so that vigorous segmentation of the obtained images is accurately required. Methods: In this research, an extended algorithm in region growing was executed on CT lung tumors to investigate precise tumor region and edges. First, a new threshold via definition of greater target region around the initial tumor was implemented in MATLAB software. Second, nearby points were settled in an array and then these points were updated established upon the tumor growth to delineate the fresh tumor edges. Here, farthest distance from the center of color intensity point of the initial tumor was selected to grow the region in the algorithm. Third, fresh tumor boundary was determined via an interpolation between these fresh points by sketching lines from the tumor midpoint. Then, the edge correction was implemented and the fresh region was attached to the principal region to attain a segmented tumor exterior. Results: The proposed technique enhanced the tumor recognition by 96% and 91% maximum and minimum accuracy, respectively, in comparison with basilar method. In inclusive algorithm, the percentage of conformity had a positive effect on realization of the threshold value and renewal of the relative amount by 13% enhancement over accuracy assessment. Also when compared to basilar algorithm, it was found that at least 12% of the percentage differences in conformity segment the tumor area in lung CT images. The proposed algorithm with sufficient accuracy accelerates the segmentation process to delineate and improve the tumor edges by growing multiple selected regions. The algorithm also guarantees the independence of the results from the starting point. Conclusion: According to the definition of the center of mass of the tumor color intensity, the proposed extended algorithm may be generalized to the 3D images regardless of the matrix size and the image thickness. The combination of techniques such as machine learning is expected to improve segmentation accuracy for different types of nodule and tumor CT images. Implications for practice: Proposed extended algorithm with sufficient accuracy accelerates the segmentation process to delineate and improve the tumor edges by growing multiple selected regions.

Context Label Learning: Improving Background Class Representations in Semantic Segmentation.

Background samples provide key contextual information for segmenting regions of interest (ROIs). However, they always cover a diverse set of structures, causing difficulties for the segmentation model to learn good decision boundaries with high sensitivity and precision. The issue concerns the highly heterogeneous nature of the background class, resulting in multi-modal distributions. Empirically, we find that neural networks trained with heterogeneous background struggle to map the corresponding contextual samples to compact clusters in feature space. As a result, the distribution over background logit activations may shift across the decision boundary, leading to systematic over-segmentation across different datasets and tasks. In this study, we propose context label learning (CoLab) to improve the context representations by decomposing the background class into several subclasses. Specifically, we train an auxiliary network as a task generator, along with the primary segmentation model, to automatically generate context labels that positively affect the ROI segmentation accuracy. Extensive experiments are conducted on several challenging segmentation tasks and datasets. The results demonstrate that CoLab can guide the segmentation model to map the logits of background samples away from the decision boundary, resulting in significantly improved segmentation accuracy. Code is available.

AGHRNet: An attention ghost-HRNet for confirmation of catch‐and‐shake locations in jujube fruits vibration harvesting

The development of an intelligent jujube fruit harvesting device is a critical step in achieving the whole mechanization process. Catch‐and‐shake harvesting, as an efficient and stable vibration harvesting method, has widely been used to save labor and improve harvesting efficiency in large-scale jujube orchards. However, existing catch‐and‐shake harvesters still rely heavily on the operator's naked eyes to determine the shaking position, which is subjective, highly inefficient and highly labor intensive. To address this problem, this study proposes a computer vision system including truck and branches segmentation, skeleton extraction and vibration position selection to automatically identify appropriate shaking positions. A lightweight attention Ghost-HRNet (AGHRNet) based on deep learning is designed to separate the truck and branches from the complex orchard background. In AGHRNet, an attention Ghost block is proposed to lighten the model and improve segmentation accuracy. Moreover, we also construct a hybrid loss function to solve the pixel-level category imbalance of the trunks, branches and background. To verify the effectiveness of the AGHRNet, ten state-of-the-art semantic segmentation methods, including the FCN8s, SegNet, U-Net, PSPNet, DeepLabv3+, BiSeNet, DDRNet, DANet, SegFormer and HRNet, are compared. Comparative experimental results show that compared with the other state-of-the-art methods, the proposed AGHRNet has higher segmentation accuracy (77.79 % mIoU, 89.46 % mPA) and a smaller model size (9.53 MB) on our datasets. Finally, mask medial axis extraction and the minimum inscribed circle operation are applied to obtain the appropriate vibration location. The computer vision system can provide technical references for catch‐and‐shake harvesting and developing jujube fruit-harvesting robots.