Segmentation Model Research Articles

A semantic segmentation network based on CNN and Swin transformer for landslide detection

Abstract To address the issues of poor object localization, difficulty in object recognition, and inadequate segmentation performance of CNN-based semantic segmentation methods in landslide detection, a semantic segmentation model based on the Swin transformer and ResUNet architectures for landslide detection in remote sensing images is proposed in this paper. This method combines the global feature capabilities of the Swin transformer with the local feature extraction abilities of CNN and integrates our proposed RCCT module for landslide segmentation in remote sensing images. We apply this model to the identification and extraction of landslide disasters in Menyuan County and the Sanjiangyuan region of Qinghai Province, China, achieving a promising F1 Score of 81.91. Furthermore, we conducted a comprehensive evaluation of our proposed model on the BiJie Landslide dataset in comparison to four recently developed methods for landslide recognition. The experimental findings indicate the superior performance and effectiveness of our model over the comparative approaches.

Artificial intelligence model for enhancing the accuracy of transvaginal ultrasound in detecting endometrial cancer and endometrial atypical hyperplasia

ObjectivesTransvaginal ultrasound is typically the initial diagnostic approach in patients with postmenopausal bleeding for detecting endometrial atypical hyperplasia/cancer. Although transvaginal ultrasound demonstrates notable sensitivity, its specificity remains limited. The objective...

Improved semantic segmentation method for weld penetration prediction of TIG welding with dual ellipsoid heat source

TIG welding is a manufacturing process that uses argon as a shielding gas and tungsten as an electrode to join metals at high temperatures. The weld penetration is the key to determining the quality of the weld. However, the lack of sensing technology makes weld penetration difficult to predict, which imposes a major challenge to process stability and weld quality. To address this challenge, a semantic segmentation-based approach for calibrating the weld penetration prediction model under dual ellipsoidal heat source is proposed to improve the prediction accuracy. To realize this, a semantic segmentation model Self-Attention-Structural-Reparameterization-BiSeNet (SASR-BiSeNet) is built for molten pool parameter extraction. The model introduces a self-attention mechanism as an enhancement to the convolution module. Meanwhile, the RepVGG-style structural reparameterization design decouples the training-time and inference-time architectures. Further, a mask feature extraction method is designed to obtain the calibration parameters of the molten pool and to correct the prediction model. The accuracy of the calibrated prediction model is verified by welding experiments using 304L steel materials on a robotic welding system. The results show that the maximum Mean Intersection over Union (mIoU) of the SASR-BiSeNet is 92.51 %, which is a 13.87 % improvement compared to the BiSeNet, and the maximum depth of molten pool error decreasing from 16.59 % to 9.84 %. This method can improve the precision of welding penetration control and plays an important role in promoting welding intelligence.

Self-supervised learning-based re-parameterization instance segmentation for hazardous object in terahertz image

Recently, terahertz imaging technology has shown widespread potential applications in the public security screening field. Previous terahertz hazardous object detection techniques primarily relied on manual recognition and image processing methods. Some solutions incorporating deep learning technologies depend on large quantities of high-quality data, making it challenging to achieve low-cost and high-performance detection. To solve the issue of dependency on large amounts of high-quality data, we propose a diversified dynamic structure You Only Look Once (DDS-YOLO) model based on masked image modeling and structural reparameterization for the instance segmentation of hazardous objects in terahertz security inspection images. To address the scarcity and annotating difficulty problems of terahertz security inspection image samples, we combine automatic data strategies with masked image modeling for self-supervised learning. We propose a multilevel feature refinement fusion mechanism to enhance the quality of learned feature representations. The backbone parameter transfer and fine-tuning training strategies are employed to achieve hazardous object instance segmentation on the terahertz dataset. To address the low detection accuracy issue caused by the poor terahertz image quality, we develop a reparameterizable hierarchical structure for the backbone, improve a multi-scale feature-integrating neck, and design a dynamically decoupled head with lower computational requirements to enhance the performance of the instance segmentation model. Experimental results demonstrate that the proposed model accurately outputs detection boxes, categories, and segmentation masks for hazardous objects with minimal training samples. The comparative experimental results indicate that the proposed model outperforms existing state-of-the-art methods in terms of detection performance. The proposed DDS-YOLO model achieves 59.3 % in mask mean Average Precision (mAP) and 61.3 % in box mAP, and the model parameters and computational requirements also meet practical application scenarios.

Advancing MRI segmentation with CLIP-driven semi-supervised learning and semantic alignment

Precise segmentation and reconstruction of multi-structures within MRI are crucial for clinical applications such as surgical navigation. However, medical image segmentation faces several challenges. Although semi-supervised methods can reduce the annotation workload, they often suffer from limited robustness. To address this issue, this study proposes a novel CLIP-driven semi-supervised model, that includes two branches and a module. In the image branch, copy-paste is used as data augmentation method to enhance consistency learning. In the text branch, patient-level information is encoded via CLIP to drive the image branch. Notably, a novel cross-modal fusion module is designed to enhance the alignment and representation of text and image. Additionally, a semantic spatial alignment module is introduced to register segmentation results from different axial MRIs into a unified space. Three multi-modal datasets (one private and two public) were constructed to demonstrate the model’s performance. Compared to previous state-of-the-art methods, this model shows a significant advantage with both 5% and 10% labeled data. This study constructs a robust semi-supervised medical segmentation model, particularly effective in addressing label inconsistency and abnormal organ deformations. It also tackles the axial non-orthogonality challenges inherent in MRI, providing a consistent view of multi-structures.

RS-SLAM: real time semantic slam with driverless car using LiDAR-camera-IMU sensing

Abstract Accurate and robust Simultaneous Localization and Mapping (SLAM) technology is a critical component of driverless cars, and semantic information plays a vital role in their analysis and understanding of the scene. In the actual scene, the moving object will produce the shadow phenomenon in the mapping process. The positioning accuracy and mapping effect will be affected. Therefore, we propose a semantic SLAM framework combining LiDAR, IMU, and camera, which includes a semantic fusion front-end odometry module and a closed-loop back-end optimization module based on semantic information. An improved image semantic segmentation algorithm based on Deeplabv3+ is designed to enhance the performance of the image semantic segmentation model by replacing the backbone network and introducing an attention mechanism to ensure the accuracy of point cloud segmentation. Dynamic objects are detected and eliminated by calculating the similarity score of semantic labels. A loop closure detection method based on semantic information is proposed to detect key semantic features and use threshold range detection and point cloud re-matching to establish the correct loop closure detection, and finally reduce the global cumulative error and improve the global trajectory accuracy using graph optimization to ultimately obtain the global motion trajectory and realize the construction of 3D semantic maps. We evaluated it on the KITTI dataset and collected a dataset for evaluation by ourselves, which includes four different sequences. The results show that the proposed framework has good positioning accuracy and mapping effect in large-scale urban road environments.

Application of multimodal deep learning and multi-instance learning fusion techniques in predicting STN-DBS outcomes for Parkinson's disease patients

Parkinson's Disease (PD) is a progressive neurodegenerative disorder with substantial impact on patients' quality of life. Subthalamic nucleus deep brain stimulation (STN-DBS) is an effective treatment for advanced PD, but patient responses vary, necessitating predictive models for personalized care. Recent advancements in medical imaging and machine learning offer opportunities to enhance predictive accuracy, particularly through deep learning and multi-instance learning (MIL) techniques. This retrospective study included 127 PD patients undergoing STN-DBS. Medical records and imaging data were collected, and patients were categorized based on treatment outcomes. Advanced segmentation models were trained for automated region of interest (ROI) delineation. A novel 2.5D deep learning approach incorporating multi-slice representation was developed to extract detailed ROI features. Multi-instance learning fusion techniques integrated predictions across multiple slices, combining radiomics and deep learning features to enhance model performance. Various machine learning algorithms were evaluated, and model robustness was assessed using cross-validation and hyperparameter optimization. The MIL model achieved an area under the curve (AUC) of 0.846 for predicting STN-DBS outcomes, surpassing the radiomics model's AUC of 0.825. Integration of MIL and radiomics features in the DLRad model further improved discriminative ability to an AUC of 0.871. Calibration tests showed good model reliability, and decision curve analysis demonstrated clinical utility, affirming the model's predictive advantage. This study demonstrates the efficacy of integrating MIL, radiomics, and deep learning techniques to predict STN-DBS outcomes in PD patients. The multimodal fusion approach enhances predictive accuracy, supporting personalized treatment planning and advancing patient care.

Detection of artificial spots in fundus images using modified U-Net based semantic segmentation

Today, deep learning algorithms are playing a very crucial role in the initial stage diagnosis of several fundus diseases like glaucoma, hypertension, and diabetic retinopathy. Lots of research is going on in this area now-a-days. During the acquisition of fundus images some artificial spots (e.g. because of the device itself, dust particles in the surroundings) have been added in captured images. In this paper, artificial spots in the fundus images that are generated due to the non-standardized conditions of scanning devices, are detected with the help of a newly proposed modified UNet (mU-Net) semantic segmentation model. Initially, preprocessing methods such as Gaussian blur, thresholding, and Hough transform have been used to create artificial spots. Now these preprocessed images have been used for training the proposed model. To make the proposed model more effective, the following modifications like, Regularization techniques (early stopping, greater weight decay, and Adam optimizer), decay learning rate scheduler, categorical cross-entropy loss function, and a significant number of filters have been modified in simple U-Net model. Apart from these mentioned modifications in the base U-Net model, a feature injecting module (FIM) has been added between the expansion and the contraction section of the simple U-Net model. FIM adds the features of the input image at the time of up-sampling. The addition of FIM to a simple U-Net model improves the detection of artificial spots and enhances the performance of the model. The mU-Net has been compared with other models, namely simple U-Net, V-Net, UNet++, ResUnet-a, WideU-Net, and Swin-Unet. The Friedman test that has been conducted on IOU, DICE, MAE, PSNR, and SSIM scores, found that mU-Net balances evaluation metrics well. It appears that the nonparametric Friedman test will improve reproducibility by demonstrating statistical significance. The IOU, DICE, MAE, PSNR, and SSIM scores of the proposed model indicate superior performance as compared to other models.

Neural Memory State Space Models for Medical Image Segmentation.

With the rapid advancement of deep learning, computer-aided diagnosis and treatment have become crucial in medicine. UNet is a widely used architecture for medical image segmentation, and various methods for improving UNet have been extensively explored. One popular approach is incorporating transformers, though their quadratic computational complexity poses challenges. Recently, State-Space Models (SSMs), exemplified by Mamba, have gained significant attention as a promising alternative due to their linear computational complexity. Another approach, neural memory Ordinary Differential Equations (nmODEs), exhibits similar principles and achieves good results. In this paper, we explore the respective strengths and weaknesses of nmODEs and SSMs and propose a novel architecture, the nmSSM decoder, which combines the advantages of both approaches. This architecture possesses powerful nonlinear representation capabilities while retaining the ability to preserve input and process global information. We construct nmSSM-UNet using the nmSSM decoder and conduct comprehensive experiments on the PH2, ISIC2018, and BU-COCO datasets to validate its effectiveness in medical image segmentation. The results demonstrate the promising application value of nmSSM-UNet. Additionally, we conducted ablation experiments to verify the effectiveness of our proposed improvements on SSMs and nmODEs.

Improved ground truth annotation by multimodal image registration from 3D ultrasound to histopathology for resected tongue carcinoma.

This study's objectives are (1) to investigate the registration accuracy from intraoperative ultrasound (US) to histopathological images, (2) to assess the agreement and correlation between measurements in registered 3D US and histopathology, and (3) to train a nnUNet model for automatic segmentation of 3D US volumes of resected tongue specimens. Ten 3D US volumes were acquired, including the corresponding digitalized histopathological images (n = 29). Based on corresponding landmarks, the registrations between 3D US and histopathology images were calculated and evaluated using the target registration error (TRE). Tumor thickness and resection margins were measured based on three annotations: (1) manual histopathological tumor annotation (HTA), manual 3D US tumor annotation, and (2) the HTA registered in the 3D US. The agreement and correlation were computed between the measurements based on the HTA and those based on the manual US and registered HTA in US. A deep-learning model with nnUNet was trained on 151 3D US volumes. Segmentation metrics quantified the model's performance. The median TRE was 0.42mm. The smallest mean difference was between registered HTA in US and histopathology with 2.16mm (95% CI - 1.31; 5.63) and a correlation of 0.924 (p < 0.001). The nnUNet predicted the tumor with a Dice similarity coefficient of 0.621, an average surface distance of 1.15mm, and a Hausdorff distance of 3.70mm. Multimodal image registration enabled the HTA's registration in the US images and improved the agreement and correlation between the modalities. In the future, this could be used to annotate ground truth labels accurately.

CAPSULE NETWORK FOR BRAIN TUMORS SEGMENTATION IN MRI IMAGES

Glioma is an abnormal, irregularly shaped cell growth in the brain. Diagnosing glioma often relies on magnetic resonance imaging (MRI), which requires precise segmentation of the tumor for effective analysis. Convolutional Neural Networks (CNN), such as U-Net, is widely used in medical image processing, particularly for image segmentation tasks. However, CNNs have the limitation in effectively learning spatial relationships within images. To address this, Capsule Networks (CapsNet) is introduced, utilizing capsule dynamic routing to better capture spatial hierarchies. This paper aims to investigate the performance of SegCaps, a segmentation model based on Capsule Networks, for brain glioma segmentation in MRI images, compared to the CNN-based U-Net model. Both models were tested on the BraTS2018 glioma dataset, which includes 374 MRI images of brain tumors across four modalities (T1, T1c, T2, FLAIR). The performance of SegCaps and U-Net was evaluated using two key segmentation metrics: Dice coefficient and Jaccard index. The results show that SegCaps outperformed U-Net with a Dice coefficient of 87.96% compared to U-Nets 85.56%, demonstrating a 2.4% improvement. Additionally, SegCaps required fewer parameters than the U-Net model, highlighting its efficiency. In conclusion, SegCaps can be considered as a promising alternative for glioma segmentation in MRI images. Future work could focus on refining the SegCaps model to enhance its performance while reducing computational costs.

CDSG-SAM: A cross-domain self-generating prompt few-shot brain tumor segmentation pipeline based on SAM

In clinical practice, accurate segmentation of brain tumor regions is essential for patient treatment and survival. Accurate brain tumor segmentation is an important task and it is based on a large amount of labeled data. Clinical data are from different cities and device protocols, which leads to the poor generalization ability of the model. In addition, the small amount of clinical data also leads to the decline of the segmentation performance. In this paper, a cross-domain self-generating prompt few-shot brain tumor segmentation pipeline called CDSG-SAM is developed. This pipeline is based on the Segment Anything Model (SAM), which integrates a new Cross-domain self-attention (CDS) Adapter module and Self-Generating (SG) Prompt module. The SAM-based segmentation model is more suitable for few-shot medical image segmentation. A dynamic fuzzy support mask decoder module (DFSMD) is proposed in the decoder stage to enhance the accuracy of brain tumor segmentation edges. Furthermore, a Joint Loss function (JL) is designed to comprehensively consider multiple aspects of performance indicators to improve the accuracy and robustness of the model. Experimental results evaluated on the BraTS2021 glioma dataset, Clinical glioma brain tumor data (CBTD), and BraTS2023 metastatic tumor dataset show that the proposed pipeline has achieved excellent performance with an average dice coefficient of 0.918, 0.828, and 0.868 which outperforms than existing methods.

Deep learning segmentation model for quantification of infarct size in pigs with myocardial ischemia/reperfusion.

Infarct size (IS) is the most robust end point for evaluating the success of preclinical studies on cardioprotection. The gold standard for IS quantification in ischemia/reperfusion (I/R) experiments is triphenyl tetrazolium chloride (TTC) staining, typically done manually. This study aimed to determine if automation through deep learning segmentation is a time-saving and valid alternative to standard IS quantification. High-resolution images from TTC-stained, macroscopic heart slices were retrospectively collected from pig experiments (n = 390) with I/R without/with cardioprotection to cover a wide IS range. Existing IS data from pig experiments, quantified using a standard method of manual and subsequent digital labeling of film-scan annotations, were used as reference. To automate the evaluation process with the aim to be more objective and save time, a deep learning pipeline was implemented; the collected images (n = 3869) were pre-processed by cropping and labeled (image annotations). To ensure their usability as training data for a deep learning segmentation model, IS was quantified from image annotations and compared to IS quantified using the existing film-scan annotations. A supervised deep learning segmentation model based on dynamic U-Net architecture was developed and trained. The evaluation of the trained model was performed by fivefold cross-validation (n = 220 experiments) and testing on an independent test set (n = 170 experiments). Performance metrics (Dice similarity coefficient [DSC], pixel accuracy [ACC], average precision [mAP]) were calculated. IS was then quantified from predictions and compared to IS quantified from image annotations (linear regression, Pearson's r; analysis of covariance; Bland-Altman plots). Performance metrics near 1 indicated a strong model performance on cross-validated data (DSC: 0.90, ACC: 0.98, mAP: 0.90) and on the test set data (DSC: 0.89, ACC: 0.98, mAP: 0.93). IS quantified from predictions correlated well with IS quantified from image annotations in all data sets (cross-validation: r = 0.98; test data set: r = 0.95) and analysis of covariance identified no significant differences. The model reduced the IS quantification time per experiment from approximately 90min to 20s. The model was further tested on a preliminary test set from experiments in isolated, saline-perfused rat hearts with regional I/Rwithout/with cardioprotection (n = 27). There was also no significant difference in IS between image annotations and predictions, but the performance on the test set data from rat hearts was lower (DSC: 0.66, ACC: 0.91, mAP: 0.65). IS quantification using a deep learning segmentation model is a valid and time-efficient alternative to manual and subsequent digital labeling.

The Study of Echocardiography of Left Ventricle Segmentation Combining Transformer and Convolutional Neural Networks.

Accurate prediction of echocardiographic parameters is essential for diagnosis and treatment of cardiac disease, especially for segmentation of the left ventricle to obtain measurements such as left ventricular ejection fraction and volume. However, manually outlining left ventricle on echocardiographic images is a time-consuming and physician experience-dependent task. Therefore, it is crucial to develop an accurate and efficient automatic segmentation tool. Therefore, we aimed to explore a model to perform echocardiography of left ventricle segmentation by combining transformer and convolutional neural networks (CNN).ResNet-50 was used in CNN branch. The encoder-decoder architecture was used for transformer branch, which was fused to the corresponding feature maps of the CNN branches. Fusion module was used to effectively combine feature information from the CNN and transformer. Bridge attention used to increase sensitivity and prediction accuracy of model. The entire network was trained end-to-end using the binary cross-entropy with logits loss L.In this work, we propose an automatic left ventricular (LV) segmentation model based on Transformer and CNN that efficiently captures global dependencies and spatial details and create a fusion module using CBAM that fuses Transformer and CNN features. In addition, attention is also computed using multi-level fusion features to obtain the final attention segmentation map. The model was trained and evaluated on a large cardiac image dataset, EchoNet-Dynamic, with test dice coefficient of 92.4%.The results show that our model can better segment left ventricle. We also tested our model on clinical patient ultrasound images, and visualization results proved effectiveness of the model.

Enhanced gastric cancer classification and quantification interpretable framework using digital histopathology images

Recent developments have highlighted the critical role that computer-aided diagnosis (CAD) systems play in analyzing whole-slide digital histopathology images for detecting gastric cancer (GC). We present a novel framework for gastric histology classification and segmentation (GHCS) that offers modest yet meaningful improvements over existing CAD models for GC classification and segmentation. Our methodology achieves marginal improvements over conventional deep learning (DL) and machine learning (ML) models by adaptively focusing on pertinent characteristics of images. This contributes significantly to our study, highlighting that the proposed model, which performs well on normalized images, is robust in certain respects, particularly in handling variability and generalizing to different datasets. We anticipate that this robustness will lead to better results across various datasets. An expectation-maximizing Naïve Bayes classifier that uses an updated Gaussian Mixture Model is at the heart of the suggested GHCS framework. The effectiveness of our classifier is demonstrated by experimental validation on two publicly available datasets, which produced exceptional classification accuracies of 98.87% and 97.28% on validation sets and 98.47% and 97.31% on test sets. Our framework shows a slight but consistent improvement over previously existing techniques in gastric histopathology image classification tasks, as demonstrated by comparative analysis. This may be attributed to its ability to capture critical features of gastric histopathology images better. Furthermore, using an improved Fuzzy c-means method, our study produces good results in GC histopathology picture segmentation, outperforming state-of-the-art segmentation models with a Dice coefficient of 65.21% and a Jaccard index of 60.24%. The model’s interpretability is complemented by Grad-CAM visualizations, which help understand the decision-making process and increase the model’s trustworthiness for end-users, especially clinicians.

Real-time segmentation of biliary structure in pure laparoscopic donor hepatectomy

Pure laparoscopic donor hepatectomy (PLDH) has become a standard practice for living donor liver transplantation in expert centers. Accurate understanding of biliary structures is crucial during PLDH to minimize the risk of complications. This study aims to develop a deep learning-based segmentation model for real-time identification of biliary structures, assisting surgeons in determining the optimal transection site during PLDH. A single-institution retrospective feasibility analysis was conducted on 30 intraoperative videos of PLDH. All videos were selected for their use of the indocyanine green near-infrared fluorescence technique to identify biliary structure. From the analysis, 10 representative frames were extracted from each video specifically during the bile duct division phase, resulting in 300 frames. These frames underwent pixel-wise annotation to identify biliary structures and the transection site. A segmentation task was then performed using a DeepLabV3+ algorithm, equipped with a ResNet50 encoder, focusing on the bile duct (BD) and anterior wall (AW) for transection. The model’s performance was evaluated using the dice similarity coefficient (DSC). The model predicted biliary structures with a mean DSC of 0.728 ± 0.01 for BD and 0.429 ± 0.06 for AW. Inference was performed at a speed of 15.3 frames per second, demonstrating the feasibility of real-time recognition of anatomical structures during surgery. The deep learning-based semantic segmentation model exhibited promising performance in identifying biliary structures during PLDH. Future studies should focus on validating the clinical utility and generalizability of the model and comparing its efficacy with current gold standard practices to better evaluate its potential clinical applications.

An optimized deep focused U-Net model for image segmentation

An optimized deep focused U-Net model for image segmentation

Attention based UNet model for breast cancer segmentation using BUSI dataset.

Breast cancer, a prevalent and life-threatening disease, necessitates early detection for the effective intervention and the improved patient health outcomes. This paper focuses on the critical problem of identifying breast cancer using a model called Attention U-Net. The model is utilized on the Breast Ultrasound Image Dataset (BUSI), comprising 780 breast images. The images are categorized into three distinct groups: 437 cases classified as benign, 210 cases classified as malignant, and 133 cases classified as normal. The proposed model leverages the attention-driven U-Net's encoder blocks to capture hierarchical features effectively. The model comprises four decoder blocks which is a pivotal component in the U-Net architecture, responsible for expanding the encoded feature representation obtained from the encoder block and for reconstructing spatial information. Four attention gates are incorporated strategically to enhance feature localization during decoding, showcasing a sophisticated design that facilitates accurate segmentation of breast tumors in ultrasound images. It displays its efficacy in accurately delineating and segregating tumor borders. The experimental findings demonstrate outstanding performance, achieving an overall accuracy of 0.98, precision of 0.97, recall of 0.90, and a dice score of 0.92. It demonstrates its effectiveness in precisely defining and separating tumor boundaries. This research aims to make automated breast cancer segmentation algorithms by emphasizing the importance of early detection in boosting diagnostic capabilities and enabling prompt and targeted medical interventions.

Improved genetic algorithm for multi-threshold optimization in digital pathology image segmentation

This paper presents an improved genetic algorithm focused on multi-threshold optimization for image segmentation in digital pathology. By innovatively enhancing the selection mechanism and crossover operation, the limitations of traditional genetic algorithms are effectively addressed, significantly improving both segmentation accuracy and computational efficiency. Experimental results demonstrate that the improved genetic algorithm achieves the best balance between precision and recall within the threshold range of 0.02 to 0.05, and it significantly outperforms traditional methods in terms of segmentation performance. Segmentation quality is quantified using metrics such as precision, recall, and F1 score, and statistical tests confirm the superior performance of the algorithm, especially in its global search capabilities for complex optimization problems. Although the algorithm’s computation time is relatively long, its notable advantages in segmentation quality, particularly in handling high-precision segmentation tasks for complex images, are highly pronounced. The experiments also show that the algorithm exhibits strong robustness and stability, maintaining reliable performance under different initial conditions. Compared to general segmentation models, this algorithm demonstrates significant advantages in specialized tasks, such as pathology image segmentation, especially in resource-constrained environments. Therefore, this improved genetic algorithm offers an efficient and precise multi-threshold optimization solution for image segmentation, providing valuable reference for practical applications.

Zero-shot autonomous robot manipulation via natural language

Smart manufacturing revolutionizes advanced automation by leveraging deep learning and robotics technology. Nevertheless, deep learning implementation in manufacturing faces several challenges in preparing datasets, training a model, and adapting to various applications. Furthermore, programming languages are required to develop deep learning models and communicate with robots, which is not straightforward to quickly and precisely reflect human intentions in manufacturing processes. To alleviate the issue, this study proposes an autonomous zero-shot robot manipulation framework via natural language. Specifically, the Segmentation Anything Model (SAM), a promptable image segmentation model, first segments objects in images captured by an RGB-D camera, which are passed to Generative Pre-trained Transformers (GPTs), a Large Language Model (LLM), to identify and localize a designated object based on human commands given in natural language. Afterwards, the Robot Operating System (ROS) manipulates a robot to a localized position. With no additional programming or specialized skills in object localization and robot manipulation, the framework demonstrates an autonomous robot automation approach through intuitive communication between humans and machines through natural language.