Model-based Segmentation Method Research Articles

Attribute binding enhancement improvement for diffusion model based zero-shot image segmentation

As diffusion model's potential ability to accomplish perception tasks being discovered, many researchers have tried to apply diffusion model in segmentation tasks and achieved good results. However, there are not many methods to optimize the diffusion model for segmentation tasks by improving the text side. Even if some research have pointed out that diffusion model sometimes 'misunderstand' prompt and bind attributes to wrong objects. With the development of artificial intelligence and the gradual entry of large models into people's daily lives, in addition to the performance of the model, the model's interactivity and ability to understand natural language are also more important. Based on existing zero-shot diffusion model based segmentation method, this work introduces a new method to enhance attribute binding in the embedding of prompt improve the performance of the model. Through this method, more descriptive text will get better segmentation results, which to some extent, improves the segmentation performance of the model for inputs that are more in line with natural language description habits.

Segment Anything Model-Based Building Footprint Extraction for Residential Complex Spatial Assessment Using LiDAR Data and Very High-Resolution Imagery

With rapid urbanization, retrieving information about residential complexes in a timely manner is essential for urban planning. To develop efficiency and accuracy of building extraction in residential complexes, a Segment Anything Model-based residential building instance segmentation method with an automated prompt generator was proposed combining LiDAR data and VHR remote sensing images in this study. Three key steps are included in this method: approximate footprint detection using LiDAR data, automatic prompt generation for the SAM, and residential building footprint extraction. By applying this method, residential building footprints were extracted in Pukou District, Nanjing, Jiangsu Province. Based on this, a comprehensive assessment model was constructed to systematically evaluate the spatial layout of urban complexes using six dimensions of assessment indicators. The results showed the following: (1) The proposed method was used to effectively extract residential building footprints. (2) The residential complexes in the study area were classified into four levels. The numbers of complexes classified as Excellent, Good, Average, and Poor were 10, 29, 16, and 1, respectively. Residential complexes of different levels exhibited varying spatial layouts and building distributions. The results provide a visual representation of the spatial distribution of residential complexes that belong to different levels within the study area, aiding in urban planning.

Enhanced curve-based segmentation method for point clouds of curved and irregular structures

This paper proposes an improved method for model-based segmentation (MS) of curved and irregular mounded structures in 3D measurements. The proposed method divides the point cloud data into several levels according to the reasonable width calculated from the density of points. Then, it fits a curve model with 2D points for each level separately. The classification results of specific types are merged to obtain specific structural measurement data in 3D space. We use MS method, difference of normals based segmentation, region growing algorithm, constrained planar cuts, and locally convex connected patches as a control group. The results show that the proposed method achieves higher accuracy with a mean intersection merge ratio of more than 0.8238, at least 37.92% higher than other methods. The method proposed in this paper requires less time to process than other methods. Therefore, the proposed method effectively and efficiently segments the measurement data of curved and irregular mounded structures in 3D measurements. The method proposed in this paper has also been applied in the practical robotic grinding task. The root mean square error of the grinding amount is less than 2 mm, and good grinding results are achieved.

Comment on Lee et al. Accuracy of New Deep Learning Model-Based Segmentation and Key-Point Multi-Detection Method for Ultrasonographic Developmental Dysplasia of the Hip (DDH) Screening. Diagnostics 2021, 11, 1174.

We have read the article titled "Accuracy of New Deep Learning Model-Based Segmentation and Key-Point Multi-Detection Method for Ultrasonographic Developmental Dysplasia of the Hip (DDH) Screening" by Lee et al. [...].

Reply to Çiftci, S.; Aydin, B.K. Comment on "Lee et al. Accuracy of New Deep Learning Model-Based Segmentation and Key-Point Multi-Detection Method for Ultrasonographic Developmental Dysplasia of the Hip (DDH) Screening. Diagnostics 2021, 11, 1174".

We thank Dr. Sadettin Ciftci for his comment on the key point issues in measuring the alpha and beta angle with Graf method. We appreciated his feedback [...].

Factors That Influence the Choice of Markov Model Order in Discriminating DNA Sequences from Different Sources.

Markov models have frequently been used in genetic sequence analysis. The number of parameters of a Markov model increases exponentially with model order, so it is often recommended that the order be chosen based on the size of data being modeled, lower orders for small and higher orders for large dataset sizes. Approaches based on model selection criterion have also been proposed. An important problem in microbiology and evolutionary biology is to decipher chimeric genomes of microbes, particularly, identify segments of distinct ancestries in genomes and reconstruct the plausible evolutionary scenarios that might have shaped the chimeric genomes in the microbial world. In this study, we assessed a Markov model-based segmentation method for its ability to detect compositionally disparate segments in chimeric sequence constructs as a function of model order, sequence length, and phylogenetic divergence. Our results show that the choice of Markov model order depends on both sequence size and composition. Higher order Markov models were found to be more effective in delineating sequence segments arising from closely related organisms in longer constructs; on the other hand, lower order Markov models were found to be more appropriate in delineating sequence segments arising from distantly related organisms in shorter constructs. These findings are important and timely, with broad implications in fields such as epidemiology that has to deal with the emergence of novel pathogenic chimeras that arise by foreign DNA acquisition, and ecology where chimeric structures may arise in various ecosystems, necessitating more robust approaches for their deconstruction and interpretation.

Image Segmentation under the Optimization Algorithm of Krill Swarm and Machine Learning.

This study aims to improve the efficiency and accuracy of image segmentation, and to compare and study traditional threshold-based image segmentation methods and machine learning model-based image segmentation methods. The krill herb optimization algorithm is combined with the traditional maximum between-class variance function to form a new graph segmentation algorithm. The pet dataset is used to train the algorithm model and build an image semantic segmentation system. The results show that when the traditional Ostu algorithm performs image single-threshold segmentation, the number of iterations is about 256. When double-threshold segmentation is performed, the number of iterations increases exponentially, and the execution time is about 2 s. The number of iterations of the improved Krill Herd algorithm in single-threshold segmentation is 6.95 times, respectively. The execution time for double-threshold segmentation is about 0.24 s. The number of iterations is only improved by a factor of 0.19. The average classification accuracy of the Unet network model and the SegNet network model is 86.3% and 91.9%, respectively. The average classification accuracy of the DC-Unet network model reaches 93.1%. This shows that the proposed fusion algorithm has high optimization efficiency and stronger practicability in multithreshold image segmentation. The DC-Unet network model can improve the image detail segmentation effect. The research provides a new idea for finding an efficient and accurate image segmentation method.

Texture appearance model, a new model-based segmentation paradigm, application on the segmentation of lung nodule in the CT scan of the chest

Lung cancer causes more than one million deaths worldwide each year. Averages of 5-year survival rate of patients with Non-small cell lung cancer (NSCLC), which is the most common type of lung cancer, is 15%. Computer-Aided Detection (CAD) is a very important tool for identifying lung lesions in medical imaging. In general, the process line of a CAD system can be divided into four main stages: preprocessing, localization, feature extraction, and classification. As segmentation is required for localization in computer vision and medical image analysis, this step has become a major and challenging problem, and much research has been done on new segmentation techniques. In recent years, interest in model-based segmentation methods has increased, and the reason for this is even if some object information is lost, such gaps can be filled by using the previous information in the model. This paper proposed Texture Appearance Model (TAM), which is a new model-based method and segments all types of nodule areas accurately and efficiently, including juxta-pleural nodules, without separating the lung from the surrounding area in a CT scan of the lung. In this method, Texture Representation of Image (TRI) is obtained using tissue texture feature extraction and feature selection algorithms. The proposed method has been evaluated in 85 nodules of the dataset, received from the Iranian hospital, in which the ground-truth annotation by physicians and CT imaging data were provided. The results show that the proposed algorithm has an encouraging performance for distinguishing different types of nodules, including pleural, cavity and non-solid nodules, achieving an average dice similarity coefficient (DSC) of 84.75%.

Accuracy of New Deep Learning Model-Based Segmentation and Key-Point Multi-Detection Method for Ultrasonographic Developmental Dysplasia of the Hip (DDH) Screening

Hip joint ultrasonographic (US) imaging is the golden standard for developmental dysplasia of the hip (DDH) screening. However, the effectiveness of this technique is subject to interoperator and intraobserver variability. Thus, a multi-detection deep learning artificial intelligence (AI)-based computer-aided diagnosis (CAD) system was developed and evaluated. The deep learning model used a two-stage training process to segment the four key anatomical structures and extract their respective key points. In addition, the check angle of the ilium body balancing level was set to evaluate the system’s cognitive ability. Hence, only images with visible key anatomical points and a check angle within ±5° were used in the analysis. Of the original 921 images, 320 (34.7%) were deemed appropriate for screening by both the system and human observer. Moderate agreement (80.9%) was seen in the check angles of the appropriate group (Cohen’s κ = 0.525). Similarly, there was excellent agreement in the intraclass correlation coefficient (ICC) value between the measurers of the alpha angle (ICC = 0.764) and a good agreement in beta angle (ICC = 0.743). The developed system performed similarly to experienced medical experts; thus, it could further aid the effectiveness and speed of DDH diagnosis.

Defect detection method using laser vision with model-based segmentation for laser brazing welds on car body surface

This paper proposes a defect detection method using laser vision with model-based segmentation for laser brazing welds on car body surface, in which a dynamic ideal surface (DIS), composed of a series of dynamic ideal contours (DIC), is first established for welding surface. The DIC model is obtained by dynamically fitting welding surface contours using the weighted cubic spline regression (WCSR) model with the fast Expectation-Maximization (Fast-EM) algorithm. Then through multi-threshold analysis on the DIC model residuals, whether the contour is defective and the nature of defect can be identified, and finally the defect can be completely separated. The whole process is fully automatic and requires no prior knowledge of defect. Experimental results show that the model-based segmentation method has managed to separate multiple types of defect effectively while meeting the actual accuracy requirements. The robustness of the proposed method has been proved through the test in the case of noise interference and unstable movement of the sensor, which helps to realize online inspection of weld defects in a factory environment.

Hierarchical conditional random field model for multi‐object segmentation in gastric histopathology images

In this Letter, a hierarchical conditional random field (HCRF) model-based gastric histopathology image segmentation (GHIS) method is proposed, which can localise abnormal (cancer) regions in gastric histopathology images to assist histopathologists in medical work. First, to obtain pixel-level segmentation information, the authors retrain a convolutional neural network (CNN) to build up their pixel-level potentials. Then, to obtain abundant spatial segmentation information in patch level, they fine tune another three CNNs to build up their patch-level potentials. Thirdly, based on the pixel- and patch-level potentials, their HCRF model is structured. Finally, a graph-based post-processing is applied to further improve their segmentation performance. In the experiment, a segmentation accuracy of 78.91 % is achieved on a haematoxylin and eosin stained gastric histopathological dataset with 560 images, showing the effectiveness and future potential of the proposed GHIS method.

Incomplete-Data-Driven Speaker Segmentation for Diarization Application; A Help-Training Approach

This paper presents a new segmentation method for diarization application. This method is established on a support vector regression-based discriminative engine which bears the main duty of estimating the most possible change points. This engine is aided by a generative classifier in a help-training approach. Considering that there are no pre-labeled training samples in a segmentation task, the proposed model-based segmentation method attempts to suggest an appropriate solution to overcome this obstacle. The introduced iterative method supposes that the initial frames in a given segment belong to the associated speaker. This hypothesis permits the SVR engine to be initiated in the first iteration. In the following iterations, discriminative regression block in conjunction with the generative classifier tags the remaining frames with advantageous (positive) and disadvantageous (negative) labels. These newly labeled frames establish the working set to update the associated speaker model. In addition to the proposed segmentation method, a new strategy is introduced to estimate inserted and deleted change points. In the evaluation section, in addition to the common experimental assessment, attempts are made to achieve a unique and comprehensive insight into the statistical aspects of choosing training samples. Finally, comparison of the proposed segmentation and diarization system with similar method shows approximately 22.95% enhancement in the performance.

Automated Contouring of Contrast and Non-Contrast CT Liver Images with Fully Convolutional Neural Networks

Liver contouring can be particularly labor-intensive based on size and unique patient liver shape. While model-based segmentation methods have shown promise in the past, they can be relatively slow (> 10 minutes), and not robust to unique patient anatomy. Fully-convolutional neural networks have shown great promise in the ability to accurately segment 2D images with multiple classes in short time periods and robustness against nuisance variations. We hypothesize that fully-convolutional networks (FCNs) can be used to rapidly and to accurately contour the Manual contours of the liver were delineated on the CT scans of 58 patients who had colorectal liver metastases (CLM) by a trained radiologist. A previously trained FCN, the Visual Geometry Group (VGG)-16 network, was adapted for rapid segmentation of the liver. A skip-layer architecture, similar to that found in the popular ‘U-net’ design, was implemented in order to improve output segmentation results. Various batch sizes, learning rates, and filters were investigated before deciding on the final model. For validation, 161 contrast-enhanced image sets released by the Medical Imaging Computing and Computer Assisted Intervention (MICCAI) challenges were obtained, as well as 10 non-contrast-enhanced image sets from the institution. The contrast-enhanced image sets came specifically from the abdominal challenge (n=30) and liver tumor segmentation challenge (LiTs, n=131). As a final test, 26 patients with clinically defined liver contours were compared by a blinded diagnostic radiologist. Upon the presentation of the two contours, responses were categorized based on the contour preference, and clinical usability. The final model was selected based on its performance in dice similarity coefficient (DSC) scores. The average DSC and standard deviations for the validation groups were as follows: μAbdomen:0.93, σAbdomen:0.02, μLiTS:0.92, σLiTS:0.05, and μNon-Contrast:0.95, σNon-Contrast:0.006. Of the contours generated from this model compared to 26 clinically defined contours, 50% (13/26) were preferred to the previously created contour, 54% (14/26) would be clinically useful without edits, 92% (24/26) useable or useable with minor edits (<1 minute of work), and 8% (2/26) required major edits. Liver contours were predicted, post-processed, and transferred to the planning system within 1 minute using an Intel i5 processor (3.3 GHz) and 16 GB RAM. These results suggest that the developed FCN can rapidly and accurately contour the liver on CT scans in a manner similar to those contoured in our clinic. Beyond the scope of DSC, on a small subset of cases these contours have been shown to be clinically useful and even preferable to currently created manual contours, warranting further investigation. For the two cases of the major edits, these patients had extensive disease, which was not seen in the training data, and will be investigated further in future iterations of the model.

Unsupervised help-trained LS-SVR-based segmentation in speaker diarization system

In this paper, we propose a new segmentation method for diarization applications. In the proposed method, segmentation is performed using a discriminatively trained support vector regression, while a generative classifier helps it to estimate the probable change points. Since, there is no pre-labeled training samples in segmentation task, the proposed model-based segmentation method tries to suggest a proper solution to bridge this gap. It is assumed that initial applied samples are labeled with the first speaker in an unsupervised manner, while the subsequent training samples are chosen by applying the help-training approach. These samples are estimated to be conducive when both regression and classifier blocks, label positive/negative samples to be advantageous. These samples would be purified in next steps and speakers’ models would be updated iteratively. In addition, a new procedure is introduced to estimate deleted and inserted change points that is executed when segmentation is completed. In comparison to similar approaches, experiments have shown performance improvement about 29% in diarization error rate.

Comparison analysis of orbital shape and volume in unilateral fractured orbits

Facial fractures often result in changes of the orbital volume. These changes can be measured in three-dimensional (3D) computed tomography (CT) scans for preoperative planning and postoperative evaluation. The aim of this study was to analyze the orbital volume and shape before and after surgical treatment of unilateral orbital fractures using semi-automatic image segmentation and registration techniques. The orbital volume in 21 patients was assessed by a semi-automatic model-based segmentation method. The fractured orbit was compared relative to the contralateral orbit. The same procedure was performed for the postoperative evaluation. Two observers performed the segmentation procedure, and the inter- and intraobserver variability was evaluated. The interobserver variability (mean volume difference ± 1.96 SD) was −0.6 ± 1.0 ml in the first trial and 0.7 ± 0.8 ml in the second trial. The intra-observer variability was −0.2 ± 0.7 ml for the first observer and 1.1 ± 0.9 ml for the second observer. The average volume overlap (Dice similarity coefficient) between the fractured and contralateral side increased after surgery, while the mean and maximum surface distance decreased, indicating that the surgery contributed to a re-establishment of size and shape. In conclusion, our study shows that the semi-automatic segmentation method has precision for detecting volume differences down to 1.0 ml. The combination of semi-automatic segmentation and 3D shape analysis provides a powerful tool for planning and evaluating treatment of orbital fractures.

A new model-based framework for lung tissue segmentation in three-dimensional thoracic CT images

A new framework for model-based lung tissue segmentation in three-dimensional thoracic CT images is proposed. In the first stage, a parametric model for lung segmenting surface is created using shape representation based on level sets method. This model is constituted by the sum of a mean distance function and a number of weighted eigenshapes. Consequently, unlike the other model-based segmentation methods, there is no need to specify any marker point in this model. In the second stage, the segmenting surface is varied so as to be matched with the binarized input image. For this purpose, a region-based energy function is minimized with respect to the parameters including the weights of eigenshapes and coefficients of a three-dimensional similarity transform. Finally, the resulted segmenting surface is post-processed in order to improve its fitness with the lung borders of the input image. The experimental results demonstrated the outperformance of the proposed framework over its model-based counterparts in model matching stage. Moreover, it performed slightly better in terms of final segmentation results.

Robust x-ray image segmentation by spectral clustering and active shape model.

Extraction of bone contours from x-ray radiographs plays an important role in joint space width assessment, preoperative planning, and kinematics analysis. We present a robust segmentation method to accurately extract the distal femur and proximal tibia in knee radiographs of varying image quality. A spectral clustering method based on the eigensolution of an affinity matrix is utilized for x-ray image denoising. An active shape model-based segmentation method is employed for robust and accurate segmentation of the denoised x-ray images. The performance of the proposed method is evaluated with x-ray images from the public-use dataset(s), the osteoarthritis initiative, achieving a root mean square error of [Formula: see text] for femur and [Formula: see text] for tibia. The results demonstrate that this method outperforms previous segmentation methods in capturing anatomical shape variations, accounting for image quality differences and guiding accurate segmentation.

An integrated model-driven method for in-treatment upper airway motion tracking using cine MRI in head and neck radiation therapy.

For the first time, MRI-guided radiation therapy systems can acquire cine images to dynamically monitor in-treatment internal organ motion. However, the complex head and neck (H&N) structures and low-contrast/resolution of on-board cine MRI images make automatic motion tracking a very challenging task. In this study, the authors proposed an integrated model-driven method to automatically track the in-treatment motion of the H&N upper airway, a complex and highly deformable region wherein internal motion often occurs in an either voluntary or involuntary manner, from cine MRI images for the analysis of H&N motion patterns. Considering the complex H&N structures and ensuring automatic and robust upper airway motion tracking, the authors firstly built a set of linked statistical shapes (including face, face-jaw, and face-jaw-palate) using principal component analysis from clinically approved contours delineated on a set of training data. The linked statistical shapes integrate explicit landmarks and implicit shape representation. Then, a hierarchical model-fitting algorithm was developed to align the linked shapes on the first image frame of a to-be-tracked cine sequence and to localize the upper airway region. Finally, a multifeature level set contour propagation scheme was performed to identify the upper airway shape change, frame-by-frame, on the entire image sequence. The multifeature fitting energy, including the information of intensity variations, edge saliency, curve geometry, and temporal shape continuity, was minimized to capture the details of moving airway boundaries. Sagittal cine MR image sequences acquired from three H&N cancer patients were utilized to demonstrate the performance of the proposed motion tracking method. The tracking accuracy was validated by comparing the results to the average of two manual delineations in 50 randomly selected cine image frames from each patient. The resulting average dice similarity coefficient (93.28% ± 1.46%) and margin error (0.49 ± 0.12 mm) showed good agreement between the automatic and manual results. The comparison with three other deformable model-based segmentation methods illustrated the superior shape tracking performance of the proposed method. Large interpatient variations of swallowing frequency, swallowing duration, and upper airway cross-sectional area were observed from the testing cine image sequences. The proposed motion tracking method can provide accurate upper airway motion tracking results, and enable automatic and quantitative identification and analysis of in-treatment H&N upper airway motion. By integrating explicit and implicit linked-shape representations within a hierarchical model-fitting process, the proposed tracking method can process complex H&N structures and low-contrast/resolution cine MRI images. Future research will focus on the improvement of method reliability, patient motion pattern analysis for providing more information on patient-specific prediction of structure displacements, and motion effects on dosimetry for better H&N motion management in radiation therapy.

Manual-Protocol Inspired Technique for Improving Automated MR Image Segmentation during Label Fusion.

Recent advances in multi-atlas based algorithms address many of the previous limitations in model-based and probabilistic segmentation methods. However, at the label fusion stage, a majority of algorithms focus primarily on optimizing weight-maps associated with the atlas library based on a theoretical objective function that approximates the segmentation error. In contrast, we propose a novel method—Autocorrecting Walks over Localized Markov Random Fields (AWoL-MRF)—that aims at mimicking the sequential process of manual segmentation, which is the gold-standard for virtually all the segmentation methods. AWoL-MRF begins with a set of candidate labels generated by a multi-atlas segmentation pipeline as an initial label distribution and refines low confidence regions based on a localized Markov random field (L-MRF) model using a novel sequential inference process (walks). We show that AWoL-MRF produces state-of-the-art results with superior accuracy and robustness with a small atlas library compared to existing methods. We validate the proposed approach by performing hippocampal segmentations on three independent datasets: (1) Alzheimer's Disease Neuroimaging Database (ADNI); (2) First Episode Psychosis patient cohort; and (3) A cohort of preterm neonates scanned early in life and at term-equivalent age. We assess the improvement in the performance qualitatively as well as quantitatively by comparing AWoL-MRF with majority vote, STAPLE, and Joint Label Fusion methods. AWoL-MRF reaches a maximum accuracy of 0.881 (dataset 1), 0.897 (dataset 2), and 0.807 (dataset 3) based on Dice similarity coefficient metric, offering significant performance improvements with a smaller atlas library (< 10) over compared methods. We also evaluate the diagnostic utility of AWoL-MRF by analyzing the volume differences per disease category in the ADNI1: Complete Screening dataset. We have made the source code for AWoL-MRF public at: https://github.com/CobraLab/AWoL-MRF.

Automated grading of breast cancer histopathology using cascaded ensemble with combination of multi-level image features

We present a novel image-analysis based method for automatically distinguishing low, intermediate, and high grades of breast cancer in digitized histopathology. A multiple level feature set, including pixel-, object-, and semantic-level features derived from convolutional neural networks (CNN), is extracted from 106 hematoxylin and eosin stained breast biopsy tissue studies from 106 women patients. These multi-level features allow not only characterization of cancer morphology, but also extraction of structural and interpretable information within the histopathological images. In this study, an improved hybrid active contour model based segmentation method was used to segment nuclei from the images. The semantic-level features were extracted by a CNN approach, which described the proportions of nuclei belonging to the different grades, in conjunction with pixel-level (texture) and object-level (architecture) features, to create an integrated set of image attributes that can potentially outperform either subtype of features individually. We utilized a cascaded approach to train multiple support vector machine (SVM) classifiers using combinations of feature subtypes to enable the possibility of maximizing the performance by leveraging different feature sets extracted from multiple levels. Final class (cancer grade) was determined by combining the scores produced by the individual SVM classifiers. By employing a light (three-layer) CNN model and parallel computing, the presented approach is computationally efficient and applicable to large-scale datasets. The method achieved an accuracy of 0.92 for low versus high, 0.77 for low versus intermediate, and 0.76 for intermediate versus high, and an overall accuracy of 0.69 when discriminating low, intermediate, and high grades of histopathological breast cancer images. This suggested that our grading method could be useful in developing a computational diagnostic tool for differentiating breast cancer grades, which might enable objective and reproducible alternative for diagnosis.