Segmentation Of Bone Surfaces Research Articles

Segmentation of bone surface from ultrasound using a lightweight network UBS-Net

Objective. Ultrasound-assisted orthopaedic navigation held promise due to its non-ionizing feature, portability, low cost, and real-time performance. To facilitate the applications, it was critical to have accurate and real-time bone surface segmentation. Nevertheless, the imaging artifacts and low signal-to-noise ratios in the tomographical B-mode ultrasound (B-US) images created substantial challenges in bone surface detection. In this study, we presented an end-to-end lightweight US bone segmentation network (UBS-Net) for bone surface detection. Approach. We presented an end-to-end lightweight UBS-Net for bone surface detection, using the U-Net structure as the base framework and a level set loss function for improved sensitivity to bone surface detectability. A dual attention (DA) mechanism was introduced at the end of the encoder, which considered both position and channel information to obtain the correlation between the position and channel dimensions of the feature map, where axial attention (AA) replaced the traditional self-attention (SA) mechanism in the position attention module for better computational efficiency. The position attention and channel attention (CA) were combined with a two-class fusion module for the DA map. The decoding module finally completed the bone surface detection. Main Results. As a result, a frame rate of 21 frames per second (fps) in detection were achieved. It outperformed the state-of-the-art method with higher segmentation accuracy (Dice similarity coefficient: 88.76% versus 87.22%) when applied the retrospective ultrasound (US) data from 11 volunteers. Significance. The proposed UBS-Net for bone surface detection in ultrasound achieved outstanding accuracy and real-time performance. The new method out-performed the state-of-the-art methods. It had potential in US-guided orthopaedic surgery applications.

3D Reconstruction and Augmented Reality Display of Bone Structure Based on Ultrasound Segmentation with Feature-Reweighted U-Net

<p indent=0mm>During the minimally invasive orthopedic surgery, surgeons insert surgical instruments under the guidance of two-dimensional (2D) X-ray images of bone structures that displayed on a screen. This method not only lacks of intuitive three-dimensional (3D) information, but also exposes patients and surgeons to significant amounts of ionizing radiations. To solve these problems, this paper proposes 3D reconstruction and augmented reality display of bone structure based on Free-hand ultrasound segmentation with feature-reweighted U-Net, and provides doctors with 3D bone structure for radiation-free surgical navigation. Firstly, a feature-reweighted U-Net is developed for accurate segmentation of bone surfaces from Free-hand ultrasound scans. Then, the segmented bone surfaces are reconstructed to acquire 3D bone structure according to the locations of ultrasound images. Finally, the bone structure is further rendered using integral photography techniques for augmented reality guidance. The experiments are finished on ultrasound data set of phantom and patients. The bone segmentation accuracy of (88.51±1.44)% and bone reconstruction error of <sc>(1.29±0.11) mm</sc> are acquired during the experiments. The proposed method can provide surgeons with intuitive intraoperative 3D navigation information.

Segmentation of the distal femur in ultrasound images

Abstract Objectives Ultrasound is a widely used imaging technology that allows for fast diagnosis of a broad range of illnesses and injuries of the musculoskeletal system. However, interpreting ultrasound images remains a challenging task that requires expert knowledge and years of training for each exam. One crucial step for the long-term goal of automatic diagnosis is pixel wise semantic segmentation. Methods In this work, several state-of-the-art semantic segmentation networks were trained on a new dataset of manually annotated ultrasound images depicting the distal femur. Results PSP-Net achieved the best overall performance with an average surface distance error (SDE) of 0.64 mm. Conclusions We recommend the PSP-Net architecture for semantic segmentation of bone surfaces.

An efficient end-to-end CNN for segmentation of bone surfaces from ultrasound

The application of ultrasound (US) imaging in orthopedic surgery has always been a research direction. However, the various problems of US imaging hinder the development of computer assisted orthopedic surgery guided by US. US bone segmentation has been an important yet challenging task for many clinical applications. We propose a new end-to-end fully convolution network called BoneNet for real-time and accurate segmentation of bone surface from US image. The BoneNet employs the squeeze-and-excitation residual to realize a robust feature learning. In order to speed up the segmentation, we reduce the convolution kernel and used depth-wise separable convolution to reduce network parameters. In addition, we assessed the impact of different normalization operations and loss functions on network performance. Finally, we realize the comparison of the segmentation performance and generalization ability of the existing real-time US bone surface segmentation network under the unified dataset. We achieved an average Dice coefficient of 93.03 % on segmentation performance test, and 91.25 % on the generalization ability test. The results show that our proposed method ensures the real-time performance and achieves significant improvements in accuracy, which substantially outperformed the state-of-the-art.

Bone shadow segmentation from ultrasound data for orthopedic surgery using GAN

Real-time, two (2D) and three-dimensional (3D) ultrasound (US) has been investigated as a potential alternative to fluoroscopy imaging in various surgical and non-surgical orthopedic procedures. However, low signal to noise ratio, imaging artifacts and bone surfaces appearing several millimeters (mm) in thickness have hindered the wide spread adaptation of this safe imaging modality. Limited field of view and manual data collection cause additional problems during US-based orthopedic procedures. In order to overcome these limitations various bone segmentation and registration methods have been developed. Acoustic bone shadow is an important image artifact used to identify the presence of bone boundaries in the collected US data. Information about bone shadow region can be used (1) to guide the orthopedic surgeon or clinician to a standardized diagnostic viewing plane with minimal artifacts, (2) as a prior feature to improve bone segmentation and registration. In this work, we propose a computational method, based on a novel generative adversarial network (GAN) architecture, to segment bone shadow images from in vivo US scans in real-time. We also show how these segmented shadow images can be incorporated, as a proxy, to a multi-feature guided convolutional neural network (CNN) architecture for real-time and accurate bone surface segmentation. Quantitative and qualitative evaluation studies are performed on 1235 scans collected from 27 subjects using two different US machines. Finally, we provide qualitative and quantitative comparison results against state-of-the-art GANs. We have obtained mean dice coefficient (± standard deviation) of [Formula: see text] ([Formula: see text]) for bone shadow segmentation, showing that the method is in close range with manual expert annotation. Statistical significant improvements against state-of-the-art GAN methods (paired t-test [Formula: see text]) is also obtained. Using the segmented bone shadow features average bone localization accuracy of 0.11mm ([Formula: see text]) was achieved. Reported accurate and robust results make the proposed method promising for various orthopedic procedures. Although we did not investigate in this work, the segmented bone shadow images could also be used as an additional feature to improve accuracy of US-based registration methods. Further extensive validations are required in order to fully understand the clinical utility of the proposed method.

Robust real-time bone surfaces segmentation from ultrasound using a local phase tensor-guided CNN.

Automatic bone surfaces segmentation is one of the fundamental tasks of ultrasound (US)-guided computer-assisted orthopedic surgery procedures. However, due to various US imaging artifacts, manual operation of the transducer during acquisition, and different machine settings, many existing methods cannot deal with the large variations of the bone surface responses, in the collected data, without manual parameter selection. Even for fully automatic methods, such as deep learning-based methods, the problem of dataset bias causes networks to perform poorly on the US data that are different from the training set. In this work, an intensity-invariant convolutional neural network (CNN) architecture is proposed for robust segmentation of bone surfaces from US data obtained from two different US machines with varying acquisition settings. The proposed CNN takes US image as input and simultaneously generates two intermediate output images, denoted as local phase tensor (LPT) and global context tensor (GCT), from two branches which are invariant to intensity variations. LPT and GCT are fused to generate the final segmentation map. In the training process, the LPT network branch is supervised by precalculated ground truth without manual annotation. The proposed method is evaluated on 1227 in vivo US scans collected using two US machines, including a portable handheld ultrasound scanner, by scanning various bone surfaces from 28 volunteers. Validation of proposed method on both US machines not only shows statistically significant improvements in cross-machine segmentation of bone surfaces compared to state-of-the-art methods but also achieves a computation time of 30 milliseconds per image, [Formula: see text] improvement over state-of-the-art. The encouraging results obtained in this initial study suggest that the proposed method is promising enough for further evaluation. Future work will include extensive validation of the method on new US data collected from various machines using different acquisition settings. We will also evaluate the potential of using the segmented bone surfaces as an input to a point set-based registration method.

Generative approach for data augmentation for deep learning-based bone surface segmentation from ultrasound images.

Precise localization of cystic bone lesions is crucial for osteolytic bone tumor surgery. Recently, there is a move toward ultrasound imaging over plain radiographs (X-rays) for intra-operative navigation due to the radiation-free and cost-effectiveness of the modality. In this process, the intra-operative bone model reconstructed from the segmented ultrasound image is registered with the pre-operative bone model. Deep learning approaches have recently shown remarkable success in bone surface segmentation from ultrasound images. However, to train deep learning models effectively with limited dataset size, data augmentation is essential. This study investigates the applicability of the generative approach for data augmentation as well as identifies standard data augmentation approaches for bone surface segmentation from ultrasound images. The generative approach we used in our work is based on Pix2Pix image-to-image translation network. We have proposed a multiple-snapshot approach, which mitigates the uni-modal deterministic output issue in the Pix2Pix network without using any complex architecture and training process. We also identified standard data augmentation approaches necessary for ultrasound bone surface segmentation through experiments. We have evaluated our networks using 800 ultrasound images from trained regions (humerus bone) and 1200 ultrasound images from untrained regions (tibia and femur bones) using four different augmentation approaches. The results show that the generative augmentation approach has a positive impact on accuracy in both trained (+4.88%) and untrained regions (+25.84%) compared to using only standard augmentations. Moreover, compared to standard augmentation approaches, the addition of the generative augmentation approach also showed a similar trend in both trained (+8.74%) and untrained (+11.55%) regions. Generative approaches are very beneficial for data augmentation, where limited dataset size is prevalent, such as ultrasound bone segmentation. The proposed multiple-snapshot Pix2Pix approach has the potential to generate multimodal images, which enlarges the dataset considerably.

Voxel‐based superimposition of serial craniofacial CBCTs: Reliability, reproducibility and segmentation effect on hard‐tissue outcomes

To test the reliability and reproducibility of a fast and user-friendly voxel-based 3D superimposition method and the effect of bone segmentation on its outcomes. This prospective methodological study assessed 15 pairs of pre-existing serial CBCT images (interval: 1.69±0.37years) obtained from growing patients (initial age: 11.75±0.59years). Volumes were superimposed on the anterior cranial base using Dolphin 3D software. Reliability was assessed visually, by inspecting the overlap of the superimposition reference structures. Reproducibility was tested with intra- and inter-operator comparisons of superimposition outcomes. The method presented good reliability in all cases. The median differences between intra- and inter-operator comparisons at various tested areas ranged from 0.06 to 0.16mm and from0.15 to 0.24mm, respectively. In few individual cases, differences exceeded 0.5mm. There was no evidence that the error increased upon increase in the magnitude of the detected T0-T1 changes. However, the superimposition error increased when the distance between the measurement area and the superimposition reference also increased. For a single image, the median error of bone surface segmentation ranged in different areas between 0.05 and 0.12mm, with few exceptions where it slightly exceeded 0.25mm. The tested voxel-based superimposition method presented good efficiency, cranial base matching and reproducibility in a growing patient sample. Segmentation error was considered minimal. The total error reached clinically relevant levels in very few cases. Thus, this technique is considered appropriate for clinical use, when 3D assessment of craniofacial changes is required.

Coupled Active Shape Models for Automated Segmentation and Landmark Localization in High-Resolution CT of the Foot and Ankle.

We develop an Active Shape Model (ASM) framework for automated bone segmentation and anatomical landmark localization in weight-bearing Cone-Beam CT (CBCT). To achieve a robust shape model fit in narrow joint spaces of the foot (0.5 - 1 mm), a new approach for incorporating proximity constraints in ASM (coupled ASM, cASM) is proposed. In cASM, shape models of multiple adjacent foot bones are jointly fit to the CBCT volume. This coupling enables checking for proximity between the evolving shapes to avoid situations where a conventional single-bone ASM might erroneously fit to articular surfaces of neighbouring bones. We used 21 extremity CBCT scans of the weight-bearing foot to compare segmentation and landmark localization accuracy of ASM and cASM in leave-one-out validation. Each scan was used as a test image once; shape models of calcaneus, talus, navicular, and cuboid were built from manual surface segmentations of the remaining 20 scans. The models were augmented with seven anatomical landmarks used for common measurements of foot alignment. The landmarks were identified in the original CBCT volumes and mapped onto mean bone shape surfaces. ASM and cASM were run for 100 iterations, and the number of principal shape components was increased every 10 iterations. Automated landmark localization was achieved by applying known point correspondences between landmark vertices on the mean shape and vertices of the final active shape segmentation of the test image. Root Mean Squared (RMS) error of bone surface segmentation improved from 3.6 mm with conventional ASM to 2.7 mm with cASM. Furthermore, cASM achieved convergence (no change in RMS error with iteration) after ~40 iterations of shape fitting, compared to ~60 iterations for ASM. Distance error in landmark localization was 25% to 55% lower (depending on the landmark) with cASM than with ASM. The importance of using a coupled model is underscored by the finding that cASM detected and corrected collisions between evolving shapes in 50% to 80% (depending on the bone) of shape model fits. The proposed cASM framework improves accuracy of shape model fits, especially in complexes of tightly interlocking, articulated joints. The approach enables automated anatomical analysis in volumetric imaging of the foot and ankle, where narrow joint spaces challenge conventional shape models.

Automatic segmentation of bone surfaces from ultrasound using a filter-layer-guided CNN.

Ultrasound (US) provides real-time, two-/three-dimensional safe imaging. Due to these capabilities, it is considered a safe alternative to intra-operative fluoroscopy in various computer-assisted orthopedic surgery (CAOS) procedures. However, interpretation of the collected bone US data is difficult due to high levels of noise, various imaging artifacts, and bone surfaces response appearing several millimeters (mm) in thickness. For US-guided CAOS procedures, it is an essential objective to have a segmentation mechanism, that is both robust and computationally inexpensive. In this paper, we present our development of a convolutional neural network-based technique for segmentation of bone surfaces from in vivo US scans. The novelty of our proposed design is that it utilizes fusion of feature maps and employs multi-modal images to abate sensitivity to variations caused by imaging artifacts and low intensity bone boundaries. B-mode US images, and their corresponding local phase filtered images are used as multi-modal inputs for the proposed fusion network. Different fusion architectures are investigated for fusing the B-mode US image and the local phase features. The proposed methods was quantitatively and qualitatively evaluated on 546 in vivo scans by scanning 14 healthy subjects. We achieved an average F-score above 95% with an average bone surface localization error of 0.2mm. The reported results are statistically significant compared to state-of-the-art. Reported accurate and robust segmentation results make the proposed method promising in CAOS applications. Further extensive validations are required in order to fully understand the clinical utility of the proposed method.

Augmented reality visualisation for orthopaedic surgical guidance with pre‐ and intra‐operative multimodal image data fusion

Augmented reality (AR) has proven to be a useful, exciting technology in several areas of healthcare. AR may especially enhance the operator's experience in minimally invasive surgical applications by providing more intuitive and naturally immersive visualisation in those procedures which heavily rely on three-dimensional (3D) imaging data. Benefits include improved operator ergonomics, reduced fatigue, and simplified hand–eye coordination. Head-mounted AR displays may hold great potential for enhancing surgical navigation given their compactness and intuitiveness of use. In this work, the authors propose a method that can intra-operatively locate bone structures using tracked ultrasound (US), registers to the corresponding pre-operative computed tomography (CT) data and generates 3D AR visualisation of the operated surgical scene through a head-mounted display. The proposed method deploys optically-tracked US, bone surface segmentation from the US and CT image volumes, and multimodal volume registration to align pre-operative to the corresponding intra-operative data. The enhanced surgical scene is then visualised in an AR framework using a HoloLens. They demonstrate the method's utility using a foam pelvis phantom and quantitatively assess accuracy by comparing the locations of fiducial markers in the real and virtual spaces, yielding root mean square errors of 3.22, 22.46, and 28.30 mm in the x , y , and z directions, respectively.

FCN-based approach for the automatic segmentation of bone surfaces in ultrasound images.

A new algorithm, based on fully convolutional networks (FCN), is proposed for the automatic localization of the bone interface in ultrasound (US) images. The aim of this paper is to compare and validate this method with (1) a manual segmentation and (2) a state-of-the-art method called confidence in phase symmetry (CPS). The dataset used for this study was composed of 1738 US images collected from three volunteers and manually delineated by three experts. The inter- and intra-observer variabilities of this manual delineation were assessed. Images having annotations with an inter-observer variability higher than a confidence threshold were rejected, resulting in 1287 images. Both FCN-based and CPS approaches were studied and compared to the average inter-observer segmentation according to six criteria: recall, precision, F1 score, accuracy, specificity and root-mean-square error (RMSE). The intra- and inter-observer variabilities were inferior to 1mm for 90% of manual annotations. The RMSE was 1.32 ± 3.70 mm and 5.00 ± 7.70mm for, respectively, the FCN-based approach and the CPS algorithm. The mean recall, precision, F1 score, accuracy and specificity were, respectively, 62%, 64%, 57%, 80% and 83% for the FCN-based approach and 66%, 34%, 41%, 52% and 43% for the CPS algorithm. The FCN-based approach outperforms the CPS algorithm, and the obtained RMSE is similar to the manual segmentation uncertainty.

Ultrasound imaging and segmentation of bone surfaces: A review.

Due to its real-time, non-radiation based three-dimensional (3D) imaging capabilities, ultrasound (US) has been incorporated into various orthopedic procedures. However, imaging artifacts, low signal-to-noise ratio (SNR) and bone boundaries appearing several mm in thickness make the analysis of US data difficult. This paper provides a review about the state-of-the-art bone segmentation and enhancement methods developed for two-dimensional (2D) and 3D US data. First, an overview for the appearance of bone surface response in B-mode data is presented. Then, classification of the proposed techniques in terms of the image information being used is provided. Specifically, the focus is given on segmentation and enhancement of B-mode US data. The review is concluded by discussing future directions of research and additional challenges which need to be overcome in order to make this imaging modality more successful in orthopedics.

Structure-enhanced local phase filtering using L0 gradient minimization for efficient semiautomated knee magnetic resonance imaging segmentation.

The segmentation of bone surfaces from magnetic resonance imaging (MRI) data has applications in the quantitative measurement of knee osteoarthritis, surgery planning for patient-specific total knee arthroplasty, and its subsequent fabrication of artificial implants. However, due to the problems associated with MRI imaging, such as low contrast between bone and surrounding tissues, noise, bias fields, and the partial volume effect, segmentation of bone surfaces continues to be a challenging operation. A framework is presented for the enhancement of knee MRI scans prior to segmentation in order to obtain high contrast bone images. During the first stage, a contrast enhanced relative total variation regularization method is used in order to remove textural noise from the bone structures and surrounding soft tissue interface. This salient bone edge information is further enhanced using a sparse gradient counting method based on [Formula: see text] gradient minimization, which globally controls how many nonzero gradients are resulted in order to approximate prominent bone structures in a structure-sparsity-management manner. The last stage of the framework involves incorporation of local phase bone boundary information in order to provide an intensity invariant enhancement of contrast between the bone and surrounding soft tissue. The enhanced images are segmented using a fast random walker algorithm. Validation against expert segmentation was performed on 20 clinical knee MRI volumes and achieved a mean dice similarity coefficient of 0.949.