Ultrasound Registration Research Articles

D2BGAN: Dual Discriminator Bayesian Generative Adversarial Network for Deformable MR-Ultrasound Registration Applied to Brain Shift Compensation.

During neurosurgical procedures, the neuro-navigation system's accuracy is affected by the brain shift phenomenon. One popular strategy is to compensate for brain shift using intraoperative ultrasound (iUS) registration with pre-operative magnetic resonance (MR) scans. This requires a satisfactory multimodal image registration method, which is challenging due to the low image quality of ultrasound and the unpredictable nature of brain deformation during surgery. In this paper, we propose an automatic unsupervised end-to-end MR-iUS registration approach named the Dual Discriminator Bayesian Generative Adversarial Network (D2BGAN). The proposed network consists of two discriminators and a generator optimized by a Bayesian loss function to improve the functionality of the generator, and we add a mutual information loss function to the discriminator for similarity measurements. Extensive validation was performed on the RESECT and BITE datasets, where the mean target registration error (mTRE) of MR-iUS registration using D2BGAN was determined to be 0.75 ± 0.3 mm. The D2BGAN illustrated a clear advantage by achieving an 85% improvement in the mTRE over the initial error. Moreover, the results confirmed that the proposed Bayesian loss function, rather than the typical loss function, improved the accuracy of MR-iUS registration by 23%. The improvement in registration accuracy was further enhanced by the preservation of the intensity and anatomical information of the input images.

Ultrasound-Based Registration for the Computer-Assisted Navigated Percutaneous Scaphoid Fixation.

An ultrasound (US)-based computer-assisted approach has the potential to improve the accuracy and precision of screw placement for the percutaneous fixation of scaphoid fractures and also reduce the radiation dose for patient and clinical staff. Therefore, a surgical plan based on preoperative diagnostic computed tomography (CT) is registered with intraoperative US images, enabling a navigated percutaneous fracture fixation. However, approaches published so far rely on semimanual methods for intraoperative registration and are limited by long computation times. To address these challenges, we propose the employment of deep learning-based methods for US segmentation and registration in order to achieve a fast and fully automated yet robust registration process. For validation of the proposed US-based approach, we first provide a comparison of methods for segmentation and registration, assess their contribution to the overall error throughout our pipeline, and, finally, evaluate navigated screw placement in an in vitro study on 3-D printed carpal phantoms. Successful screw placement has been achieved for all ten screws, with deviations from the planned axis of 1.0 ± 0.6 and 0.7 ± 0.3 mm at the distal and proximal pole, respectively. The complete automation and total duration of about 12 s also allow seamless integration of our approach into the surgical workflow.

Feasibility of tracked ultrasound registration for pelvic-abdominal tumor navigation: a patient study.

Surgical navigation techniques can guide surgeons in localizing pelvic-abdominal malignancies. For abdominal navigation, accurate patient registration is crucial and is generally performed using an intra-operative cone-beam CT (CBCT). However, this method causes 15-min surgical preparation workflow interruption and radiation exposure, and more importantly, it cannot be repeated during surgery to compensate for large patient movement. As an alternative, the accuracy and feasibility of tracked ultrasound (US) registration are assessed in this patient study. Patients scheduled for surgical navigation during laparotomy of pelvic-abdominal malignancies were prospectively included. In the operating room, two percutaneous tracked US scans of the pelvic bone were acquired: one in supine and one in Trendelenburg patient position. Postoperatively, the bone surface was semiautomatically segmented from US images and registered to the bone surface on the preoperative CT scan. The US registration accuracy was computed using the CBCT registration as a reference and acquisition times were compared. Additionally, both US measurements were compared to quantify the registration error caused by patient movement into Trendelenburg. In total, 18 patients were included and analyzed. US registration resulted in a mean surface registration error of 1.2 ± 0.2mm and a mean target registration error of 3.3 ± 1.4mm. US acquisitions were 4 × faster than the CBCT scans (two-sample t-test P < 0.05) and could even be performed during standard patient preparation before skin incision. Patient repositioning in Trendelenburg caused a mean target registration error of 7.7 ± 3.3mm, mainly in cranial direction. US registration based on the pelvic bone is accurate, fast and feasible for surgical navigation. Further optimization of the bone segmentation algorithm will allow for real-time registration in the clinical workflow. In the end, this would allow intra-operative US registration to correct for large patient movement. This study is registered in ClinicalTrials.gov (NCT05637359).

PO-03-104 MRI VERSUS ULTRASOUND: EXPLORING THE INFLUENCE OF IMAGING MODALITY ON THE FIDELITY OF NON-CONTACT CHARGE DENSITY MAPPING

Non-contact charge density mapping of atrial fibrillation (AF) relies on an inverse algorithm and accurate representation of atrial geometry. Current clinical systems utilize intrachamber ultrasound (US) for endocardial surface mesh reconstruction. To compare intrachamber US and MRI derived left atrial (LA) reconstructions and investigate the impact of any geometric discrepancy on non-contact mapping derived conduction metrics. Five patients underwent LA ablation for persistent AF. Intrachamber US data was analyzed with AcQMap 8.5 (Acutus Medical) to produce individual LA surface meshes. Cardiac MRI was acquired up to 6 weeks prior to their procedure on a 3T clinical MR scanner. A proprietary graphical interface tool implemented in Matlab R2014a enabled 3D surface mesh reconstruction from 2D MRI slices. Prior to registration of MRI and US generated surface meshes, the mitral valve (MV), left atrial appendage, and pulmonary veins (PVs) were excluded. Registration was achieved with a two-step approach, using the iterative closest point algorithm followed by a rigid registration using 5 anatomic landmarks: center of 4 PVs and MV holes. Subsequently, non-contact intrachamber voltage measurements from the individuals’ procedures were processed to inversely derive cardiac activation on all surface meshes and measure the resulting conduction speeds (Figure 1A and B). The average distance between surfaces of intrachamber US and MRI reconstructed surface meshes was measured as 3.4±4.0mm. Histograms of non-contact mapping determined conduction speed over surface mesh area for each participant displayed similar overlapping distributions for intrachamber US and MRI (Figure 1C). Conduction speed statistics across the entire population did not show a significant difference between the two modalities (0.66±0.26m/s and 0.63±0.22m/s respectively) (Figure 1D). LA surface reconstructions produced by intrachamber US and cardiac MRI compare favorably. We demonstrate concordance between both modalities in subsequent application of the inverse algorithm required for non-contact charge density mapping and calculation of conduction speed.

Towards transcervical ultrasound image guidance for transoral robotic surgery.

Trans-oral robotic surgery (TORS) using the da Vinci surgical robot is a new minimally-invasive surgery method to treat oropharyngeal tumors, but it is a challenging operation. Augmented reality (AR) based on intra-operative ultrasound (US) has the potential to enhance the visualization of the anatomy and cancerous tumors to provide additional tools for decision-making in surgery. We propose a US-guided AR system for TORS, with the transducer placed on the neck for a transcervical view. Firstly, we perform a novel MRI-to-transcervical 3D US registration study, comprising (i) preoperative MRI to preoperative US registration, and (ii) preoperative to intraoperative US registration to account for tissue deformation due to retraction. Secondly, we develop a US-robot calibration method with an optical tracker and demonstrate its use in an AR system that displays anatomy models in the surgeon's console in real-time. Our AR system achieves a projection error from the US to the stereo cameras of 27.14 and 26.03 pixels (image is 540[Formula: see text]960) in a water bath experiment. The average target registration error (TRE) for MRI to 3D US is 8.90mm for the 3D US transducer and 5.85mm for freehand 3D US, and the TRE for pre-intra operative US registration is 7.90mm. We demonstrate the feasibility of each component of the first complete pipeline for MRI-US-robot-patient registration for a proof-of-concept transcervical US-guided AR system for TORS. Our results show that trans-cervical 3D US is a promising technique for TORS image guidance.

A robust and automatic CT-3D ultrasound registration method based on segmentation, context, and edge hybrid metric.

The fusion of computed tomography (CT) and ultrasound (US) image can enhance lesion detection ability and improve the success rate of liver interventional radiology. The image-based fusion methods encounter the challenge of registration initialization due to the random scanning pose and limited field of view of US. Existing automatic methods those used vessel geometric information and intensity-based metric are sensitive to parameters and have low success rate. The learning-based methods require a large number of registered datasets for training. The aim of this study is to provide a fully automatic and robust US-3D CT registration method without registered training data and user-specified parameters assisted by the revolutionary deep learning-based segmentation, which can further be used for preparing training samples for the study of learning-basedmethods. We propose a fully automatic CT-3D US registration method by two improved registration metrics. We propose to use 3D U-Net-based multi-organ segmentation of US and CT to assist the conventional registration. The rigid transform is searched in the space of any paired vessel bifurcation planes where the best transform is decided by a segmentation overlap metric, which is more related to the segmentation precision than Dice coefficient. In nonrigid registration phase, we propose a hybrid context and edge based image similarity metric with a simple mask that can remove most noisy US voxels to guide the B-spline transform registration. We evaluate our method on 42 paired CT-3D US datasets scanned with two different US devices from two hospitals. We compared our methods with other exsiting methods with both quantitative measures of target registration error (TRE) and the Jacobian determinent with paired t-test and qualitative registration imagingresults. The results show that our method achieves fully automatic rigid registration TRE of 4.895 mm, deformable registration TRE of 2.995 mm in average, which outperforms state-of-the-art automatic linear methods and nonlinear registration metrics with paired t-test's p value less than 0.05. The proposed overlap metric achieves better results than self similarity description (SSD), edge matching (EM), and block matching (BM) with p values of 1.624E-10, 4.235E-9, and 0.002, respectively. The proposed hybrid edge and context-based metric outperforms context-only, edge-only, and intensity statistics-only-based metrics with p values of 0.023, 3.81E-5, and 1.38E-15, respectively. The 3D US segmentation has achieved mean Dice similarity coefficient (DSC) of 0.799, 0.724, 0.788, and precision of 0.871, 0.769, 0.862 for gallbladder, vessel, and branch vessel,respectively. The deep learning-based US segmentation can achieve satisfied result to assist robust conventional rigid registration. The Dice similarity coefficient-based metrics, hybrid context, and edge image similarity metric contribute to robust and accurateregistration.

Ultrasound-guided breast biopsy using an adapted automated cone-based ultrasound scanner: a feasibility study.

Among available breast biopsy techniques, ultrasound (US)-guided biopsy is preferable because it is relatively inexpensive and provides live imaging feedback. The availability of magnetic resonance imaging (MRI)-3D US image fusion would facilitate US-guided biopsy even for US occult lesions to reduce the need for expensive and time-consuming MRI-guided biopsy. In this paper, we propose a novel Automated Cone-based Breast Ultrasound Scanning and Biopsy System (ACBUS-BS) to scan and biopsy breasts of women in prone position. It is based on a previously developed system, called ACBUS, that facilitates MRI-3D US image fusion imaging of the breast employing a conical container filled with a coupling medium. The purpose of this study was to introduce the ABCUS-BS system and demonstrate its feasibility for biopsy of US occult lesions. The biopsy procedure with the ACBUS-BS comprises four steps: target localization, positioning, preparation, and biopsy. The biopsy outcome can be impacted by 5 types of errors: due to lesion segmentation, MRI-3D US registration, navigation, lesion tracking during repositioning, and US inaccuracy (due to sound speed difference between the sample and the one used for image reconstruction). For the quantification, we use a soft custom-made polyvinyl alcohol phantom (PVA) containing 8 lesions (3 US-occult and 5 US-visible lesions of 10 mm in diameter) and a commercial breast mimicking phantom with a median stiffness of 7.6 kPa and 28 kPa, respectively. Errors of all types were quantified using the custom-made phantom. The error due to lesion tracking was also quantified with the commercial phantom. Finally, the technology was validated by biopsying the custom-made phantom and comparing the size of the biopsied material to the original lesion size. The average size of the 10-mm-sized lesions in the biopsy specimen was 7.00 ± 0.92 mm (6.33 ± 1.16 mm for US occult lesions, and 7.40 ± 0.55 mm for US-visible lesions). For the PVA phantom, the errors due to registration, navigation, lesion tracking during repositioning, and US inaccuracy were 1.33 mm, 0.30 mm, 2.12 mm, and 0.55 mm. The total error was 4.01 mm. For the commercial phantom, the error due to lesion tracking was estimated at 1.10 mm, and the total error was 4.11 mm. Given these results, the system is expected to successfully biopsy lesions larger than 8.22 mm in diameter. Patient studies will have to be carried out to confirm this in vivo. The ACBUS-BS facilitates US-guided biopsy of lesions detected in pre-MRI and therefore might offer a low-cost alternative to MRI-guided biopsy. We demonstrated the feasibility of the approach by successfully taking biopsies of 5 US-visible and 3 US-occult lesions embedded in a soft breast-shaped phantom. This article is protected by copyright. All rights reserved.

Multimodal registration of ultrasound and MR images using weighted self-similarity structure vector

ProposeMultimodal registration of 2D Ultrasound (US) and 3D Magnetic Resonance (MR) for fusion navigation can improve the intraoperative detection accuracy of lesion. However, multimodal registration remains a challenge because of the poor US image quality. In the study, a weighted self-similarity structure vector (WSSV) is proposed to registrate multimodal images. MethodThe self-similarity structure vector utilizes the normalized distance of symmetrically located patches in the neighborhood to describe the local structure information. The texture weights are extracted using the local standard deviation to reduce the speckle interference in the US images. The multimodal similarity metric is constructed by combining a self-similarity structure vector with a texture weight map. ResultsExperiments were performed on US and MR images of the liver from 88 groups of data including 8 patients and 80 simulated samples. The average target registration error was reduced from 14.91 ± 3.86 mm to 4.95 ± 2.23 mm using the WSSV-based method. ConclusionsThe experimental results show that the WSSV-based registration method could robustly align the US and MR images of the liver. With further acceleration, the registration framework can be potentially applied in time-sensitive clinical settings, such as US-MR image registration in image-guided surgery.

Nondeformed Ultrasound Image Production Method for Ultrasound-Guided Radiotherapy.

Purpose: During ultrasound (US)-guided radiotherapy, the tissue is deformed by probe pressure, and the US image is limited by changes in tissue and organ position and geometry when the US image is aligned with computed tomography (CT) image, leading to poor alignment. Accordingly, a pixel displacement-based nondeformed US image production method is proposed. Methods: The correction of US image deformation is achieved by calculating the pixel displacement of an image. The positioning CT image (CTstd) is used as the gold standard. The deformed US image (USdef) is inputted into the Harris algorithm to extract corner points for selecting feature points, and the displacement of adjacent pixels of feature points in the US video stream is calculated using the Lucas-Kanade optical flow algorithm. The moving least squares algorithm is used to correct USdef globally and locally in accordance with image pixel displacement to generate a nondeformed US image (USrev). In addition, USdef and USrev were separately aligned with CTstd to evaluate the improvement of alignment accuracy through deformation correction. Results: In the phantom experiment, the overall and local average correction errors of the US image under the optimal probe pressure were 1.0944 and 0.7388 mm, respectively, and the registration accuracy of USdef and USrev with CTstd was 0.6764 and 0.9016, respectively. During the volunteer experiment, the correction error of all 12 patients' data ranged from -1.7525 to 1.5685 mm, with a mean absolute error of 0.8612 mm. The improvement range of US and CT registration accuracy, before and after image deformation correction in the 12 patients evaluated by a normalized correlation coefficient, was 0.1232 to 0.2476. Conclusion: The pixel displacement-based deformation correction method can solve the limitation imposed by image deformation on image alignment in US-guided radiotherapy. Compared with USdef, the alignment results of USrev with CT were better.

Demographic Profile of Suspected Female Breast Cancer Patients Seeking Ultrasound Services in Selected Public Hospitals in Lusaka District, Zambia

Background: Globally, breast cancer among females is the number one cause of morbidity at 11.6% and is the leading cause of mortality at 6.6%. In Zambia, it is the most common malignancy among women. However, the demographic characteristics of suspected breast cancer female patients accessing ultrasound breast cancer services in Zambia are still unclear. Objective: This study aimed to determine the demographic characteristics of suspected female breast cancer patients. The demographic characteristics may then be used by responsible authorities such as the Ministry of Health (MoH) in Zambia and aligned Non-governmental organisations (NGOs) such as the Zambia cancer association in concentrating their efforts to curtail the scourge. Methods: Systematic random sampling was used for the selection of participants in this study. The selected participant’s demographic information was obtained from their medical records such as the medical files, Cancer Diseases Hospital (CDH) ultrasound register and University Teaching Hospital (UTH) histology registers. Further, the researcher personally interviewed the selected participants to obtain the demographic information required for the study. Results: The median age for the study participants was 43, with the interquartile range (IQR)=36-50. The median parity status was 3, with IQR=1-5. The majority of breast cancer patients were married (57) (55.9%) and employed (80) (78.4%). There was no significant difference in education level and classification of residence for study participants between participants with a histology-positive and histology-negative result, P greater than 0.05. Conclusion: This study has found evidence that the reproductive age group, low to no parity status, married marital status and employed employment status are associated with breast cancer. Education level and classification of residence did not show any association with breast cancer in this study.

Automatic 3D MRI-Ultrasound Registration for Image Guided Arthroscopy

Registration of partial view intra-operative ultrasound (US) to pre-operative MRI is an essential step in image-guided minimally invasive surgery. In this paper, we present an automatic, landmark-free 3D multimodal registration of pre-operative MRI to 4D US (high-refresh-rate 3D-US) for enabling guidance in knee arthroscopy. We focus on the problem of initializing registration in the case of partial views. The proposed method utilizes a pre-initialization step of using the automatically segmented structures from both modalities to achieve a global geometric initialization. This is followed by computing distance maps of the procured segmentations for registration in the distance space. Following that, the final local refinement between the MRI-US volumes is achieved using the LC2 (Linear correlation of linear combination) metric. The method is evaluated on 11 cases spanning six subjects, with four levels of knee flexion. A best-case error of 1.41 mm and 2.34∘ and an average registration error of 3.45 mm and 7.76∘ is achieved in translation and rotation, respectively. An inter-observer variability study is performed, and a mean difference of 4.41 mm and 7.77∘ is reported. The errors obtained through the developed registration algorithm and inter-observer difference values are found to be comparable. We have shown that the proposed algorithm is simple, robust and allows for the automatic global registration of 3D US and MRI that can enable US based image guidance in minimally invasive procedures.

Deep hashing for global registration of untracked 2D laparoscopic ultrasound to CT

PurposeThe registration of Laparoscopic Ultrasound (LUS) to CT can enhance the safety of laparoscopic liver surgery by providing the surgeon with awareness on the relative positioning between critical vessels and a tumour. In an effort to provide a translatable solution for this poorly constrained problem, Content-based Image Retrieval (CBIR) based on vessel information has been suggested as a method for obtaining a global coarse registration without using tracking information. However, the performance of these frameworks is limited by the use of non-generalisable handcrafted vessel features.MethodsWe propose the use of a Deep Hashing (DH) network to directly convert vessel images from both LUS and CT into fixed size hash codes. During training, these codes are learnt from a patient-specific CT scan by supplying the network with triplets of vessel images which include both a registered and a mis-registered pair. Once hash codes have been learnt, they can be used to perform registration with CBIR methods.ResultsWe test a CBIR pipeline on 11 sequences of untracked LUS distributed across 5 clinical cases. Compared to a handcrafted feature approach, our model improves the registration success rate significantly from 48% to 61%, considering a 20 mm error as the threshold for a successful coarse registration.ConclusionsWe present the first DH framework for interventional multi-modal registration tasks. The presented approach is easily generalisable to other registration problems, does not require annotated data for training, and may promote the translation of these techniques.

Prevalence and Predictors of Cerebral Microangiopathy Determined by Pulsatility Index in an Asymptomatic Population From the ILERVAS Project.

Background: Little is known about the prevalence of cerebral microangiopathy (CM), which is related to cognitive impairment, in an asymptomatic population. Pulsatility index (PI) is an easily measurable parameter of cerebral vascular resistance in transcranial duplex of the middle cerebral artery (MCA) study. We aimed to determine the prevalence of CM measured by PI of MCA in low to moderate vascular risk subjects.Methods: We included 3,721 subjects between 45 and 70 years without previous history of vascular disease or diabetes mellitus and with at least one other vascular risk factor from the cross-sectional study ILERVAS (Lleida, Spain). Patients underwent transcranial duplex to determine MCA-PI. Possible CM was defined by MCA-PI >1.1. Carotid and femoral arteries ultrasound registration was done to determine the presence, the number, and the area of atheromatous plaques. Body mass index (BMI), pulse pressure (PP) and laboratory data were also recorded.Results: 439 (11.8%) subjects were excluded due to the low quality of transcranial duplex images. Median age was 57 [IQR 52, 62] years. Possible CM was found in 424 (12.9%) subjects. CM patients had higher prevalence of plaques than non-CM (77.4 vs. 66.4%, p < 0.001). PI showed a positive linear correlation with the number of territories with plaques (r = 0.130, p < 0.001), and the total plaque area (r = 0.082, p < 0.001). The predictors of possible CM were the age, male gender, and PP.Conclusions: In low-to-moderate vascular risk asymptomatic population, the proportion of abnormal brain microvascular bed determined by MCA-PI is not negligible. The planned 10-year follow-up will describe the clinical relevance of these findings.

Technical note: Validation of 3D ultrasound for image registration during oncological liver surgery.

Registration of pre- and intraoperative images is a crucial step of surgical liver navigation, where rigid registration of vessel centerlines is currently commonly used. When using 3D ultrasound (US), accuracy during navigation might be influenced by the size of the intraoperative US volume, yet the relationship between registration accuracy and US volume size is understudied. In this study, we specify an optimal 3D US volume size for registration using varying volumes of liver vasculature. While previous studies measured accuracy at registered fiducials, in this work, accuracy is determined at the target lesion which is clinically the most relevant structure. Three-dimensional US volumes were acquired in 14 patients after laparotomy and liver mobilization. Manual segmentation of vasculature and centerline extraction was performed. Intraoperative and preoperative vasculature centerlines were registered with coherent point drift, using different sub-volumes (sphere with radius r=30, 40, …, 120mm). Accuracy was measured by fiducial registration error (FRE) between vessel centerlines and target registration error (TRE) at the center of the target lesion. The lowest FRE for vessel registration was reached with r=50mm (6.5±2.5mm), the highest with r=120mm (7.1±2.1mm). Clinical accuracy at the target lesion, resulted most accurate (TRE=8.8±5.0mm) in sub-volumes with a radius of 50mm. Smaller US sub-volumes resulted in lower average TREs when compared to larger US sub-volumes (Pearson's correlation coefficient R=0.91, p<0.001). Our results indicate that there is a linear correlation between US volume size and registration accuracy at the tumor. Volumes with radii of 50mm around the target lesion yield higher accuracy (p<0.05) (Trial number IRBd18032, 11 September 2018).

Three-dimensional ultrasound for knee osteophyte depiction: a comparative study to computed tomography

PurposeOsteophytes are common radiographic markers of osteoarthritis. However, they are not accurately depicted using conventional imaging, thus hampering surgical interventions that rely on pre-operative images. Studies have shown that ultrasound (US) is promising at detecting osteophytes and monitoring the progression of osteoarthritis. Furthermore, three-dimensional (3D) ultrasound reconstructions may offer a means to quantify osteophytes. The purpose of this study was to compare the accuracy of osteophyte depiction in the knee joint between 3D US and conventional computed tomography (CT).MethodsEleven human cadaveric knees were pre-screened for the presence of osteophytes. Three osteoarthritic knees were selected, and then, 3D US and CT images were obtained, segmented, and digitally reconstructed in 3D. After dissection, high-resolution structured light scanner (SLS) images of the joint surfaces were obtained. Surface matching and root mean square (RMS) error analyses of surface distances were performed to assess the accuracy of each modality in capturing osteophytes. The RMS errors were compared between 3D US, CT and SLS models.ResultsAverage RMS error comparisons for 3D US versus SLS and CT versus SLS models were 0.87 mm ± 0.33 mm (average ± standard deviation) and 0.95 mm ± 0.32 mm, respectively. No statistical difference was found between 3D US and CT. Comparative observations of imaging modalities suggested that 3D US better depicted osteophytes with cartilage and fibrocartilage tissue characteristics compared to CT.ConclusionUsing 3D US can improve the depiction of osteophytes with a cartilaginous portion compared to CT. It can also provide useful information about the presence and extent of osteophytes. Whilst algorithm improvements for automatic segmentation and registration of US are needed to provide a more robust investigation of osteophyte depiction accuracy, this investigation puts forward the potential application for 3D US in routine diagnostic evaluations and pre-operative planning of osteoarthritis.

Vessel segmentation for automatic registration of untracked laparoscopic ultrasound to CT of the liver

Purpose:Registration of Laparoscopic Ultrasound (LUS) to a pre-operative scan such as Computed Tomography (CT) using blood vessel information has been proposed as a method to enable image-guidance for laparoscopic liver resection. Currently, there are solutions for this problem that can potentially enable clinical translation by bypassing the need for a manual initialisation and tracking information. However, no reliable framework for the segmentation of vessels in 2D untracked LUS images has been presented.Methods:We propose the use of 2D UNet for the segmentation of liver vessels in 2D LUS images. We integrate these results in a previously developed registration method, and show the feasibility of a fully automatic initialisation to the LUS to CT registration problem without a tracking device.Results:We validate our segmentation using LUS data from 6 patients. We test multiple models by placing patient datasets into different combinations of training, testing and hold-out, and obtain mean Dice scores ranging from 0.543 to 0.706. Using these segmentations, we obtain registration accuracies between 6.3 and 16.6 mm in 50% of cases.Conclusions:We demonstrate the first instance of deep learning (DL) for the segmentation of liver vessels in LUS. Our results show the feasibility of UNet in detecting multiple vessel instances in 2D LUS images, and potentially automating a LUS to CT registration pipeline.

Bowel wall thickening and hyperemia assessed by high-frequency ultrasound indicate histological inflammation in Crohn\u2019s ileitis

PurposeWe evaluated ileal bowel wall thickness and semiquantitative vascularization by ultrasound in correlation with the presence or absence of histopathological inflammation in patients with Crohn’s disease (CD).MethodsWe conducted a retrospective analysis of 221 ultrasound examinations of the terminal ileum or neoterminal ileum in CD patients with biopsies of the ileum during colonoscopies within 8 weeks of the ultrasound. Ultrasound data were obtained from an inflammatory bowel disease ultrasound register from 2011 to 2017. Bowel wall ultrasound was performed by a high-frequency, linear transducer (7–12 MHz). Presence of bowel wall thickening (> 3 mm), vascularization by the Limberg score, and presence of ileal histopathological inflammation were analyzed.ResultsIn 221 bowel wall ultrasound examinations of CD patients (128 female, 93 male, mean age 37.5 years), a thickened bowel wall was found in 140 (63.3%) and hypervascularization (corresponding to a Limberg score ≥ 2) in 96 (43.4%) cases. In 187 (84.6%) cases, ileal inflammation was confirmed by histopathology and in 34 (15.4%) cases no inflammation was shown. Bowel wall thickening showed a sensitivity of 70.1%, a negative predictive value (NPV) of 30.9%, a specificity of 73.5% and a positive predictive value (PPV) of 93.6% for the detection of histopathological ileal inflammation. Hypervasularization had a low sensitivity (49.7%) and NPV (24.8%), but high specificity (91.2%) and PPV (96.9%).ConclusionIn this CD subcohort of an ultrasound register, pathologic ultrasound findings were quite common. Bowel wall thickening (> 3 mm) and hypervascularization are good predictors of histopathological inflammation within the terminal ileum or neoterminal ileum. Normal ultrasound findings without bowel wall thickening and without hypervascularization do not rule out histopathological inflammation.

Label-driven magnetic resonance imaging (MRI)-transrectal ultrasound (TRUS) registration using weakly supervised learning for MRI-guided prostate radiotherapy

Registration and fusion of magnetic resonance imaging (MRI) and transrectal ultrasound (TRUS) of the prostate can provide guidance for prostate brachytherapy. However, accurate registration remains a challenging task due to the lack of ground truth regarding voxel-level spatial correspondence, limited field of view, low contrast-to-noise ratio, and signal-to-noise ratio in TRUS. In this study, we proposed a fully automated deep learning approach based on a weakly supervised method to address these issues. We employed deep learning techniques to combine image segmentation and registration, including affine and nonrigid registration, to perform an automated deformable MRI-TRUS registration. To start with, we trained two separate fully convolutional neural networks (CNNs) to perform a pixel-wise prediction for MRI and TRUS prostate segmentation. Then, to provide the initialization of the registration, a 2D CNN was used to register MRI-TRUS prostate images using an affine registration. After that, a 3D UNET-like network was applied for nonrigid registration. For both the affine and nonrigid registration, pairs of MRI-TRUS labels were concatenated and fed into the neural networks for training. Due to the unavailability of ground-truth voxel-level correspondences and the lack of accurate intensity-based image similarity measures, we propose to use prostate label-derived volume overlaps and surface agreements as an optimization objective function for weakly supervised network training. Specifically, we proposed a hybrid loss function that integrated a Dice loss, a surface-based loss, and a bending energy regularization loss for the nonrigid registration. The Dice and surface-based losses were used to encourage the alignment of the prostate label between the MRI and the TRUS. The bending energy regularization loss was used to achieve a smooth deformation field. Thirty-six sets of patient data were used to test our registration method. The image registration results showed that the deformed MR image aligned well with the TRUS image, as judged by corresponding cysts and calcifications in the prostate. The quantitative results showed that our method produced a mean target registration error (TRE) of 2.53 ± 1.39 mm and a mean Dice loss of 0.91 ± 0.02. The mean surface distance (MSD) and Hausdorff distance (HD) between the registered MR prostate shape and TRUS prostate shape were 0.88 and 4.41 mm, respectively. This work presents a deep learning-based, weakly supervised network for accurate MRI-TRUS image registration. Our proposed method has achieved promising registration performance in terms of Dice loss, TRE, MSD, and HD.

In Vivo Measurement of the Human Lumbar Spine Using Magnetic Resonance Imaging to Ultrasound Registration

ObjectiveThis study aimed to refine a magnetic resonance imaging (MRI)-ultrasound registration (ie, alignment) technique to make noninvasive, nonionizing, 3-dimensional measurement of the lumbar segmental motion in vivo. MethodsFive healthy participants participated in this validation study. We scanned the lumbar region of each participant 5 times using an ultrasound probe while he or she kept a prone lying posture on a plinth. Participant-specific models of L1-L5 were constructed from magnetic resonance (MR) images and aligned with the 3-dimensional ultrasound dataset of each scan using 4 variants of MRI-ultrasound registration approach (simplified intensity-based registration [1] with and [2] without including the transverse processes and their surrounding soft tissues [denoted as TP complex]; and hierarchical intensity-based registration [3] with and [4] without including the TP complex). The robustness and precision of these registration approaches were compared. ResultsAlthough all registration approaches converged to a similar solution, excluding the TP complex improved the percentage of successful registration from 92% to 100%. There was no significant difference in the precision among the 4 MRI-ultrasound registration variants. For the simplified intensity-based registration without including the TP complex, average precision at each degree of freedom was 1.33° (flexion-extension), 2.48° (lateral bending), 1.32° (axial rotation), 2.15 mm (left/right), 1.08 mm (anterior-posterior), and 1.16 (superior-inferior), respectively. ConclusionGiven that using simplified intensity-based MRI-ultrasound registration can substantially streamline the registration process and excluding the TP complex would improve the robustness of the registration, we conclude that this combination is the method of choice for in vivo human applications.

Research of obtaining pseudo-CT images based on ultrasound deformation fields in radiotherapy

Objective An improved method for obtaining pseudo-computed tomography (CTps) based on ultrasound deformation field. Methods The three-dimensional image data of computed tomography and ultrasound for three postoperative cervical cancer patients were selected, including the CT (CTsim) and ultrasound (USsim) images obtained during the simulated positioning stage, and the cone beam CT (CBCT) and ultrasound images obtained during the positioning verification stage of the treatment one week later. Binary masks of the OROI and OROW were created and applied in ultrasound image registration; thus, the deformation field was obtained. The deformation field was applied to CTsim images and different pseudo-CT images were obtained. Similarities between these pseudo-CT images and those of CBCT were compared, and registration accuracies between pseudo-CT images under different binary masks and CTsim were discussed. Results The averages of the correlation coefficient between pseudo-CT based on OROI, OROW, no binary mask and CBCT were 0.95, 0.82 and 0.64 respectively. The average of the normalized mean square Error were 0.12, 0.42 and 0.57 respectively. Conclusion The pseudo-CT based on OROI binary mask matches the best with CTsim and achieves the highest similarity with CBCT. Key words: Tomography, X-ray computed; Three-dimensional ultrasound image; Pseudo-CT imagingimage guided radiation therapy