Multimodality Correlation Research Articles

An Interpretable Fusion Siamese Network for Multi-Modality Remote Sensing Ship Image Retrieval

With the increasing number of remote sensing ship images, it’s vitally important to search for the ship objects that users are interested in from the remote sensing image big data. The existing works are focused on single-modality remote sensing ship image processing. But there is no method of retrieving ship images from different remote sensing image modalities. Besides, the models in existing works only output a predicted result without offering a reasonable explanation. In this work, we propose an interpretable fusion siamese network (IFSN) for addressing the multi-modality remote sensing ship image retrieval (MRSSIR). 1) An interpretable attention feature representation module is proposed to generate multiple attention maps and aggregate the filters of the last convolutional layer, which can focus on the ship’s discriminative parts and make each divided convolutional filter group express specific visual information. 2) A multi-modality correlation learning module is proposed to overcome the intra-modality and inter-modality variations by designing some constraints. 3) A discriminative region mining module is proposed to exhaustively explore all the ship’s discriminative parts available for decision-making of the proposed network. We construct a multi-modality remote sensing ship images dataset (MRSSID) to evaluate the performance of the proposed IFSN. The experimental results exhibit that our IFSN outperforms the existing methods in retrieval accuracy and provides reasonable and intuitive interpretations for the retrieval results.

Characterization of Alar Ligament in Young Adult on 3.0T MRI: A Cross-sectional Study in IIUM Medical Centre, Kuantan

Introduction: Alar ligament is a paired craniocervical junction ligaments which stabilizes the atlantooccipital and atlantoaxial joints. The main purpose of the study was to compare the normal anatomy of alar ligament on MRI between male and female. The prevalence of alar ligament visualized on MRI and its characteristics were also studied apart from determining the association between the heights of respondents with alar ligament signal intensity and dimensions. Methods: Fifty healthy volunteers were studied using 3.0T MR scanner (Siemens Magnetom Spectra) by 2-mm proton density, T2 and fat-suppression sequences. Alar ligament visualization, dimensions and variability of the ligament courses, shapes and signal intensity characteristics were determined. Results: The orientation of the ligament was laterally ascending in most of the subjects (60%), predominantly oval in shaped (54%) and 67% showed inhomogenous signal. Females are 70% less likely to exhibit alar ligament signal inhomogeneity than males. There were positive correlation between height and the craniocaudal diameter of the alar ligament as well as the anteroposterior diameter, which were statistically significant (r = 0.25, n = 100, p = 0.01 and r = 0.201, n = 100, p = 0.045 respectively). Conclusion: Tremendous variability of alar ligament shows that clinical and multimodality correlation needs to be exercised, especially in evaluating alar ligament MR signal in male. Taller individuals otherwise tend to have longer and thicker ligaments. Future studies with larger samples of alar ligaments including trauma cases are also recommended to supplant a new classification system of alar ligament injury.

Cardiac Computed Tomography Angiography for Device-Related Thrombus Assessment After WATCHMAN FLX™ Occluder Device Implantation: A Single-Center Retrospective Observational Study

Background/purposeDevice-related thrombosis (DRT) is one of the greatest challenges of transcatheter left atrial appendage device occlusion. Due to the invasive nature of transesophageal echocardiography (TEE), cardiac computed tomography angiography (CCTA) is being increasingly utilized in several centers for assessing adequate left atrial appendage closure and monitoring for DRT. There is a paucity of data regarding the standardized definition of DRT on CCTA for the WATCHMAN FLX™ device. Methods/materialsA retrospective review was conducted on 43 patients receiving WATCHMAN FLX™ device implantation with CCTA performed at the first follow-up at our institution. A comparative review of DRT predictors was performed on 10 patients who had both CCTA and TEE at the time of follow-up. ResultsHypoattenuated thickening (HAT) was a common finding on CCTA and was noted to be present in 95.35% of the patients. The combination of a large device size, peridevice gap >4 mm, and HAT located on the device gutter and 1 shoulder were characteristics present on CCTA observed in 2 patients with confirmed DRT on TEE. ConclusionCCTA is a noninvasive imaging modality for DRT monitoring, with guidelines still in development. We report potential predictors of DRT on CCTA. Additional studies are needed to further determine standardized parameters for DRT detection on CCTA and the significance of HAT with multimodality correlation.

Atypical Metastases in the Abdomen and Pelvis From Biochemically Recurrent Prostate Cancer: 11C-Choline PET/CT With Multimodality Correlation.

PET with targeted radiotracers has become integral to mapping the location and burden of recurrent disease in patients with biochemical recurrence (BCR) of prostate cancer (PCa). PET with 11C-choline is part of the National Comprehensive Cancer Network and European Association of Urology guidelines for evaluation of BCR. With advances in PET technology, increasing use of targeted radiotracers, and improved survival of patients with BCR because of novel therapeutics, atypical sites of metastases are being increasingly encountered, challenging the conventional view that prostate cancer rarely metastasizes beyond bones or lymph nodes. The purpose of this article is to describe such atypical metastases in the abdomen and pelvis on 11C-choline PET (including metastases to the liver, pancreas, genital tract, urinary tract, peritoneum, abdominal wall, and perineural spread) and to present multimodality imaging features and relevant imaging pitfalls. Given atypical metastases' inconsistent relationship with the serum PSA level and the nonspecific presenting symptoms, atypical metastases are often first detected on imaging. Awareness of their imaging features is important because their detection affects clinical management, patient counseling, prognosis, and clinical trial eligibility. Such awareness is particularly critical because the role of radiologists in the imaging and management of BCR will continue to increase given the expanding regulatory approvals of other targeted and theranostic radiotracers.

Cocaine-induced pulmonary complications

Pulmonary complications of cocaine among users are common. Manifestations include lung congestion, intra-alveolar edema, and diffuse alveolar hemorrhage (DAH). Direct cellular toxicity, eosinophilia, barotrauma, and vasoactive effects of cocaine are believed to induce DAH. We present a rare case of cocaine-associated focal alveolar hemorrhage mimicking malignancy on imaging. Initially contemplated biopsy was avoided based on rapid growth of concerning lung lesion, with subsequent near resolution on follow-up. This case illustrates the importance of epidemiologic and temporal multimodality correlation when evaluating indeterminate lung lesions.

Spatial-temporal Regularized Multi-modality Correlation Filters for Tracking with Re-detection

The development of multi-spectrum image sensing technology has brought great interest in exploiting the information of multiple modalities (e.g., RGB and infrared modalities) for solving computer vision problems. In this article, we investigate how to exploit information from RGB and infrared modalities to address two important issues in visual tracking: robustness and object re-detection. Although various algorithms that attempt to exploit multi-modality information in appearance modeling have been developed, they still face challenges that mainly come from the following aspects: (1) the lack of robustness to deal with large appearance changes and dynamic background, (2) failure in re-capturing the object when tracking loss happens, and (3) difficulty in determining the reliability of different modalities. To address these issues and perform effective integration of multiple modalities, we propose a new tracking-by-detection algorithm called Adaptive Spatial-temporal Regulated Multi-Modality Correlation Filter. Particularly, an adaptive spatial-temporal regularization is imposed into the correlation filter framework in which the spatial regularization can help to suppress effect from the cluttered background while the temporal regularization enables the adaptive incorporation of historical appearance cues to deal with appearance changes. In addition, a dynamic modality weight learning algorithm is integrated into the correlation filter training, which ensures that more reliable modalities gain more importance in target tracking. Experimental results demonstrate the effectiveness of the proposed method.

Imaging of cerebral complications of extracorporeal membrane oxygenation in infants with congenital heart disease \u2014 ultrasound with multimodality correlation

Cranial ultrasound on neonatal intensive care units is generally performed by intensive care physicians, but radiologists often provide this crucial bedside test to children on specialist paediatric cardiac intensive care units. On a paediatric cardiac intensive care unit, complex congenital cardiac conditions are commonly encountered in both pre- and postoperative scenarios, often with the use of extracorporeal membrane oxygenation (ECMO), which both increases the risks of a number of neurologic complications and results in significant changes in vascular physiology. The aim of this pictorial essay is to discuss cranial ultrasound technique, demonstrate the changes in Doppler flow profiles resulting from veno-arterial extracorporeal membrane oxygenation and congenital cardiac conditions, and illustrate commonly encountered intracranial complications of extracorporeal membrane oxygenation support in congenital cardiac care.

It's Not All Deep Vein Thrombosis: Sonography of the Painful Lower Extremity With Multimodality Correlation.

The purpose of this pictorial essay is to review different etiologies for lower extremity pain encountered on lower extremity venous sonography including acute deep venous thrombosis, chronic postthrombotic change, central venous disease, common arterial pathologies, and nonvascular abnormalities.

Comprehensive Shoulder US Examination: A Standardized Approach with Multimodality Correlation for Common Shoulder Disease.

Shoulder pain is one of the most common musculoskeletal conditions encountered in primary care and specialty orthopedic clinic settings. Although magnetic resonance (MR) imaging is typically the modality of choice for evaluating the soft-tissue structures of the shoulder, ultrasonography (US) is becoming an important complementary imaging tool in the evaluation of superficial soft-tissue structures such as the rotator cuff, subacromial-subdeltoid bursa, and biceps tendon. The advantages of US driving its recent increased use include low cost, accessibility, and capability for real-time high-resolution imaging that enables dynamic assessment and needle guidance. As more radiologists are considering incorporating shoulder US into their practices, the development of a standardized approach to performing shoulder US should be a priority to facilitate the delivery of high-quality patient care. Familiarity with and comfort in performing a standardized shoulder US examination, as well as knowledge of the types of anomalies that can be evaluated well with US, will enhance the expertise of those working in musculoskeletal radiology practices and add value in the form of increased patient and health care provider satisfaction. This review describes the utility and benefits of shoulder US as a tool that complements MR imaging in the assessment of shoulder pain. A standardized approach to the shoulder US examination is also described, with a review of the basic technique of this examination, normal anatomy of the shoulder, common indications for shoulder US, and characteristic US findings of common shoulder diseases-with select MR imaging and arthroscopic correlation. Online supplemental material is available for this article. ©RSNA, 2016.

Multi-modality morphological correlation of axillary lymph nodes

The goal of this study is to develop a computerized method that identifies a specific axillary lymph node (ALN) seen on ultrasound (US) with its most likely corresponding node on breast MRI (BMRI). This goal is an important step in developing a preoperative non-invasive method for staging breast cancer on the basis of multi-modality imaging. Twenty patients with newly diagnosed breast cancer were scanned on US and MRI. Two expert breast imaging radiologists independently correlated ALNs seen on US with BMRI, and this correlation was used as the gold standard. To correlate ALNs on US and BMRI, the cortex and hilum of each ALN was segmented using an ellipse fitting algorithm, then the ALN long and short axes and maximum cortical thickness (MCT) were computed. Three ALNs were chosen as candidates from the BMRI datasets for each lymph node seen on US. Finally, the Euclidean distances across all measurements between the US ALN and each of the three BMRI candidates were computed, and the smallest distance was reported as the correlation result. Using the expert radiologists identified correlated BMRI slice as the ground truth, the shortest Euclidean distance successfully identified the same lymph node as the radiologists in 13 out of 16 ALNs (81.25%). In negative ALNs, the standard deviation for long and short axes was relatively large but that of maximum cortical thickness was small. Average maximum cortical thickness and its standard deviation measured in US were very close to those measured in MRI. There were no significant differences among the long axis, short axis, and MCT measurements between US and MRI-T2 weighted sequence (P > 0.05 paired t-test). We performed a feasibility study which showed that computerized measurements of ALNs might be used to identify the same ALN on different modalities such as US and BMRI. This type of correlation would be valuable as it would allow the use of combined imaging parameters to be applied to the evaluation of ALNs in patients with breast cancer. It is hoped that the combined multi-modality information would provide a more robust non-invasive method of staging the axilla than is currently available.

Age-Related Structural and Functional Changes in the Breast: Multimodality Correlation With Digital Mammography, Computed Tomography, Magnetic Resonance Imaging, and Positron Emission Tomography

Preliminary results generated from digital mammography, computed tomography, magnetic resonance imaging, and (18)F-fluorodeoxyglucose positron emission tomography demonstrate concordant findings of decreasing glandular tissue and decreasing metabolic activity with increasing age. These results are presented in the context of a detailed literature review summarizing age-related changes in the breast, both from the histologic/physiologic and the imaging perspectives. We also discuss potential applications of this approach and emphasize the importance of new advanced imaging technologies to offer high levels of quantitative precision for tissue characterization for research and clinical purposes.

Experience with image management networks at three universities: is the cup half-empty or half-full? 1989.

Experience with image management networks at three universities: is the cup half-empty or half-full? 1989.