IEEE ISBI Research Articles

SellaMorph-Net: A Novel Machine Learning Approach for Precise Segmentation of Sella Turcica Complex Structures in Full Lateral Cephalometric Images

Background: The Sella Turcica is a critical structure from an orthodontic perspective, and its morphological characteristics can help in understanding craniofacial deformities. However, accurately extracting Sella Turcica shapes can be challenging due to the indistinct edges and indefinite boundaries present in X-ray images. This study aimed to develop and validate an automated Sella Morphology Network (SellaMorph-Net), a hybrid deep learning pipeline for segmenting Sella Turcica structure and extracting different morphological types; Methods: The SellaMorph-Net model proposed in this study combined attention-gating and recurrent residual convolutional layers (AGM and RrCL) to enhance the encoder’s abilities. The model’s output was then passed through a squeeze-and-excitation (SE) module to improve the network’s robustness. In addition, dropout layers were added to the end of each convolution block to prevent overfitting. A Zero-shot classifier was employed for multiple classifications, and the model’s output layer used five colour codes to represent different morphological types. The model’s performance was evaluated using various quantitative metrics, such as global accuracy and mean pixel-wise Intersection over Union (IoU) and dice coefficient, based on qualitative results; Results: The study collected 1653 radiographic images and categorised them into four classes based on the predefined shape of Sella Turcica. These classes were further divided into three subgroups based on the complexity of the Sella structures. The proposed SellaMorph-Net model achieved a global accuracy of 97.570, mean pixel-wise IoU scores of 0.7129, and a dice coefficient of 0.7324, significantly outperforming the VGG-19 and InceptionV3 models. The publicly available IEEE ISBI 2015 challenge dataset and our dataset were used to evaluate the test performance between the state-of-the-art and proposed models. The proposed model provided higher testing results, which were 0.7314 IoU and 0.7768 dice for our dataset and 0.7864 IoU and 0.8313 dice for the challenge dataset; Conclusions: The proposed hybrid SellaMorph-Net model provides an accurate and reliable pipeline for detecting morphological types of Sella Turcica using full lateral cephalometric images. Future work will focus on further improvement and utilisation of the developed model as a prognostic tool for predicting anomalies related to Sella structures.

DMNet: Dual-Stream Marker Guided Deep Network for Dense Cell Segmentation and Lineage Tracking.

Accurate segmentation and tracking of cells in microscopy image sequences is extremely beneficial in clinical diagnostic applications and biomedical research. A continuing challenge is the segmentation of dense touching cells and deforming cells with indistinct boundaries, in low signal-to-noise-ratio images. In this paper, we present a dual-stream marker-guided network (DMNet) for segmentation of touching cells in microscopy videos of many cell types. DMNet uses an explicit cell marker-detection stream, with a separate mask-prediction stream using a distance map penalty function, which enables supervised training to focus attention on touching and nearby cells. For multi-object cell tracking we use M2Track tracking-by-detection approach with multi-step data association. Our M2Track with mask overlap includes short term track-to-cell association followed by track-to-track association to re-link tracklets with missing segmentation masks over a short sequence of frames. Our combined detection, segmentation and tracking algorithm has proven its potential on the IEEE ISBI 2021 6th Cell Tracking Challenge (CTC-6) where we achieved multiple top three rankings for diverse cell types. Our team name is MU-Ba-US, and the implementation of DMNet is available at, http://celltrackingchallenge.net/participants/MU-Ba-US/.

Evaluation of automated cephalometric analysis based on the latest deep learning method

To compare an automated cephalometric analysis based on the latest deep learning method of automatically identifying cephalometric landmarks (AI) with previously published AI according to the test style of the worldwide AI challenges at the International Symposium on Biomedical Imaging conferences held by the Institute of Electrical and Electronics Engineers (IEEE ISBI). This latest AI was developed by using a total of 1983 cephalograms as training data. In the training procedures, a modification of a contemporary deep learning method, YOLO version 3 algorithm, was applied. Test data consisted of 200 cephalograms. To follow the same test style of the AI challenges at IEEE ISBI, a human examiner manually identified the IEEE ISBI-designated 19 cephalometric landmarks, both in training and test data sets, which were used as references for comparison. Then, the latest AI and another human examiner independently detected the same landmarks in the test data set. The test results were compared by the measures that appeared at IEEE ISBI: the success detection rate (SDR) and the success classification rates (SCR). SDR of the latest AI in the 2-mm range was 75.5% and SCR was 81.5%. These were greater than any other previous AIs. Compared to the human examiners, AI showed a superior success classification rate in some cephalometric analysis measures. This latest AI seems to have superior performance compared to previous AI methods. It also seems to demonstrate cephalometric analysis comparable to human examiners.

Cell segmentation and tracking using CNN-based distance predictions and a graph-based matching strategy.

The accurate segmentation and tracking of cells in microscopy image sequences is an important task in biomedical research, e.g., for studying the development of tissues, organs or entire organisms. However, the segmentation of touching cells in images with a low signal-to-noise-ratio is still a challenging problem. In this paper, we present a method for the segmentation of touching cells in microscopy images. By using a novel representation of cell borders, inspired by distance maps, our method is capable to utilize not only touching cells but also close cells in the training process. Furthermore, this representation is notably robust to annotation errors and shows promising results for the segmentation of microscopy images containing in the training data underrepresented or not included cell types. For the prediction of the proposed neighbor distances, an adapted U-Net convolutional neural network (CNN) with two decoder paths is used. In addition, we adapt a graph-based cell tracking algorithm to evaluate our proposed method on the task of cell tracking. The adapted tracking algorithm includes a movement estimation in the cost function to re-link tracks with missing segmentation masks over a short sequence of frames. Our combined tracking by detection method has proven its potential in the IEEE ISBI 2020 Cell Tracking Challenge (http://celltrackingchallenge.net/) where we achieved as team KIT-Sch-GE multiple top three rankings including two top performances using a single segmentation model for the diverse data sets.

DeepHistReg: Unsupervised Deep Learning Registration Framework for Differently Stained Histology Samples

Background and objectiveThe use of several stains during histology sample preparation can be useful for fusing complementary information about different tissue structures. It reveals distinct tissue properties that combined may be useful for grading, classification, or 3-D reconstruction. Nevertheless, since the slide preparation is different for each stain and the procedure uses consecutive slices, the tissue undergoes complex and possibly large deformations. Therefore, a nonrigid registration is required before further processing. The nonrigid registration of differently stained histology images is a challenging task because: (i) the registration must be fully automatic, (ii) the histology images are extremely high-resolution, (iii) the registration should be as fast as possible, (iv) there are significant differences in the tissue appearance, and (v) there are not many unique features due to a repetitive texture. MethodsIn this article, we propose a deep learning-based solution to the histology registration. We describe a registration framework dedicated to high-resolution histology images that can perform the registration in real-time. The framework consists of an automatic background segmentation, iterative initial rotation search and learning-based affine/nonrigid registration. ResultsWe evaluate our approach using an open dataset provided for the Automatic Non-rigid Histological Image Registration (ANHIR) challenge organized jointly with the IEEE ISBI 2019 conference. We compare our solution to the challenge participants using a server-side evaluation tool provided by the challenge organizers. Following the challenge evaluation criteria, we use the target registration error (TRE) as the evaluation metric. Our algorithm provides registration accuracy close to the best scoring teams (median rTRE 0.19% of the image diagonal) while being significantly faster (the average registration time is about 2 seconds). ConclusionsThe proposed framework provides results, in terms of the TRE, comparable to the best-performing state-of-the-art methods. However, it is significantly faster, thus potentially more useful in clinical practice where a large number of histology images are being processed. The proposed method is of particular interest to researchers requiring an accurate, real-time, nonrigid registration of high-resolution histology images for whom the processing time of traditional, iterative methods in unacceptable. We provide free access to the software implementation of the method, including training and inference code, as well as pretrained models. Since the ANHIR dataset is open, this makes the results fully and easily reproducible.

Prototypes for content-based image retrieval in clinical practice.

Content-based image retrieval (CBIR) has been proposed as key technology for computer-aided diagnostics (CAD). This paper reviews the state of the art and future challenges in CBIR for CAD applied to clinical practice.We define applicability to clinical practice by having recently demonstrated the CBIR system on one of the CAD demonstration workshops held at international conferences, such as SPIE Medical Imaging, CARS, SIIM, RSNA, and IEEE ISBI. From 2009 to 2011, the programs of CADdemo@CARS and the CAD Demonstration Workshop at SPIE Medical Imaging were sought for the key word “retrieval” in the title. The systems identified were analyzed and compared according to the hierarchy of gaps for CBIR systems.In total, 70 software demonstrations were analyzed. 5 systems were identified meeting the criterions. The fields of application are (i) bone age assessment, (ii) bone fractures, (iii) interstitial lung diseases, and (iv) mammography. Bridging the particular gaps of semantics, feature extraction, feature structure, and evaluation have been addressed most frequently.In specific application domains, CBIR technology is available for clinical practice. While system development has mainly focused on bridging content and feature gaps, performance and usability have become increasingly important. The evaluation must be based on a larger set of reference data, and workflow integration must be achieved before CBIR-CAD is really established in clinical practice.

Mapping correlations between ventricular expansion and CSF amyloid and tau biomarkers in 240 subjects with Alzheimer's disease, mild cognitive impairment and elderly controls

We aimed to improve on the single-atlas ventricular segmentation method of (Carmichael, O.T., Thompson, P.M., Dutton, R.A., Lu, A., Lee, S.E., Lee, J.Y., Kuller, L.H., Lopez, O.L., Aizenstein, H.J., Meltzer, C.C., Liu, Y., Toga, A.W., Becker, J.T., 2006. Mapping ventricular changes related to dementia and mild cognitive impairment in a large community-based cohort. IEEE ISBI. 315–318) by using multi-atlas segmentation, which has been shown to lead to more accurate segmentations (Chou, Y., Leporé, N., de Zubicaray, G., Carmichael, O., Becker, J., Toga, A., Thompson, P., 2008. Automated ventricular mapping with multi-atlas fluid image alignment reveals genetic effects in Alzheimer's disease, NeuroImage 40(2): 615–630); with this method, we calculated minimal numbers of subjects needed to detect correlations between clinical scores and ventricular maps. We also assessed correlations between emerging CSF biomarkers of Alzheimer's disease pathology and localizable deficits in the brain, in 80 AD, 80 mild cognitive impairment (MCI), and 80 healthy controls from the Alzheimer's Disease Neuroimaging Initiative. Six expertly segmented images and their embedded parametric mesh surfaces were fluidly registered to each brain; segmentations were averaged within subjects to reduce errors. Surface-based statistical maps revealed powerful correlations between surface morphology and 4 variables: (1) diagnosis, (2) depression severity, (3) cognitive function at baseline, and (4) future cognitive decline over the following year. Cognitive function was assessed using the mini-mental state exam (MMSE), global and sum-of-boxes clinical dementia rating (CDR) scores, at baseline and 1-year follow-up. Lower CSF Aβ1–42 protein levels, a biomarker of AD pathology assessed in 138 of the 240 subjects, were correlated with lateral ventricular expansion. Using false discovery rate (FDR) methods, 40 and 120 subjects, respectively, were needed to discriminate AD and MCI from normal groups. 120 subjects were required to detect correlations between ventricular enlargement and MMSE, global CDR, sum-of-boxes CDR and clinical depression scores. Ventricular expansion maps correlate with pathological and cognitive measures in AD, and may be useful in future imaging-based clinical trials.