Medical Imaging Information Research Articles

Question-guided feature pyramid network for medical visual question answering

Medical VQA (VQA-Med) is a critical multi-modal task that raises attention from the community. Existing models utilized just one high-level feature map (i.e., the last layer feature map) extracted by CNN and then fused it with semantic features through the co-attention mechanism. However, only using the high-level feature as a visual feature often ignores the details of the image, which are crucial for VQA-Med. In addition, questions often serve as a guide to targets of attention in the medical image. Therefore, in this paper, we propose a question-guided Feature Pyramid Network (QFPN) for VQA-Med. It extracts multi-level visual features with a feature pyramid network (FPN). In this way, the multi-scale information of medical images can be captured by using the high resolution of low-level features and rich semantic information of high-level features. Besides, a novel question-guided dynamic filter network (DFN) is designed to modulate the fusion progress of multi-level visual features and semantic features with respect to the raised question. Extensive results have demonstrated the effectiveness of the QFPN. Especially, we beat the winner of the 2019 ImageCLEF challenge and achieved 63.8% Accuracy and 65.7% BLEU in the ImageCLEF 2019 VQA-Med dataset.

Malaria Parasite Identification from Red Blood Cell Images Using Transfer Learning Models

Malaria is a dangerous viral disease caused by Plasmodium protozoan parasites that are spread by the bite of an infected female Anopheles mosquito. This pandemic disease's fast and precise identification is essential for effective treatment. The most reliable method for diagnosing malaria is a microscopic examination of a thick and thin blood smear, which looks for the parasite and counts the number of infected cells. The ability to wholly or partially automate the identification of the disease using the information in medical images highlights the critical role that computer-aided diagnosis plays in modern medicine, in which machine learning and deep learning play a critical role. In this study, we have presented an in-depth overview of the techniques and methods used to diagnose the malaria parasite through blood slides automatically. One of the techniques is using transfer learning models to detect the malaria parasite. We have compared the performance of transfer learning models on identifying infected malaria cells by feeding the models a large dataset of uninfected and parasite cell images. The results show that the DensNet models have the edge over the other models, with DenseNet-201 achieving the highest accuracy and F1 score of 0.9339 and 0.9321, respectively. Also, DenseNet-169 outperformed the other models with 0.9594 in precision, and finally, Densenet-121 had the highest recall with 0.9490.

Robust watermarking algorithm for medical images based on accelerated‐KAZE discrete cosine transform

With the continuous progress and development in the field of Internet technology, the area of medical image processing has also developed along with it. Specially, digital watermarking technology plays an essential role in the field of medical image processing and greatly improves the security of medical image information. A medical image watermarking algorithm based on an accelerated-KAZE discrete cosine transform (AKAZE-DCT) is proposed to address the poor robustness of medical image watermarking algorithms to geometric attacks, which leads to low security of the information contained in medical images. First, the AKAZE-DCT algorithm is used to extract the feature vector of the medical image and then combined with the perceptual hashing technique to obtain the feature sequence of the medical image; then, the watermarking image is encrypted with logistic chaos dislocation to get the encrypted watermarking image, which ensures the security of the watermarking information; finally, the watermarking is embedded and extracted with the zero-watermarking technique. The experimental results show that the algorithm can effectively extract the watermark under conventional and geometric attacks, reflecting better robustness and invisibility, and has certain practicality in the medical field compared with other algorithms.

AHU-MultiNet: Adaptive loss balancing based on homoscedastic uncertainty in multi-task medical image segmentation network

Medical image segmentation is an important field in medical image analysis and a vital part of computer-aided diagnosis. Due to the challenges in acquiring image annotations, semi-supervised learning has attracted high attention in medical image segmentation. Despite their impressive performance, most existing semi-supervised approaches lack attention to ambiguous regions (e.g., some edges or corners around the organs). To achieve better performance, we propose a novel semi-supervised method called Adaptive Loss Balancing based on Homoscedastic Uncertainty in Multi-task Medical Image Segmentation Network (AHU-MultiNet). This model contains the main task for segmentation, one auxiliary task for signed distance, and another auxiliary task for contour detection. Our multi-task approach can effectively and sufficiently extract the semantic information of medical images by auxiliary tasks. Simultaneously, we introduce an inter-task consistency to explore the underlying information of the images and regularize the predictions in the right direction. More importantly, we notice and analyze that searching an optimal weighting manually to balance each task is a difficult and time-consuming process. Therefore, we introduce an adaptive loss balancing strategy based on homoscedastic uncertainty. Experimental results show that the two auxiliary tasks explicitly enforce shape-priors on the segmentation output to further generate more accurate masks under the adaptive loss balancing strategy. On several standard benchmarks, the 2018 Atrial Segmentation Challenge and the 2017 Liver Tumor Segmentation Challenge, our proposed method achieves improvements and outperforms the new state-of-the-art in semi-supervised learning.

Global weighted average pooling network with multilevel feature fusion for weakly supervised brain tumor segmentation

Medical image segmentation plays a vital role in computer-aided diagnosis and intelligent medical treatment. It can preprocess medical images to help doctors better diagnose diseases. Class activation map (CAM) is an important technology in weakly supervised segmentation, which can achieve image segmentation without pixel-level label training. This technology can well meet the needs of medical image segmentation. However, CAM obtaining is still unperfect due to global average pooling (GAP). GAP will cause important and non-important regions to be given equal attention during the training process. So, CAM cannot demarcate the boundary of the target regions well. In order to solve this problem, a global weighted average pooling network fusing the grayscale information of medical images is proposed. The proposed network can solve the problem that GAP has the same concern for important regions and non-important regions of the feature map, because the different weights can be learned for different positions of the feature map before the GAP in the proposed model. At the same time, because of the grayscale difference between the tumor area and the non-tumor area in the brain tumor image, the low-level grayscale information of the medical image is fused with the high-level semantic information extracted by the network to learn the weights. This operation gives full play to the advantages of feature maps of different levels. The experiment results on the popular medical image dataset BraTS2019 show that the proposed method can well improve the performance of CAM and help CAM fit the boundaries of objects. Meanwhile, in the DSC evaluation, the proposed method achieves a score of 64.1%, which is a 4.6% improvement over a recent research method.

HEA-Net: Attention and MLP Hybrid Encoder Architecture for Medical Image Segmentation.

The model, Transformer, is known to rely on a self-attention mechanism to model distant dependencies, which focuses on modeling the dependencies of the global elements. However, its sensitivity to the local details of the foreground information is not significant. Local detail features help to identify the blurred boundaries in medical images more accurately. In order to make up for the defects of Transformer and capture more abundant local information, this paper proposes an attention and MLP hybrid-encoder architecture combining the Efficient Attention Module (EAM) with a Dual-channel Shift MLP module (DS-MLP), called HEA-Net. Specifically, we effectively connect the convolution block with Transformer through EAM to enhance the foreground and suppress the invalid background information in medical images. Meanwhile, DS-MLP further enhances the foreground information via channel and spatial shift operations. Extensive experiments on public datasets confirm the excellent performance of our proposed HEA-Net. In particular, on the GlaS and MoNuSeg datasets, the Dice reached 90.56% and 80.80%, respectively, and the IoU reached 83.62% and 68.26%, respectively.

Deep evidential fusion network for medical image classification

The multi-modality characteristic of medical images calls for the application of information fusion theory in computer aided diagnosis (CAD) algorithm design. Recently, the research of uncertainty estimation in deep neural networks provides a new perspective for information fusion in deep learning algorithms. For medical image classification tasks, due to the difficulty in collecting large-scale datasets, it is a challenging job in the study of deep learning multi-modality medical image classification model. In this paper, we investigate the fusion method based on the belief/uncertainty estimation framework of evidential deep learning (EDL) and Dempster's rule of combination. We also propose a deep evidential fusion method to best utilize the belief assignment and uncertainty estimation for combining the information of multi-modality medical images when only small-scale and even incomplete multi-modality medical image dataset is available. The proposed method has been tested on two real-world medical image classification tasks. To maximize the use of available medical imaging resources, we extended our model to handle the modality missing problem for multi-modality learning. Experiments show that, with the proposed weighted mass calibration method, our fusion model can handle the modality missing problem in real-world applications, making it possible to incorporate more incomplete data for learning.

Multi-input Unet model based on the integrated block and the aggregation connection for MRI brain tumor segmentation

With the growth of data information and the development of computer equipment, it is extremely time-consuming and laborious to rely on the traditional manual segmentation of brain medical images. To solve the above problems, this paper proposes a multi-input Unet model based on the integrated block and the aggregation connection to achieve efficient and accurate segmentation of tumor structure. Besides, this paper solves the low-resolution problem in sagittal and coronal planes, which can effectively improve memory efficiency. The proposed algorithm is innovative in three aspects. Firstly, by inputting the mask images which can effectively represent the tumor location characteristics, it can provide more information about the spatial relationship to alleviate the problems of fuzzy boundary and low contrast between the lesion region and healthy brain tissue. Then, the integrated block extracts the tumor local information in different receptive domains by a multi-scale convolution kernel. The aggregation connection realizes the implicit deep connection of context information, which combines the shallow and deep information of the brain with strong geometric spatial relationships. Meanwhile, to effectively alleviate the waste of memory resources caused by redundant and background information in medical images, the amount of calculation in model training is reduced by dimension reduction of the feature map. An ablation experiment is used to verify the innovation of the proposed algorithm on the BraTS dataset, which compares with the state-of-the-art brain tumor segmentation methods. The precision of the proposed multi-input Unet model for the whole tumor and core lesion is 0.92 and 0.90, respectively.

Deep Learning Algorithm in Biomedical Engineering in Intelligent Automatic Processing and Analysis of Sports Images

In order to improve the detection and identification ability of sports injury ultrasound medicine, a segmentation method of sports injury ultrasound medical image based on local features is proposed, and the research on the sports injury ultrasound medical detection and identification ability is carried out. Methods of the sports injury ultrasound medical image segmentation model are established; the sports injury ultrasound medical image information is enhanced by using the sports skeletal muscle block matching technology; the image features are extracted; and the characteristics of sports injury ultrasound medical images are analyzed by CT bright spot feature transmission. In detail, combined with the deep convolutional neural network training method, the extracted sports injury points are automatically detected for sports injury ultrasound medical images, and the sports injury ultrasound medical image segmentation is realized. The simulation results show that this method has high accuracy for sports injury ultrasound medical image segmentation, the error value can be controlled below 0.103, and finally, the effect of zero error is achieved. It is confirmed that the method proposed in this paper has high resolution and accuracy for sports injury point detection and has strong practical application ability.

Real-time medical image denoising and information hiding model based on deep wavelet multiscale autonomous unmanned analysis

Real-time medical image denoising and information hiding model based on deep wavelet multiscale autonomous unmanned analysis

Sports Medical Image Modeling of Injury Prevention in Dance Learning and Sports Training.

In order to effectively prevent injuries in dance learning and sports training, this paper proposes a method based on sports medical image modeling. This method solves the problem of injury prevention in dance learning by studying the association analysis algorithm, medical image information system, and CT technology and analyzing the role of data mining technology in the medical image information system. The experimental results show that the average prediction error of CT and US is about 5%, which can be considered that the model can predict accurately. The error of MR is as high as 28.2%, and the prediction is relatively inaccurate. Conclusion. the model can effectively prevent the injury in training.

Robust convolutional neural networks against adversarial attacks on medical images

Convolutional neural networks (CNNs) have been widely applied to medical images. However, medical images are vulnerable to adversarial attacks by perturbations that are undetectable to human experts. This poses significant security risks and challenges to CNN-based applications in clinic practice. In this work, we quantify the scale of adversarial perturbation imperceptible to clinical practitioners and investigate the cause of the vulnerability in CNNs. Specifically, we discover that noise (i.e., irrelevant or corrupted discriminative information) in medical images might be a key contributor to performance deterioration of CNNs against adversarial perturbations, as noisy features are learned unconsciously by CNNs in feature representations and magnified by adversarial perturbations. In response, we propose a novel defense method by embedding sparsity denoising operators in CNNs for improved robustness. Tested with various state-of-the-art attacking methods on two distinct medical image modalities, we demonstrate that the proposed method can successfully defend against those unnoticeable adversarial attacks by retaining as much as over 90% of its original performance. We believe our findings are critical for improving and deploying CNN-based medical applications in real-world scenarios.

LEGAN: A Light and Effective Generative Adversarial Network for medical image synthesis

Medical image synthesis plays an important role in clinical diagnosis by providing auxiliary pathological information. However, previous methods usually utilize the one-step strategy designed for wild image synthesis, which are not sensitive to local details of tissues within medical images. In addition, these methods consume a great number of computing resources in generating medical images, which seriously limits their applicability in clinical diagnosis. To address the above issues, a Light and Effective Generative Adversarial Network (LEGAN) is proposed to generate high-fidelity medical images in a lightweight manner. In particular, a coarse-to-fine paradigm is designed to imitate the painting process of humans for medical image synthesis within a two-stage generative adversarial network, which guarantees the sensitivity to local information of medical images. Furthermore, a low-rank convolutional layer is introduced to construct LEGAN for lightweight medical image synthesis, which utilizes principal components of full-rank convolutional kernels to reduce model redundancy. Additionally, a multi-stage mutual information distillation is devised to maximize dependencies of distributions between generated and real medical images in model training. Finally, extensive experiments are conducted in two typical tasks, i.e., retinal fundus image synthesis and proton density weighted MR image synthesis. The results demonstrate that LEGAN outperforms the comparison methods by a significant margin in terms of Fréchet inception distance (FID) and Number of parameters (NoP).

Combining Molecular, Imaging, and Clinical Data Analysis for Predicting Cancer Prognosis.

Simple SummaryThe rise of Big Data, the widespread use of Machine Learning, and the cheapening of omics techniques have allowed for the creation of more sophisticated and accurate models in biomedical research. This article presents the state-of-the-art predictive models of cancer prognosis that use multimodal data, considering clinical, molecular (omics and non-omics), and image data. The subject of study, the data modalities used, the data processing and modelling methods applied, the validation strategies involved, the integration strategies encompassed, and the evolution of prognostic predictive models are discussed. Finally, we discuss challenges and opportunities in this field of cancer research, with great potential impact on the clinical management of patients and, by extension, on the implementation of personalised and precision medicine.Cancer is one of the most detrimental diseases globally. Accordingly, the prognosis prediction of cancer patients has become a field of interest. In this review, we have gathered 43 state-of-the-art scientific papers published in the last 6 years that built cancer prognosis predictive models using multimodal data. We have defined the multimodality of data as four main types: clinical, anatomopathological, molecular, and medical imaging; and we have expanded on the information that each modality provides. The 43 studies were divided into three categories based on the modelling approach taken, and their characteristics were further discussed together with current issues and future trends. Research in this area has evolved from survival analysis through statistical modelling using mainly clinical and anatomopathological data to the prediction of cancer prognosis through a multi-faceted data-driven approach by the integration of complex, multimodal, and high-dimensional data containing multi-omics and medical imaging information and by applying Machine Learning and, more recently, Deep Learning techniques. This review concludes that cancer prognosis predictive multimodal models are capable of better stratifying patients, which can improve clinical management and contribute to the implementation of personalised medicine as well as provide new and valuable knowledge on cancer biology and its progression.

Fuzzy Medical Computer Vision Image Restoration and Visual Application

In order to shorten the image registration time and improve the imaging quality, this paper proposes a fuzzy medical computer vision image information recovery algorithm based on the fuzzy sparse representation algorithm. Firstly, by constructing a computer vision image acquisition model, the visual feature quantity of the fuzzy medical computer vision image is extracted, and the feature registration design of the fuzzy medical computer vision image is carried out by using the 3D visual reconstruction technology. Then, by establishing a multidimensional histogram structure model, the wavelet multidimensional scale feature detection method is used to achieve grayscale feature extraction of fuzzy medical computer vision images. Finally, the fuzzy sparse representation algorithm is used to automatically optimize the fuzzy medical computer vision images. The experimental results show that the proposed method has a short image information registration time, less than 10 ms, and has a high peak PSNR. When the number of pixels is 700, its peak PSNR can reach 83.5 dB, which is suitable for computer image restoration.

Key Technology of the Medical Image Wise Mining Method Based on the Meanshift Algorithm.

Mean-shift originally refers to the mean vector of the offset. The algorithm idea is to assume that the data sets of different clusters conform to different probability density distributions, and the area with high sample density corresponds to the center of the cluster. With the wide application of hospital information system, especially the popularity of the meanshift algorithm in the outpatient system, it has greatly improved the efficiency of medical staff. Medical imaging refers to the technology and process of obtaining internal tissue images of the human body or a certain part of the human body in a noninvasive manner for medical treatment or medical research. It contains the following two relatively independent research directions: medical imaging system and medical image processing. In this paper, we expect to improve the mining ability of medical image information with the help of the meanshift algorithm based on the key technology of the medical image intelligent mining algorithm. This paper proposes a method to enhance image feature extraction and data mining and how to apply relevant analysis rules for mining. Applying this integrated algorithm to extract simplified rules is more beneficial to people's understanding than the raw data and helps doctors quickly understand the patient's condition.

Benign and malignant diagnosis of spinal tumors based on deep learning and weighted fusion framework on MRI

BackgroundThe application of deep learning has allowed significant progress in medical imaging. However, few studies have focused on the diagnosis of benign and malignant spinal tumors using medical imaging and age information at the patient level. This study proposes a multi-model weighted fusion framework (WFF) for benign and malignant diagnosis of spinal tumors based on magnetic resonance imaging (MRI) images and age information.MethodsThe proposed WFF included a tumor detection model, sequence classification model, and age information statistic module based on sagittal MRI sequences obtained from 585 patients with spinal tumors (270 benign, 315 malignant) between January 2006 and December 2019 from the cooperative hospital. The experimental results of the WFF were compared with those of one radiologist (D1) and two spine surgeons (D2 and D3).ResultsIn the case of reference age information, the accuracy (ACC) (0.821) of WFF was higher than three doctors’ ACC (D1: 0.686; D2: 0.736; D3: 0.636). Without age information, the ACC (0.800) of the WFF was also higher than that of the three doctors (D1: 0.750; D2: 0.664; D3:0.614).ConclusionsThe proposed WFF is effective in the diagnosis of benign and malignant spinal tumors with complex histological types on MRI.

Multimodal Medical Image Fusion Based on Content-based and PCA-sigmoid.

The objective of any multimodal medical image fusion algorithm is to assist a radiologist for better decision-making during the diagnosis and therapy by integrating the anatomical (magnetic resonance imaging) and functional (positron emission tomography/ single-photon emission computed tomography) information. We proposed a new medical image fusion method based on content-based decomposition, Principal Component Analysis (PCA), and sigmoid function. We considered Empirical Wavelet Transform (EWT) for content-based decomposition purposes since it can preserve crucial medical image information such as edges and corners. PCA is used to obtain initial weights corresponding to each detail layer. In our experiments, we found that direct usage of PCA for detail layer fusion introduces severe artifacts into the fused image due to weight scaling issues. In order to tackle this, we considered using the sigmoid function for better weight scaling. We considered 24 pairs of MRI-PET and 24 pairs of MRI-SPECT images for fusion, and the results are measured using four significant quantitative metrics. Finally, we compared our proposed method with other state-of-the-art transformbased fusion approaches, using traditional and recent performance measures. An appreciable improvement is observed in both qualitative and quantitative results compared to other fusion methods.

Visual expertise modulates baseline brain activity: a preliminary resting-state fMRI study using expertise model of radiologists

Backgroundvisual expertise and experience modulate evoked brain activity in response to training-related stimuli. However, few studies have considered how the visual experience is represented in the resting state brain activity. This study tried to investigate the way visual experience, i.e., visual recognition expertise, modulates baseline brain neuronal activity in the resting state using the model of radiologists.MethodsThe amplitude of low-frequency (< 0.08 Hz) fluctuation (ALFF) was used as the metric of baseline brain activity and a visual expertise model of radiologists to investigated this question. The visual recognition skill enables them to accurately identify pathological information in medical images. After the behavior measurement, a cohort group of radiology interns (n = 22) and a group of matched layperson (n = 22) were selected for inclusion in the study. The resting state functional magnetic resonance imaging (fMRI) scans were performed for all of the subjects.ResultsHigher ALFF in the right fusiform gyrus and the left orbitofrontal cortex were observed, and the ALFF in the fusiform gyrus was correlated with the intern radiologists’ behavioral expertise(all results corrected for multiple comparisons).ConclusionsVisual experience modulates the baseline brain activity in both high-level visual cortex and high-order cognitive cortex, indicating the engagement of both top-down and bottom-up facilitation. We provide a novel perspective to how visual experience modulated cortical brain activity by introducing the resting state changes. Also, we propose that our current study may provide novel ideas for the development of new training protocols in medical school.

Using Quantitative Imaging for Personalized Medicine in Pancreatic Cancer: A Review of Radiomics and Deep Learning Applications

Simple SummaryWith a five-year survival rate of only 3% for the majority of patients, pancreatic cancer is a global healthcare challenge. Radiomics and deep learning, two novel quantitative imaging methods that treat medical images as minable data instead of just pictures, have shown promise in advancing personalized management of pancreatic cancer through diagnosing precursor diseases, early detection, accurate diagnosis, and treatment personalization. Radiomics and deep learning methods aim to collect hidden information in medical images that is missed by conventional radiology practices through expanding the data search and comparing information across different patients. Both methods have been studied and applied in pancreatic cancer. In this review, we focus on the current progress of these two methods in pancreatic cancer and provide a comprehensive narrative review on the topic. With better regulation, enhanced workflow, and larger prospective patient datasets, radiomics and deep learning methods could show real hope in the battle against pancreatic cancer through personalized precision medicine.As the most lethal major cancer, pancreatic cancer is a global healthcare challenge. Personalized medicine utilizing cutting-edge multi-omics data holds potential for major breakthroughs in tackling this critical problem. Radiomics and deep learning, two trendy quantitative imaging methods that take advantage of data science and modern medical imaging, have shown increasing promise in advancing the precision management of pancreatic cancer via diagnosing of precursor diseases, early detection, accurate diagnosis, and treatment personalization and optimization. Radiomics employs manually-crafted features, while deep learning applies computer-generated automatic features. These two methods aim to mine hidden information in medical images that is missed by conventional radiology and gain insights by systematically comparing the quantitative image information across different patients in order to characterize unique imaging phenotypes. Both methods have been studied and applied in various pancreatic cancer clinical applications. In this review, we begin with an introduction to the clinical problems and the technology. After providing technical overviews of the two methods, this review focuses on the current progress of clinical applications in precancerous lesion diagnosis, pancreatic cancer detection and diagnosis, prognosis prediction, treatment stratification, and radiogenomics. The limitations of current studies and methods are discussed, along with future directions. With better standardization and optimization of the workflow from image acquisition to analysis and with larger and especially prospective high-quality datasets, radiomics and deep learning methods could show real hope in the battle against pancreatic cancer through big data-based high-precision personalization.