Can Signal-to-Noise Ratio Perform as a Baseline Indicator for Medical Image Quality Assessment

Abstract

Natural image quality assessment (NIQA) wins increasing attention, while NIQA models are rarely used in the medical community. A couple of studies employ the NIQA methodologies for medical image quality assessment (MIQA), but building the benchmark data sets necessitates considerable time and professional skills. In particular, the characteristics of synthesized distortions are different from those of clinical distortions, which make the results not so convincing. In clinic, signal-to-noise ratio (SNR) is widely used, which is defined as the quotient of the mean signal intensity measured in a tissue region of interest (ROI) and the standard deviation of the signal intensity in an air region outside the imaged object, and both regions are outlined by specialists. We take advantage of the knowledge that SNR is routinely used and concern whether SNR measure can perform as a baseline metric for the development of MIQA algorithms. To address the issue, the inter-observer reliability of SNR measure is investigated regarding to different tissue ROIs [white matter (WM); cerebral spinal fluid (CSF)] in magnetic resonance (MR) images. A total of 192 T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> , 88 T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , 76 T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> and 55 contrast-enhanced T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> (T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> C) weighted images are analyzed. Statistical analysis indicates that SNR values show consistency between different observers to the same ROI in each modality (Wilcoxon rank sum test, p <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">w</sub> ≥ 0.11; and paired sample t-test, pp 0.28). Moreover, whether off-the-shelf NIQA models can predict MR image quality is considered by using SNR values as the reference scores. Four NIQA models (BIQI, BLIINDS-II, BRISQUE, and NIQE) are evaluated, and the correlation between SNR values and NIQA results is evaluated. Pearson correlation coefficient (r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ) shows that WM-based SNR values correlates well with BIQI, BLIINDS-II and BRISQUE in T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> images (rp 0.77), BRISQUE and NIQE in T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> images (r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ≥ 0.75), BLIINDS-II in T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> images (rp 0.67), and BRISQUE and NIQE in T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> C images (r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ≥ 0.58), while CSF-based SNR values correlates well with BLIINDS-II in T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> images (r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ≥ 0.64) and T <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> images (r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ≥ 0.60), and all p <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> <; 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-4</sup> . The prediction performance analysis further proves the result from the correlation analysis. Conclusively, SNR measure is reliable to different observations and can perform as a baseline indicator for the development of MIQA algorithms. In general, BRISQUE and BLIINDS-II are full of potential to be conditionally used as objective MIQA models toward human brain MR images. This paper presents the first attempt of using SNR measure to bridge the gap between NIQA and MIQA, and large-scale experiments should be further conducted to confirm the conclusion in this paper.