Structural Similarity Feature Research Articles

MSTG: Multi-Scale Transformer with Gradient for joint spatio-temporal enhancement

Despite the widespread availability of HDR (High Dynamic Range) display devices, the majority of video sources are still stored in SDR (Standard Dynamic Range) format, leading to an urgent need for UHD (Ultra High Definition) video reconstruction. A simple solution is to divide the task into two sub-tasks: video super-resolution (SR) and video Bit-Depth Enhancement (BDE). While some joint enhancement tasks led by SR have been explored in multiple dimensions, the quantitative dimension has often been overlooked. In this paper, we address the first joint task of BDE and SR and propose a succinct network called MSTG. We conduct a systematic analysis to explain how this effect can be achieved and overcome the core challenge arising from distortions exacerbated by the differing optimization directions of BDE and SR. This challenge is attributed to indistinguishable contours and detailed textures. To tackle this, we employ a dual-branch module that leverages both gradient and image information to discern between BDE and SR. Furthermore, we propose a novel model named Cross-scale Transformer (CTF) embedded with Cross-scale Attention (CSA). This model facilitates information aggregation while adaptively utilizing structural similarity features. It jointly extracts, aligns, and fuses frame features at multiple scales, forming the MST module. Experiments demonstrate that the proposed algorithm outperforms the cascaded video enhancement methods by large margins on both synthetic and realistic datasets.

Robust single-photon 3D imaging based on full-scale feature integration and intensity edge guidance

Active single-photon light detection and ranging (LiDAR) imaging technology adopting emerging single-photon avalanche diode (SPAD) has been used extensively for 3D imaging of complex scenes. However, the different signal-to-noise data acquired by SPAD under complex scenes and the larger pixel size of the SPAD pose a great challenge for robustly estimating the scene depth with rich detailed information. In this paper, we have designed a two-stage network including a multi-scale encoder-decoder sub-network and an intensity-guided edge refinement sub-network using a sensor fusion strategy to improve the robustness of the depth estimation network. In the first stage of the sub-network, we use the UNet3+ network to integrate fine-grained detail information and coarse-grained semantic information from full-scale through skip connections across scales on the decoder. In the meantime, the deep boosting network is added to the encoder to extract rich contextual information from the spatial-temporal domain to alleviate semantic gaps between the encoder and decoder. Moreover, in the second stage of the sub-network, we adopt an inverted y-shaped backbone network with the same structure between the encoder and decoder coupled with the same weight at both encoders. The purpose of this design is to extract structural similarity features with spatial consistency from intensity and depth to compensate for the missing detail of reconstructed depth by the first stage sub-network. The experimental results conducted on both simulated and real-world datasets demonstrate that our network outperforms previous networks. In addition, networks are trained on single or multiple signal-to-background ratio datasets to estimate the real-world data respectively, and the results show that our proposed network is more robust.

DeepFusion: A deep learning based multi-scale feature fusion method for predicting drug-target interactions.

Predicting drug-target interactions (DTIs) is essential for both drug discovery and drug repositioning. Recently, deep learning methods have achieved relatively significant performance in predicting DTIs. Generally, it needs a large amount of approved data of DTIs to train the model, which is actually tedious to obtain. In this work, we propose DeepFusion, a deep learning based multi-scale feature fusion method for predicting DTIs. To be specific, we generate global structural similarity feature based on similarity theory, convolutional neural network and generate local chemical sub-structure semantic feature using transformer network respectively for both drug and protein. Data experiments are conducted on four sub-datasets of BIOSNAP, which are 100%, 70%, 50% and 30% of BIOSNAP dataset. Particularly, using 70% sub-dataset, DeepFusion achieves ROC-AUC and PR-AUC by 0.877 and 0.888, which is close to the performance of some baseline methods trained by the whole dataset. In case study, DeepFusion achieves promising prediction results on predicting potential DTIs in case study.

Unsupervised Learning-Based CBCT-CT Deformable Image Registration for CBCT-Guided Abdominal Radiotherapy

Daily Cone beam CT (CBCT) imaging provides necessary anatomical information for accurate patient setup. Image quality of CBCT is usually far inferior to simulation CT scans. A workaround is to register the CT to the CBCT such that the contours and Hounsfield Unit (HU) values of the CT can be propagated to the CBCT. However, the inconsistent HU values across CT and CBCT make it less effective to use conventional image similarity measures. We aim to develop an unsupervised registration network to overcome this challenge in multimodal CT-CBCT image registration.We propose to integrate directional local structural similarity into an unsupervised learning framework to perform abdominal CT-CBCT image registration. Directional local structural similarity measures the image's self-similarity which reflects the underlying structural similarity regardless of the modality in use. The CBCT and CT images were separately processed to extract directional local structural similarity feature maps in different directions. We concatenated the directional local structural similarity feature maps and the original images as network input. Taking both the original images and their respective structural similarity feature maps as input allows the network to fully explore the potential correlations between CBCT and CT for accurate deformation vector field (DVF) prediction. Salient features learnt through previous iterations were highlighted by attention gates across layers to expedite the learning process. A 3D bicubic interpolation was used to up-sample and smooth the predicted DVF. We performed a leave-one-out cross validation with an image dataset of 45 patients to evaluate the proposed registration method. Normalized cross correlation (NCC) and target registration error (TRE) between CBCT and deformed CT were calculated to quantify the registration accuracy.Our results show that the alignment between the abdominal soft tissues has been greatly improved after registration for all patients. The mean and standard deviation of NCC and TRE were 0.97 (range 0.95-0.99) and 1.88 (range 1.03-2.67) mm. The proposed network allows for many datasets to be used as training datasets since ground truth DVF is not needed for the training process. The proposed network can predict a final DVF via a single forward prediction, which is faster than the conventional iterative registration algorithms.We have developed a novel unsupervised multimodal image registration method for CT-CBCT abdominal image registration, which does not need ground truth DVF for training, and demonstrated its feasibility. Taking both CBCT and CT images and their respective directional local structural similarity features as input, the proposed network performs direct DVF prediction to register the abdominal CT to CBCT images. This tool will be useful for future CBCT-guided radiotherapy of abdominal malignancies.

Structure Based Drug Design Approach to Identify Potential SARS-CoV-2 Polymerase Inhibitors

Aims: The research work aims to apply the current virtual screening approaches for rapid screening of available compounds as inhibitors of the novel coronavirus (COVID-19). Background: The worldwide pandemic, uncontrolled spread, and lack of effective therapeutics demand novel SARS-CoV-2 inhibitory anti-viral agents. Objective: The major objectives of the present work are – i) effective utilization of open-source computer- aided drug design (CADD) tools; ii) to prepare a database according to chemical structure similarity to the reported anti-viral drug, Favipiravir; and iii) to investigate potential inhibitors of the novel coronavirus. Methods: The dataset was prepared based on the chemical structure similarity feature of ChemSpider. The virtual screening was carried out using molecular docking and ADMET properties. For performing molecular docking studies, the standard docking protocol of iGEMDOCK was used. Result: Based on chemical structure similarity search to Favipiravir, a small library of 40 compounds was designed. The docking score and ADMET properties were analyzed to prioritize the compounds. Conclusion: The virtual screening resulted in the identification of potential anti-viral compounds. Among the designed library of compounds based on structural similarity to Favipiravir, 70% of compounds were found to possess docking scores more than that of Favipiravir. The amino acid residues involved in binding at the RNA dependent RNA polymerase (RdRp) were identified. The compounds have shown acceptable ADME properties and are potentially non-toxic. Other: The study has successfully applied the open source CADD tools to investigate novel SARS-CoV-2 polymerase inhibitors.

Comparison of two-dimensional synthesized mammograms versus original digital mammograms: a quantitative assessment.

This study objectively evaluates the similarity between standard full-field digital mammograms and two-dimensional synthesized digital mammograms (2DSM) in a cohort of women undergoing mammography. Under an institutional review board-approved data collection protocol, we retrospectively analyzed 407 women with digital breast tomosynthesis (DBT) and full-field digital mammography (FFDM) examinations performed from September 1, 2014, through February 29, 2016. Both FFDM and 2DSM images were used for the analysis, and 3216 available craniocaudal (CC) and mediolateral oblique (MLO) view mammograms altogether were included in the dataset. We analyzed the mammograms using a fully automated algorithm that computes 152 structural similarity, texture, and mammographic density-based features. We trained and developed two different global mammographic image feature analysis-based breast cancer detection schemes for 2DSM and FFDM images, respectively. The highest structural similarity features were obtained on the coarse Weber Local Descriptor differential excitation texture feature component computed on the CC view images (0.8770) and MLO view images (0.8889). Although the coarse structures are similar, the global mammographic image feature-based cancer detection scheme trained on 2DSM images outperformed the corresponding scheme trained on FFDM images, with area under a receiver operating characteristic curve (AUC) = 0.878 ± 0.034 and 0.756 ± 0.052, respectively. Consequently, further investigation is required to examine whether DBT can replace FFDM as a standalone technique, especially for the development of automated objective-based methods.

Quality Prediction on Deep Generative Images.

In recent years, deep neural networks have been utilized in a wide variety of applications including image generation. In particular, generative adversarial networks (GANs) are able to produce highly realistic pictures as part of tasks such as image compression. As with standard compression, it is desirable to be able to automatically assess the perceptual quality of generative images to monitor and control the encode process. However, existing image quality algorithms are ineffective on GAN generated content, especially on textured regions and at high compressions. Here we propose a new "naturalness"-based image quality predictor for generative images. Our new GAN picture quality predictor is built using a multi-stage parallel boosting system based on structural similarity features and measurements of statistical similarity. To enable model development and testing, we also constructed a subjective GAN image quality database containing (distorted) GAN images and collected human opinions of them. Our experimental results indicate that our proposed GAN IQA model delivers superior quality predictions on the generative image datasets, as well as on traditional image quality datasets.

Low rank theory-based interframe forgery detection for blurry video

It is difficult to extract appropriate features of blurry digital video because of the low quality image for which traditional forgery detection methods are not effective. The video is often blurred due to various complicated conditions, such as limited bandwidth or low pixel complementary metal oxide semiconductor sensors. To address the feature extraction problem for blurry video, we explore one low rank theory to deblur video, which fuses multiple fuzzy kernels of key frame images by low rank decomposition. Further, we extract mean structural similarity features and information fidelity criterion features sequentially by the double detection mechanism to detect forgery points on blurred video frames. We can determine the forgery form of the video and locate the source replication area of the copy-move forgery. Experimental evaluation on two public video databases and videos that we recorded show that this method is robust in blurry video forgery detection, and compared with traditional video forgery detection methods, the efficiency is improved.

Developing a new case based computer-aided detection scheme and an adaptive cueing method to improve performance in detecting mammographic lesions

The purpose of this study is to evaluate a new method to improve performance of computer-aided detection (CAD) schemes of screening mammograms with two approaches. In the first approach, we developed a new case based CAD scheme using a set of optimally selected global mammographic density, texture, spiculation, and structural similarity features computed from all four full-field digital mammography images of the craniocaudal (CC) and mediolateral oblique (MLO) views by using a modified fast and accurate sequential floating forward selection feature selection algorithm. Selected features were then applied to a ‘scoring fusion’ artificial neural network classification scheme to produce a final case based risk score. In the second approach, we combined the case based risk score with the conventional lesion based scores of a conventional lesion based CAD scheme using a new adaptive cueing method that is integrated with the case based risk scores. We evaluated our methods using a ten-fold cross-validation scheme on 924 cases (476 cancer and 448 recalled or negative), whereby each case had all four images from the CC and MLO views. The area under the receiver operating characteristic curve was AUC = 0.793 ± 0.015 and the odds ratio monotonically increased from 1 to 37.21 as CAD-generated case based detection scores increased. Using the new adaptive cueing method, the region based and case based sensitivities of the conventional CAD scheme at a false positive rate of 0.71 per image increased by 2.4% and 0.8%, respectively. The study demonstrated that supplementary information can be derived by computing global mammographic density image features to improve CAD-cueing performance on the suspicious mammographic lesions.

Monogenic signal theory based feature similarity index for image quality assessment

Image quality assessment (IQA) aims to establish generic metrics consistently with subjective evaluations using computational models. Recent phase congruency, which is a dimensionless, normalized feature of a local structure, is used as the structure similarity feature. This paper proposes a novel feature similarity index that is Riesz transform based monogenic phase congruency feature similarity index (RMFSIM). RMFSIM is based on monogenic signal theory for full reference IQA. Monogenic phase congruency (MPC) map, instead of phase congruency map, is constructed utilizing the local phase, the local orientation and the local energy information of the 2D monogenic signal. The corresponding 1st-order and 2nd-order coefficients of the MPC map are obtained by Riesz transform. The local feature coefficients similarities are computed by the similarity measure, and then a final single quality score is derived from weighting feature coefficients similarities. Experimental results demonstrate that the proposed similarity index is highly consistent with human subjective evaluations and achieves good performance with the existing state-of-the-art methods in terms of prediction monotonicity and accuracy.