Dataset Of Pairs Research Articles

Unsupervised disease tags for automatic radiology report generation

Recently, the automatic radiology report generation system that aims to accurately describe medical observations for given X-ray images has gained increasing attention. Compared to image captioning, radiology report generation poses greater challenges for data-driven neural-based methods due to the significant data biases. Faced with these challenges, conventional approaches typically utilize a supervised multi-label classification process to accurately recognize disease tags, thereby facilitating the detection of abnormal findings. However, these methods depend heavily on paired image-tag datasets that are costly to acquire. To tackle this challenge, we propose the unsupervised disease tags (UDT) model for automatic radiology report generation. In detail, our UDT model introduces three major modules: the General Disease Tags Cluster (GDTC) module for the clustering of general disease tags, which is a more generalizable way to learn and store medical knowledge; the Parallel Dual Attention (PDA) module for generating the attended visual and disease tag features in a parallel manner; and the Parallel Visual-Tag Hybrid (PVTH) module for enhancing the final feature representation by the joint utilization of attended visual and disease tag features. Experiments on two benchmark datasets show that our proposal achieves state-of-the-art (SOTA) performance on various natural language generation evaluation (NLG) metrics. Further clinical efficacy (CE) metrics also validate that our proposal can provide more accurate abnormal findings. The source code will be available at https://github.com/SKD-HPC/UDT.

Forex market directional trends forecasting with Bidirectional-LSTM and enhanced DeepSense network using all member-based optimizer

This study focuses on successful Forex trading by emphasizing the importance of identifying market trends and utilizing trend analysis for informed decision-making. The authors collected low-correlated currency pair datasets to mitigate multicollinearity risk. Authors developed a two-stage predictive model that combines regression and classification tasks, using the predicted closing price to determine entry and exit points. The model incorporates Bi-directional long short-term memory (Bi-LSTM) for improved price forecasting and higher highs and lower lows (HHs-HLs and LHs-LLs) to identify trend changes. They proposed an enhanced DeepSense network (DSN) with all member-based optimization (AMBO-DSN) to optimize decision variables of DSN. The performance of the models was compared to various machine learning, deep learning, and statistical approaches including support vector regressor (SVR), artificial neural network (ANN), auto-regressive integrated moving average (ARIMA), vanilla-LSTM (V-LSTM), and recurrent neural network (RNN). The optimized form of DSN using genetic algorithm (GA), particle swarm optimization (PSO), and differential evolution (DE) was compared with AMBO-DSN, yielding satisfactory results that demonstrated comparable quality to the observed trends on the original currency pairs. The effectiveness and reliability of the AMBO-DSN approach in forecasting trends for USD/EUR, AUD/JPY, and CHF/INR currency pairs were validated through statistical analysis while considering computational cost.

A RGB-Thermal based adaptive modality learning network for day–night wildfire identification

Wildfires have long been a danger to the atmosphere and ecological environment. With the advancement of deep learning technology and sensor equipment, wildfire identification methods based on optical (RGB) or thermal infrared (TIR) images have made significant progress. However, single-modal identification technologies often experience serious performance degradation in challenging scenarios. Therefore, a multi-modal framework that combines both RGB and TIR is necessary for accurate wildfire identification. Despite this, current RGB-T based identification methods focus solely on developing fusion strategies to learn shared representations of the two modalities, while ignoring the effect of modality-specific features. To address this issue, we propose a novel wildfire identification framework that can adaptively learn modality-specific and shared features. Specifically, our model utilizes two parallel encoders to extract multi-scale RGB and TIR features, and integrates these modality-specific features into the fusion feature layer. Each modality-specific decoder and shared decoder corresponds to three independent label supervisions that oversee feature adaptive learning. To validate the effectiveness of our approach, we propose a new RGB-T paired wildfire semantic segmentation dataset that includes flames of different scales captured under varying lighting conditions from multiple UAV-based and ground-based cameras. Finally, we evaluate our proposed method alongside benchmarking single-modal methods and state-of-the-art multi-modal methods on two tasks: a common test setting under various lighting conditions, and a generalization test across three subsets. The experimental results demonstrate the superiority of our proposed method, with the average ＋6.41% (IoU) and ＋3.39% (F1-score) improvement over the second-best RGB-T semantic segmentation method.

TMIC-84. A DEEP LEARNING-BASED METHOD FOR RAPID, PATIENT-SPECIFIC ASSAY OF MACROPHAGE INFILTRATION IN HIGH-GRADE GLIOMA USING LABEL-FREE STIMULATED RAMAN HISTOLOGY

Abstract BACKGROUND 5-Aminolevulinic acid (5-ALA), the most widely used fluorophore in image-guided glioma surgery, identifies tumor-associated macrophages (TAM) in high-grade glioma (HGG) tissue. Infiltrating macrophages, the predominant immune cells in glioma, are highly implicated in tumor progression, recurrence, and treatment response – particularly in immunotherapy. Using a unique, paired dataset of label-free stimulated Raman histology (SRH) and two-photon microscopy (TPEF) images sharing one-to-one spatial resolution, we developed a deep-learning approach to automatically quantify TAM infiltration from intraoperatively acquired SRH images. METHODS We compiled a dataset of 906 paired whole-slide SRH/TPEF images from 79 patients with HGG. We first trained a pix2pix generative adversarial network to convert raw, label-free SRH into synthetic TPEF with identifiable macrophages. Next, we trained a MaskR-CNN model to locate and segment TAMs. The pix2pix network was trained using 5,531 hand-picked 300-by-300-pixel fields-of-view (FOV) best exemplifying TAMs. We used a human-in-the-loop approach to train the segmentation network on 1,000 hand-labeled FOVs. RESULTS Macrophage segmentation from purely histologic data was near-identical to the fluorescence-based ground truth, with a mean dice score of 90.3%. Predicted TAM density was highly correlated with r=0.735. Subgroup analyses of TAM density in 346,836 non-overlapping FOVs revealed significantly lower TAM density in IDH-mutant (p< 0.01) and MGMT-hypermethylated (p=0.03) tumors. Our end-to-end algorithm uses fresh, unlabeled tissue specimens in the operating room and takes just three minutes to analyze whole-slide SRH images. CONCLUSION We demonstrate how deep neural networks can be used to rapidly and quantitatively evaluate macrophage infiltration in HGG in the operating room. Our software enables analysis of glioma patients’ tumor immune environment without immunohistochemistry or fluorescent labels, and may be leveraged to study the effects of immune-modulating therapies in the tumor microenvironment. Future work will focus on evaluating macrophage density and distribution as biomarkers for response to immunotherapies showing efficacy in glioma patients.

Image-to-Image Translation-Based Structural Damage Data Augmentation for Infrastructure Inspection Using Unmanned Aerial Vehicle

As technology advances, the use of unmanned aerial vehicles (UAVs) and image sensors for structural monitoring and diagnostics is becoming increasingly critical. This approach enables the efficient inspection and assessment of structural conditions. Furthermore, the integration of deep learning techniques has been proven to be highly effective in detecting damage from structural images, as demonstrated in our study. To enable effective learning by deep learning models, a substantial volume of data is crucial, but collecting appropriate instances of structural damage from real-world scenarios poses challenges and demands specialized knowledge, as well as significant time and resources for labeling. In this study, we propose a methodology that utilizes a generative adversarial network (GAN) for image-to-image translation, with the objective of generating synthetic structural damage data to augment the dataset. Initially, a GAN-based image generation model was trained using paired datasets. When provided with a mask image, this model generated an RGB image based on the annotations. The subsequent step generated domain-specific mask images, a critical task that improved the data augmentation process. These mask images were designed based on prior knowledge to suit the specific characteristics and requirements of the structural damage dataset. These generated masks were then used by the GAN model to produce new RGB image data incorporating various types of damage. In the experimental validation conducted across the three datasets to assess the image generation for data augmentation, our results demonstrated that the generated images closely resembled actual images while effectively conveying information about the newly introduced damage. Furthermore, the experimental validation of damage detection with augmented data entailed a comparative analysis between the performance achieved solely with the original dataset and that attained with the incorporation of additional augmented data. The results for damage detection consistently demonstrated that the utilization of augmented data enhanced performance when compared to relying solely on the original images.

Cross-Domain Unpaired Learning for Low-Dose CT Imaging.

Supervised deep-learning techniques with paired training datasets have been widely studied for low-dose computed tomography (LDCT) imaging with excellent performance. However, the paired training datasets are usually difficult to obtain in clinical routine, which restricts the wide adoption of supervised deep-learning techniques in clinical practices. To address this issue, a general idea is to construct a pseudo paired training dataset based on the widely available unpaired data, after which, supervised deep-learning techniques can be adopted for improving the LDCT imaging performance by training on the pseudo paired training dataset. However, due to the complexity of noise properties in CT imaging, the LDCT data are difficult to generate in order to construct the pseudo paired training dataset. In this paper, we propose a simple yet effective cross-domain unpaired learning framework for pseudo LDCT data generation and LDCT image reconstruction, which is denoted as CrossDuL. Specifically, a dedicated pseudo LDCT sinogram generative module is constructed based on a data-dependent noise model in the sinogram domain, and then instead of in the sinogram domain, a pseudo paired dataset is constructed in the image domain to train an LDCT image restoration module. To validate the effectiveness of the proposed framework, clinical datasets are adopted. Experimental results demonstrate that the CrossDuL framework can obtain promising LDCT imaging performance in both quantitative and qualitative measurements.

ENHANCING IMAGE SUPER-RESOLUTION WITH DEEP CONVOLUTIONAL NEURAL NETWORKS

In computer vision, image super-resolution plays a pivotal role in improving the visual quality of low-resolution images, thereby enhancing various applications such as medical imaging, surveillance, and digital entertainment. The problem at hand involves the inherent limitations of conventional methods in restoring high-frequency information lost during image downscaling. This research aims to bridge this gap by leveraging DCNNs, exploiting their ability to learn complex mappings between low and high-resolution image spaces. This study addresses the challenge of image super-resolution through the application of Deep Convolutional Neural Networks (DCNNs). The research involves the design and training of a novel DCNN architecture tailored specifically for image super-resolution. We employ a large dataset of low and high-resolution image pairs to facilitate supervised learning. The network is trained to intelligently infer high-frequency details from low-resolution inputs, enabling the generation of visually compelling super-resolved images. Results from extensive experiments showcase the superior performance of the proposed DCNN-based approach compared to traditional methods. Quantitative metrics, such as peak signal-to-noise ratio (PSNR) and structural similarity index (SSI), demonstrate significant improvements in image quality. Additionally, qualitative assessments highlight the network’s ability to reconstruct fine details, edges, and textures, resulting in visually pleasing super-resolved images.

Self-Supervised Non-Uniform Low-Light Image Enhancement Combining Image Inversion and Exposure Fusion

Low-light image enhancement is a challenging task in non-uniform low-light conditions, often resulting in local overexposure, noise amplification, and color distortion. To obtain satisfactory enhancement results, most models must resort to carefully selected paired or multi-exposure data sets. In this paper, we propose a self-supervised framework for non-uniform low-light image enhancement to address these issues, only requiring low-light images on their own for training. We first design a robust Retinex model-based image exposure enhancement network (EENet) to obtain global brightness enhancement and noise removal of images by carefully designing the loss function of each decomposition map. Then, to correct overexposed areas in the enhanced image, we incorporate the inverse image of the low-light image for enhancement using EENet. Furthermore, a three-branch asymmetric exposure fusion network (TAFNet) is designed. The two enhanced images and the original image are used as the TAFNet inputs to obtain a globally well-exposed and detail-rich image. Experimental results demonstrate that our framework outperforms some state-of-the-art methods in visual and quantitative comparisons.

Identifying Fibroblast Growth Factor Receptor 3 as a Mediator of Periosteal Osteochondral Differentiation through the Construction of microRNA-Based Interaction Networks.

Human periosteum-derived progenitor cells (hPDCs) have the ability to differentiate towards both the chondrogenic and osteogenic lineages. This coordinated and complex osteochondrogenic differentiation process permits endochondral ossification and is essential in bone development and repair. We have previously shown that humanised cultures of hPDCs enhance their osteochondrogenic potentials in vitro and in vivo; however, the underlying mechanisms are largely unknown. This study aimed to identify novel regulators of hPDC osteochondrogenic differentiation through the construction of miRNA-mRNA regulatory networks derived from hPDCs cultured in human serum or foetal bovine serum as an alternative in silico strategy to serum characterisation. Sixteen differentially expressed miRNAs (DEMis) were identified in the humanised culture. In silico analysis of the DEMis with TargetScan allowed for the identification of 1503 potential miRNA target genes. Upon comparison with a paired RNAseq dataset, a 4.5% overlap was observed (122 genes). A protein-protein interaction network created with STRING interestingly identified FGFR3 as a key network node, which was further predicted using multiple pathway analyses. Functional analysis revealed that hPDCs with the activating mutation FGFR3N540K displayed increased expressions of chondrogenic gene markers when cultured under chondrogenic conditions in vitro and displayed enhanced endochondral bone formation in vivo. A further histological analysis uncovered known downstream mediators involved in FGFR3 signalling and endochondral ossification to be upregulated in hPDC FGFR3N540K-seeded implants. This combinational approach of miRNA-mRNA-protein network analysis with in vitro and in vivo characterisation has permitted the identification of FGFR3 as a novel mediator of hPDC biology. Furthermore, this miRNA-based workflow may also allow for the identification of drug targets, which may be of relevance in instances of delayed fracture repair.

Benchmarking algorithms for joint integration of unpaired and paired single-cell RNA-seq and ATAC-seq data

BackgroundSingle-cell RNA-sequencing (scRNA-seq) measures gene expression in single cells, while single-nucleus ATAC-sequencing (snATAC-seq) quantifies chromatin accessibility in single nuclei. These two data types provide complementary information for deciphering cell types and states. However, when analyzed individually, they sometimes produce conflicting results regarding cell type/state assignment. The power is compromised since the two modalities reflect the same underlying biology. Recently, it has become possible to measure both gene expression and chromatin accessibility from the same nucleus. Such paired data enable the direct modeling of the relationships between the two modalities. Given the availability of the vast amount of single-modality data, it is desirable to integrate the paired and unpaired single-modality datasets to gain a comprehensive view of the cellular complexity.ResultsWe benchmark nine existing single-cell multi-omic data integration methods. Specifically, we evaluate to what extent the multiome data provide additional guidance for analyzing the existing single-modality data, and whether these methods uncover peak-gene associations from single-modality data. Our results indicate that multiome data are helpful for annotating single-modality data. However, we emphasize that the availability of an adequate number of nuclei in the multiome dataset is crucial for achieving accurate cell type annotation. Insufficient representation of nuclei may compromise the reliability of the annotations. Additionally, when generating a multiome dataset, the number of cells is more important than sequencing depth for cell type annotation.ConclusionsSeurat v4 is the best currently available platform for integrating scRNA-seq, snATAC-seq, and multiome data even in the presence of complex batch effects.

The potential of high temporal resolution automatic measurements of PM2.5 composition as an alternative to the filter-based manual method used in routine monitoring

Under the EU Air Quality Directive (AQD) 2008/50/EC member states are required to undertake routine monitoring of PM2.5 composition at background stations. The AQD states for PM2.5 speciation this should include at least: nitrate (NO3−), sulfate (SO42−), chloride (Cl−), ammonium (NH4+), sodium (Na+), potassium (K+), magnesium (Mg2+), calcium (Ca2+), elemental carbon (EC) and organic carbon (OC). Until 2017, it was the responsibility of each country to determine the methodology used to report the composition for the inorganic components of PM2.5. In August 2017 a European standard method of measurement of PM2.5 inorganic chemical components (NO3−, SO42−, Cl−, NH4+, Na+, K+, Mg2+, Ca2+) as deposited on filters (EN16913:2017) was published. From August 2019 this then became the European standard method. This filter method is labour-intensive and provides limited time resolution and is prone to losses of volatile compounds. There is therefore increasing interest in the use of alternative automated methods. For example, the UK reports hourly PM2.5 chemical composition using the Monitor for AeRosols and Gases in Ambient air (MARGA, Metrohm, NL). This study is a pre-assessment review of available data to demonstrate if or to what extent equivalence is possible using either the MARGA or other available automatic methods, including the Aerosol Chemical Speciation Monitor (ACSM, Aerodyne Research Inc. US) and the Ambient Ion Monitor (AIM, URG, US).To demonstrate equivalence three objectives were to be met. The first two objectives focused on data capture and were met by all three instruments. The third objective was to have less than a 50% expanded uncertainty compared to the reference method for each species. Analysis of this objective was carried out using existing paired datasets available from different regions around the world. It was found that the MARGA (2006–2019 model) had the potential to demonstrate equivalence for all species in the standard, though it was only through a combination of case studies that it passed uncertainty criteria. The ACSM has the potential to demonstrate equivalence for NH4+, SO42−, and in some conditions NO3−, but did not for Cl− due to its inability to quantify refractory aerosol such as sea salt. The AIM has the potential for NH4+, NO3−, SO42−, Cl− and Mg2+. Future investigations are required to determine if the AIM could be optimised to meet the expanded uncertainty criterion for Na+, K+ and Ca2+.The recommendation is that a second stage to demonstrate equivalence is required which would include both laboratory and field studies of the three candidate methods and any other technologies identified with the potential to report the required species.

Correction of Arterial-Phase Motion Artifacts in Gadoxetic Acid-Enhanced Liver MRI Using an Innovative Unsupervised Network.

This study aims to propose and evaluate DR-CycleGAN, a disentangled unsupervised network by introducing a novel content-consistency loss, for removing arterial-phase motion artifacts in gadoxetic acid-enhanced liver MRI examinations. From June 2020 to July 2021, gadoxetic acid-enhanced liver MRI data were retrospectively collected in this center to establish training and testing datasets. Motion artifacts were semi-quantitatively assessed using a five-point Likert scale (1 = no artifact, 2 = mild, 3 = moderate, 4 = severe, and 5 = non-diagnostic) and quantitatively evaluated using the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR). The datasets comprised a training dataset (308 examinations, including 58 examinations with artifact grade = 1 and 250 examinations with artifact grade ≥ 2), a paired test dataset (320 examinations, including 160 examinations with artifact grade = 1 and paired 160 examinations with simulated motion artifacts of grade ≥ 2), and an unpaired test dataset (474 examinations with artifact grade ranging from 1 to 5). The performance of DR-CycleGAN was evaluated and compared with a state-of-the-art network, Cycle-MedGAN V2.0. As a result, in the paired test dataset, DR-CycleGAN demonstrated significantly higher SSIM and PSNR values and lower motion artifact grades compared to Cycle-MedGAN V2.0 (0.89 ± 0.07 vs. 0.84 ± 0.09, 32.88 ± 2.11 vs. 30.81 ± 2.64, and 2.7 ± 0.7 vs. 3.0 ± 0.9, respectively; p < 0.001 each). In the unpaired test dataset, DR-CycleGAN also exhibited a superior motion artifact correction performance, resulting in a significant decrease in motion artifact grades from 2.9 ± 1.3 to 2.0 ± 0.6 compared to Cycle-MedGAN V2.0 (to 2.4 ± 0.9, p < 0.001). In conclusion, DR-CycleGAN effectively reduces motion artifacts in the arterial phase images of gadoxetic acid-enhanced liver MRI examinations, offering the potential to enhance image quality.

Improving toxicity prediction of metal-contaminated sediments by incorporating sediment properties

For the purpose of sediment quality assessment, the prediction of toxicity risk-levels for aquatic organisms based on simple environmental measurements is desirable. One commonly used approach is the comparison of total contaminant concentrations with corresponding water and sediment quality guideline values, serving as a Line of Evidence (LoE) based on chemistry-toxicity effects relationships. However, the accuracy of toxicity predictions can be improved by considering the factors that modify contaminant bioavailability. In this study we used paired chemistry-ecotoxicity data sets for sediments to evaluate the improvement in toxicity risk predictions using bioavailability-modified guidelines. The sediments were predominantly contaminated with metals, and measurements of sediment particle size, total organic carbon (TOC) and acid volatile sulfide (AVS) were used to modify hazard quotients (HQ). To further assess the predictive efficacy of the bioavailability-modified guideline models, sediments with differing contamination levels were tested for toxicity to a benthic amphipod's reproduction. To account for differences between laboratory exposure and field exposure scenarios, where the latter creates greater dilution, both static-renewal and flow-through test procedures were employed, and flow-through resulted in lower dissolved metal concentrations in the overlying waters. We also investigated how lower AVS concentration by oxidation modified the toxicity. This study reaffirmed that consideration of factors that influence contaminant bioavailability improves toxicity risk predictions, however the improvements may be modest. The sediment particle size data had the greatest influence on the modified HQ, indicating that higher percentage of fine particle size (<63 μm) contributed most to a lower predicted toxicity. The comparison of the static-renewal and flow-through test results continue to raise important questions about the relevance of static or static-renewal toxicity test results for risk assessment decisions, as both these test designs may cause unrealistically high contributions of dissolved metals in overlying waters to toxicity. Overall, this study underscores the value of incorporating outcomes from simple and routine sediment analysis (e.g., particle size, TOC, and consideration of AVS) to enhance the predictive efficacy of toxicity risk assessments in the context of sediment quality risk assessment.

Convolutional Neural Network‐Based Adaptive Localization for an Ensemble Kalman Filter

AbstractFlow‐dependent background error covariances estimated from short‐term ensemble forecasts suffer from sampling errors due to limited ensemble sizes. Covariance localization is often used to mitigate the sampling errors, especially for high dimensional geophysical applications. Most applied localization methods, empirical or adaptive ones, multiply the Kalman gain or background error covariances by a distance‐dependent parameter, which is a simple linear filtering model. Here two localization methods based on convolutional neural networks (CNNs) learning from paired data sets are proposed. The CNN‐based localization function (CLF) aims to minimize the sampling error of the estimated Kalman gain, and the CNN‐based empirical localization function (CELF) aims to minimize the posterior error of state variables. These two CNN‐based localization methods can provide localization functions that are nonlinear, spatially and temporally adaptive, and non‐symmetric with respect to displacement, without requiring any prior assumptions for the localization functions. Results using the Lorenz05 model show that CLF and CELF can better capture the structures of the Kalman gain than the best Gaspari and Cohn (GC) localization function and the adaptive reference localization method. For both perfect‐ and imperfect‐model experiments, CLF produces smaller errors of the Kalman gain, prior and posterior than the best GC and reference localization, especially for spatially averaged observations. Without model error, CELF has smaller prior and posterior errors than the best GC and reference localization for spatially averaged observations, while with model error, CELF has smaller prior and posterior errors than the best GC and reference localization for single‐point observations.

Blind deep-learning based preprocessing method for Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is a technique for tackling the trade-off between the resolution and the imaging field of view by combining the techniques from aperture synthesis and phase retrieval to estimate the complex object from a series of low-resolution intensity images captured under angle-varied illumination. The captured images are commonly corrupted by multiple noise, leading to the degradation of the reconstructed image quality. Typically speaking, the noise model and noise level of the experimental images are unknown, and the traditional image denoising methods have limited effect. In this paper we model the FPM forward imaging process corrupted by noise and divide the noise in the captured images into two parts: the signal-dependent part and the signal-independent part. Based on the noise model we propose a novel blind deep-learning based Fourier ptychographic microscopy preprocessing method, termed BDFP, for removing these two components of noise. First, from a portion of the captured low-resolution images, a set of blocks corresponding to the smooth area of the object are extracted to model signal-independent noise. Second, under the assumption that the signal-dependent noise follows a Poisson distribution, we add Poisson noise and signal-independent noise blocks to clean images to form a paired training dataset, which is then used for training a deep convolutional neural network (CNN) model to reduce both signal-dependent noise and signal-independent noise. The proposed blind preprocessing method, combining with typical FPM reconstruction algorithms, is tested on simulated data and experimental images. Experimental results show that our preprocessing method can significantly reduce the noise in the captured images and bring about effective improvements in reconstructed image quality.

Deep learning enhanced achromatic imaging with a singlet flat lens.

Correction of chromatic aberration is an important issue in color imaging and display. However, realizing broadband achromatic imaging by a singlet lens with high comprehensive performance still remains challenging, though many achromatic flat lenses have been reported recently. Here, we propose a deep-learning-enhanced singlet planar imaging system, implemented by a 3 mm-diameter achromatic flat lens, to achieve relatively high-quality achromatic imaging in the visible. By utilizing a multi-scale convolutional neural network (CNN) imposed to an achromatic multi-level diffractive lens (AMDL), the white light imaging qualities are significantly improved in both indoor and outdoor scenarios. Our experiments are fulfilled via a large paired imaging dataset with respect to a 3 mm-diameter AMDL, which guaranteed with achromatism in a broad wavelength range (400-1100 nm) but a relative low efficiency (∼45%). After our CNN enhancement, the imaging qualities are improved by ∼2 dB, showing competitive achromatic and high-quality imaging with a singlet lens for practical applications.

EasyFuse: Easy-to-learn visible and infrared image fusion framework based on unpaired set

The major hurdle in building a visible and infrared fusion model is the necessity of large-scale pixel-aligned and time-synchronized image pairs. In this paper, we propose an easy-to-learn visible and infrared image fusion framework that does not require pairs for training. In addition to easy training using unpaired sets, our framework provides fusion images with more textures and meaningful scene information compared to previous works. In particular, to mix features from each spectrum, we newly present a feature line-up module to identify important information in each source. Additionally, in order to provide a new option for benchmark evaluation, we construct a new sequence-based visible and infrared paired dataset that is aligned and synchronized. Finally, we perform extensive experiments to verify the performance of the proposed method by both public and proposed datasets.

Imbalanced Anomaly Detection Using Augmented Deeper FCDDs for Wooden Sleeper Deterioration Prognostics

Maintaining high standards for user safety during daily railway operations is crucial for railway managers. To aid in this endeavor, top or side-view cameras and GPS positioning systems have facilitated progress toward automating periodic inspections of defective features and assessing the deteriorating status of railway components. However, collecting data on deteriorated status can be time-consuming and requires repeated data acquisition because of the extreme temporal occurrence imbalance. In supervised learning, thousands of paired data sets containing defective raw images and annotated labels are required. Concretely, the one-class classification approach offers the advantage of requiring quite a few anomalous images to optimize parameters for training large normal images. The deeper fully-convolutional data descriptions (FCDDs) were applicable to several damage data sets of concrete/steel components in structures, and fallen tree, and wooden building collapse in disasters. However, it is not yet known to feasible to railway components. In this study, we devised a prognostic discriminator pipeline to automate one class classification using the augmented deeper FCDDs for defective railway components. We also performed sensitivity analysis of the mixture and erasing augmentations, and the deeper backbone rather than the shallow baseline of convolutional neural network (CNN) with 27 layers. Furthermore, we visualized defective railway features by using transposed Gaussian upsampling. We demonstrated our application to railway inspection using a video acquisition dataset that contains wooden sleeper deterioration. Finally, we examined the usability of our approach for prognostic monitoring and fu ture work on railway component inspection.

Low-Dose CT Image Synthesis for Domain Adaptation Imaging Using a Generative Adversarial Network With Noise Encoding Transfer Learning.

Deep learning (DL) based image processing methods have been successfully applied to low-dose x-ray images based on the assumption that the feature distribution of the training data is consistent with that of the test data. However, low-dose computed tomography (LDCT) images from different commercial scanners may contain different amounts and types of image noise, violating this assumption. Moreover, in the application of DL based image processing methods to LDCT, the feature distributions of LDCT images from simulation and clinical CT examination can be quite different. Therefore, the network models trained with simulated image data or LDCT images from one specific scanner may not work well for another CT scanner and image processing task. To solve such domain adaptation problem, in this study, a novel generative adversarial network (GAN) with noise encoding transfer learning (NETL), or GAN-NETL, is proposed to generate a paired dataset with a different noise style. Specifically, we proposed a method to perform noise encoding operator and incorporate it into the generator to extract a noise style. Meanwhile, with a transfer learning (TL) approach, the image noise encoding operator transformed the noise type of the source domain to that of the target domain for realistic noise generation. One public and two private datasets are used to evaluate the proposed method. Experiment results demonstrated the feasibility and effectiveness of our proposed GAN-NETL model in LDCT image synthesis. In addition, we conduct additional image denoising study using the synthesized clinical LDCT data, which verified the merit of the proposed synthesis in improving the performance of the DL based LDCT processing method.

ScTIGER: A Deep-Learning Method for Inferring Gene Regulatory Networks from Case versus Control scRNA-seq Datasets.

Inferring gene regulatory networks (GRNs) from single-cell RNA-seq (scRNA-seq) data is an important computational question to find regulatory mechanisms involved in fundamental cellular processes. Although many computational methods have been designed to predict GRNs from scRNA-seq data, they usually have high false positive rates and none infer GRNs by directly using the paired datasets of case-versus-control experiments. Here we present a novel deep-learning-based method, named scTIGER, for GRN detection by using the co-differential relationships of gene expression profiles in paired scRNA-seq datasets. scTIGER employs cell-type-based pseudotiming, an attention-based convolutional neural network method and permutation-based significance testing for inferring GRNs among gene modules. As state-of-the-art applications, we first applied scTIGER to scRNA-seq datasets of prostate cancer cells, and successfully identified the dynamic regulatory networks of AR, ERG, PTEN and ATF3 for same-cell type between prostatic cancerous and normal conditions, and two-cell types within the prostatic cancerous environment. We then applied scTIGER to scRNA-seq data from neurons with and without fear memory and detected specific regulatory networks for BDNF, CREB1 and MAPK4. Additionally, scTIGER demonstrates robustness against high levels of dropout noise in scRNA-seq data.