Existing Diffusion Models Research Articles

NeuroDM: Decoding and visualizing human brain activity with EEG-guided diffusion model

Background and Objective:Brain–Computer Interface (BCI) technology has recently been advancing rapidly, bringing significant hope for improving human health and quality of life. Decoding and visualizing visually evoked electroencephalography (EEG) signals into corresponding images plays a crucial role in the practical application of BCI technology. The recent emergence of diffusion models provides a good modeling basis for this work. However, the existing diffusion models still have great challenges in generating high-quality images from EEG, due to the low signal-to-noise ratio and strong randomness of EEG signals. The purpose of this study is to address the above-mentioned challenges by proposing a framework named NeuroDM that can decode human brain responses to visual stimuli from EEG-recorded brain activity. Methods:In NeuroDM, an EEG-Visual-Transformer (EV-Transformer) is used to extract the visual-related features with high classification accuracy from EEG signals, then an EEG-Guided Diffusion Model (EG-DM) is employed to synthesize high-quality images from the EEG visual-related features. Results:We conducted experiments on two EEG datasets (one is a forty-class dataset, and the other is a four-class dataset). In the task of EEG decoding, we achieved average accuracies of 99.80% and 92.07% on two datasets, respectively. In the task of EEG visualization, the Inception Score of the images generated by NeuroDM reached 15.04 and 8.67, respectively. All the above results outperform existing methods. Conclusions:The experimental results on two EEG datasets demonstrate the effectiveness of the NeuroDM framework, achieving state-of-the-art performance in terms of classification accuracy and image quality. Furthermore, our NeuroDM exhibits strong generalization capabilities and the ability to generate diverse images.

High-Frequency Space Diffusion Model for Accelerated MRI.

Diffusion models with continuous stochastic differential equations (SDEs) have shown superior performances in image generation. It can serve as a deep generative prior to solving the inverse problem in magnetic resonance (MR) reconstruction. However, low-frequency regions of k -space data are typically fully sampled in fast MR imaging, while existing diffusion models are performed throughout the entire image or k -space, inevitably introducing uncertainty in the reconstruction of low-frequency regions. Additionally, existing diffusion models often demand substantial iterations to converge, resulting in time-consuming reconstructions. To address these challenges, we propose a novel SDE tailored specifically for MR reconstruction with the diffusion process in high-frequency space (referred to as HFS-SDE). This approach ensures determinism in the fully sampled low-frequency regions and accelerates the sampling procedure of reverse diffusion. Experiments conducted on the publicly available fastMRI dataset demonstrate that the proposed HFS-SDE method outperforms traditional parallel imaging methods, supervised deep learning, and existing diffusion models in terms of reconstruction accuracy and stability. The fast convergence properties are also confirmed through theoretical and experimental validation. Our code and weights are available at https://github.com/Aboriginer/HFS-SDE.

Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference

Due to the recent success of diffusion models, text-to-image generation is becoming increasingly popular and achieves a wide range of applications. Among them, text-to-image editing, or continuous text-to-image generation, attracts lots of attention and can potentially improve the quality of generated images. It's common to see that users may want to slightly edit the generated image by making minor modifications to their input textual descriptions for several rounds of diffusion inference. However, such an image editing process suffers from the low inference efficiency of many existing diffusion models even using GPU accelerators. To solve this problem, we introduce Fast Image Semantically Edit (FISEdit), a cached-enabled sparse diffusion model inference engine for efficient text-to-image editing. The key intuition behind our approach is to utilize the semantic mapping between the minor modifications on the input text and the affected regions on the output image. For each text editing step, FISEdit can 1) automatically identify the affected image regions and 2) utilize the cached unchanged regions' feature map to accelerate the inference process. For the former, we measure the differences between cached and ad hoc feature maps given the modified textual description, extract the region with significant differences, and capture the affected region by masks. For the latter, we develop an efficient sparse diffusion inference engine that only computes the feature maps for the affected region while reusing the cached statistics for the rest of the image. Finally, extensive empirical results show that FISEdit can be 3.4 times and 4.4 times faster than existing methods on NVIDIA TITAN RTX and A100 GPUs respectively, and even generates more satisfactory images.

Towards Efficient Diffusion-Based Image Editing with Instant Attention Masks

Diffusion-based Image Editing (DIE) is an emerging research hot-spot, which often applies a semantic mask to control the target area for diffusion-based editing. However, most existing solutions obtain these masks via manual operations or off-line processing, greatly reducing their efficiency. In this paper, we propose a novel and efficient image editing method for Text-to-Image (T2I) diffusion models, termed Instant Diffusion Editing (InstDiffEdit). In particular, InstDiffEdit aims to employ the cross-modal attention ability of existing diffusion models to achieve instant mask guidance during the diffusion steps. To reduce the noise of attention maps and realize the full automatics, we equip InstDiffEdit with a training-free refinement scheme to adaptively aggregate the attention distributions for the automatic yet accurate mask generation. Meanwhile, to supplement the existing evaluations of DIE, we propose a new benchmark called Editing-Mask to examine the mask accuracy and local editing ability of existing methods. To validate InstDiffEdit, we also conduct extensive experiments on ImageNet and Imagen, and compare it with a bunch of the SOTA methods. The experimental results show that InstDiffEdit not only outperforms the SOTA methods in both image quality and editing results, but also has a much faster inference speed, i.e., +5 to +6 times. Our code available at https://anonymous.4open.science/r/InstDiffEdit-C306

Zero-Shot Medical Image Translation via Frequency-Guided Diffusion Models.

Recently, the diffusion model has emerged as a superior generative model that can produce high quality and realistic images. However, for medical image translation, the existing diffusion models are deficient in accurately retaining structural information since the structure details of source domain images are lost during the forward diffusion process and cannot be fully recovered through learned reverse diffusion, while the integrity of anatomical structures is extremely important in medical images. For instance, errors in image translation may distort, shift, or even remove structures and tumors, leading to incorrect diagnosis and inadequate treatments. Training and conditioning diffusion models using paired source and target images with matching anatomy can help. However, such paired data are very difficult and costly to obtain, and may also reduce the robustness of the developed model to out-of-distribution testing data. We propose a frequency-guided diffusion model (FGDM) that employs frequency-domain filters to guide the diffusion model for structure-preserving image translation. Based on its design, FGDM allows zero-shot learning, as it can be trained solely on the data from the target domain, and used directly for source-to-target domain translation without any exposure to the source-domain data during training. We evaluated it on three cone-beam CT (CBCT)-to-CT translation tasks for different anatomical sites, and a cross-institutional MR imaging translation task. FGDM outperformed the state-of-the-art methods (GAN-based, VAE-based, and diffusion-based) in metrics of Fréchet Inception Distance (FID), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index Measure (SSIM), showing its significant advantages in zero-shot medical image translation.

DIFFUSION-MODEL-ASSISTED SUPERVISED LEARNING OF GENERATIVE MODELS FOR DENSITY ESTIMATION

We present a supervised learning framework of training generative models for density estimation. Generative models, including generative adversarial networks (GANs), normalizing flows, and variational auto-encoders (VAEs), are usually considered as unsupervised learning models, because labeled data are usually unavailable for training. Despite the success of the generative models, there are several issues with the unsupervised training, e.g., requirement of reversible architectures, vanishing gradients, and training instability. To enable supervised learning in generative models, we utilize the score-based diffusion model to generate labeled data. Unlike existing diffusion models that train neural networks to learn the score function, we develop a training-free score estimation method. This approach uses mini-batch-based Monte Carlo estimators to directly approximate the score function at any spatial-temporal location in solving an ordinary differential equation (ODE), corresponding to the reverse-time stochastic differential equation (SDE). This approach can offer both high accuracy and substantial time savings in neural network training. Once the labeled data are generated, we can train a simple, fully connected neural network to learn the generative model in the supervised manner. Compared with existing normalizing flow models, our method does not require the use of reversible neural networks and avoids the computation of the Jacobian matrix. Compared with existing diffusion models, our method does not need to solve the reverse-time SDE to generate new samples. As a result, the sampling efficiency is significantly improved. We demonstrate the performance of our method by applying it to a set of 2D datasets as well as real data from the University of California Irvine (UCI) repository.

A Recycling Training Strategy for Medical Image Segmentation with Diffusion Denoising Models

Denoising diffusion models have found applications in image segmentation by generating segmented masks conditioned on images. Existing studies predominantly focus on adjusting model architecture or improving inference, such as test-time sampling strategies. In this work, we focus on improving the training strategy and propose a novel recycling method. During each training step, a segmentation mask is first predicted given an image and a random noise. This predicted mask, which replaces the conventional ground truth mask, is used for denoising task during training. This approach can be interpreted as aligning the training strategy with inference by eliminating the dependence on ground truth masks for generating noisy samples. Our proposed method significantly outperforms standard diffusion training, self-conditioning, and existing recycling strategies across multiple medical imaging data sets: muscle ultrasound, abdominal CT, prostate MR, and brain MR. This holds for two widely adopted sampling strategies: denoising diffusion probabilistic model and denoising diffusion implicit model. Importantly, existing diffusion models often display a declining or unstable performance during inference, whereas our novel recycling consistently enhances or maintains performance. We show for the first time that, under a fair comparison with the same network architectures and computing budget, the proposed recycling-based diffusion models achieved on-par performance with non-diffusion-based supervised training. Furthermore, by ensembling the proposed diffusion model and the non-diffusion counterpart, significant improvements to the non-diffusion models have been observed across all applications, demonstrating the value of this novel training method. This paper summarizes these quantitative results and discusses their values, with a fully reproducible JAX-based implementation, released at <a href='https://github.com/mathpluscode/ImgX-DiffSeg'>https://github.com/mathpluscode/ImgX-DiffSeg</a>

Nonlinear diffusion system for simultaneous restoration and binarization of degraded document images

Existing diffusion models can only do tasks for either restoration or binarization of degraded document images; in this paper, we pay close attention to the problem of simultaneous restoration and binarization. We first introduce a model of image formation for degraded document images, which decomposes a degraded document image into the product of background and foreground components. Then a weakly coupled nonlinear diffusion system is developed for simultaneous restoration and binarization of degraded document images. In this diffusion system, the first equation is used to estimate background component and the second equation is responsible for the foreground component. The restored and binarized images are finally obtained by the estimated background and foreground components, respectively. Two coupled diffusion equations are solved alternately by explicit finite differencing. The proposed model is tested on some degraded document images that represent five types of typical degradations including bleed-through, complex background, page stains, stamps and smeared ink, and is compared with eight closely related models for degraded document binarization, qualitatively and quantitatively. Experimental results illustrate that the proposed model has shown promising results in terms of restoration and binarization of degrade document images.

MDM: Meta diffusion model for hard-constrained text generation

Hard-constrained text generation (Hard-CTG) task aims to generate text with given keywords, which is helpful for summarization, data augmentation, story writing, etc. Existing Hard-CTG models face two significant challenges. Firstly, hard-CTG models based on the editing method suffer from error propagation, resulting in low generation quality. Secondly, Hard-CTG models utilizing the prompt method cannot guarantee high keyword coverage. To tackle these challenges, we propose Meta Diffusion Model (MDM), a non-autoregressive diffusion model with novel meta-learning module. Specifically, we fix the embedding of keywords in the generation process, while all non-keyword tokens evolve simultaneously and contribute to each other towards the target sentence under given keywords, addressing the above issues. Moreover, existing diffusion models for the text domain have an inconsistency in the training and inference stages. To unify the training and inference processes, we present an adaptively denoising method derived from meta-learning, and further improves generation quality. Experiments on three representative datasets demonstrate that our method achieves state-of-the-art performance evaluated on empirical metrics. Especially, compared with strong baselines, MDM significantly improves the BLEU-4, CIDEr, and ROUGE by 0.48%—11.56%, 17.33%—82.87%, and 23.15%–29.78%, respectively. In terms of keyword coverage, our MDM outperforms ChatGPT by 2.93%–7.88%.

Study on multicomponent leakage and diffusion characteristics of hydrogen-blended natural gas in utility tunnels

With the rapid development of utility tunnels and hydrogen energy, hydrogen-blended natural gas (HBNG) enters gas compartments just around the corner. However, all safety requirements for gas compartments in current standards are designed for natural gas, and the applicability to HBNG is unclear. In addition, most of the existing diffusion models assume that the CH4/H2/Air mixture is single-component homogeneous gas, which cannot accurately describe the diffusion behavior differences between CH4 and H2. In this paper, an improved diffusion coefficient model for CH4/H2/Air non-ideal multi-component mixtures is proposed, which considers the interaction between CH4 and H2 components. After that, a 200 m long and narrow three-dimensional utility tunnel model is established to study the HBNG concentration distribution, alarm response time, and explosion hazardous area under different ventilation conditions. The effects of hydrogen blending ratio (HBR), leakage diameter, pipeline operating pressure, leakage location, and ventilation frequency on leakage and diffusion characteristics of utility tunnels are analyzed. The results show that under natural ventilation, the HBNG diffuses symmetrically to both sides with the leak hole as the center and covers the whole compartment after 1500 s. Normal mechanical ventilation reduces the distribution range and HBNG concentration, and the leaked gas covers the compartment downstream within 250 s. With the increase of HBR, the CH4/H2 mixture concentration increase at each monitoring point downstream of the leakage location, which causes the advance of the alarm response time. When the leakage reaches a steady state, the CH4/H2 mixture concentration for HBR of 5%, 10%, 15%, and 20% is 2.15%, 4.14%, 7.76%, and 10.97% higher than that of natural gas. The leakage location affects the hazardous area range, but little affects the CH4/H2 mixture concentration at the final steady state. The larger leakage diameter and the higher pipeline operating pressure can accelerate the HBNG diffusion rate in utility tunnels. When the pipeline operating pressure increases to 1.6 MPa, the current minimum accident ventilation frequency of 12 times/h is insufficient to reduce the CH4/H2 mixture concentration below the lower explosion limit (LEL). Increasing the ventilation frequency can effectively reduce the leaked gas concentration in utility tunnels, so it is recommended to increase the minimum accident ventilation frequency from 12 times/h to 15 times/h. The research results can guide the design and safety management of HBNG pipelines in utility tunnels.

Revisiting Denoising Diffusion Probabilistic Models for Speech Enhancement: Condition Collapse, Efficiency and Refinement

Recent literature has shown that denoising diffusion probabilistic models (DDPMs) can be used to synthesize high-fidelity samples with a competitive (or sometimes better) quality than previous state-of-the-art approaches. However, few attempts have been made to apply DDPM for the speech enhancement task. The reported performance of the existing works is relatively poor and significantly inferior to other generative methods. In this work, we first reveal the difficulties in applying existing diffusion models to the field of speech enhancement. Then we introduce DR-DiffuSE, a simple and effective framework for speech enhancement using conditional diffusion models. We present three strategies (two in diffusion training and one in reverse sampling) to tackle the condition collapse and guarantee the sufficient use of condition information. For efficiency, we introduce the fast sampling technique to reduce the sampling process into several steps and exploit a refinement network to calibrate the defective speech. Our proposed method achieves the state-of-the-art performance to the GAN-based model and shows a significant improvement over existing DDPM-based algorithms.

Information cascade final size distributions derived from urn models

Bipolarization is a phenomenon in which either a large or very small information cascade appears randomly when the retweet rate is high. This phenomenon, which has been observed only in simulations, has the potential to significantly advance the prediction of final cascade sizes because forecasters need only focus on the two peaks in the final cascade size distribution rather than considering the effects of various details, such as network structure and user behavioral patterns. The phenomenon also suggests the difficulty of identifying factors that lead to the emergence of large-scale cascades. To verify the existence of bipolarization, this paper theoretically derives mathematical expressions of the cascade final size distribution using urn models, which simplify the diffusion behavior of actual online social networks. Under the assumption of infinite network size, the distribution exhibits power-law behavior, consistent with the results of existing diffusion models and previous Twitter analytical outcomes. Under the assumption of finite network size, bipolarization is observed.

ExoFIA: Deep Exogenous Assistance in the Prediction of the Influence of Fake News with Social Media Explainability

The growth of social platforms has lowered the barrier of entry into the media sector, allowing for the spread of false information and putting democratic politics and social security at peril. Preliminary analysis shows that posts sharing real news and fake news are disseminated on social media. Moreover, posts pointing to fake news spread faster, so this paper aims to predict the impact of posts citing fake news on social platforms. In this study, we take into account that exogenous factors, in addition to endogenous factors, can potentially determine how influential a post is. For example, the occurrence of social events can generate public resonance and discussion, thereby increasing the impact of relevant posts. Given that Google Trends can obtain search trends that reflect social popularity, this work aims to use Google Trends as the source of our exogenous factors. We propose a deep learning model called the deep exogenous aid in fake news (ExoFIA) model, which combines multi-modal features and utilizes an attention mechanism to provide model interpretability and analyze the influencing factors. Applying the model to real-world data from Twitter demonstrates that our model outperforms existing diffusion models. Furthermore, further examination of the relevant aspects of true and fake news reveals that the two are influenced by distinct variables.

Fuzzy sign-aware diffusion models for influence maximization in signed social networks

The diffusion models in the influence maximization problem are a trending topic in many companies' viral marketing to raise their business promotion. The majority of existing diffusion models have focused only on trust relationships, and a few models have considered distrust relationships. Nevertheless, the latter models lack appropriate theoretical support to cover social influences resulting from different user-relationship types and still do not provide proper predictions. In this study, a fuzzy-based approach is first introduced to model the influence propagation for different user-relationship types. Then, four novel fuzzy sign-aware diffusion models named FSC-SB, FSC-N, FST-SB, and FST-N are proposed by the introduced fuzzy-based approach in two categories: cascade and threshold-based models. In the proposed models, the user-relationship type is determined by a fuzzy expert system in which a natural multi-trust level relationship is applied instead of a commonly used crisp relationship. Moreover, new rules and equations are defined to determine a user's state by information received from its active neighbors. The performance of proposed models was compared with some state-of-the-art models conducted by two real-world networks, Bitcoin OTC and Bitcoin Alpha. The experimental results showed that the proposed models enhance the prediction accuracy and make effective decisions in viral marketing.

A model-based approach to disentangling facilitation and interference effects in conflict tasks.

Conflict tasks have become one of the most dominant paradigms within cognitive psychology, with their key finding being the conflict effect: That participants are slower and less accurate when task-irrelevant information conflicts with task-relevant information (i.e., incompatible trials), compared to when these sources of information are consistent (i.e., compatible trials). However, the conflict effect can consist of two separate effects: Facilitation effects, which is the amount of benefit provided by consistent task-irrelevant information, and interference effects, which is the amount of impairment caused by conflicting task-irrelevant information. While previous studies have attempted to disentangle these effects using neutral trials, which contrast compatible and incompatible trials to trials that are designed to have neutral task-irrelevant information, these analyses rely on the assumptions of Donder's subtractive method, which are difficult to verify and may be violated in some circumstances. Here, we develop a model-based approach for disentangling facilitation and interference effects, which extends the existing diffusion model for conflict tasks (DMC) framework to allow for different levels of automatic activation in compatible and incompatible trials. Comprehensive parameter recovery assessments display the robust measurement properties of our model-based approach, which we apply to nine previous data sets from the flanker (6) and Simon (3) tasks. Our findings suggest asymmetric facilitation and interference effects, where interference effects appear to be present for most participants across most studies, whereas facilitation effects appear to be small or nonexistent. We believe that our novel model-based approach provides an important step forward for understanding how information processing operates in conflict tasks, allowing researchers to assess the convergence or divergence between experimental-based (i.e., neutral trials) and model-based approaches when investigating facilitation and interference effects. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Unsupervised learning of anomalous diffusion data: an anomaly detection approach

The characterization of diffusion processes is a keystone in our understanding of a variety of physical phenomena. Many of these deviate from Brownian motion, giving rise to anomalous diffusion. Various theoretical models exists nowadays to describe such processes, but their application to experimental setups is often challenging, due to the stochastic nature of the phenomena and the difficulty to harness reliable data. The latter often consists on short and noisy trajectories, which are hard to characterize with usual statistical approaches. In recent years, we have witnessed an impressive effort to bridge theory and experiments by means of supervised machine learning techniques, with astonishing results. In this work, we explore the use of unsupervised methods in anomalous diffusion data. We show that the main diffusion characteristics can be learnt without the need of any labelling of the data. We use such method to discriminate between anomalous diffusion models and extract their physical parameters. Moreover, we explore the feasibility of finding novel types of diffusion, in this case represented by compositions of existing diffusion models. At last, we showcase the use of the method in experimental data and demonstrate its advantages for cases where supervised learning is not applicable.

A New Approach to Constructing Models of Electron Diffusion by EMIC Waves in the Radiation Belts

AbstractElectromagnetic ion cyclotron (EMIC) waves play an important role in relativistic electron losses in the radiation belts through diffusion via resonant wave‐particle interactions. We present a new approach for calculating bounce and drift‐averaged EMIC electron diffusion coefficients. We calculate bounce‐averaged diffusion coefficients, using quasi‐linear theory, for each individual Combined Release and Radiation Effects Satellite (CRRES) EMIC wave observation using fitted wave properties, the plasma density and the background magnetic field. These calculations are then combined into bounce‐averaged diffusion coefficients. The resulting coefficients therefore capture the combined effects of individual spectra and plasma properties as opposed to previous approaches that use average spectral and plasma properties, resulting in diffusion over a wider range of energies and pitch angles. These calculations, and their role in radiation belt simulations, are then compared against existing diffusion models. The new diffusion coefficients are found to significantly improve the agreement between the calculated decay of relativistic electrons and Van Allen Probes data.

Rigidification of Poly(p-phenylene)s through ortho-Phenyl Substitution.

A sterically π-congested ortho-phenylated poly(p-phenylene) (PPP) has been synthesized with unprecedentedly high molecular weights up to 29 kDa after fractionation, as confirmed by gel permeation chromatography coupled with a multiangle laser light scattering detector. The chain translation diffusion coefficient obtained from dynamic light scattering experiments displayed strong scaling (∼Lw–0.8) to the chain contour length, indicating a rodlike shape with remarkably high rigidity of this novel PPP. These results provide an interesting insight into the relationship between the structure and the chain stiffness of PPP-based polymers and challenge the validity of the existing diffusion models in polymer physics.

Size-controlled non-Fickian diffusion in a combined micro- and mesoporous material

In the porous media, the diffusion is affected by the pore size. Particularly, the diffusion coefficient is treated as a function of the pore size. In this letter, we have studied the mass transfer of the diffusing substances with different molecule size through the media characterized by the presence of the micropores and the mesopores. The diffusant with the molecule size larger than the micropore size demonstrates the standard Fickian behavior, whereas the diffusant with the molecule size smaller than the micropore size gives the non-Fickian diffusion kinetics. It reveals that the pore size has a specific impact on the mass transfer deviation from the classical Fickian law. It also affects the expression of the conservation law for the diffusion flux. The obtained findings demonstrate the size-controllability of the diffusion regime, contrary to the existing diffusion models based on the dependence of the diffusion coefficient on the pore size only.

Factoring out nondecision time in choice reaction time data: Theory and implications.

Choice reaction time (RT) experiments are an invaluable tool in psychology and neuroscience. A common assumption is that the total choice response time is the sum of a decision and a nondecision part (time spent on perceptual and motor processes). While the decision part is typically modeled very carefully (commonly with diffusion models), a simple and ad hoc distribution (mostly uniform) is assumed for the nondecision component. Nevertheless, it has been shown that the misspecification of the nondecision time can severely distort the decision model parameter estimates. In this article, we propose an alternative approach to the estimation of choice RT models that elegantly bypasses the specification of the nondecision time distribution by means of an unconventional convolution of data and decision model distributions (hence called the D*M approach). Once the decision model parameters have been estimated, it is possible to compute a nonparametric estimate of the nondecision time distribution. The technique is tested on simulated data, and is shown to systematically remove traditional estimation bias related to misspecified nondecision time, even for a relatively small number of observations. The shape of the actual underlying nondecision time distribution can also be recovered. Next, the D*M approach is applied to a selection of existing diffusion model application articles. For all of these studies, substantial quantitative differences with the original analyses are found. For one study, these differences radically alter its final conclusions, underlining the importance of our approach. Additionally, we find that strongly right skewed nondecision time distributions are not at all uncommon.