Diffusion Step Research Articles

SAMControl: Controlling Pose and Object for Image Editing with Soft Attention Mask

To achieve content-consistent results in text-conditioned image editing, existing methods typically employ a reconstruction branch to capture the source image details via diffusion inversion and a generation branch to synthesize the target image based on the given textual prompt and the masked source image details. However, accurately segmenting source details is challenging with the current fixed-threshold mask strategy. Additionally, the inadequacies in the inversion process can lead to insufficient retention of source details. In this paper, we propose a method called SAMControl ( S oft A ttention M ask) to adaptively control the pose and object details for image editing. SAMControl dynamically learns flexible attention masks for different images at various diffusion steps. Furthermore, in the reconstruction branch, we utilize a direct inversion technique to ensure the fidelity of source details within SAM. Extensive qualitative and quantitative results demonstrate the effectiveness of the proposed method.

First-principles study on atomic formation and manganese diffusion in MnS/Mg-Ti-oxide complex inclusions in steel

Mg-Ti oxide/MnS complex inclusions effectively serve as nucleation sites for intragranular ferrite. This study employs first-principles calculation methods to investigate the inducing properties of complex inclusions, including the growth of MnS on Ti oxide, the absorption of Mn atoms by Ti oxide, and the diffusion behavior of Mn atoms within Ti oxide, both untreated and Mg-treated. Ti2O3 indicates the untreated group, while MgTiO3 indicates the Mg-treated group. Computational analysis indicated that MnS is more likely to adsorb and grow on MgTiO3 than on Ti2O3. MnS vertically grows on the O-Mg-Mg surface of MgTiO3. By calculating the dissociation energy, it can be determined that the Mn in the oxide of the MnS/oxide complex inclusions originates from the steel matrix rather than from MnS. By analyzing the diffusion behavior of vacancies and Mn atoms through energy barrier calculations, it is evident that the addition of Mg alters the diffusion pathways of Mn atoms in Ti oxide and confirms that the bonding between Mn and O is the limiting step for Mn atom diffusion within the oxides. The diffusion path of Mn atoms in MgTiO3 is Path I → Path II → Path I. Path I: Mn atoms diffuse within the cationic layer. Path II: Mn atoms diffuse by traversing across two layers of O atoms.

Amine-modified SiO2 aerogel for efficient adsorption of low-pressure CO2: Response surface methodology optimization and adsorption mechanism

Amine-modified SiO2 aerogel (AMSA) has shown tremendous potential in CO2 capture due to its unique three-dimensional network structure and excellent stability. However, it still faces many challenges in the preparation optimization, adsorption enhancement at low CO2 partial pressure, and mechanism elucidation. Hence, in this paper, the response surface methodology is employed for the first time and the optimal preparation conditions for AMSA are determined. A regression model is successfully established to accurately predict the CO2 adsorption capacity of AMSA under different synthesis conditions (with a relative error of only 0.98 %). Additionally, the AMSA prepared at optimal conditions exhibits high CO2 adsorption capacity of 2.11 mmol/g and excellent stability (only decreased by 1.44 % after 8 cycles) at 10 % CO2 partial pressure, indicating enormous application potential. Notably, the CO2 adsorption mechanism by AMSA at different temperatures and partial pressures are revealed in detail through fitting with the Freundlich isotherm and the double-exponential model, as well as thermodynamic parameter calculations. Firstly, the CO2 adsorption behavior belongs to the multi-layer adsorption behavior with uneven energy of active sites. Secondly, the CO2 adsorption capacity and rate are jointly controlled by molecular motion and chemical reaction equilibrium, diffusion and chemical reaction steps, respectively. Finally, the adsorption process is exothermic and spontaneous, and the level of disorder decreases at the gas-solid interface. This study not only achieves efficient adsorption of low-pressure CO2, but also deepens the understanding of gas-solid interface adsorption mechanisms, providing scientific basis and technological support for the development of novel efficient AMSA adsorbent materials.

When guided diffusion model meets zero-shot image super-resolution

Existing deep learning-based single-image super-resolution (SR) methods typically rely on vast quantities of paired data. As an essential solution, zero-shot SR methods require only a single image to handle image-specific degradation. However, these methods still struggle to recover fine-grained details due to the lack of supervised information. In this work, we propose a novel guided Diffusion model for Zero-shot image SR (ZeroDiff) to explicitly direct image quality enhancement. Specifically, we elaborate two key guidance strategies: (1) high-frequency guidance and (2) content-consistent guidance. The former concentrates on boosting fine-grained textures by embedding high-frequency information into the cross-attention mechanism of the noise estimator. The latter avoids the sampling deviating from the original image in terms of structure and low-frequency content. Specifically, the noisy images at each diffusion step are injected into the corresponding sampling step, encouraging the sampled image to be consistent with that of the corresponding diffusion step. Moreover, we design a progressive zoom-in paradigm by gradually enlarging the image size and enriching the image details to boost the sampling efficiency of diffusion models, while enabling high-quality image reconstruction. Extensive experiments reveal that our method achieves comparable results with other state-of-the-art methods in quantitative and qualitative evaluations on both face and natural images from synthetic and real-world datasets.

Study on the adsorption efficiency and mechanism of Sb(V) in aqueous solutions using enhanced surfactant-modified iron-calcium composite

The presence of Sb(V) in aqueous solutions poses a significant threat to the surrounding environment, and current treatment methods are inadequate. In this study, a magnetic surfactant (CTAB)-modified iron-calcium composite (CTAB-IC) was successfully synthesized using iron-calcium composite as the base material. This novel composite was used for the efficient removal of Sb(V) from textile wastewater solutions. Characterization analyses revealed that the CTAB-IC material exhibits a rich long-prismatic structure and superparamagnetic properties, classifying it as a soft magnetic material. Post-adsorption particle agglomerates were found to comprise Ca, S, and O. Sequential batch experiments demonstrated a maximum adsorption capacity of 54.05 mg/g, with adsorption kinetics data fitting the pseudo-second-order model. The intraparticle diffusion model indicated the presence of multiple diffusion steps during the adsorption process. Additionally, the adsorption of Sb(V) by CTAB-IC was identified as a heterogeneous surface adsorption process, best described by the Freundlich model. The primary adsorption mechanisms involved the formation of surface Ca-O-Sb complexes and inner-sphere Fe-O-Sb complexes, as well as amorphous surface precipitation and electrostatic adsorption. Notably, the treatment of textile wastewater often results in iron-calcium-rich sludge, which is challenging to manage and valorize. This study explored the potential for resource recycling by utilizing CTAB to harness the Fe elements in textile wastewater sludge, thereby promoting waste-to-resource conversion.

Coarse-to-fine mechanisms mitigate diffusion limitations on image restoration

Recent years have witnessed the remarkable performance of diffusion models in various vision tasks. However, for image restoration that aims to recover clear images with sharper details from given degraded observations, diffusion-based methods may fail to recover promising results due to inaccurate noise estimation. Moreover, simple constraining noises cannot effectively learn complex degradation information, which subsequently hinders the model capacity. To solve the above problems, we propose a coarse-to-fine diffusion Transformer (C2F-DFT) to mitigate diffusion limitations mentioned before on image restoration. Specifically, the proposed C2F-DFT contains diffusion self-attention (DFSA) and diffusion feed-forward network (DFN) within a new coarse-to-fine training mechanism. The DFSA and DFN with embedded diffusion steps respectively capture the long-range diffusion dependencies and learn hierarchy diffusion representation to guide the restoration process in different time steps. In the coarse training stage, our C2F-DFT estimates noises and then generates the final clean image by a sampling algorithm. To further improve the restoration quality, we propose a simple yet effective fine training pipeline. It first exploits the coarse-trained diffusion model with fixed steps to generate restoration results, which then would be constrained with corresponding ground-truth ones to optimize the models to remedy the unsatisfactory results affected by inaccurate noise estimation. Extensive experiments show that C2F-DFT significantly outperforms diffusion-based restoration method IR-SDE and achieves competitive performance compared with Transformer-based state-of-the-art methods on 3 tasks, including image deraining, image deblurring, and real image denoising. The source codes and visual results are available at https://github.com/wlydlut/C2F-DFT.

Multi-Modal Modality-Masked Diffusion Network for Brain MRI Synthesis With Random Modality Missing.

Synthesis of unavailable imaging modalities from available ones can generate modality-specific complementary information and enable multi-modality based medical images diagnosis or treatment. Existing generative methods for medical image synthesis are usually based on cross-modal translation between acquired and missing modalities. These methods are usually dedicated to specific missing modality and perform synthesis in one shot, which cannot deal with varying number of missing modalities flexibly and construct the mapping across modalities effectively. To address the above issues, in this paper, we propose a unified Multi-modal Modality-masked Diffusion Network (M2DN), tackling multi-modal synthesis from the perspective of "progressive whole-modality inpainting", instead of "cross-modal translation". Specifically, our M2DN considers the missing modalities as random noise and takes all the modalities as a unity in each reverse diffusion step. The proposed joint synthesis scheme performs synthesis for the missing modalities and self-reconstruction for the available ones, which not only enables synthesis for arbitrary missing scenarios, but also facilitates the construction of common latent space and enhances the model representation ability. Besides, we introduce a modality-mask scheme to encode availability status of each incoming modality explicitly in a binary mask, which is adopted as condition for the diffusion model to further enhance the synthesis performance of our M2DN for arbitrary missing scenarios. We carry out experiments on two public brain MRI datasets for synthesis and downstream segmentation tasks. Experimental results demonstrate that our M2DN outperforms the state-of-the-art models significantly and shows great generalizability for arbitrary missing modalities.

Suppressing the HIFU interference in ultrasound guiding images with a diffusion-based deep learning model

Background and objectivesIn ultrasound guided high-intensity focused ultrasound (HIFU) surgery, it is necessary to transmit sound waves at different frequencies simultaneously using two transducers: one for the HIFU therapy and another for the ultrasound imaging guidance. In this specific setting, real-time monitoring of non-invasive surgery is challenging due to severe contamination of the ultrasound guiding images by strong acoustic interference from the HIFU sonication. MethodsThis paper proposed the use of a deep learning (DL) solution, specifically a diffusion implicit model, to suppress the HIFU interference. We considered the images contaminated with HIFU interference as low-resolution images, and those free from interference as high-resolution. While suppressing HIFU interference using the diffusion implicit (HIFU-Diff) model, the task was transformed into generating a high-resolution image through a series of forward diffusion steps and reverse sampling. A series of ex-vivo and in-vivo experiments, conducted under various parameters, were designed to validate the performance of the proposed network. ResultsQuantitative evaluation and statistical analysis demonstrated that the HIFU-Diff network achieved superior performance in reconstructing interference-free images under a variety of ex-vivo and in-vivo conditions, compared to the most commonly used notch filtering and the recent 1D FUS-Net deep learning network. The HIFU-Diff maintains high performance with ‘unseen’ datasets from separate experiments, and its superiority is more pronounced under strong HIFU interferences and in complex in-vivo situations. Furthermore, the reconstructed interference-free images can also be used for quantitative attenuation imaging, indicating that the network preserves acoustic characteristics of the ultrasound images. ConclusionsWith the proposed technique, HIFU therapy and the ultrasound imaging can be conducted simultaneously, allowing for real-time monitoring of the treatment process. This capability could significantly enhance the safety and efficacy of the non-invasive treatment across various clinical applications. To the best of our knowledge, this is the first diffusion-based model developed for HIFU interference suppression.

Super-resolution reconstruction of ultrasound image using a modified diffusion model

Objective. This study aims to perform super-resolution (SR) reconstruction of ultrasound images using a modified diffusion model, designated as the diffusion model for ultrasound image super-resolution (DMUISR). SR involves converting low-resolution images to high-resolution ones, and the proposed model is designed to enhance the suitability of diffusion models for this task in the context of ultrasound imaging. Approach. DMUISR incorporates a multi-layer self-attention (MLSA) mechanism and a wavelet-transform based low-resolution image (WTLR) encoder to enhance its suitability for ultrasound image SR tasks. The model takes interpolated and magnified images as input and outputs high-quality, detailed SR images. The study utilized 1,334 ultrasound images from the public fetal head-circumference dataset (HC18) for evaluation. Main results. Experiments were conducted at 2× , 4× , and 8× magnification factors. DMUISR outperformed mainstream ultrasound SR methods (Bicubic, VDSR, DECUSR, DRCN, REDNet, SRGAN) across all scales, providing high-quality images with clear structures and rich detailed textures in both hard and soft tissue regions. DMUISR successfully accomplished multiscale SR reconstruction while suppressing over-smoothing and mode collapse problems. Quantitative results showed that DMUISR achieved the best performance in terms of learned perceptual image patch similarity, with a significant decrease of over 50% at all three magnification factors (2× , 4× , and 8× ), as well as improvements in peak signal-to-noise ratio and structural similarity index measure. Ablation experiments validated the effectiveness of the MLSA mechanism and WTLR encoder in improving DMUISR’s SR performance. Furthermore, by reducing the number of diffusion steps, the computational time of DMUISR was shortened to nearly one-tenth of its original while maintaining image quality without significant degradation. Significance. This study demonstrates that the modified diffusion model, DMUISR, provides superior performance for SR reconstruction of ultrasound images and has potential in improving imaging quality in the medical ultrasound field.

Numerical simulation of 3D vorticity dynamics with the Diffused Vortex Hydrodynamics method

In this paper the three-dimensional extension of the Diffused Vortex Hydrodynamics (DVH) is discussed along with free vorticity dynamics simulations. DVH is a vortex particle method developed in-house and widely validated in a 2D framework. The DVH approach has been embedded in a new frontend to the open-source code PEPC, the Pretty Efficient Parallel Coulomb solver. Within this parallel Barnes–Hut tree code, a superposition of elementary heat equation solutions in a cubic support is performed during the diffusion step. This redistribution avoids excessive clustering or rarefaction of the vortex particles, providing robustness and high accuracy of the method. An ascending vortex dipole at various resolutions is selected as a test-case and heuristic convergence measurements are performed, taking into account the conservation of prime integrals and the energy–enstrophy balance.

Adaptive denoising autoencoder for robust fault detection

In real chemical processes, the collected data is often subject to interference from ambient environmental noise, resulting in a decline in the detection performance. Although denoising autoencoder can realize robust fault detection to a certain extent, its performance is limited by the addition of artificial noise, which refers to Gaussian noise or the dropout noise. Moreover, the artificial noise added is often far from the distribution of real data, making it difficult to suppress the noise in real data. To address this issue, an adaptive denoising autoencoder (ADAE) based on the diffusion model is proposed. ADAE adopts a chain-like structure, concatenating traditional DAEs by replacing the manually injected noise in each DAE block with noise generated by diffusion model at various diffusion steps. In order to better capture the key features of the original process data, a novel noise prediction network is proposed within the diffusion model, utilizing dilated and gated convolutions to capture the dynamic information of the process data. Ultimately, adaptation is achieved through a chain of DAEs, suppressing noise similar to the distribution of the original data. The superiority of ADAE in noise suppression is then verified through numerical examples and the Tennessee Eastman process (TEP), comparing it with other methods.

An efficient image encryption model based on 6D hyperchaotic system and symmetric matrix for color and gray images

The security of images is one of the predominant pivotal aspects in the mammoth and still expanding digital domain. Due to chaotic system properties i.e. randomness and unpredictability is very appropriate to encrypt the images. In this research article, we construct an encryption model via 6D hyperchaotic map and a symmetric matrix for both color and grayscale images. We utilize the 6D hyperchaotic map in the confusion stage to change the pixel location and the symmetric matrix is used for changing the pixel value in the diffusion step for each RGB channel extraction from plain or original image. The image encryption model is checked over differential attacks (NPCR and UACI). Histogram analysis, correlation coefficients, and entropy analysis are also performed as statistical attacks. In conclusion, the image pixels are uniformly distributed, and the average entropy value are 7.9992 and 7.9973 for color and grayscale images, subsequently. The average NPCR and UACI for color images are 99.5956 and 33.4061, correspondingly, while the values for grayscale images are 99.5934 and 33.3054, respectively. These values are in the vicinity of optimal ranges. The suggested scheme's great efficiency and the proposed algorithm's resilience to a wide range of cryptanalytic attacks are implied by experimental results, statistical analysis, and differential attacks.

Revealing the Influence of Molecular Chemisorption Direction on the Reaction Selectivity of Dehalogenative Coupling on Au(111): Polymerization versus Cyclization.

The control of reaction selectivity is of great interest in chemistry and depends crucially on the revelation of key influencing factors. Based on well-defined molecule-substrate model systems, various influencing factors have been elucidated, focusing primarily on the molecular precursors and the underlying substrates themselves, while interfacial properties have recently been shown to be essential as well. However, the influence of molecular chemisorption direction on reaction selectivity, as a subtle interplay between molecules and underlying substrates, remains elusive. In this work, by a combination of scanning tunneling microscopy imaging and density functional theory calculations, we report the influence of molecular chemisorption direction on the reaction selectivity of two types of dehalogenative coupling on Au(111), i.e., polymerization and cyclization, at the atomic level. The diffusion step of a reactive dehalogenated intermediate in two different chemisorption directions was theoretically revealed to be the key to determining the corresponding reaction selectivity. Our results highlight the important role of molecular chemisorption directions in regulating the on-surface dehalogenative coupling reaction pathways and products, which provides fundamental insights into the control of reaction selectivity by exploiting some subtle interfacial parameters in on-surface reactions for the fabrication of target low-dimensional carbon nanostructures.

Quaternary branched polyethylenimine‐based nanocomposite for removal of NO3− ions from aqueous solutions

AbstractThis study synthesized a nanocomposite based on quaternary branched polyethyleneimine containing sodium alginate and graphene oxide and characterized by FE‐SEM, EDX‐MAP, XRD, FT‐IR, and TGA and used as an efficient sorbent for the removal of nitrate ions. The maximum ion exchange in optimized conditions was discovered to be 75%. The experimental studies showed that nitrate removal follows the pseudo‐first‐order model due to more consonance. Also, Weber–Morris and Boyd models suggested that the nitrate ions adsorption on the surface of nanocomposite is not controlled only by the intraparticle diffusion step. Moreover, Langmuir, Redlich–Peterson, and Hill models exhibited high regression coefficients of 0.991, 0.993, and 0.992, respectively. Thermodynamic studies confirmed the adsorption system was spontaneous and exothermic. Recovery assay showed that more than 98% of the nitrate ion exchanged by the nanocomposite was regenerated and nanocomposite could be reused seven cycles without efficacy reducing significantly. The nanocomposite showed excellent antibacterial activity against Staphylococcus aureus and Escherichia coli cells, with minimum inhibitory concentrations of 62.5 and 125 μg mL−1, and zone of growth inhibition of 17.5 ± 0.5 and 10.5 ± 0.5 mm, respectively. The results represented that the introduced nanocomposite is a useful material for the removal of nitrate ions from aqueous solution and has antibacterial properties.

Composites of sodium alginate based - Functional materials towards sustainable adsorption of benzene phenol derivatives - Bisphenol A/Triclosan

The present study evaluates the adsorption efficiency of low-cost carbonaceous adsorbents as fly ash (FA), saw dust biochar (SDB) (untreated and alkali - treated), live/dead pulverized white rot fungus Hypocrea lixii biomass encapsulated in sodium alginate (SA) against the commercially available activated carbon (AC) and graphene oxide (GO) SA beads for removal of benzene phenol derivatives - Bisphenol A (BPA)/triclosan (TCS). Amongst bi - and tri - composites SA beads, tri-composite beads comprising of untreated flyash - dead fungal biomass - sodium alginate (UFA - DB - SA) showed at par results with commercial composite beads. The tri - composite beads with point zero charge (Ppzc) of 6.2 was characterized using FTIR, XRD, surface area BET and SEM-EDX. The batch adsorption using tri - composite beads revealed removal of 93% BPA with adsorption capacity of 16.6 mg/g (pH 6) and 83.72% TCS with adsorption capacity of 14.23 mg/g (pH 5), respectively at 50 ppm initial concentration with 6 % adsorbent dose in 5 h. Freundlich isotherm favoring multilayered adsorption provided a better fit with r2 of 0.9674 for BPA and 0.9605 for TCS respectively. Intraparticle diffusion model showed adsorption of BPA/TCS molecules to follow pseudo - second order kinetics with boundary layer diffusion governed by first step of fast adsorption and intraparticle diffusion within pores by second slow adsorption step. Thermodynamic parameters (ΔH°, ΔS°, ΔG°) revealed adsorption process as exothermic, orderly and spontaneous. Methanol showed better desorbing efficiency leading to five cycles reusability. The phytotoxicity assay revealed increased germination rate of mung bean (Vigna radiata) seeds, sprinkled with post adsorbed treated water (0 h, 5 h and 7 h) initially spiked with 50 ppm BPA/TCS. Overall, UFA - DB - SA tri - composite beads provides a cost effective and eco - friendly matrix for effective removal of hydrophobic recalcitrant compounds.

FSRDiff: A fast diffusion-based super-resolution method using GAN

Single image super-resolution with diffusion probabilistic models (SRDiff) is a successful diffusion model for image super-resolution that produces high-quality images and is stable during training. However, due to the long sampling time, it is slower in the testing phase than other deep learning-based algorithms. Reducing the total number of diffusion steps can accelerate sampling, but it also causes the inverse diffusion process to deviate from the Gaussian distribution and exhibit a multimodal distribution, which violates the diffusion assumption and degrades the results. To overcome this limitation, we propose a fast SRDiff (FSRDiff) algorithm that integrates a generative adversarial network (GAN) with a diffusion model to speed up SRDiff. FSRDiff employs conditional GAN to approximate the multimodal distribution in the inverse diffusion process of the diffusion model, thus enhancing its sampling efficiency when reducing the total number of diffusion steps. The experimental results show that FSRDiff is nearly 20 times faster than SRDiff in reconstruction while maintaining comparable performance on the DIV2K test set.

Adsorption of oxytetracycline in hyporheic zone sediments mediated by microplastics: Experimental revelations and mechanistic insights

Microplastics (MPs), serving as carriers, possess a mediating effect on altering antibiotics (ATs) migration, yet this novel effect has not received due attention in the vulnerable hyporheic zones where surface water and groundwater converge. This study collected sediment layers from the typical hyporheic zone of the Weihe River, focusing on oxytetracycline (OTC) detected therein, and employed three types of MPs (polyethylene (PE), polypropylene (PP), and polymethyl methacrylate (PMMA) to establish various mediated adsorption systems. Through batch equilibrium adsorption experiments, morphological and spectroscopic characterizations combined with the impact of external environmental factors, the changes in OTC adsorption characteristics and the interactions before and after MPs mediation were systematically examined. The results indicate that MPs-mediated adsorption led to the formation of a rough, porous amorphous structure on the sediment surface, with potential internal hydrogen bonds. This exhibited an additional micropore diffusion step that delayed the adsorption kinetics equilibrium time, increased the isothermal maximum adsorption capacity along with desorption hysteresis effects, shifted thermodynamic enthalpy from physically dominated to semi-chemical adsorption participation, and resulted in an increased tendency towards disorder in entropy. The pH, ionic strength, and aging of MPs all affect the charge balance within the adsorption system, while MPs, OTC, and sediments can interact through the negative-charge-assisted hydrogen bond ((−)CAHB). The mediating effect of MPs stems from their role as receptors, facilitating the formation of (−)CAHB and thus crosslinking an amorphous structure available for OTC micropore filling. Compared to PE and PP, PMMA provides receptors through an additional free radical pathway, hence possessing a stronger mediating effect. These findings contribute to reevaluating the environmental fate of ATs and MPs from a new perspective, offering theoretical references for risk assessment and management of emerging pollutants.

Influential lead uptake using dried and inactivated-fungal biomass obtained from Panaeolus papilionaceus: biological activity, equilibrium, and mechanism

AbstractIn this study, the use of fungal (Panaeolus papilionaceus) biomass as a biosorbent was investigated to effectively remove Pb2+ ions from aquatic medium. The removal of Pb2+ ions using a fungal biosorbent was examined in a batch system in terms of initial solution pH, temperature, time, and initial Pb2+ concentration. Optimal operating conditions for biosorption of Pb2+ ions; pH: 4.5, T: 25 °C, and t: 24 h. The max biosorption capacity for Pb2+ ions was found to be 31.2 mg g−1 from the Langmuir model. Thermodynamic studies showed that Pb2+ ions biosorption into fungal biomass was possible, spontaneous, and endothermic. Additionally, the antimicrobial activity and antibiofilm activity of the extract of fungus were also investigated. It was determined that the fungal extract did not have antimicrobial properties. On the other hand, the extract has been shown to have the potential to prevent biofilm formation. 1 mg of the extract prevented the biofilm formation of Staphylococcus aureus by 87.85%. It has been observed that the biosorption mechanism of Pb2+ ions into fungal biomass includes the steps of surface biosorption, film diffusion, and intra-particle diffusion.

Score mismatching for generative modeling

We propose a new score-based model with one-step sampling. Previously, score-based models were burdened with heavy computations due to iterative sampling. For substituting the iterative process, we train a standalone generator to compress all the time steps with the gradient backpropagated from the score network. In order to produce meaningful gradients for the generator, the score network is trained to simultaneously match the real data distribution and mismatch the fake data distribution. This model has the following advantages: (1) For sampling, it generates a fake image with only one step forward. (2) For training, it only needs 10 diffusion steps. (3) Compared with consistency model, it is free of the ill-posed problem caused by consistency loss. On the popular CIFAR-10 dataset, our model outperforms Consistency Model and Denoising Score Matching, which demonstrates the potential of the framework. We further provide more examples on the MINIST and LSUN datasets. The code is available on GitHub.

LMD: Faster Image Reconstruction with Latent Masking Diffusion

As a class of fruitful approaches, diffusion probabilistic models (DPMs) have shown excellent advantages in high-resolution image reconstruction. On the other hand, masked autoencoders (MAEs), as popular self-supervised vision learners, have demonstrated simpler and more effective image reconstruction and transfer capabilities on downstream tasks. However, they all require extremely high training costs, either due to inherent high temporal-dependence (i.e., excessively long diffusion steps) or due to artificially low spatial-dependence (i.e., human-formulated high mask ratio, such as 0.75). To the end, this paper presents LMD, a faster image reconstruction framework with Latent Masking Diffusion. First, we propose to project and reconstruct images in latent space through a pre-trained variational autoencoder, which is theoretically more efficient than in the pixel-based space. Then, we combine the advantages of MAEs and DPMs to design a progressive masking diffusion model, which gradually increases the masking proportion by three different schedulers and reconstructs the latent features from simple to difficult, without sequentially performing denoising diffusion as in DPMs or using fixed high masking ratio as in MAEs, so as to alleviate the high training time-consumption predicament. Our approach allows for learning high-capacity models and accelerate their training (by 3x or more) and barely reduces the original accuracy. Inference speed in downstream tasks also significantly outperforms the previous approaches.