Diffusion Model Research Articles

A behavioral conditional diffusion probabilistic model for human motion modeling in multi-action mixed human-robot collaboration

Collision avoidance is priority for human-robot collaboration (HRC). Human motions have stochastic, making it difficult for robots to recognize humans. To establish accurate human motion models is important to recognize humans. In the collaboration process, the action categories may suddenly and continuously switch to adapt to complex production tasks, namely, the stochastic of human motions, and we call this collaboration process as multi-action mixed HRC. The previous HRC rarely considered the stochastic of human motions. However, the stochastic in multi-action mixed HRC brings great challenges to human motion modeling. On the one hand, the same observed motions may develop into different action categories, the motion distribution need to be predicted. On the other hand, the connected motions between two action categories are missing, and need to be predicted. To address these two problems, this study introduced diffusion probabilistic models to capture the stochastic of human motions. However, the current diffusion probabilistic models cannot realize the high-accuracy human motion prediction. Most of them did not consider the condition distribution in diffusion process, and did not comprehensively capture the spatiotemporal characteristics between human and robot. To solve the above problems, a behavioral conditional probabilistic diffusion model (BCDPM) is proposed. The BCDPM model can capture the stochastic of human motions to model the human motion distribution. A mask mechanism is combined with diffusion prior generated by BCDPM model to predict the missing connected motions. The BCDPM model consists of the designed constrain network, the denoising network, and enhanced memory graph neural network. The constrain network can generate the condition distribution, the denoising network and enhanced memory graph neural network can capture the short term and long term spatiotemporal characteristics, respectively. Moreover, the overlapping motion primitives are explicitly encoded into the implicit BCDPM model to mine commonalities of motions. The results show the proposed method can obtain excellent accuracy in prediction and outstanding F1 score in collision detection experiment.

Trajectory-based fish event classification through pre-training with diffusion models

This study contributes to advancing the field of automatic fish event recognition in natural underwater videos, addressing the current gap in studying fish interaction and competition, including predator-prey relationships and mating behaviors. We used the corkwing wrasse (Symphodus melops) as a model, a marine species of commercial importance that reproduces in sea-weed nests built and cared for by a single male. These nests attract a wide range of visitors and are the focal point for behavior such as spawning, chasing, and maintenance. We propose a deep learning methodology to analyze the movement trajectories of the nesting male and classify the associated events observed in their natural habitat. Our approach leverages unsupervised pre-training based on diffusion models, leading to improved feature learning. Additionally, we introduce a dataset comprising 16,937 trajectories across 12 event classes, making it the largest in terms of event class diversity. Our results demonstrate the superior performance of our method compared to several deep architectures. The code for the proposed method and the trajectories can be found at https://github.com/NoeCanovi/Fish_Behaviors_Generative_Models.

FloorDiffusion: Diffusion model-based conditional floorplan image generation method using parameter-efficient fine-tuning and image inpainting

The conditional generation of high-quality floorplan images using deep-learning methods is challenging because the generated floorplans are required to match specific conditions, such as floorplan silhouettes and spatial layouts. Recently, diffusion models have emerged as alternatives of conditional generative adversarial networks in image generation, offering higher image quality, pairing-free training datasets, and adaptability to various image domains via parameter fine-tuning of pretrained diffusion models. However, diffusion models are rarely used for floorplan generation because when fine-tuning them on image domains that were not learned in pretraining, such as floorplans, the quality of the generated images is poor and tuning takes a long time. This phenomenon arises from the so-called catastrophic forgetting problem, where traditional fine-tuning methods that update all parameters easily destroy the knowledge of pretrained diffusion models. To address this problem, we propose FloorDiffusion, a diffusion model-based conditional floorplan generation method. In this method, only a few key parameters of the pre-trained diffusion model are fine-tuned, which allows adaptation to the floorplan domain while retaining its useful knowledge. Then, the fine-tuned diffusion model performs conditional floorplan generation by inpainting the unfinished regions of the input conditional image. Comparative experiments with existing methods demonstrate that our method can produce more architecturally realistic floorplan images with up to 72 % image quality improvement. It can also generate various floorplan images for a single input condition image. Finally, ablation studies show that all components of the proposed method are essential for optimal operation.

Vivid-persona: customizable persona tool with interactive and immersive experiences

In the design field, persona is widely used to help design teams understand their target users. However, existing persona tools have limitations such as insufficient interactivity, inability to arouse designers’ empathy, and inability to finely segment users. We developed Vivid-Persona, which is a customisable persona tool offering an interactive and immersive experience. Designers can set the demographics and scenarios of the target user to customise a persona. The tool generates a realistic image of the persona using a Diffusion model. Embedded with the ChatGPT and SadTalker models, the persona can engage in real-time video chats with designers. The persona is augmented with facial expressions, based on the analysis of the emotions in utterances. A user study showed that Vivid-Persona effectively solves the problem of user segmentation, significantly enhancing the designer’s perception of persona in terms of authenticity, clarity, empathy and willingness to use, as well as the quality of the output design solutions.

Holistic regulatory framework for distributed generation based on multi-objective optimization

Regulatory changes associated with distributed generation have occurred in several countries (e.g., the USA, Germany, the UK, and Australia). However, there is a lack of robust and holistic analytical models that can be used to implement the best regulatory framework among possible options. In this context, the present paper proposes a cutting-edge regulatory framework for distributed generation based on multi-objective optimization, taking into account socioeconomic (socioeconomic welfare created by the regulated electricity market and electricity tariff affordability) and environmental (global warming potential) indicators. Such indicators are modeled primarily based on the optimized tariff model (socioeconomic regulated electricity market model), Bass diffusion model (forecasting model of distributed generation deployment), and life cycle assessment (environmental impact assessment method). The design variables are assumed to be the regulated electricity tariff and remuneration of the electricity injected into the grid over the years. First, the proposed methodology is applied to fifteen large-scale Brazilian concession areas with a significant deployment of distributed generation assuming two approaches, a multi-compensation scenario, where the compensation is set individually for each concession area, and a single-compensation scenario, where the compensation is set equally for all concession areas. Then, the optimal solutions are compared to Ordinary Law 14300, which is a recently implemented regulatory framework for distributed generation in Brazil. Results demonstrate that Ordinary Law 14300 is a dominated or non-optimal solution since it is not located on the optimal Pareto frontiers for any of the assessed concession areas. Assuming the Euclidian knee points, benefits averaging 33% and 15% were achieved in terms of electricity tariff affordability for the multi and single-compensation scenarios, respectively, with small losses of 8% and 3% in terms of socioeconomic welfare and global warming potential. Though the proposed methodology is applied in the Brazilian context, it can also be applied to other countries with regulated electricity markets; thus, it is expected to be valuable for researchers, government institutions, and regulatory agencies worldwide.

Clam shell waste recycling and valorization for sustainable Hg remediation

Shellfish aquaculture world production is constantly growing due to the increase in demand for seafood and reached over 18 million tons in 2022. The suitable management of the shell waste is one of the main environmental challenging issues as most of this waste is sent to landfills with emanation of foul odors, pathogens proliferation and reduction of available space. However, the conversion of this biowaste to new value-added materials could provide significant environmental and economic benefits. Clam shell waste was the starting material for the synthesis of hydroxyapatite (CSHAP) applied as an adsorbent for Hg2+ removal from aqueous solutions. Adsorption experiments were performed in batch using simulated wastewaters prepared from HgCl2 to investigate the effects of contact time and initial Hg2+ concentration on the removal process. Mineralogical composition, morphological features and elemental composition of CSHAP before and after the experiments were investigated by XRPD, SEM-EDS and FTIR analysis. The concentrations of Hg2+ and Ca2+ in the solutions were analyzed by ICP-AES. The adsorption kinetics of Hg2+ was simulated with the pseudo-first-order rate model, the pseudo-second order model and the intraparticle diffusion model. The results of the kinetics study showed that the Hg2+ adsorption followed the pseudo-second-order kinetics model and reached equilibrium within 40 min. The Langmuir model fitted the experimental results better than the Freundlich, Temkin and Dubinin-Radushkevic isotherm models, with a maximum adsorption capacity of 65.8 mg/g which is generally higher than other waste-derived adsorbents used for the removal of Hg2+ ions from water. The removal mechanism includes rapid surface complexation on CSHAP grains, followed by a slow incorporation of the Hg2+ ions in the crystalline structure. The results of this study could contribute to delineate a new research direction for a more sustainable management of clam shell biowaste.

Data augmentation based on diffusion probabilistic model for remaining useful life estimation of aero-engines

Predicting the remaining useful life (RUL) of aero-engines is essential for their prognostics and health management (PHM). Deep learning technologies are effective in this area, but their success depends critically on acquiring sufficient engine monitoring data from operation to failure, a process that is expensive and challenging in practice. Insufficient data limit the training of deep learning methods, thereby affecting their predictive performance. To address this issue, this study proposes a novel method named DiffRUL for augmenting multivariate engine monitoring data and generating high-quality samples mimicking degradation trends in real data. Initially, a specialized degradation trend encoder is designed to extract degradation trend representations from monitoring data, which serve as generative conditions. Subsequently, the diffusion model is adapted to the scenario of generating multivariate monitoring data, reconstructing and synthesizing data from Gaussian noise through a reverse process. Additionally, a denoising network is developed to incorporate generative conditions and capture spatio-temporal correlations in the data, accurately estimating noise levels during the reverse process. Experimental results on C-MAPSS and N-CMAPSS datasets show that DiffRUL successfully generates high-fidelity multivariate monitoring data. Furthermore, these generated data effectively support the RUL prediction task and significantly enhance the predictive ability of the underlying deep learning models.

Denoising diffusion delensing: reconstructing the non-Gaussian CMB lensing potential with diffusion models

ABSTRACT Optimal extraction of cosmological information from observations of the cosmic microwave background (CMB) critically relies on our ability to accurately undo the distortions caused by weak gravitational lensing. In this work, we demonstrate the use of denoising diffusion models in performing Bayesian lensing reconstruction. We show that score-based generative models can produce accurate, uncorrelated samples from the CMB lensing convergence map posterior, given noisy CMB observations. To validate our approach, we compare the samples of our model to those obtained using established Hamiltonian Monte Carlo methods, which assume a Gaussian lensing potential. We then go beyond this assumption of Gaussianity, and train and validate our model on non-Gaussian lensing data, obtained by ray-tracing N-body simulations. We demonstrate that in this case, samples from our model have accurate non-Gaussian statistics beyond the power spectrum. The method provides an avenue towards more efficient and accurate lensing reconstruction, which does not rely on an approximate analytical description of the posterior probability. The reconstructed lensing maps can be used as an unbiased tracer of the matter distribution, and to improve delensing of the CMB, resulting in more precise cosmological parameter inference.

An agent-based model of cross-platform information diffusion and moderation

Social media platforms are highly interconnected because many users maintain a presence across multiple platforms. Consequently, efforts to limit the spread of misinformation taken by individual platforms can have complex consequences on misinformation diffusion across the social media ecosystem. This is further complicated by the diverse social structures, platform standards, and moderation mechanisms provided on each platform. We study this issue by extending our previous model of Reddit interactions and community-specific moderation measures. By adding a followership-based model of Twitter interactions and facilitating cross-platform user participation, we simulate information diffusion across heterogeneous social media platforms. While incorporating platform-specific moderation mechanisms, we simulate interactions at the user level and specify user-specific attributes. This allows practitioners to conduct experiments with various types of actors and different combinations of moderation. We show how the model can simulate the impacts of such features on discussions facilitated by Reddit and Twitter and the cross-platform spread of misinformation. To validate this model, we use a combination of empirical datasets from three U.S. political events and prior findings from user surveys and studies.

Effects of different decaffeination methods on caffeine contents, physicochemical, and sensory properties of coffee

Abstract Coffee consumption could provide various benefits for human health, but also could contribute to several health problems. The growing trend of coffee consumption has created a rising demand for decaffeinated coffee that is safe for consumers with low caffeine tolerance. Decaffeination process, however, can result in the alteration of several properties of coffee which affect overall coffee taste. This review discussed current decaffeination methods such as water decaffeination, solvent decaffeination, supercritical decaffeination, and biodecaffeination which includes their mechanisms, benefits, and drawbacks as well as their effect in the physicochemical and sensory characteristics of coffee. Solvent decaffeination has showed potential improvements in the future such as the incorporation of membrane and ultrasonic technology. In addition, the mathematical model for caffeine diffusion has been arranged according to Fick’s second law of diffusion, based upon spherical and rectangular coordinates with several assumptions. Further research should be aimed to maintain the properties of coffee after decaffeination process. Furthermore, utilizing new solvents that are safe and non-toxic will potentially be favorable research in the development of decaffeination methods in the future.

Kinetics study on the low-temperature soda roasting of boron-rich blast furnace slag

The method of preparing borax by low-temperature soda roasting and water leaching of boron-rich blast furnace slag (BRBFS) is a novel method for extracting boron from BRBFS. In order to further improve the water leaching rate of boron, this article mainly studied the low-temperature soda roasting kinetics of BRBFS. The effects of roasting temperature and Na2CO3 addition on the water leaching rate of boron were investigated. The results demonstrate that when the amount of Na2CO3 added is four times of the theoretical amount, the kinetics of NaBO2 formation can be described by the Nucleation (Avrami) model in the temperature range of 600–700 °C. The corresponding apparent activation energy is 54.45 kJ/mol, and the apparent frequency factor is 215.16 h−1. It was found that at a roasting temperature of 700 °C,when the amount of sodium carbonate added is twice, three times, and four times of the theoretical amount, the kinetics of NaBO2 formation matches with 3-D Diffusion (Jander) model, Nucleation and Growth (Avrami-Erofeev) model, and Nucleation (Avrami) model, respectively. With an increase in the amount of Na2CO3 added, the rate-controlling step for the formation of NaBO2 transitions from being diffusion-controlled to nucleation-controlled.

Mg-Al layered double hydroxides modified by 2,4,6-trimercapto-s-triazine for effective removal heavy metals from soil

Since 2,4,6-trimercapto-s-triazine (TMT) possesses advantages of multiple adsorption sites and strong binding ability with heavy metals, in this study, layered double hydroxides modified by TMT are prepared (LDHs-TMT) for removing Cu(II)/Pb(II) from contaminated soil. The characterization of structure indicated TMT is intercalated between LDHs layers, and affects growth of crystal structure. The data of kinetics fits pseudo-second-order and intraparticle diffusion models. The isotherms is conformed to Langmuir model, and the adsorption capacity reaches 102.8 mg/g for Cu(II) and 105.8 mg/g for Pb(II). For adsorption mechanism, when concentration of Cu(II)/Pb(II) is low, Cu(II)/Pb(II) is preferentially immobilized by interlayered TMT molecule. As concentration increases, since adsorption sites on TMT are fully occupied, Cu(II)/Pb(II) grab the hydroxy of LDHs and form hydroxide precipitation. The effects of pH, coexistence ions, column leaching and leaching experiments are conducted. The results indicate LDHs-TMT have high adsorption stability and possesses appreciable interception for heavy metal migration.

Comparison of modeling methods for the effective diffusivities of IO3 − estimated in compacted bentonite using through-diffusion tests under aerobic conditions

Abstract Diffusion is the predominant mechanism governing the transport of 129I through geosphere. Consequently, the assessment of the experimental findings on two-compartment diffusion reservoirs of IO3 − within compacted bentonite involved the application of three distinct diffusion models: CC–CC (constant concentration), CC–VC (variable concentration), and VC–VC. To ensure the reliability of the obtained diffusion coefficients, multiple laboratory tests were performed for internal comparison. The experimental results revealed that IO3 − diffusion coefficients were ranging from 3.83 × 10−13 to 1.91 × 10−11 m2/s. These techniques using three mathematical models could be conducted to estimate the diffusion coefficients of non- or weakly-sorbing radionuclides on compacted bentonite for safety assessment of radioactive waste final disposal.

Diffusing on Two Levels and Optimizing for Multiple Properties: A Novel Approach to Generating Molecules with Desirable Properties.

In the past decade, Artificial Intelligence (AI) driven drug design and discovery has been a hot research topic in the AI area, where an important branch is molecule generation by generative models, from GAN-based models and VAE-based models to the latest diffusion-based models. However, most existing models pursue mainly the basic properties like validity and uniqueness of the generated molecules, a few go further to explicitly optimize one single important molecular property (e.g., QED or PlogP), which makes most generated molecules little usefulness in practice. In this paper, we present a novel approach to generating molecules with desirable properties, which expands the diffusion model framework with multiple innovative designs. The novelty is two-fold. On the one hand, considering that the structures of molecules are complex and diverse, and molecular properties are usually determined by some substructures (e.g., pharmacophores), we propose to perform diffusion on two structural levels: molecules and molecular fragments respectively, with which a mixed Gaussian distribution is obtained for the reverse diffusion process. To get desirable molecular fragments, we develop a novel electronic effect based fragmentation method. On the other hand, we introduce two ways to explicitly optimize multiple molecular properties under the diffusion model framework. First, as potential drug molecules must be chemically valid, we optimize molecular validity by an energy-guidance function. Second, since potential drug molecules should be desirable in various properties, we employ a multi-objective mechanism to optimize multiple molecular properties simultaneously. Extensive experiments with two benchmark datasets QM9 and ZINC250k show that the molecules generated by our proposed method have better validity, uniqueness, novelty, Fr´echet ChemNet Distance (FCD), QED, and PlogP than those generated by current SOTA models. The Code of D2L-OMP is available at https://github.com/bz99bz/D2L-OMP.

Inexpensive high fidelity melt pool models in additive manufacturing using generative deep diffusion

Defects in Laser Powder Bed Fusion (L-PBF) parts often result from the meso-scale dynamics of the molten alloy near the laser, known as the melt pool. Experimental in-situ monitoring of the three-dimensional melt pool physical fields is challenging, due to the short length and time scales involved in the process. Multi-physics simulation methods can describe the three-dimensional dynamics of the melt pool, but are computationally expensive at the mesh refinement required for accurate predictions of complex effects. Therefore, we develop a generative deep learning model based on the probabilistic diffusion framework to map low-fidelity simulation information to the high-fidelity counterpart. By doing so, we bypass the computational expense of conducting multiple high-fidelity simulations for analysis by upscaling lightweight coarse mesh simulations. We demonstrate the preservation of key metrics of the melting process between the ground truth simulation data and the diffusion model output, such as the temperature field, the melt pool dimensions and the variability of the keyhole vapor cavity. We predict the melt pool depth within 3 μm based on low-fidelity input data 4× coarser than the high-fidelity simulations, reducing analysis time by two orders of magnitude.

DiffPROTACs is a deep learning-based generator for proteolysis targeting chimeras.

PROteolysis TArgeting Chimeras (PROTACs) has recently emerged as a promising technology. However, the design of rational PROTACs, especially the linker component, remains challenging due to the absence of structure-activity relationships and experimental data. Leveraging the structural characteristics of PROTACs, fragment-based drug design (FBDD) provides a feasible approach for PROTAC research. Concurrently, artificial intelligence-generated content has attracted considerable attention, with diffusion models and Transformers emerging as indispensable tools in this field. In response, we present a new diffusion model, DiffPROTACs, harnessing the power of Transformers to learn and generate new PROTAC linkers based on given ligands. To introduce the essential inductive biases required for molecular generation, we propose the O(3) equivariant graph Transformer module, which augments Transformers with graph neural networks (GNNs), using Transformers to update nodes and GNNs to update the coordinates of PROTAC atoms. DiffPROTACs effectively competes with existing models and achieves comparable performance on two traditional FBDD datasets, ZINC and GEOM. To differentiate the molecular characteristics between PROTACs and traditional small molecules, we fine-tuned the model on our self-built PROTACs dataset, achieving a 93.86% validity rate for generated PROTACs. Additionally, we provide a generated PROTAC database for further research, which can be accessed at https://bailab.siais.shanghaitech.edu.cn/service/DiffPROTACs-generated.tgz. The corresponding code is available at https://github.com/Fenglei104/DiffPROTACs and the server is at https://bailab.siais.shanghaitech.edu.cn/services/diffprotacs.

ConDiff-rPPG: Robust Remote Physiological Measurement to Heterogeneous Occlusions.

Remote photoplethysmography (rPPG) is a contactless technique that facilitates the measurement of physiological signals and cardiac activities through facial video recordings. This approach holds tremendous potential for various applications. However, existing rPPG methods often did not account for different types of occlusions that commonly occur in real-world scenarios, such as temporary movement or actions of humans in videos or dust on camera. The failure to address these occlusions can compromise the accuracy of rPPG algorithms. To address this issue, we proposed a novel Condiff-rPPG to improve the robustness of rPPG measurement facing various occlusions. First, we compressed the damaged face video into a spatio-temporal representation with several types of masks. Second, the diffusion model was designed to recover the missing information with observed values as a condition. Moreover, a novel low-rank decomposition regularization was proposed to eliminate background noise and maximize informative features. ConDiff-rPPG ensured optimization goal consistency during the training process. Through extensive experiments, including intra- and cross-dataset evaluations, as well as ablation tests, we demonstrated the robustness and generalization ability of our proposed model.

Improving the radiographic image analysis of the classic metaphyseal lesion via conditional diffusion models

The classic metaphyseal lesion (CML) is a unique fracture highly specific for infant abuse. This fracture is often subtle in radiographic appearance and commonly occurs in the distal tibia. The development of an automated model that can accurately identify distal tibial radiographs with CMLs is important to assist radiologists in detecting these fractures. However, building such a model typically requires a large and diverse training dataset. To address this problem, we propose a novel diffusion model for data augmentation called masked conditional diffusion model (MaC-DM). In contrast to previous generative models, our approach produces a wide range of realistic-appearing synthetic images of distal tibial radiographs along with their associated segmentation masks. MaC-DM achieves this by incorporating weighted segmentation masks of the distal tibias and CML fracture sites as image conditions for guidance. The augmented images produced by MaC-DM significantly enhance the performance of various commonly used classification models, accurately distinguishing normal distal tibial radiographs from those with CMLs. Additionally, it substantially improves the performance of different segmentation models, accurately labeling areas of the CMLs on distal tibial radiographs. Furthermore, MaC-DM can control the size of the CML fracture in the augmented images.

Cross-domain data fusion generation: A novel composite label-guided generative solution for adaptation diagnosis

Unsupervised domain adaptation (UDA) has been widely studied in the fault diagnosis community, and remarkable success has been witnessed. Nonetheless, existing UDA methods always assumes that data is sufficient in source and target domains. In real industrial scenarios, access to massive amount of data is expensive, especially for the target domain. A direct and effective solution is to expand the amount of the original dataset through the generative models. However, when the available information on the target domain is exceptionally scarce, obtaining a perform-well generative model is significantly challenging. In light of this, we propose a novel composite-label guided cross-domain data fusion generation (CCDFG) method, which trains a latent diffusion model by integrating the data, class labels, pseudo-labels, and domain labels from the source and target domains. Notably, we redesign the classifier-free guidance conditional generative strategy for diffusion models, enhancing the data generation quality of the target domain by borrowing information from the source domain. Subsequently, we present a sample selection strategy based on entropy-oriented prediction uncertainty, aiming to improve the transfer performance of the UDA models by retaining high-value generated data. Comprehensive experiments and evaluations on three mechanical systems demonstrate the effectiveness and application prospects of our method.

A reaction–diffusion model of major emerging infectious diseases in a spatially heterogeneous environment and case study

During the outbreak of major emerging infectious diseases, virus droplets can survive in the environment as aerosols for hours to days, and their impact on human infectious diseases is often overlooked. In addition, the speed of transmission of infectious diseases is often closely related to transportation. Therefore, studying the impact of environmental viruses and transportation on disease development is significant for effective infectious disease prevention and control. We proposed a degenerate reaction–diffusion infectious disease model ([Formula: see text]) considering environmental virus interference and established a well-posedness and threshold system for this model. We have obtained the system solution approaches the disease-free equilibrium ([Formula: see text]) when the basic reproduction number [Formula: see text]. The system has at least one positive steady state solution (PSS) when [Formula: see text]. This paper used a non-standard finite difference method discretization model while data fitting and parameter estimation were performed based on data provided by the Health Commission. Further sensitivity analysis was conducted on [Formula: see text]. At the same time, we also discussed the impact of various parameters in the early stages of the outbreak of major emerging infectious diseases on the development of the disease. Research found that even if the contact rate between people is controlled at a shallow level, the disease may persist. In the early stages of major emerging outbreak of infectious diseases, immediately reducing the use of transportation can effectively reduce the speed of disease spread.