Sub-visible particle analysis using flow imaging microscopy combined with deep learning has proven effective in identifying particle types, enabling the distinction of harmless components such as silicone oil from protein particles. However, the scarcity of available data and severe imbalance between particle types within datasets remain substantial hurdles when applying multi-class classifiers to such problems, often forcing researchers to rely on less effective methods. The aforementioned issue is particularly challenging for particle types that appear unintentionally and in lower numbers, such as silicone oil and air bubbles, as opposed to protein particles, where obtaining large numbers of images through controlled settings is comparatively straightforward. In this work, we develop a state-of-the-art diffusion model to address data imbalance by generating high-fidelity images that can augment training datasets, enabling the effective training of multi-class deep neural networks. We validate this approach by demonstrating that the generated samples closely resemble real particle images in terms of visual quality and structure. To assess the effectiveness of using diffusion-generated images in training datasets, we conduct large-scale experiments on a validation dataset comprising 500,000 protein particle images and demonstrate that this approach improves classification performance with no observable downside. Finally, to promote open research and reproducibility, we publicly release both our diffusion models and the trained multi-class deep neural network classifiers, along with a straightforward interface for easy integration into future studies, at https://github.com/utkuozbulak/svp-generative-ai.

• Introduce energy-conserving neural network closure for turbulence. • Skew-symmetric term redistributes energy; negative definite term dissipates energy. • Outperforms standard machine learning models; delivers accurate long-time LES. • Neural network training procedure consistently yields accurate and stable LES. Machine learning-based closure models for large eddy simulation (LES) have shown promise in capturing complex turbulence dynamics but often suffer from instabilities and physical inconsistencies. In this work, we develop a novel skew-symmetric neural architecture as closure model that enforces stability while preserving key physical conservation laws. Our approach leverages a discretization that ensures mass, momentum, and energy conservation, along with a face-averaging filter to maintain mass conservation in coarse-grained velocity fields. We compare our model against several conventional data-driven closures (including unconstrained convolutional neural networks), and the physics-based Smagorinsky model. Performance is evaluated on decaying turbulence and Kolmogorov flow for multiple coarse-graining factors. In these test cases, we observe that unconstrained machine learning models suffer from numerical instabilities. In contrast, our skew-symmetric model remains stable across all tests, though at the cost of increased dissipation. Despite this trade-off, we demonstrate that our model still outperforms the Smagorinsky model in unseen scenarios. These findings highlight the potential of structure-preserving machine learning closures for reliable long-time LES.

Brain tumour segmentation is a key application of AI in neuroimaging. Recently, federated learning (FL) has emerged as a strategic and increasingly relevant paradigm in neural computing due to its ability to address key challenges in large-scale neural network training, such as data access, privacy, collaborative learning, and model robustness. However, its adoption is currently hindered by high communication costs and the heterogeneity of client data. In this study, we investigated an efficient FL framework for brain tumour segmentation based on communication-aware optimization. We evaluated FedWSOComp, which integrates sparsification, quantization, and entropy-based encoding, in combination with a 3D U-Net architecture under both homogeneous and heterogeneous data distributions. The multi-institutional FeTS 2024 dataset was employed and partitioned into independent and identically distributed (IID) and non-IID settings, with an independent test set of 67 patients. An overall of 18 configurations combined sparsification rates and quantization levels. Performance was measured using Dice Similarity Coefficient (DSC) and 95th percentile Hausdorff Distance (HD95). Experimental results demonstrated that aggressive compression caused severe degradation in segmentation quality, with HD95 exceeding 60 mm. In contrast, higher retention with finer quantization achieved the best balance between efficiency and accuracy, reaching a DSC and HD95 mm on the test set under non-IID conditions. The findings demonstrated that, when configured with moderate-to-fine quantization and high sparsification retention, FedWSOComp enabled accurate and communication-efficient federated brain tumour segmentation. This study provides quantitative evidence and practical guidance for the deployment of FL-based segmentation models in privacy-sensitive and bandwidth-constrained clinical settings. • Analyse the impact of FedWSOComp, an integrated strategy combining top-k sparsification, quantization, and entropy-based encoding. • The performance of 18 different configurations (including IID and non-IID) was evaluated systematically. • High retention (60%) with fine quantization (64 clusters) optimizes performance.

Abstract In complex frictional systems, friction-induced vibration (FIV) and noise are ubiquitous and intricate issues. Achieving high-precision simulation of the vibration response is crucial for the diagnosis of system dynamic properties and vibration control. However, frictional surfaces with multiple contact points introduce nonsmoothness, resulting in unpredictable vibration responses and posing significant challenges for numerical methods to maintain accuracy over long-term analyses. This study proposes a new physics-informed neural network (PINN) method designed to enhance the adaptability between physical constraints and neural network training. The method introduces loss functions with state transition boundary modification (STBM) derived from the physical governing equations. Additionally, a data expansion and regression (DER) strategy for processing linear complementarity problem (LCP) is implemented in the optimizer, significantly improving simulation accuracy for complex stick–slip vibration processes in multicontact frictional systems. By combining these two innovations, the proposed method, referred to as BMDER-PINN, was validated through simulations of stick–slip vibration in a two-degree-of-freedom (2DoF) frictional system. Compared with conventional time-stepping methods, this approach ensures higher accuracy in longer simulations while also enabling large time steps, thereby offering a promising calculation method for improving nonsmooth dynamics simulations.

To date, several Field-Programmed Gate Array (FPGA) implementable computational architectures have been proposed that can be used for neural network training in real-time by the backpropagation algorithm. However, they are intended for small neural networks or have a significant reduction in maximum clock frequency as network sizes increase. The novelty of this work lies in addressing the problems of ensuring a predictable maximum clock frequency and minimizing its degradation when scaling the computational architecture. The proposed architecture solves these problems at the level of computational organization. The architecture comprises an array of computational blocks which are based on FPGA digital signal processing blocks and perform most computations in parallel. The architecture also contains the shared block that sequentially processes the computation results received from the array blocks. The equations were derived showing that the latency of computations increases linearly with neural network sizes. After a computational block instance, the shared block and neural networks containing various numbers of computational blocks had been implemented on the FPGA, their timing characteristics were assessed. It has been determined that the data path delays of the buses connecting the shared block with the array blocks are the primary factors constraining the maximum clock frequencies of neural networks. When the number of the array blocks lies in the range 3–240, the maximum clock frequency is from 112 down to 77 MHz. Compared to the closest counterpart, the critical paths in the proposed architecture are shortened because some computations are transferred to the sequential mode; however, this transfer may increase the latency of calculating the local gradients of the hidden layers neurons. When the number of the array computational blocks grows from 3 to 128, the maximum clock frequency decreases by 27 % compared to 52 % for the closest counterpart. Growing the number of computational blocks in the proposed architecture from 128 to 240 reduces the maximum clock frequency by no more than 5 %. FPGA-based neural networks of the proposed architecture are suitable for object tracking and system identification, which are typical applications of neural networks trained in real-time mode.

The reliability of machines largely depends on the accuracy of predicting the stress–strain state of components in tribofatigue systems, especially under high operating loads. Traditional finite element analysis provides high accuracy but requires significant computational resources and offers limited flexibility for rapid parameter variation. In recent years, machine learning methods have been increasingly applied in engineering practice. Among them, neural networks are of particular interest, as they allow nonlinear relationships between loads and stresses to be captured while significantly reducing computation time compared to traditional models. This work proposes an approach for predicting maximum stresses in the “shaft–insert” system by combining three-dimensional finite element modeling with subsequent neural network training. A database was created containing the results of numerical experiments for different combinations of bending and contact loads. A fully connected neural network with three hidden layers and different activation functions was used for training. The quality of the model was assessed using standard metrics: Mean Squared Error, Mean Absolute Error (MAE), and the coefficient of determination R2. The trained neural network demonstrated high accuracy in predicting maximum stresses both in the shaft and in the insert. For the training set, the R2 value reached 0.99991, and for the test set it was 0.99984, confirming minimal deviations from finite element results. The MAE was less than 0.006, while the maximum relative error in the test set did not exceed 3.2 %. The developed neural network model demonstrated the ability to reproduce the results of finite element analysis for the “shaft–insert” system while providing a substantial reduction in computation time compared to traditional finite element simulations. The model was constructed for a limited range of loads; therefore, further research should focus on expanding the dataset and including additional materials, which will make it possible to evaluate the scalability of the approach and its robustness under more complex conditions.

Abstract Recent efforts to improve the efficiency of neuromorphic and machine learning systems have centred on developing of specialised hardware for neural networks. These systems typically feature architectures that go beyond the von Neumann model employed in general-purpose hardware such as GPUs, offering potential efficiency and performance gains. However, neural networks developed for specialised hardware must consider its specific characteristics. This requires novel training algorithms and accurate hardware models, since they cannot be abstracted as a general-purpose computing platform. In this work, we present a bottom-up approach to training neural networks for hardware-based spiking neurons and synapses, built using ferroelectric capacitors (FeCAPs) and resistive random-access memories (RRAMs), respectively. Unlike the common approach of designing hardware to fit abstract neuron or synapse models, we start with compact models of the physical device to model the computational primitives. Based on these models, we have developed a training algorithm (BRUNO) that can reliably train the networks, even when applying hardware limitations, such as stochasticity or low bit precision. We analyse and compare BRUNO with Backpropagation Through Time. We test it on different spatio-temporal datasets. First on a music prediction dataset, where a network composed of ferroelectric leaky integrate-and-fire (FeLIF) neurons is used to predict at each time step the next musical note that should be played. The second dataset consists on the classification of the Braille letters using a network composed of quantised RRAM synapses and FeLIF neurons. The performance of this network is then compared with that of networks composed of LIF neurons. Experimental results show the potential advantages of using BRUNO by reducing the time and memory required to detect spatio-temporal patterns with quantised synapses.

Over the years, Music Information Retrieval (MIR) research community has released various models pretrained on large amounts of music data. Transfer learning showcases the proven effectiveness of pretrained backend models for a broad spectrum of downstream tasks, including auto-tagging and genre classification. However, MIR papers generally do not explore the efficiency of pretrained models for Music Recommender Systems (MRS). In addition, the Recommender Systems community tends to favour traditional end-to-end neural network training. Our research addresses this gap and evaluates the performance of nine pretrained backend models (MusicFM, Music2Vec, MERT, EncodecMAE, Jukebox, MusiCNN, MULE, MuQ and MuQ-MuLan) in the context of MRS. We assess them using five recommendation approaches: K-Nearest Neighbours (KNN), Shallow Neural Network, Contrastive Multi-Modal projection, a Hybrid model, and BERT4Rec both for the hot and cold-start scenarios. Our findings suggest that pretrained audio representations exhibit significant performance disparity between traditional MIR tasks and both hot and cold music recommendations, indicating that valuable aspects of musical information captured by backend models may differ depending on the task. This study establishes a foundation for further exploration of pretrained audio representations to enhance music recommendation systems.

This study addresses challenges in fluid catalytic cracking (FCC) units, including inaccurate quantification of carbon emissions, difficulties in implementing low-carbon operational optimization, and low computational efficiency in solving complex process kinetics. A low-carbon operation optimization method based on physics-informed neural networks (PINNs) is proposed. First, a unit-level carbon footprint assessment model is established using process life cycle assessment (PLCA) to achieve high-resolution quantification of both direct and indirect carbon emissions. Second, a multi-objective low-carbon operation optimization model is developed considering carbon tax scenarios, incorporating carbon emissions and corresponding carbon tax costs into the optimization objectives to achieve economic and low-carbon synergistic optimization. Finally, a PINN-assisted surrogate model is designed by embedding material balance constraints into the neural network training process, enabling efficient approximation of complex product yield kinetics and improving optimization solution efficiency and predictive reliability. The proposed method is applied to optimize the operation of an FCC unit at a refinery site. The results indicate an increase of 12,048.851 CNY/h in profit, a reduction of 1088.921 kgCO2eq/h in CO2 emissions, and a decrease of 324.281 m3/h in steam consumption. Meanwhile, the PINN model exhibits excellent performance in product yield prediction, with an average R2 of 0.9968 and an average RMSE of 0.1482, outperforming conventional data-driven methods. The proposed approach balances carbon emission quantification accuracy, physical consistency in yield prediction, and optimization solution efficiency, providing a systematic and implementable technical framework for low-carbon operation optimization of FCC units.

The explosive growth of temporal graph data has led to significant training overheads for Dynamic Graph Neural Networks (DGNNs), a bottleneck primarily driven by massive data movement between host processors and storage arrays across conventional PCIe I/O buses. While near-data processing with Computational Storage Devices (CSDs) can alleviate this bottleneck, a single CSD is inherently incapable of meeting the terabyte-scale capacity requirements and complex sequence modeling demands of modern large-scale DGNNs. Horizontal scaling with multi-CSD clusters over standard PCIe topologies presents a viable, cost-effective solution, yet our in-depth profiling identifies two critical architectural bottlenecks in naive multi-CSD architectures: host-bounced memory copies significantly compromise inter-device communication efficiency, and sparse graph sampling frequently exceeds the capacity of the tightly constrained local DRAM of CSDs, resulting in excessive flash I/O and performance degradation. To address these interconnected bottlenecks, we propose M-DGNN, a hardware–software co-designed architecture optimized for standard PCIe interconnects. First, M-DGNN orchestrates direct peer-to-peer (P2P) DMA dataflows for inter-CSD hidden state exchange, completely bypassing host operating system intervention and reducing the context-switching overhead. Second, we design a host-assisted caching strategy with a Host-Pinned Memory Extension (HPME) mechanism, which leverages host-pinned memory as an asynchronous DMA extension pool to shield resource-constrained CSDs from high-latency flash I/O during structural subgraph sampling. Extensive experimental evaluations across seven large-scale dynamic graph datasets demonstrate that M-DGNN delivers up to a 6.2× end-to-end speedup over the state-of-the-art DGNN systems. This work establishes an efficient, scalable near-data computing paradigm for large-scale DGNN training.

Neural network training relies on gradient computation through backpropagation, yet memory requirements for storing layer activations present significant scalability challenges. We present the first adaptation of control-theoretic matrix sketching to neural network layer activations, enabling memory-efficient gradient reconstruction in backpropagation. This work builds on recent matrix sketching frameworks for dynamic optimization problems, where similar state trajectory storage challenges motivate sketching techniques. Our approach sketches layer activations using three complementary sketch matrices maintained through exponential moving averages with adaptive rank adjustment, automatically balancing memory efficiency against approximation quality. Empirical evaluation on MNIST, CIFAR-10, and physics-informed neural networks demonstrates a controllable accuracy-memory tradeoff. We demonstrate a gradient monitoring application on MNIST showing how sketched activations enable real-time gradient norm tracking with minimal memory overhead. These results establish that sketched activation storage provides a viable path toward memory-efficient neural network training and analysis.

Abstract Can regularization terms in the training of invertible neural networks lead to known Bayesian point estimators in reconstruction? Invertible networks are attractive for inverse problems due to their inherent stability and interpretability. Recently, optimization strategies for invertible neural networks that approximate either a reconstruction map or the forward operator have been studied from a Bayesian perspective, but each has limitations. To address this, we introduce and analyze two regularization terms for the network training that, upon inversion of the network, recover properties of classical Bayesian point estimators: while the first can be connected to the posterior mean, the second resembles the MAP estimator. Our theoretical analysis characterizes how each loss shapes both the learned forward operator and its inverse reconstruction map. Numerical experiments support our findings and demonstrate how these loss‑term regularizers introduce data-dependence in a stable and interpretable way.

Optoelectronic devices that unify sensing, memory, and computation offer a promising route toward intelligent and data-local edge systems. Here, a multifunctional metal-semiconductor-metal neuromorphic photodetector based on the persistent photoconductivity (PPC) of κ-phase gallium oxide (κ-Ga2O3) is reported, enabling in-sensor information processing and long-term state retention within a single device element. Owing to pronounced PPC effect, nominally identical devices exhibit reproducible yet device-distinguishable temporal photocurrent responses. These responses are exploited for hardware-level authentication using a hybrid 1D deep embedding network, which achieves robust cross-cycle verification performance with an Area Under the Curve (AUC) of about 0.97 and an Equal Error Rate (EER) of about 9%. Beyond authentication, the neuromorphic inference capability of the devices is evaluated using a hardware-aware simulation framework, in which experimentally extracted conductance states are mapped to a quantization-aware trained (QAT) artificial neural network (ANN) with 16 discrete levels. The quantized network achieves 98.17% accuracy and is subsequently converted into a leaky integrate-and-fire (LIF) spiking neural network (SNN), retaining 96.80% accuracy under device-constrained operation. By performing sensing, authentication, and inference at device level, the κ-Ga2O3 synaptic photodetectors establish a materials-enabled pathway toward compact, intelligent, and privacy-enhancing optoelectronic hardware for next-generation edge systems.

Abstract Accurate modeling of hydrometeor bulk scattering properties (BSPs) is essential for the effective assimilation of satellite microwave observations in cloudy and precipitating conditions. Nevertheless, current radiative transfer models use oversimplified hydrometeor BSP parameterizations, leading to significant simulation errors and biases that limit the full potential of all‐sky assimilation. To address this challenge, this study developed PhySCAT‐Net, a physics‐informed deep learning (DL) framework that integrates forward and Jacobian operators of a physical radiative transfer model into the neural network training, enabling efficient optimization of the BSP models against satellite observations. Applied to vertically and horizontally polarized radiances from the Global Precipitation Measurement Microwave Imager 166.5 GHz channels, the framework selectively fine‐tunes the snow BSP model while temporarily fixing other hydrometeor types. Results demonstrate that the optimized DL model substantially improves agreement between simulated and observed brightness temperatures. Across most regions globally, mean observation‐minus‐background (O‐B) biases are reduced to within ±1K, and the Jensen‐Shannon divergence decreases by orders of magnitude. The error distributions, which were previously highly skewed and therefore problematic for data assimilation, are now roughly symmetrical. Furthermore, PhySCAT‐Net enables the DL model to extract polarimetric information of non‐spherical ice particles directly from observed radiances, demonstrating superior performance compared to existing empirical schemes. It successfully reproduces the distributions of polarization differences and their non‐monotonic relationship with brightness temperature.

Physics-informed neural networks in water and wastewater systems: a critical review.

This paper presents a feed-forward neural network (NN) for hosting capacity analysis (HCA) in real-time using information from a limited number of buses, addressing the challenge of data scarcity and the need for efficient HCA with minimal input data. A limited number of buses in the distribution network is identified, referred to as pilot buses, using an approach that maximizes observability and controllability while minimizing voltage deviations at load buses to ensure network robustness. A large number of loading and electrical generation scenarios are generated using Monte Carlo simulation to train the NN with voltage information as inputs and HC as output. This trained NN can then predict the hosting capacity for distribution networks within seconds with new inputs, offering a highly efficient and practical solution for distribution utilities. The effectiveness of the proposed approach is demonstrated on the IEEE 123 bus distribution network and a rural feeder in New South Wales, Australia. Results show that the model predicts hosting capacity in 0.04 s for the IEEE 123 network and 0.18 s for the real network, with an average error of less than 1%, showcasing its accuracy and reliability. This rapid and precise prediction capability enhances distribution network management and operational efficiency for utilities. • A feed-forward neural network is proposed for quick hosting capacity analysis. • Only voltage information from pilot buses is used to train the neural network. • The method is tested on the IEEE 123 bus system and a rural feeder in Australia. • The proposed method shows promising results and requires fewer inputs.

Explainable machine learning algorithm for classifying resting-state functional MRI in amyotrophic lateral sclerosis.

Abstract Reconstructing the plasma state is a central challenge in nuclear fusion experiments, as it is essential for understanding and predicting plasma behaviour. Physics-Informed Neural Networks (PINNs), especially when combined with a multi-diagnostic approach, offer powerful advantages for addressing this problem. PINNs embed the governing physical laws directly into the learning process through differential equation constraints, enabling them to integrate sparse or noisy measurements while maintaining physical consistency. This makes them particularly suitable for equilibrium reconstruction, where they can incorporate diagnostic data as boundary conditions and naturally enforce the structure of the magnetohydrodynamic equations. Moreover, the use of multiple diagnostics helps over-constrain the system, reducing uncertainties and mitigating the illposedness characteristic of the plasma core region. Starting from results obtained in previous works where the capabilities of multi-diagnostics equilibrium reconstruction through PINNs were demonstrated, in this work we perform several parametric studies to optimise both the neural network architecture and the training procedure. We examine the impact of automatically adjusting the relative weights between data and physics losses during training, the role of specialised physics-based network's layers informed by prior knowledge or plasma state hypotheses, the choice of hidden layers' activation function, and the benefits of initialising training from a pre-trained network. These analyses provide guidelines for designing the most effective neural network and training strategy for specific plasma conditions.

This paper presents a methodology that employs inductive spatial geometric deep learning networks to detect multiple avian vocalizations from field recordings. Initially, a graph is constructed from the Mel-spectrogram of each audio file using a trained deep convolutional neural network (Deep CNN). The extracted features are used to build a node-feature graph, which is then processed by two spatial inductive graph-based models: graph sample and aggregation (GraphSAGE) and the graph attention network (GAT), for multi-label classification. To enhance the robustness and generalization of the Deep CNN, SpecAugment is applied to generate additional Mel-spectrograms via data augmentation. The proposed framework is evaluated on the Xeno-canto bird sound database and compared against state-of-the-art methods. The results demonstrate that the proposed inductive spatial graph-based approach outperforms existing techniques, achieving macro F1-scores of 0.90 with GraphSAGE and 0.92 with GAT. We further replaced Deep CNN with AudioProtoPNet-20 and evaluated GAT on the Xeno-canto dataset, obtaining a macro F1-score of 0.93.

Synthetically trained convolutional neural networks for time-resolved aortic segmentation of 4D flow MRI.