Constructive Neural Network Research Articles

Diffraction‐Driven Parallel Convolution Processing with Integrated Photonics

AbstractTraditional electronic processors often struggle with bandwidth limitations and high power consumption when executing extensive linear operations for deep learning tasks. Optical computing has emerged as a promising alternative, offering parallel and energy‐efficient computation capabilities. Yet, the development of high‐density optical computing architectures on integrated photonic platforms remains limited, hindered by constraints in neuron scalability and control engineering complexities. Addressing these challenges, this work presents a diffraction‐driven multi‐kernel optical convolution unit (MOCU) that enables on‐chip parallel convolution processing. By utilizing cascaded silica 1D metalines as pre‐trained large‐scale weights and employing spatial multiplexing at the output, MOCU allows simultaneous passive computation of diverse convolutions within a single unit. This architecture facilitates the construction of optical convolutional neural networks (OCNNs), enabling efficient machine vision processing with a streamlined design. To mitigate errors in MOCU‐embedded OCNNs, a lightweight electronic neural network operates concurrently to calibrate systematic deviations via a low‐rank adaptation (LoRA) algorithm, with minimal overhead. The fabricated MOCU chip demonstrates the highest independent 8‐kernel convolutions in parallel, each with a kernel size and occupying just 0.06 . This architecture effectively merges photonic and electronic technologies, offering a scalable design pathway for energy‐efficient, high‐density deep learning hardware.

Identification and Analysis of Potential Biomarkers Associated with Neutrophil Extracellular Traps in Cervicitis.

Early diagnosis of cervicitis is important. Previous studies have found that neutrophil extracellular traps (NETs) play pro-inflammatory and anti-inflammatory roles in many diseases, suggesting that they may be involved in the inflammation of the uterine cervix and NETs-related genes may serve as biomarkers of cervicitis. However, what NETs-related genes are associated with cervicitis remains to be determined. Transcriptome analysis was performed using samples of exfoliated cervical cells from 15 patients with cervicitis and 15 patients without cervicitis as the control group. First, the intersection of differentially expressed genes (DEGs) and neutrophil extracellular trap-related genes (NETRGs) were taken to obtain genes, followed by functional enrichment analysis. We obtained hub genes through two machine learning algorithms. We then performed Artificial Neural Network (ANN) and nomogram construction, confusion matrix, receiver operating characteristic (ROC), gene set enrichment analysis (GSEA), and immune cell infiltration analysis. Moreover, we constructed ceRNA network, mRNA-transcription factor (TF) network, and hub genes-drug network. We obtained 19 intersecting genes by intersecting 1398 DEGs and 136 NETRGs. 5 hub genes were obtained through 2 machine learning algorithms, namely PKM, ATG7, CTSG, RIPK3, and ENO1. Confusion matrix and ROC curve evaluation ANN model showed high accuracy and stability. A nomogram containing the 5 hub genes was established to assess the disease rate in patients. The correlation analysis revealed that the expression of ATG7 was synergistic with RIPK3. The GSEA showed that most of the hub genes were related to ECM receptor interactions. It was predicted that the ceRNA network contained 2 hub genes, 3 targeted miRNAs, and 27 targeted lnRNAs, and that 5 mRNAs were regulated by 28 TFs. In addition, 36 small molecule drugs that target hub genes may improve the treatment of cervicitis. In this study, five hub genes (PKM, ATG7, CTSG, RIPK3, ENO1) provided new directions for the diagnosis and treatment of patients with cervicitis.

Experimental and ANN modeling of kerosene fuel desulfurization using a manganese oxide-tin oxide catalyst

This research pioneered the use of a nanocatalyst composed of manganese oxide (MnO2) and stannic oxide (SnO2) to effectively remove dibenzothiophene (DBT) from kerosene fuel through the catalytic oxidative desulfurization (ODS) process, using hydrogen peroxide (H2O2) as the oxidant. Impregnating SnO2 with varying amounts of MnO2 was used to manufacture the catalyst. The oxidation experiment ran in a batch reactor with varying reaction times and temperatures to determine optimal conditions. High MnO2 dispersion over SnO2 was shown by catalyst characterization data. Under optimal operating parameters (catalyst type: 5 % MnO2/SnO2, reaction temperature: 75 °C, and reaction duration: 100 min), the results demonstrated a maximum DBT removal efficiency of 82.84 % from kerosene fuel. This research also provides the construction of Artificial Neural Network (ANN) model to simulate the upgrading of kerosene fuel via desulfurization process. There has been a growing trend toward the diversified use of ANN to represent steady state systems in chemical engineering. MATLAB's code was employed for matching the experimental data to the artificial neural network (ANN) model. The resulted data showed significant agreement between the experimental and predicted outcomes, with regression coefficients (R2) of 0.99902, 0.99986, and 0.99961 and mean square errors (MSE) of 0.266, 0.272, and 0.104 for 0 % MnO2/SnO2, 1 % MnO2/SnO2, and 5 % MnO2/SnO2 respectively. This interactive model provided a solid foundation for understanding the novel behavior of the oxidation process.

On the construction of an artificial neural network for solving a system of equations Navier–Stokes in the case of incompressible fluid

On the construction of an artificial neural network for solving a system of equations Navier–Stokes in the case of incompressible fluid

Identification of potential biomarkers and drug of ischemic stroke in patients with COVID-19 through machine learning

The relationship between COVID-19 and ischemic stroke (IS) has attracted significant attention, yet the precise mechanism at the gene level remains unclear. This study aims to reveal potential biomarkers and drugs for COVID-19-related IS through bioinformatics methods. We collected two gene expression profiling datasets, GSE16561 and GSE213313, and selected GSE179879 and GSE196822 as validation sets for analysis. Through analysis, we identified 77 differentially expressed genes (DEGs) shared between COVID-19 and IS. Further gene enrichment analysis revealed that these genes are primarily involved in immune regulation. By constructing a protein-protein interaction network, we screened out nine hub genes, including FCGR3A, KLRB1, IL2RB, CD2, IL7R, CCR7, CD3D, GZMK, and ITK. In LASSO regression analysis, we evaluated the ROC curve's area under the curve (AUC) scores of key genes to assess their diagnostic accuracy. Subsequently, we performed random forest (RF), Support Vector Machine Recursive Feature Elimination (SVM-RFE), and neural network construction on hub genes to ensure accurate diagnosis of IS. Finally, by intersecting the results of three algorithms (LASSO regression, random forest, and SVM), CD3D and ITK were identified as the ultimate key genes. Based on this, we predicted potential targeted drug Blinatumomab. These research findings provide clues for a deeper understanding of the biological mechanisms of COVID-19-related IS and offer new insights for exploring novel treatment approaches.

Optimal strategies design of active target defense differential game

In this paper, the optimal strategies are investigated for the differential game of active target defense. Specifically, we describe a pursuitevasion game with three agents (namely, an attacker, a target and a defender). For the three agents engagement scenario, two different differential game guidance laws–pure pursuit (PP) and proportional navigation (PN)–are presented for defender. Firstly, the relative motion kinematic models of the three agents are established. We subsequently derive the coupled Hamilton-Jacobi (HJ) equations to design the optimal control strategies for the three agents. Through the construction of actor-critic neural networks, optimal solutions are obtained. Then, the stability of the three-agent system is ensured, and the convergence to the Nash equilibrium equations is demonstrated. Finally, we simulate the optimal strategies employed by the attacker and the defender-target team, thereby highlighting the contrasting outcomes and superiority of different defensive approaches.

Gate-based quantum neurons in hybrid neural networks

Abstract Quantum computing is conceived as a promising and powerful next-generation platform for information processing and it has been shown that it could bring significant accelerations to certain tasks, compared to its classical counterparts. With recent advances in noisy intermediate-scale quantum (NISQ) devices, we can process classical data from real-world problems using hybrid quantum systems. In this work, we investigate the critical problem of designing a gate-based hybrid quantum neuron under NISQ constraints to enable the construction of scalable hybrid quantum deep neural networks (HQDNNs). We explore and characterize diverse quantum circuits for hybrid quantum neurons and discuss related critical components of HQDNNs. We also utilize a new schema to infer multiple predictions from a single hybrid neuron. We further compose a highly customizable platform for simulating HQDNNs via Qiskit and test them on diverse classification problems including the iris and the wheat seed datasets. The results show that even HQDNNs with the simplest neurons could lead to superior performance on these tasks. Finally, we show that the HQDNNs are robust to certain levels of noise, making them preferred on NISQ devices. Our work provides a comprehensive investigation of building scalable near-term gate-based HQDNNs and paves the way for future studies of quantum deep learning via both simulations on classical computers and experiments on accessible NISQ devices.

Better approximation of sigmoid function for privacy-preserving neural networks

Abstract With the development of artificial intelligence technology, privacy protection has become a prominent research direction. Homomorphic encryption is an important and promising technology in the construction of privacy-preserving neural networks. However, computing nonlinear activation functions remains a challenging task in the context of homomorphic encryption. Considering Sigmoid is one of the most commonly used activation functions, this paper proposes a polynomial approximation method for the Sigmoid function and applies it to the homomorphic encryption scheme of CKKS. We design polynomial approximation functions using Improved Taylor Expansion Segmentation (ITES), Least Squares method (LS), and Random Noise Addition method, respectively. These methods are implemented through the CKKS scheme in the homomorphic encryption library of SEAL. The experimental results show that the ITES achieves high fitting accuracy, reducing the average absolute error by 11.09% and the mean squared error by 38.06% compared to previous fitting methods. Therefore, experiments have identified methods with better fitting results in the CKKS scheme, which can meet the requirements of using homomorphic encryption in deep learning technologies. Finally, our analysis points out that different technologies have unique advantages and adapt to different scenarios, e.g., ITES has higher accuracy, LS has higher performance in scenarios with limited computing resources.

Construction of an Artificial Neural Network for Solving the Incompressible Navier–Stokes Equations

Construction of an Artificial Neural Network for Solving the Incompressible Navier–Stokes Equations

Hypergraph clustering based multi-label cross-modal retrieval

Most existing cross-modal retrieval methods face challenges in establishing semantic connections between different modalities due to inherent heterogeneity among them. To establish semantic connections between different modalities and align relevant semantic features across modalities, so as to fully capture important information within the same modality, this paper considers the superiority of hypergraph in representing higher-order relationships, and proposes an image-text retrieval method based on hypergraph clustering. Specifically, we construct hypergraphs to capture feature relationships within image and text modalities, as well as between image and text. This allows us to effectively model complex relationships between features of different modalities and explore the semantic connectivity within and across modalities. To compensate for potential semantic feature loss during the construction of the hypergraph neural network, we design a weight-adaptive coarse and fine-grained feature fusion module for semantic supplementation. Comprehensive experimental results on three common datasets demonstrate the effectiveness of the proposed method.

Machine learning-based predictive control of an electrically-heated steam methane reforming process

Hydrogen plays a crucial role in improving sustainability and offering a clean and efficient energy carrier that significantly reduces greenhouse gas emissions. However, the primary method of industrial hydrogen production, steam methane reforming (SMR), relies on the combustion of hydrocarbons as the heating source for the reforming reactions, resulting in significant carbon emissions. To address this issue, an experimental setup of an electrically-heated steam methane reformer (e-SMR) has been constructed at UCLA, and a lumped first-principle dynamic process model was built based on parameters estimated from the experimental data in a previous study. Subsequently, the first-principle dynamic process model was implemented into the computational model predictive control (MPC) scheme, successfully driving the hydrogen production rate to the desired setpoint. While these works are important and pave the way for developing MPC for large-scale e-SMR processes, the first-principle process model may not accurately reflect the actual process behavior, particularly as the process behavior changes with time. Therefore, the development and establishment of an adaptive data-driven approach for implementing model predictive control in the e-SMR process is necessary. To address this need, the present work investigates the construction of recurrent neural network (RNN) models for an e-SMR process in-depth, utilizing data from an experimentally-validated first-principle model. Specifically, a long short-term memory (LSTM) layer was utilized in the RNN model to effectively capture the complex correlations present in long-term sequential data. Subsequently, this LSTM-based RNN process model was employed to design an MPC, and its performance was evaluated through comparison with proportional–integral (PI) control. To address potential disturbances and variability in a typical e-SMR process, three distinct approaches were developed: MPC with an integrator, MPC with real-time online retraining (transfer learning), and offset-free MPC. These approaches effectively eliminated the offset caused by disturbances. Overall, this study underscores the effectiveness of utilizing RNN models to capture process dynamics in an experimental e-SMR process. It also outlines strategies for employing RNN-based control and multiple approaches to address disturbances in general processes with partially infrequent and delayed measurement feedback. This approach is particularly valuable in scenarios where developing first-principle models for a new process may be challenging.

Complex Dynamical Behavior of Locally Active Discrete Memristor-Coupled Neural Networks with Synaptic Crosstalk: Attractor Coexistence and Reentrant Feigenbaum Trees

In continuous neural modeling, memristor coupling has been investigated widely. Yet, there is little research on discrete neural networks in the field. Discrete models with synaptic crosstalk are even less common. In this paper, two locally active discrete memristors are used to couple two discrete Aihara neurons to form a map called DMCAN. Then, the synapse is modeled using a discrete memristor and the DMCAN map with crosstalk is constructed. The DMCAN map is investigated using phase diagram, chaotic sequence, Lyapunov exponent spectrum (LEs) and bifurcation diagrams (BD). Its rich and complex dynamical behavior, which includes attractor coexistence, state transfer, Feigenbaum trees, and complexity, is systematically analyzed. In addition, the DMCAN map is implemented in hardware on a DSP platform. Numerical simulations are further validated for correctness. Numerical and experimental findings show that the synaptic connections of neurons can be modeled by discrete memristor coupling which leads to the construction of more complicated discrete neural networks.

Assessment of the industrial policy to the military-industrial complex effectiveness based on neural networks based on fuzzy logic

The relevance of the topic under consideration on the development of universal applied methods for assessing the effectiveness and efficiency of industrial policy in the military-industrial complex (hereinafter - MIC) is due to its high practical significance in the context of the challenges of technological development associated with digitalization and informatization. It is also due to the peculiarities of the current stage of Russia’s economic development in the conditions of financial and technological sanctions from Western states. The purpose of the study is to develop the main provisions of a methodological approach to assessing the effectiveness of industrial policy in the defense industry based on the use of economic-mathematical modeling apparatus, which involves the construction of neural networks based on fuzzy logic. In the course of the research the authors have solved the following problems: mathematically formalized the object of analysis; developed an algorithm for determining the effectiveness of industrial policy in the MIC using neural networks; formalized the model for assessing the effectiveness of such industrial policy based on fuzzy sets; proposed a system of indicators for assessing industrial policy in the MIC; determined the sequence of actions in the construction the fuzzy model for assessing the effectiveness of industrial policy in the MIC by means of Fuzzy Logic on the software platform MatLab.

ConGCNet: Convex geometric constructive neural network for Industrial Internet of Things

The intersection of the Industrial Internet of Things (IIoT) and artificial intelligence (AI) has garnered ever-increasing attention and research interest. Nevertheless, the dilemma between the strict resource-constrained nature of IIoT devices and the extensive resource demands of AI has not yet been fully addressed with a comprehensive solution. Taking advantage of the lightweight constructive neural network (LightGCNet) in developing fast learner models for IIoT, a convex geometric constructive neural network with a low-complexity control strategy, namely, ConGCNet, is proposed in this article via convex optimization and matrix theory, which enhances the convergence rate and reduces the computational consumption in comparison with LightGCNet. Firstly, a low-complexity control strategy is proposed to reduce the computational consumption during the hidden parameters training process. Secondly, a novel output weights evaluated method based on convex optimization is proposed to guarantee the convergence rate. Finally, the universal approximation property of ConGCNet is proved by the low-complexity control strategy and convex output weights evaluated method. Simulation results, including four benchmark datasets and the real-world ore grinding process, demonstrate that ConGCNet effectively reduces computational consumption in the modelling process and improves the model’s convergence rate.

Symmetry-enforcing neural networks with applications to constitutive modeling

The use of machine learning techniques to homogenize the effective behavior of arbitrary microstructures has been shown to be not only efficient but also accurate. In a recent work, we demonstrated how to combine state-of-the-art micromechanical modeling and advanced machine learning techniques to homogenize complex microstructures exhibiting non-linear and history dependent behaviors (Logarzo et al., 2021). The resulting homogenized model, termed smart constitutive law (SCL), enables the adoption of microstructurally informed constitutive laws into finite element solvers at a fraction of the computational cost required by traditional concurrent multiscale approaches. In this work, the capabilities of SCLs are expanded via the introduction of a novel methodology that enforces material symmetries at the neuron level, applicable across various neural network architectures. This approach utilizes tensor-based features in neural networks, facilitating the concise and accurate representation of symmetry-preserving operations, and is general enough to be extend to problems beyond constitutive modeling. Details on the construction of these tensor-based neural networks and their application in learning constitutive laws are presented for both elastic and inelastic materials. The superiority of this approach over traditional neural networks is demonstrated in scenarios with limited data and strong symmetries, through comprehensive testing on various materials, including isotropic neo-Hookean materials and tensegrity lattice metamaterials. This work is concluded by a discussion on the potential of this methodology to discover symmetry bases in materials and by an outline of future research directions.

Development Artificial Neural Network (ANN) computing model to analyses men's 100¬meter sprint performance trends

Coaches and analysts face a significant challenge of inaccurate estimation when analyzing Men's 100 Meter Sprint Performance, particularly when there is limited data available. This necessitates the use of modern technologies to address the problem of inaccurate estimation. Unfortunately, current methods used to estimate Men's 100 Meter Sprint Performance indexes in Iraq are ineffective, highlighting the need to adopt new and advanced technologies that are fast, accurate, and flexible. Therefore, the objective of this study was to utilize an advanced method known as artificial neural networks to estimate four key indexes: Accelerate First of 10 meters, Speed Rate, Time First of 10 meters, and Reaction Time. The application of artificial neural networks in the sports industry in the Republic of Iraq is crucial to ensure successful players. In this study, an artificial neural network model was built to predict Men's 100 meter indexes. Several factors related to the construction of artificial neural networks were studied, including network architecture and internal factors and their impact on the performance of the models. As a result, four easy equations were developed to calculate the four key indexes. The findings of the study indicate that these networks can predict Men's 100 Meter indexes with a high degree of reliability 98.034% and accounting coefficients R = 0.9143.

Efficient construction and convergence analysis of sparse convolutional neural networks

In this paper, a new variant of convolutional neural network (CNN) based on L1 regularization is proposed. The main consideration is to generate a sparse weight matrix, i.e., to generate a sparse neural network model. To some extent, L1 regularization prevents the occurrence of overfitting and effectively improves the learning efficiency of the network. Unfortunately, the L1 regularization term is non-smooth, which leads to the occurrence of oscillations in experiments and poses a great challenge to the theoretical analysis. So, we overcome this drawback by using polishing techniques and rigorously prove the monotonicity of the error function and the strong and weak convergence theorems of the algorithm. Finally, numerical simulations on several data sets support our theoretical results and the superiority of the proposed algorithm.

Biological neuron modeling based on bifunctional memristor and its application in secure communication

In recent years, memristors have shown great potential in building artificial neural networks. Many researchers have worked on improving and innovating the design of memristors. The bifunctional memristor proposed in this paper provides new ideas for establishing artificial neural networks. And a novel second-order memristor Hindmarsh–Rose neuron (SOM-HR) is constructed on this basis. The SOM-HR model can well simulate the dynamical behavior of the memristive autapse HR neuron under electromagnetic radiation (EMR), and many types of firing patterns, coexisting firing patterns and state transition are found. In addition, the correctness of the numerical simulation is verified by circuit simulation and digital signal processor (DSP) implementation. Finally, the chaotic sequences generated by this model are used for image encryption, which guarantees the security of the encryption scheme. The results of the study provide a reference for construction and application of complex neural network.

Novel process optimization based on machine learning: A study on biohydrogen production from waste resources

The biomass poultry litter and sewage sludge co-gasification is a good thermochemical method to produce hydrogen energy meanwhile to mitigate the consumption of fossil fuels. However, the operation optimization in the complex process system is important while computationally difficult because of high nonlinearity and many first-principle constraints. To address the optimization of this biohydrogen production process, a methodology framework for achieving the optimal operations is thus presented. The complete process system is first simulated and decomposed into upstream gasification process and downstream hydrogen purification for reducing computational complexity through the thermodynamic equilibrium-based simulation. Totally 1400 data points by pairing total 15 operating conditions with a composite sustainability index objective are generated through the random sampling and data classification strategies, which are then used for the construction of the artificial neural network (ANN)-based prediction models. ANN models of two subprocesses demonstrate the satisfactory prediction accuracy with R2 value of 0.98 and 0.99, respectively, which are then integrated with mixed-integer linear programming (MILP) for the optimization of upstream process and downstream process step-by-step. The MILP problems based on the ANN models are solved with lower optimization time of 45∼100 s compared to the heuristic algorithm optimization (3–6 h) based on the thermodynamic equilibrium-based simulation. The optimal sustainability index values of two processes are 0.80 and 0.91 which are both improved compared to the existing optimization results (0.79 and 0.84). This study emphasizes the optimization potential of the integration approach of machine learning-based modelling and mathematical programming for developing the optimal waste-to-energy processes.

Construction of artificial neural network (ANN) based on predictive value of prognostic nutritional index (PNI) and neutrophil-to-lymphocyte ratio (NLR) in patients with cervical squamous cell carcinoma.

To explore the analytical worth of prognostic nutritional index (PNI) and neutrophil-to-lymphocyte ratio (NLR) in patients with cervical squamous cell carcinoma. The clinical data of 539 patients with cervical cancer in the Affiliated Tumor Hospital of Nantong University from December 2007 to October 2016 were analyzed retrospectively. The ROC is used to select the best cutoff values of PNI and NLR, which are 48.95 and 2.4046. Cox regression analysis was used for univariate and multivariate analysis. Survival differences were assessed by Kaplan-Meier (KM) survival method. Finally, a 3-layer artificial neural network (ANN) model is established. In cervical squamous cell carcinoma, the KM survival curve showed that the overall survival (OS) rate of high-level PNI group was significantly higher than that of low-level PNI group (P < .001), while the OS rate of low-level NLR group was significantly higher than that of high-level NLR group (P = .002). In non-squamous cell carcinoma, there was no significant difference in OS between the 2 groups (P > .005). According to Cox multivariate analysis, preliminary diagnosed PNI and NLR were independent prognostic factors of cervical squamous cell carcinoma (P < .001, P = .008), and pathological type and International Federation of Gynecology and Obstetrics (FIGO) stage also had a certain impact on tumor progression (P = .042, P = .048). The increase of PNI and the decrease of NLR will help patients with cervical squamous cell carcinoma live longer. ANN showed that PNI and NLR were of great importance in predicting survival. Preoperative PNI and NLR are independent predictors of cervical squamous cell carcinoma patients related to clinicopathological features, and have particular value in judging prognosis.