Backpropagation Training Research Articles

SkipNet: an adaptive neural network equalization algorithm for future passive optical networking

In this paper, we propose an original adaptive neural network equalizer (NNE) algorithm named SkipNet, which is suitable for rapid training on a packet-by-packet basis for burst-mode non-linear equalization in upstream PON transmission. SkipNet uses the simple LMS algorithm and avoids complex neural network training algorithms such as backpropagation and mini-batch training. We demonstrate SkipNet on captured continuous mode 100 Gbit/s PAM4 signals using an SOA preamplifier to achieve the challenging 29 dB PON optical loss budget. The adaptive SkipNet equalizer is shown to overcome combinations of severe SOA patterning effects and fiber dispersion impairments to achieve >29dB dynamic range back-to-back and >22.9dB dynamic range for up to 81.6 ps/nm accumulated dispersion. It can adapt in as little as 250 training symbols to each impairment scenario, which is equivalent to existing FFE/DFE solutions, while matching the non-linear performance of previously proposed static NNE solutions. To the best of our knowledge, SkipNet is the first ever adaptive NNE framework that can realistically be trained and adapted on a packet-by-packet basis and within strict PON packet preamble lengths.

Continuous and funnel-gate configurations of a permeable reactive barrier for reclamation of groundwater laden with tetracycline: experimental and simulation approaches

The current study investigates removing tetracycline from water using batch, column, and tank experiments with statistical modelling using ANN for continuous tests. An artificial neural network (ANN) using the Levenberg-Marquardt back-propagation (LMA) training algorithm is constructed to compare the effectiveness of Tetracycline removal from aqueous solution using the sorption technique with prepared adsorbent. Several characterization analyses XRD, FT-IR, and SEM are employed for prepared Brownmillerite (Ca2Fe2O5)–Na alginate beads. The operating conditions of batch tests involved, contact time (0.1–3 h), initial of tetracycline (Co) of (100–250 mg/L), pH (3–12), agitation speed (50–250) rpm and dosage of adsorbent (0.2–1.2 g/50 mL). The outcomes of experiments have demonstrated that the optimum conditions for the batch test to achieve the maximum adsorbent capacity (qmax =7.845 mg/g) are achieved at pH 7, contact time 1.5 h, adsorbent dose 1.2 g/50 mL, agitation speed of 200 rpm, and initial concentration of TC 100 mg/L. Minimum mean square error (MSE) values of 7.09E-04 for 30 hidden neurons and 0.0029 for 59 hidden neurons in the 1D and 2D systems are accomplished, respectively. The artificial neural network model has exhibited excellent performance with correlation coefficients exceeding 0.980 for the operating variables, demonstrating its accuracy and effectiveness in predicting the experimental outcomes. According to sensitivity analysis, the influential parameter in the column test (1D) is the flow rate (mL/min), with a relative importance of 32.769%. However, in the tank test (2D), time (day) is signified as an influential parameter with a relative importance of 31.207%.

Investigation of Vessel Manoeuvring Abilities in Shallow Depths by Applying Neural Networks

A set of planar motion mechanism experiments of the Duisburg Test Case Post-Panamax container model executed in a towing tank with shallow depth is applied to train a neural network to analyse the ability of the proposed model to learn the effects of different depth conditions on ship’s manoeuvring capabilities. The motivation of the work presented in this paper is to contribute an alternative and effective approach to model non-linear systems through artificial neural networks that address the manoeuvring simulation of ships in shallow water. The system is developed using the Levenberg–Marquardt backpropagation training algorithm and the resilient backpropagation scheme to demonstrate the correlation between the vessel forces and the respective trajectories and velocities. Sensitivity analyses were performed to identify the number of layers necessary for the proposed model to predict the vessel manoeuvring characteristics in two different depths. The outcomes achieved with the proposed system have shown excellent accuracy and ability in predicting ship manoeuvring with varying depths of shallow water.

Load Control Battery Strategy based on Backpropagation and Simulated Annealing Training Performance

Load Control Battery Strategy based on Backpropagation and Simulated Annealing Training Performance

A framework for model maintenance using kernel-based forward propagating neural networks

Deep learning models possess limited flexibility in computational burden in model adaptation owing to the conventional use of backpropagation for model training. To address this problem, we propose an alternate training methodology inspired by the forward–forward algorithm originally designed for classification tasks. We extend this concept through a kernel-based modification, enabling its application to regression tasks, which are commonly encountered in process system modeling. Our proposed Kernel-based Forward Propagating Neural Network (K-FP-NN) eliminates backpropagation, using layer-wise updates for better adaptability. We introduce a real-time (RT) updating framework, RT-K-FP-NN, to continuously refine model parameters with new data. Results indicate that when applied to model predictive control of a continuous stirred tank reactor (CSTR) system, our approach updates the model within 100 s, achieving better performance metrics compared to backpropagation-based real-time models, which require 326 s. This framework can be applied to various dynamic systems, enhancing real-time decision-making by improving predictive accuracy and system adaptability.

Performance Analysis of Cooperative Spectrum Sensing using Empirical Mode Decomposition and Artificial Neural Network in Wireless Regional Area Network

Background: Radio spectrum is natural and the most precious means in wireless communication systems. Optimal spectrum utilization is a key concern for today's cutting-edge wireless communication networks. The impending problem of the lack of available spectrum has prompted the development of a new idea called “Cognitive Radio” (CR). Cooperative spectrum sensing (CSS) is utilized to improve the detection performance of the system. Several fusion algorithms of decision-making are proposed for sensing the licensed user, but they do not work well under low signal-to-noise ratio (SNR). Objectives: To address the issue of poor detection performance under low SNR, Empirical mode decomposition (EMD) and artificial neural network (ANN) based CSS under Rayleigh multipath fading channel in IEEE 802.22 wireless regional area network (WRAN) is proposed in this paper. Method: In this work, we propose the use of ANN as a fusion center. First, the received signal's energy is calculated using EMD. The computed energy, SNR, and false alarm probability are combined to form a data set of 2048 samples. They are utilized to train Levenberg- Marquardt back propagation training algorithm-based feed-forward neural network (FFNN). Using this trained network, CSS in WRAN is simulated under Rayleigh multipath fading. Results: Simulation results show that the proposed CSS method based on EMD-ANN outperforms the standard fast Fourier transform (FFT) and EMD detection-based cooperative spectrum sensing with a hard "OR" fusion at low SNR. With Pf =0.01, 100% detection accuracy with proposed techniques is obtained at SNR= -22dB. Conclusion: The findings show that the suggested approach outperforms EMD and FFT based energy detection scheme-based traditional CSS in low SNR environments.

Artificial neural networks in the retention of anthocyanins and total phenolics in the osmotic pre-treatment of Biloxi variety blueberry (Vaccinium corymbosum L.) jam

Blueberries are a fruit that is an important source of bioactive components beneficial to the human diet, such as anthocyanins and total phenolics, which are altered by the use of high temperatures during processing. This study aimed to evaluate the use of artificial neural networks in the optimization of sucrose concentration and time for the osmotic pre-treatment of blueberries of the Biloxi variety, to retain the greatest amount of anthocyanins and total phenolics in the subsequent preparation of jam. Artificial neural networks of the feedforward type were used, with a Backpropagation training algorithm with Levenberg-Marquardt weight adjustment, to achieve the optimal predicted combination that maximizes the retention of these bioactive components. The model achieved its best performance with 11 neurons in the hidden layer, achieving an R2 coefficient of 0.98 and a mean square error of 4.76, indicating a strong ability for generalization. Artificial neural networks allowed to obtain the best optimal combination of predicted multiple responses of factors consisting of a sucrose concentration of 1.64 M and a time of 211.52 min, which retained a higher content of total monomeric anthocyanins with 70.98 mg cyanidin-3-O-glucoside 100 g-1 of jam and total phenolics with 110.54 mg GAE g-1 of jam. On the other hand, through single-response optimization was obtained that the combination of experimental factors that maximized total anthocyanins (71.59 mg cyanidin-3-O-glucoside 100 g-1 of jam) was 1.54 M of sucrose and 232.73 min and for total phenols (111.06 mg GAE g-1 of jam) 1.79 M of sucrose and 196.36 min. The use of artificial neural networks is an excellent alternative for modeling phenomena, compared to traditional methods.

A reconfigurable FPGA-based spiking neural network accelerator

The spiking neural network (SNN) is suitable for the intelligent edge computing applications because of its low-power characteristic. This work designs a reconfigurable spiking neural network accelerator supporting the spatiotemporal backpropagation (STBP) training method. The reconfigurable architecture is proposed between the spatial convolution module and the temporal accumulation module of the SNN accelerator. A sparse zero-hopping mechanism is designed to exploit the input sparsity of SNN datasets, and a mask mechanism is introduced between the forward inference computation and the backward training computation to exploit the output sparsity. During the training process, the peak and average performances of the SNN accelerator are 5.57 TOPS and 4.96 TOPS respectively, the power consumption is 6.124 W and the energy efficiency is 0.81 TOPS/W. The peak and average performances of the SNN accelerator are 5.98 TOPS and 5.14 TOPS respectively, the power consumption is 6.943 W and the energy efficiency is 0.74 TOPS/W, during the inference process.

Hardware implementation of backpropagation using progressive gradient descent for in situ training of multilayer neural networks.

Neural network training can be slow and energy-expensive due to the frequent transfer of weight data between digital memory and processing units. Neuromorphic systems can accelerate neural networks by performing multiply-accumulate operations in parallel using nonvolatile analog memory. However, executing the widely used backpropagation training algorithm in multilayer neural networks requires information-and therefore storage-of the partial derivatives of the weight values preventing suitable and scalable implementation in hardware. Here, we propose a hardware implementation of the backpropagation algorithm that progressively updates each layer using in situ stochastic gradient descent, avoiding this storage requirement. We experimentally demonstrate the in situ error calculation and the proposed progressive backpropagation method in a multilayer hardware-implemented neural network. We confirm identical learning characteristics and classification performance compared to conventional backpropagation in software. We show that our approach can be scaled to large and deep neural networks, enabling highly efficient training of advanced artificial intelligence computing systems.

Modeling of CI engine performance and emission parameters using artificial neural network powered by catalytic co-pyrolytic renewable fuel

Emission and performance parameters of a 4-stroke CI engine operated on a blend of catalytic co-pyrolysis oil with pure diesel, produced through Azadirachta indica seed, waste LDPE (low-density polyethylene), and aluminium oxide (Al2O3) as a catalyst, are modelled in the current work using an Artificial Neural Network (ANN). At 500°C temperature, the highest oil output obtained was 93.91 wt%. The produced liquid fuel possessed similar physical features to that of pure diesel, including density (794 kg/m3) and heating value (44.42 MJ/kg), but lower flash and fire points that would assist in a better and complete combustion of the fuel blend resulting in a better performance and combustion characteristics. Using inputs including brake mean effective pressure, load, brake power, and torque, a developed ANN model was applied to forecast the performance (Brake Thermal Efficiency and Brake Specific Fuel Consumption) along with emission characteristics (Smoke and NOx). The Levenberg-Marquardt back-propagation training technique was applied for emissions and performance characteristics prediction having the best accuracy. Regression coefficients (R2) for predicting BTE, BSFC, NOx, and smoke were all very near to 1: 0.99801, 0.9983, 0.95753, and 0.97467. The study determines that the proposed alternative fuel could be utilized in blend with the pure diesel to in an unmodified diesel engine. It has also been found that artificial neural networks (ANN) could prove to be useful to model and forecast the performance or emissions of renewable fuels in diesel engines, with the potential for these fuels to be employed in transportation.

Ultra-short-term prediction of microgrid source load power considering weather characteristics and multivariate correlation

Multiple microgrids interconnect to form a microgrid cluster. To fully exploit the comprehensive benefits of the microgrid cluster, it is imperative to optimize dispatch based on the matching degree between the sources and loads of each microgrid. The power of distributed energy sources such as wind and photovoltaic systems and the sensitive loads in microgrids is related to the regional weather characteristics. Given the relatively small geographical scope of microgrid areas and the fact that distributed energy sources and loads within the grid share the same weather characteristics, simultaneous ultra-short-term forecasting of power for both sources and loads is essential in the same environmental context. Firstly, the introduction of the multi-variable uniform information coefficient (MV-UIC) is proposed for extracting the correlation between weather characteristics and the sequences of source and load power. Subsequently, the application of factor analysis (FA) is introduced to reduce the dimensionality of input feature variables. Drawing inspiration from the concept of combination forecasting models, a combined forecasting model based on Error Back Propagation Training (BP), Long Short-Term Memory (LSTM), and Bidirectional Long Short-Term Memory Neural Network (BiLSTM) is constructed. This model is established on the MV-UIC-FA foundation for the joint ultra-short-term forecasting of source and load power in microgrids. Simulation is conducted using the DTU 7K 47-bus system as an example to analyze the accuracy, applicability, and effectiveness of the proposed joint forecasting method for sources and loads.

Artificial Neural Network (ANN)-Based Water Quality Index (WQI) for Assessing Spatiotemporal Trends in Surface Water Quality—A Case Study of South African River Basins

Artificial neural networks (ANNs) are powerful data-oriented “black-box” algorithms capable of assessing and delineating linear and multifaceted non-linear correlations between the dependent and explanatory variables. Through the years, neural networks have proven to be effective and robust analytical techniques for establishing artificial intelligence-based tools for modelling, estimating, and projecting spatial and temporal variations in water bodies. Accordingly, ANN-based algorithms gained increased attention and have emerged as practical alternatives to traditional approaches for hydro-chemical analysis. ANNs are among the widely used computer systems for modelling surface water quality. Considering their wide recognition, resilience, flexibility, and accuracy, the current study employs a neural network-based methodology to construct a novel water quality index (WQI) model suitable for analysing South African rivers. The feed-forward, back-propagated multilayered perceptron model has three parallel-distributed neuron layers interconnected with seventy weighted links orientated laterally from left to right. First, the input layer includes thirteen neuro-nodes symbolising thirteen explanatory variables, including NH3, Ca, Cl, Chl-a, EC, F, CaCO3, Mg, Mn, NO3, pH, SO4, and turbidity (NTU). Second, the hidden layer consists of eleven neuro-nodes accountable for computational tasks. Lastly, the output layer features one neuron responsible for conveying network outcomes using a single-digit WQI rating extending from zero to one hundred, where zero represents substandard water quality and one hundred denotes exceptional water quality. The AI-based model was developed using water quality data obtained from six monitoring locations within four drainage basins under the management of the Umgeni Water Board in the KwaZulu-Natal Province of South Africa. The dataset comprises 416 samples randomly divided into training, testing, and validation sets using a proportional split of 70:15:15%. The Broyden–Fletcher–Goldfarb–Shanno (BFGS) technique was utilised to conduct backpropagation training and adjust synapse weights. The dependent variables are the WQI scores from the universal water quality index (UWQI) model developed specifically for South African river basins. The ANN demonstrated enhanced efficiency through an overall correlation coefficient (R) of 0.985. Furthermore, the neural network attained R-values of 0.987, 0.992, and 0.977 for the training, testing, and validation intervals. The ANN model achieved a Nash–Sutcliffe efficiency (NSE) value of 0.974 and coefficient of determination (R2) of 0.970. Sensitivity analysis provided additional validation of the preparedness and computational competence of the ANN model. The typical target-to-output error tolerance for the ANN model is 0.242, demonstrating an adequate predictive ability to deliver results comparable with the target UWQI, having the lowest and highest index ratings of 75.995 and 94.420, respectively. Accordingly, the three-layer neural network is scientifically sound, with index values and water quality evaluations corresponding to the UWQI results. The current research project seeks to document the processes used and the outcomes obtained.

Artificial Neural Networks and Experimental Analysis of the Resistance Spot Welding Parameters Effect on the Welded Joint Quality of AISI 304.

The automobile industry relies primarily on spot welding operations, particularly resistance spot welding (RSW). The performance and durability of the resistance spot-welded joints are significantly impacted by the welding quality outputs, such as the shear force, nugget diameter, failure mode, and the hardness of the welded joints. In light of this, the present study sought to determine how the aforementioned welding quality outputs of 0.5 and 1 mm thick austenitic stainless steel AISI 304 were affected by RSW parameters, such as welding current, welding time, pressure, holding time, squeezing time, and pulse welding. In order to guarantee precise evaluation and experimental analysis, it is essential that they are supported by a numerical model using an intelligent model. The primary objective of this research is to develop and enhance an intelligent model employing artificial neural network (ANN) models. This model aims to provide deeper knowledge of how the RSW parameters affect the quality of optimum joint behavior. The proposed neural network (NN) models were executed using different ANN structures with various training and transfer functions based on the feedforward backpropagation approach to find the optimal model. The performance of the ANN models was evaluated in accordance with validation metrics, like the mean squared error (MSE) and correlation coefficient (R2). Assessing the experimental findings revealed the maximum shear force and nugget diameter emerged to be 8.6 kN and 5.4 mm for the case of 1-1 mm, 3.298 kN and 4.1 mm for the case of 0.5-0.5 mm, and 4.031 kN and 4.9 mm for the case of 0.5-1 mm. Based on the results of the Pareto charts generated by the Minitab program, the most important parameter for the 1-1 mm case was the welding current; for the 0.5-0.5 mm case, it was pulse welding; and for the 0.5-1 mm case, it was holding time. When looking at the hardness results, it is clear that the nugget zone is much higher than the heat-affected zone (HZ) and base metal (BM) in all three cases. The ANN models showed that the one-output shear force model gave the best prediction, relating to the highest R and the lowest MSE compared to the one-output nugget diameter model and two-output structure. However, the Levenberg-Marquardt backpropagation (Trainlm) training function with the log sigmoid transfer function recorded the best prediction results of both ANN structures.

Compare the Performance of Distinct Neural Networks Techniques to diagnose the kidney stone disease

Artificial Neural Networks are excellent at identifying patterns or trends in data, which makes them perfect for forecasting or prediction. Thus, neural networks have extensive application in biological systems. The application of neural networks to kidney stone diagnosis is emphasized in this article. Kidney stone issues can be diagnosed with neural networks by applying technological concepts such as MLP, SVM, RBF, and BPA. The purpose of this research is to use three different neural network algorithms—each with its own specific design and set of properties to identify kidney stone disease. The performance of the three neural networks is compared in this research with respect to training data set size, model creation time, and accuracy. Kidney stone sickness will be diagnosed using radial basis function (RBF) networks, two layers feed forward perceptrons trained with the back propagation training algorithm, and learning vector quantization (LVQ). However, determining the best approach for any particular diagnostic had never been an easy task. Like many other illnesses, kidney stones have already been diagnosed using neural network algorithms. The main objective of this work is to recommend the best medical diagnostic instrument, such as kidney stone detection, to reduce diagnosis times and improve accuracy and efficiency.

GINN-LP: A Growing Interpretable Neural Network for Discovering Multivariate Laurent Polynomial Equations

Traditional machine learning is generally treated as a black-box optimization problem and does not typically produce interpretable functions that connect inputs and outputs. However, the ability to discover such interpretable functions is desirable. In this work, we propose GINN-LP, an interpretable neural network to discover the form and coefficients of the underlying equation of a dataset, when the equation is assumed to take the form of a multivariate Laurent Polynomial. This is facilitated by a new type of interpretable neural network block, named the “power-term approximator block”, consisting of logarithmic and exponential activation functions. GINN-LP is end-to-end differentiable, making it possible to use backpropagation for training. We propose a neural network growth strategy that will enable finding the suitable number of terms in the Laurent polynomial that represents the data, along with sparsity regularization to promote the discovery of concise equations. To the best of our knowledge, this is the first model that can discover arbitrary multivariate Laurent polynomial terms without any prior information on the order. Our approach is first evaluated on a subset of data used in SRBench, a benchmark for symbolic regression. We first show that GINN-LP outperforms the state-of-the-art symbolic regression methods on datasets generated using 48 real-world equations in the form of multivariate Laurent polynomials. Next, we propose an ensemble method that combines our method with a high-performing symbolic regression method, enabling us to discover non-Laurent polynomial equations. We achieve state-of-the-art results in equation discovery, showing an absolute improvement of 7.1% over the best contender, by applying this ensemble method to 113 datasets within SRBench with known ground-truth equations.

Adaptive Threshold Learning in Frequency Domain for Classification of Breast Cancer Histopathological Images

Breast cancer has become the most common cancer in the world, and biopsy is the most reliable and widely used technique for detecting breast cancer. However, observation of histopathological images is time-consuming and labor-intensive. Currently, CNN has become the mainstream method for breast cancer histopathological image classification research. However, some studies have found that the optical microscope-generated histopathological images have noise, and the output of a well-trained convolutional neural network in image classification tasks can change drastically due to small variations in the input. Therefore, the quality of the image significantly affects the accuracy of the classification. Wavelet transform is a commonly used denoising method, but the selection of the threshold is a difficult problem, and traditional methods are difficult to find the appropriate threshold quickly and accurately. This paper proposes an adaptive threshold selection method that combines threshold selection steps with deep learning methods by using the threshold as a parameter in the CNN model to train. In this way, we associate the threshold with the classification result of the model and find the appropriate value for that image and task by back-propagation in training. The method was experimented on publicly available datasets BreaKHis and BACH. The results in BreaKHis (40x: 94.37%, 100x: 93.85%, 200x: 91.63%, 400x: 93.31%), and BACH (91.25%) demonstrate that our adaptive threshold selection method can improve classification accuracy and is significantly superior to traditional threshold selection methods.

Modeling of deionization of aqueous compounds using electrodialysis process (data collection for enhanced clarity)

ABSTRACT The deionization process is increasingly recognized as a solution to global water shortages. This study aims to investigate the impact of various parameters on the deionization efficiency. Specifically, volume flow rates (F) were tested (1, 2.5, 5, 10, 15, and 20 ml min−1), solute concentrations (C) (200, 500, and 1000 mg l−1), temperatures (T) (50, 60, and 70°C), voltages (V) (10, 20, and 30 V), and pressures (P) (200, 400, and 800 Pa). In another experiment examined F (100, 250, and 500 mol l−1 m2), V (0.1, 0.5, and 1 V), C (100, 200, 300, 400, and 500 moles m3), and time durations (D) (16, 32, 48, 64, 80, and 96 min). These experiments aimed to determine the separation percentage of potassium chloride cations and anions and were based on data extracted from scientific articles. In the subsequent step, the ANFIS model with the FCM network creation method and the error backpropagation training algorithm was employed to evaluate the impact of each cell input on the separation percentage, output flux, and the separation percentage of potassium chloride cations and anions. This evaluation was conducted through 31 and 15 iterations, respectively. Based on the evaluation criteria, it was observed that the volume flow rate as the sole input yielded the highest prediction accuracy for the separation percentage and output flux. Additionally, the FCTVP model results indicated a slight reduction in the error criteria compared to the FCTV, FCT, FC, and single models. Consequently, it can be concluded that the cell pressure factor may play a decisive role, either reducing or increasing the separation process (R2 = 0.93,MBE = −0.3, and RMSE = 6.4), as well as the output flux (R2 = 0.85, MBE = 0.0003, and RMSE = 0.07). Furthermore, the statistical criteria values of the FVCT model for the separation percentage of potassium chloride cations and anions demonstrated a proper correlation (R2 = 0.96 and 0.95), slight underestimation, and low error MBE = −0.57 and −1.2) (RMSE = 15.2 and 18.5, respectively).

Comprehensive investigation of isotherm, RSM, and ANN modeling of CO2 capture by multi-walled carbon nanotube

Chemical vapor deposition was used to produce multi-walled carbon nanotubes (MWCNTs), which were modified by Fe–Ni/AC catalysts to enhance CO2 adsorption. In this study, a new realm of possibilities and potential advancements in CO2 capture technology is unveiled through the unique combination of cutting-edge modeling techniques and utilization of the recently synthesized Fe–Ni/AC catalyst adsorbent. SEM, BET, and FTIR were used to analyze their structure and morphology. The surface area of MWCNT was found to be 240 m2/g, but after modification, it was reduced to 11 m2/g. The modified MWCNT showed increased adsorption capacity with higher pressure and lower temperature, due to the introduction of new adsorption sites and favorable interactions at lower temperatures. At 25 °C and 10 bar, it reached a maximum adsorption capacity of 424.08 mg/g. The optimal values of the pressure, time, and temperature parameters were achieved at 7 bar, 2646 S and 313 K. The Freundlich and Hill models had the highest correlation with the experimental data. The Second-Order and Fractional Order kinetic models fit the adsorption results well. The adsorption process was found to be exothermic and spontaneous. The modified MWCNT has the potential for efficient gas adsorption in fields like gas storage or separation. The regenerated M-MWCNT adsorbent demonstrated the ability to be reused multiple times for the CO2 adsorption process, as evidenced by the study. In this study, a feed-forward MLP artificial neural network model was created using a back-propagation training approach to predict CO2 adsorption. The most suitable and efficient MLP network structure, selected for optimization, consisted of two hidden layers with 25 and 10 neurons, respectively. This network was trained using the Levenberg–Marquardt backpropagation algorithm. An MLP artificial neural network model was created, with a minimum MSE performance of 0.0004247 and an R2 value of 0.99904, indicating its accuracy. The experiment also utilized the blank spreadsheet design within the framework of response surface methodology to predict CO2 adsorption. The proximity between the Predicted R2 value of 0.8899 and the Adjusted R2 value of 0.9016, with a difference of less than 0.2, indicates a high level of similarity. This suggests that the model is exceptionally reliable in its ability to predict future observations, highlighting its robustness.

DT-SCNN: dual-threshold spiking convolutional neural network with fewer operations and memory access for edge applications.

The spiking convolutional neural network (SCNN) is a kind of spiking neural network (SNN) with high accuracy for visual tasks and power efficiency on neuromorphic hardware, which is attractive for edge applications. However, it is challenging to implement SCNNs on resource-constrained edge devices because of the large number of convolutional operations and membrane potential (Vm) storage needed. Previous works have focused on timestep reduction, network pruning, and network quantization to realize SCNN implementation on edge devices. However, they overlooked similarities between spiking feature maps (SFmaps), which contain significant redundancy and cause unnecessary computation and storage. This work proposes a dual-threshold spiking convolutional neural network (DT-SCNN) to decrease the number of operations and memory access by utilizing similarities between SFmaps. The DT-SCNN employs dual firing thresholds to derive two similar SFmaps from one Vm map, reducing the number of convolutional operations and decreasing the volume of Vms and convolutional weights by half. We propose a variant spatio-temporal back propagation (STBP) training method with a two-stage strategy to train DT-SCNNs to decrease the inference timestep to 1. The experimental results show that the dual-thresholds mechanism achieves a 50% reduction in operations and data storage for the convolutional layers compared to conventional SCNNs while achieving not more than a 0.4% accuracy loss on the CIFAR10, MNIST, and Fashion MNIST datasets. Due to the lightweight network and single timestep inference, the DT-SCNN has the least number of operations compared to previous works, paving the way for low-latency and power-efficient edge applications.

Weighted Hermite Variable Projection Networks for Classifying Visually Evoked Potentials.

The occipital cortex responds to visual stimuli regardless of a patient's level of consciousness or attention, offering a noninvasive diagnostic tool for both ophthalmologists and neurologists. This response signal manifests as a unique waveform referred to as the visually evoked potential (VEP), which can be extracted from the electroencephalogram (EEG) activity of a human being. We propose a trainable VEP representation to disentangle the underlying explanatory factors of the data. To enhance the learning process with domain knowledge, we present an innovative parameterization of classical Hermite functions that effectively captures VEP pattern variations arising from patient-specific factors, disorders, and measurement setup influences. Then, we introduce a differentiable variable projection (VP) layer to fuse Hermite basis function expansions (BFEs) of VEP signals with machine learning (ML) approaches. We prove the existence of an optimal set of parameters in the least-squares sense, assess the representation power of such layers, and calculate their analytical derivatives, which allows us to utilize backpropagation for training. Finally, we evaluate the effectiveness of the proposed learning framework in VEP-based color classification. To achieve this, we have designed a novel measurement system dedicated to intraoperative clinical use cases, which presents new ways for patient monitoring during neurosurgical procedures.