Input-output Mapping Research Articles

A Cepstrum-Informed neural network for Vibration-Based structural damage assessment

Data-driven methods for vibration-based Structural Health Monitoring (SHM) have gained significant popularity for their straightforward modeling process and real-time tracking capabilities. However, developing complex models such as deep neural networks can pose challenges, including limited interpretability and substantial computational demands, due to the large number of parameters and deep layer stacking. This study introduces a novel Cepstrum-Informed Attention-Based Network (CIABN) developed to model power cepstral coefficients of structural acceleration responses, guided by cepstrum-based physical properties to facilitate efficient structural damage assessment. The CIABN integrates three key components: a unique input–output mapping based on weighted cepstral coefficients, a novel cepstral positional encoding mechanism, and a multi-head self-attention mechanism. The unique input–output mapping enables appreciable model generalization in overall structural characteristics, with the weighted cepstral coefficients serving as informative and compact data for efficient neural network modeling. The developed cepstral positional encoding scientifically guides the model to capture the coefficient indices, and the underlying trend of cepstral coefficients primarily governed by overall structural characteristics. The multi-head attention mechanism enables computationally efficient parallel analysis of interdependencies among coefficients, facilitating the development of a lightweight network. The effectiveness and superiority of the method have been validated using both simulated and experimental structural data.

Incremental learning and granular computing from evolving data streams: An application to speech-based bipolar disorder diagnosis

We apply an evolving granular-computing modeling approach, called evolving Optimal Granular System (eOGS), to bipolar mood disorder (BD) diagnosis based on speech data streams. The eOGS online learning algorithm reveals information granules in the flow and design the structure and parameters of a granular rule-based model with a certain degree of interpretability based on acoustic attributes obtained from phone calls made over 7 months to the Psychiatry department of a hospital. A multi-objective programming problem that trades-off information specificity, model compactness, and numerical and granular error indices is presented. Spectral and prosodic attributes are ranked and selected based on a hybrid Pearson-Spearman correlation coefficient. Low attribute-class correlation, ranging from 0.03 to 0.07, is observed, as well as high class overlap, which is typical in the psychiatric field. eOGS models for BD recognition overcome alternative computational-intelligence models, namely, Dynamic Evolving Neural-Fuzzy Inference System (DENFIS) and Fuzzy-set-Based evolving Modeling (FBeM-Gauss), by a small margin in both best and average cases; followed by eXtended Takagi-Sugeno (xTS) and evolving Takagi Sugeno (eTS) types of models. The proposed eOGS model using only 8 of the original acoustic attributes, and about 15 ‘If-Then’ inference rules, has exhibited the best root mean square error, 0.1361, and 91.8% accuracy in sharp BD class estimates. Granules associated to linguistic labels and a granular input-output map offer human understandability with relation to the inherent process of generating class estimates. Linguistically readable eOGS rules may assist physicians in explaining symptoms and making a diagnosis.

Fuzzy Neural Logic Reasoning for Robust Classification

The efficacy of neural networks is widely recognized across a multitude of machine learning tasks, yet their black-box nature impedes the understanding of their decision-making processes. Such lack of explainability limits their use in high-stake fields such as medicine and finance, where transparent decision-making is essential. In contrast, traditional rule-based models offer clear input-output mappings, but often lag in performance when compared to their neural network counterparts. To address this challenge, this study introduces Fuzzy Neural Logic Reasoning (FNLR), a novel architecture that combines the best of both rule-based and deep learning models to achieve performance, interpretability, and noise robustness simultaneously. At its core, FNLR employs a “Symbolic Pre-training + Neural Fine-tuning” paradigm. Initially, the model adapts a pre-fitted binary decision tree. It then performs a “neuralization” process, replacing each node of the tree with a corresponding neural network equivalent. This transformation is facilitated through three shallow MLP modules, which are trained to emulate the relational operators intrinsic to decision trees. The model architecture is also extensible, allowing it to further boost expressiveness. Furthermore, FNLR incorporates fuzzy logic by proposing novel fuzzy relational operators, accounting for satisfaction degrees of propositions and thus eliminating rigid decision boundaries. This approach enhances model flexibility, enabling all paths of the decision tree to contribute to the target prediction in a weighted manner. Empirical evaluations on tabular datasets from various domains demonstrate that FNLR performs comparably to, or better than, state-of-the-art deep learning models designed for tabular data, while also exhibiting strong robustness to noise.

A new approach for spatio-temporal interface treatment in fluid–solid interaction using artificial neural networks employing coupled partitioned fluid–solid solvers

Partitioned fluid–solid interaction (FSI) problems involving non-conforming grids pose formidable challenge in interface treatment, especially for information exchange, interface tracking, and field variable interpolation between solvers in both space and time. These demand special considerations for accurate and efficient simulations. This paper presents an application of artificial neural networks (ANN) for the interface treatment in a coupled FSI problem employing partitioned solvers. A shallow time-series ANN (nonlinear auto-regressive model with exogenous inputs, NARX) scheme is proposed to handle the exchange of Neumann/Dirichlet information at the coupling interface. This scheme involves two interface treatment models that were developed and analysed. The proposed models interpolate and transfer loads from the fluid to the solid domains, and conversely, displacements from the solid to the fluid domains between non-collocated grids. To validate this approach, we tested it on a 3D FSI problem, which involved damped oscillations of a flexible flap submerged in a fluid cavity. Adequately trained NARX interface models demonstrate reliable input–output mapping and accurate prediction of transient behaviour at the interface. Additionally, we explored the concept of reduced-order modelling (ROM) in the time domain. This allowed us to reduce the model’s complexity by half. Different training algorithms were evaluated to enhance the efficiency and performance of the proposed scheme. The study demonstrates that NARX networks trained with Bayesian Regularization (BR) and Levenberg–Marquardt (LM) algorithms exhibit the best accuracy, while the scaled conjugate gradient (SCG)-based training method provides better computational efficiency with acceptable accuracy. Overall, the NARX interface models provide precise performance and offer a viable potential for applications in FSI problems requiring accurate and faster computations.

How Age of Acquisition Affects Compound Word Recognition

ABSTRACT Purpose Adults recognize words that are acquired during childhood more quickly than words acquired during adulthood. This is known as the Age of Acquisition (AoA) effect. The AoA effect, according to the integrated account, manifests in tasks necessitating greater semantic processing and in tasks with arbitrary input-output mapping. Compound words allow us to investigate this account due to the arbitrary input-output mapping between the compound word itself and its morphemes, which requires greater semantic processing. Method Forty-eight British English students in each experiment completed an unspaced (Experiment 1; n = 48; 83% female; Mage = 19.73), spaced (Experiment 2; n = 48; 83% female; Mage = 19.04), auditory (Experiment 3; n = 48; 63% female; Mage = 19.83), and cross-modal (Experiment 4; n = 48; 52% female; Mage = 19.81) lexical decision task (LDT) using a regression design on 226 compound words. Results We observed that the AoA of the compound word affected accuracy across all tasks, whereas the AoA of the compound word influenced recognition latencies across all tasks except cross-modal LDT. Discussion The results suggest that the influence of the AoA effect and of semantic predictors is largest in unspaced compound words and smallest in cross-modal LDT. This indicates that the AoA effect in word recognition is in line with the integrated account.

A graph network-based learnable simulator for spatial-temporal prediction of rigid projectile penetration

Predicting plate penetration by rigid projectiles (PPRP) is crucial in terminal ballistics, with broad applications in civil and military engineering. Empirical and analytical methods face challenges in predicting field variables like displacement and stress in target plates. Although numerical methods offer high accuracy, they suffer from low computational efficiency. Herein, we introduce an efficient data-driven machine learning (ML) method based on graph neural networks (GNNs), named PGN, specifically tailored to address the PPRP problem. Unlike traditional ML methods that establish direct input-output mappings, PGN predicts comprehensive spatial-temporal information pertaining to the projectile-target interaction process. A thorough analysis of PGN's performance in terms of accuracy, computational efficiency and generalization ability was performed. Compared to validated results of numerical simulations, PGN maintained high precision with RMSE for displacement, stress, and strain predictions below 0.5 %, 9.5 %, and 2.1 %, respectively. It also achieved R2 values exceeding 0.92 for the time history of projectile velocity and acceleration, while requiring only 9.8 % of the computation time compared to LS-DYNA. In generalization tests, PGN exhibited remarkable adaptability in tackling challenging scenarios that extend far beyond the training data distribution, with overall RMSE between 11 % and 13 %. Furthermore, we find that the maximum information propagation capacity of a simulated physical system must meet or exceed the information propagation need of the real-world physical phenomenon it aims to replicate. Consequently, an approach was proposed to determine the critical connectivity radius of the massage passing method directly from the wave speed in the target medium, which greatly improved the accuracy and efficiency of PGN.

Encryption-decryption scheme in a single-pixel system based on polarization and Laguerre-Gaussian mode modulation

Optical cryptosystems are crucial for ensuring the security of optical information transmission and storage. The indirect measurement mechanism of single-pixel imaging (SPI) offers a feasible implementation channel for optical cryptosystems. Illumination patterns are encryption keys projected onto the plaintext object, while the intensity collected by the single-pixel detector forms the ciphertext. However, the variations in the object's angular position during SPI measurement generally introduce certain inaccuracies in image reconstruction. And due to SPI's input-output linear mapping relationship, the plaintext is vulnerable to exposure. This proposes an encryption-decryption scheme in a single-pixel system based on polarization and Laguerre-Gaussian (LG) mode modulation. The inherent circular symmetry of LG mode makes the angular position of the object information that can be encrypted, while the intrinsic properties of the object can be represented by polarization. Our system characterizes various polarization parameters of samples serving as reliable plaintext with an error of less than 4.2%, including depolarization, diattenuation, and retardance. For encryption demonstration, LG modes are randomly divided into 5 groups, corresponding to an object at different rotational states. This, combined with 16 polarization modulations, constructs pattern-angle-polarization joint keys, enabling high-security encryption as well as high-fidelity decryption of the mask image, optical axis orientation, and retardance of the test sample. Experimental results demonstrate the effectiveness of our scheme in enhancing the security and information complexity of optical cryptography, offering valuable insights for optical communication and quantum information security.

Estimation of Soil Moisture during Different Growth Stages of Summer Maize under Various Water Conditions Using UAV Multispectral Data and Machine Learning

Accurate estimation of soil moisture content (SMC) is vital for effective farmland water management and informed irrigation decision-making. The utilization of unmanned aerial vehicle (UAV)-based remote sensing technology to monitor SMC offers advantages such as mobility, high timeliness, and high spatial resolution, thereby compensating for the limitations of in-situ observations and satellite remote sensing. However, previous research has primarily focused on SMC diagnostics for the entire crop growth period, often neglecting the development of targeted soil moisture modeling paradigms that account for the specific characteristics of the canopy and root zone at different growth stages. Furthermore, the variations in soil moisture status between fields, resulting from the hysteresis of water flow in irrigation channels at different levels, may influence the development of soil moisture modeling schemes, an area that has been seldom explored. In this study, SMC models based on UAV spectral information were constructed using Random Forest (RF) and Particle Swarm Optimization-Support Vector Machine (PSO-SVM) algorithms. The soil moisture modeling paradigms (i.e., input–output mapping) under different growth stages and soil moisture conditions of summer maize were systematically compared and discussed, along with the corresponding physical interpretability. Our results showed that (1) the SMC modeling schemes differ significantly across the various growth stages, with distinct input–output mappings recommended for the early (i.e., jointing, tasselling, and silking stages), middle (i.e., blister and milk stages), and late (i.e., maturing stage) periods. (2) these machine learning-based models performed best at the jointing stage, while subsequently, their accuracy generally exhibited a downward trend as the maize grew. (3) the RF model demonstrates superior robustness in estimating soil moisture status across different fields (moisture conditions), achieving optimal estimation accuracy in fields with overall higher SMC in line with the PSO-SVM model. (4) unlike the RF model’s robustness in spatial SMC diagnostics, the PSO-SVM model more reliably captured the temporal dynamics of SMC across different growth stages of summer maize. This study offers technical references for future modelers in UAV-based SMC modeling across various spatial and temporal conditions, addressing both the types of models as well as their input features.

Delivering holistic energy-saving operation for heterogeneous all-parallel chiller plant systems through two-stage optimization assisted by input convex neural networks

With the rapid growth of building cooling demand and energy consumption, the potential for energy-saving operations gave rise to the popularity of implementing heterogeneous all-parallel chiller plant systems in large-scale public buildings. Meanwhile, heterogeneity, multi-dimensionality, and tight coupling interactions considerably complicate the holistically optimal operation of the system. Although artificial neural networks (ANN) have achieved great success in complex system identification, the endogenous non-convexity of classical feedforward neural networks (FNN), which are widely applied in model-based optimization for chiller plant operation, makes conventional optimizers prone to falling into local optima, forcing previous methodologies to sacrifice model precision for computational tractability. In this study, a special type of ANN named input convex neural network (ICNN) was leveraged to identify five individual convex models for energy-consuming units in the chiller plant system, providing convex input-output mappings while preserving high-resolution representations. Then, a two-stage optimizer with a sequential integration of a global searcher and a local optimizer was developed to effectively solve the holistic energy-saving problem by optimizing ten decision variables including chilled water supply temperature, partial load ratio of chillers, speed ratios of pumps and cooling tower fans, cooling water flow rate of cooling towers, and on/off states of devices. Finally, the methodology was deployed to a case study using on-site operation data and was compared with conventional candidates. Average holistic energy-saving ratios of 13.37% and 9.14% were achieved with average runtimes of 1.12 s and 1.07 s by the proposed optimizer in the high-load and light-load scenarios, respectively, showcasing more advantageous in resolving the operation optimization of complicated chiller plant systems.

Terminal sequence consistency verification method for small diameter abreast optical fibers based on computer vision

In a kind of precision industrial equipment, small diameter abreast optical fibers are used for high-speed communication among functional nodes. The arrangement order at both terminals of the abreast optical fibers need to comply with communication protocols. In this paper, we propose an automatic terminal sequence consistency verification method based on computer vision. The Hue Saturation Value (HSV) color space is used for improving the image feature extraction capability. An abreast optical fiber sequence dictionary which converts the protocol logic into an input-output mapping table is provided to follow protocol confidentiality and improve inspecting speed. A light control baffle position adaptive algorithm is designed for improving the accuracy of optical fiber incident light control. The experimental results show that the method can achieve the conductivity inspection of 1 optical fiber every 50 seconds, and the inspection accuracy is over 96.5%, which generally improves the inspection efficiency by 45% compared with manual inspection.

Simulation-based Flexible Needle Control with Single-core FBG Feedback for Spinal Injections.

We present a general framework of simultaneous needle shape reconstruction and control input generation for robot-assisted spinal injection procedures, without continuous imaging feedback. System input-output mapping is generated with a real-time needle-tissue interaction simulation, and single-core FBG sensor readings are used as local needle shape feedback within the same simulation framework. FBG wavelength shifts due to temperature variation is removed by exploiting redundancy in fiber arrangement. Targeting experiments performed on both plastisol lumbar phantoms as well as an ex vivo porcine lumbar section achieved in-plane tip errors of and , and total tip errors of and for the two testing environments. Our clinically inspired control strategy and workflow is self-contained and not dependent on the modality of imaging guidance. The generalizability of the proposed approach can be applied to other needle-based interventions where medical imaging cannot be reliably utilized as part of a closed-loop control system for needle guidance.

Projection-based reduced order modeling of multi-species mixing and combustion

High-fidelity simulations of mixing and combustion processes are computationally demanding and time-consuming, hindering their wide application in industrial design and optimization. This study proposes projection-based reduced order models (ROMs) to predict spatial distributions of physical fields for multi-species mixing and combustion problems in a fast and accurate manner. The developed ROMs explore the suitability of various regression methods, including kriging, multivariate polynomial regression (MPR), k-nearest neighbors (KNN), deep neural network (DNN), and support vector regression (SVR), for the functional mapping between input parameters and reduced model coefficients of mixing and combustion problems. The ROMs are systematically examined using two distinct configurations: steam-diluted hydrogen-enriched oxy-combustion from a triple-coaxial nozzle and fuel-flexible combustion in a practical gas-turbine combustor. The projected low-dimensional manifolds are capable of capturing important combustion physics, and the response surfaces of reduced model coefficients present pronounced nonlinear characteristics of the flowfields with varying input parameters. The ROMs with kriging present a superior performance of establishing the input–output mapping to predict almost all physical fields, such as temperature, velocity magnitude, and combustion products for both test problems. The accuracy of DNN is less encouraging owing to the stringent requirement on the size of training database. KNN performs well in the region near the design points but its effectiveness diminishes when the test points are distant from the sampling points, whereas SVR and MPR exhibit large prediction errors. For the spatial prediction at unseen design points, the ROMs achieve a prediction time of up to eight orders of magnitude faster than conventional numerical simulations, rendering an efficient tool for the fast prediction of mixing and combustion fields and potentially an alternative of a full-order numerical solver.

Towards transferable metamodels for water distribution systems with edge-based graph neural networks

Data-driven metamodels reproduce the input-output mapping of physics-based models while significantly reducing simulation times. Such techniques are widely used in the design, control, and optimization of water distribution systems. Recent research highlights the potential of metamodels based on Graph Neural Networks as they efficiently leverage graph-structured characteristics of water distribution systems. Furthermore, these metamodels possess inductive biases that facilitate generalization to unseen topologies. Transferable metamodels are particularly advantageous for problems that require an efficient evaluation of many alternative layouts or when training data is scarce. However, the transferability of metamodels based on GNNs remains limited, due to the lack of representation of physical processes that occur on edge level, i.e. pipes. To address this limitation, our work introduces Edge-Based Graph Neural Networks, which extend the set of inductive biases and represent link-level processes in more detail than traditional Graph Neural Networks. Such an architecture is theoretically related to the constraints of mass conservation at the junctions. To verify our approach, we test the suitability of the edge-based network to estimate pipe flowrates and nodal pressures emulating steady-state EPANET simulations. We first compare the effectiveness of the metamodels on several benchmark water distribution systems against Graph Neural Networks. Then, we explore transferability by evaluating the performance on unseen systems. For each configuration, we calculate model performance metrics, such as coefficient of determination and speed-up with respect to the original numerical model. Our results show that the proposed method captures the pipe-level physical processes more accurately than node-based models. When tested on unseen water networks with a similar distribution of demands, our model retains a good generalization performance with a coefficient of determination of up to 0.98 for flowrates and up to 0.95 for predicted heads. Further developments could include simultaneous derivation of pressures and flowrates.

Utilizing a CNN-RNN machine learning approach for forecasting time-series outlet fluid temperature monitoring by long-term operation of BHEs system

The Borehole Heat Exchanger (BHE) plays a pivotal role in enhancing heat exchange efficiency within Ground Source Heat Pump (GSHP) systems. The accurate prediction of the BHE's outlet fluid temperature is crucial for optimizing GSHP performance, energy storage, and resource conservation. However, conventional machine learning methods encounter challenges in manual feature extraction, learning complex nonlinear relationships, and adapting to real-world scenarios. To address these limitations, this research proposes a crossbreed model integrating Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) architectures to forecast long-term outlet fluid temperature in BHE systems. The model framework encompasses data preprocessing, utilizing refined data in the CNN module for temporal feature extraction, subsequently passed to the RNN module to capture sequential and temporal patterns from each dataset. Specifically, the advanced CNN-RNN architecture is designed to establish a comprehensive input-output mapping, leveraging essential input features such as inlet fluid, ambient air, and subsurface temperatures at varying depths (0, 10, and 20 m). Performance evaluation metrics, including R2, RMSE, MAE, and AARE, are employed to compare and assess prediction accuracy across various models, including LSTM, CNN, and SimpleRNN. The obtained results demonstrate the superior performance of the proposed model, achieving an RSME of 0.818, MAE of 0.642, AARE of 0.0305, and an R2 value of 98.75 %. This surpasses the performance of traditional prediction models (LSTM, CNN, and SimpleRNN) by 3.01 %, 5.80 %, and 19.52 %, respectively. Notably, the remarkably low MAE of 0.642 exhibited by a CNN-RNN model underscores its capability to outperform traditional approaches, especially when handling large datasets. These findings emphasize the significance of the developed model in facilitating efficient operation, positioning it as a valuable tool for advancing the long-term sustainability of BHE systems.

FDT: Improving the transferability of adversarial examples with frequency domain transformation

The development of Deep Neural Networks (DNNs) has facilitated profound advancements in computer vision. While DNNs demonstrate substantial capabilities, their susceptibility to adversarial attacks can introduce significant errors and hinder their applicability in real-world scenarios. The characteristic of black-box attacks is that they rely on the input–output mapping of the model without accessing its parameters or gradients. Due to the disparities between the substitute and target models, black-box attacks typically exhibit reduced success rates. To address the challenges outlined previously, we propose the Frequency Domain Transformation (FDT) method that employs the Discrete Cosine Transform (DCT) for transforming the input image into the frequency domain. It innovatively applies a grid mask to generate adversarial samples within the frequency domain. This transformation diminishes the spatial correlation among the image pixels and offers a fresh perspective for enhancing the transferability of adversarial examples. Experiments on adversarial attacks using the ImageNet dataset reveal that adversarial examples, when created through methods that transform data in the frequency domain, show enhanced transferability compared to those produced by transformations in the spatial domain. In attacks on six classification models using the single-model approach, FDT achieved an average success rate of 82.9%, corresponding to a 14.4% improvement over the spectrum simulation attack (SSA) method. For ensemble-model attacks with momentum-based techniques, the average success rate using the proposed strategy was 94.4%, corresponding to a 5.7% improvement over the SSA method.

Exploring Simplicity Bias in 1D Dynamical Systems.

Arguments inspired by algorithmic information theory predict an inverse relation between the probability and complexity of output patterns in a wide range of input-output maps. This phenomenon is known as simplicity bias. By viewing the parameters of dynamical systems as inputs, and the resulting (digitised) trajectories as outputs, we study simplicity bias in the logistic map, Gauss map, sine map, Bernoulli map, and tent map. We find that the logistic map, Gauss map, and sine map all exhibit simplicity bias upon sampling of map initial values and parameter values, but the Bernoulli map and tent map do not. The simplicity bias upper bound on the output pattern probability is used to make a priori predictions regarding the probability of output patterns. In some cases, the predictions are surprisingly accurate, given that almost no details of the underlying dynamical systems are assumed. More generally, we argue that studying probability-complexity relationships may be a useful tool when studying patterns in dynamical systems.

Inverse analysis of granular flows using differentiable graph neural network simulator

Inverse problems in granular flows, such as landslides and debris flows, involve estimating material parameters or boundary conditions based on a target runout profile. Traditional high-fidelity simulators for these inverse problems are computationally expensive, restricting the number of possible simulations. These simulators are also non-differentiable, making efficient gradient-based optimization methods for high-dimensional spaces inapplicable. Machine learning-based surrogate models offer computational efficiency and differentiability. However, they often struggle to generalize beyond their training data due to relying on low-dimensional input–output mappings that fail to capture the complete physics of granular flows. We propose a novel differentiable graph neural network simulator (GNS) that combines reverse mode automatic differentiation of graph neural networks with gradient-based optimization for solving inverse problems. GNS learns the dynamics of granular flow by representing the system as a graph and predicts the evolution of the graph at the next timestep, given the current state. The differentiable GNS demonstrates optimization capabilities beyond the training data. We demonstrate the effectiveness of our method for inverse estimation across single and multi-parameter optimization problems, including evaluating material properties and boundary conditions for a target runout distance and designing baffle locations to limit a landslide runout. Our proposed differentiable GNS framework solves inverse problems with orders of magnitude faster convergence than the conventional gradient-based optimization approach using finite difference.

A bilevel fast-convergent optimizer via high-fidelity convex models: Application on optimal operation of all-parallel heterogeneous chiller-pump systems

To cope with buildings’ growing and diverse cooling demand, heterogeneous multi-chiller systems with non-dedicated pumps are commonly applied in large-scale centralized chiller plants. All-parallel heterogeneous systems offer high operation flexibility but pose challenges to optimal operation planning. For the deficiencies in convexity tractability, the classical neural networks (NNs) achieved great success in system identification yet were restricted in model-based optimization. In this study, an optimization-oriented convex modeling approach based on input convex neural networks (ICNN) to identify energy consumption models for energy units of chiller-pump systems was presented, which provides convex input–output mappings while leveraging the high-fidelity capability of NNs. Using the convex models, the energy minimization issue subjected to multiple constraints was formulated, the optimality of which was mathematically proven. A bilevel deterministic optimizer was developed to determine the global optimal. Comprehensive data experiments were conducted using practical and simulated operation data to evaluate the modeling and optimization performances of the proposed methods compared with conventional candidates. Numerical results suggest that the ICNN-based convex models exhibit superior modeling performances than physics-based models. A better overall energy saving ratio of 8.86 % and faster convergence time of 3.53 s for average achieved by the proposed bilevel optimizer compared with rule-based and meta-heuristic optimizers under identical external conditions.

Bidirectional machine learning-assisted sensitivity-based stochastic searching approach for groundwater DNAPL source characterization.

In this study, we designed a machine learning-based parallel global searching method using the Bayesian inversion framework for efficient identification of dense non-aqueous phase liquid (DNAPL) source characteristics and contaminant transport parameters in groundwater. Swarm intelligence organized hybrid-kernel extreme learning machine (SIO-HKELM) was proposed to approximate the forward and inverse input-output correlation with a high accuracy using the DNAPL transport numerical simulation model. An adaptive inverse-HKELM was established for preliminary estimation of the source characteristics and contaminant transport parameters to correct prior information and generate high-quality initial starting points of parallel searching. A local accurate forward-HKELM surrogate of the numerical model was embedded in the searching system for avoiding repetitive CPU-demanding likelihood evaluations. A sensitivity-based Metropolis criterion (MC), incorporating the dynamic particle swarm optimization (SD-PSO) algorithm, was developed for improving the search ergodicity and realizing precise inversion of all the unknown variables with drastic variations in sensitivity to the likelihood function. Results showed that the generalization capability and robustness of SIO-HKELM were superior to those of the traditional machine learning methods, including KELM and support vector regression (SVR), and it sufficiently approximated the forward and inverse input-output mapping of the numerical model with testing determination coefficients of 0.9944 and 0.6440, respectively. With high-quality prior information and initial starting points generated by the adaptive inverse-HKELM feed approach, the uncertainty in the inversion outputs was reduced, and the searching process rapidly converged to reasonable posterior distributions in around 60 iterations. Compared with the widely used multichain Markov chain Monte Carlo (MCMC) approach, the parallel searching lines generated by SD-PSO-MC adequately covered the searching space, and the "equifinality" effect was more effectively restrained by reducing the relative errors of all the point estimations to less than 8%. Therefore, the real source information reflected by the statistical characteristics of the SD-PSO-MC inversion outputs was more precise than that obtained using the multichain MCMC approach.

Approximating Nonlinear Functions With Latent Boundaries in Low-Rank Excitatory-Inhibitory Spiking Networks.

Deep feedforward and recurrent neural networks have become successful functional models of the brain, but they neglect obvious biological details such as spikes and Dale's law. Here we argue that these details are crucial in order to understand how real neural circuits operate. Towards this aim, we put forth a new framework for spike-based computation in low-rank excitatory-inhibitory spiking networks. By considering populations with rank-1 connectivity, we cast each neuron's spiking threshold as a boundary in a low-dimensional input-output space. We then show how the combined thresholds of a population of inhibitory neurons form a stable boundary in this space, and those of a population of excitatory neurons form an unstable boundary. Combining the two boundaries results in a rank-2 excitatory-inhibitory (EI) network with inhibition-stabilized dynamics at the intersection of the two boundaries. The computation of the resulting networks can be understood as the difference of two convex functions and is thereby capable of approximating arbitrary non-linear input-output mappings. We demonstrate several properties of these networks, including noise suppression and amplification, irregular activity and synaptic balance, as well as how they relate to rate network dynamics in the limit that the boundary becomes soft. Finally, while our work focuses on small networks (5-50 neurons), we discuss potential avenues for scaling up to much larger networks. Overall, our work proposes a new perspective on spiking networks that may serve as a starting point for a mechanistic understanding of biological spike-based computation.