Input Space Research Articles

Preserving Fairness in AI under Domain Shift

Existing algorithms for ensuring fairness in AI use a single-shot training strategy, where an AI model is trained on an annotated training dataset with sensitive attributes and then fielded for utilization. This training strategy is effective in problems with stationary distributions, where both the training and testing data are drawn from the same distribution. However, it is vulnerable with respect to distributional shifts in the input space that may occur after the initial training phase. As a result, the time-dependent nature of data can introduce biases and performance degradation into the model predictions, even if the model is initially fair. Model retraining from scratch using a new annotated dataset is a naive solution that is expensive and time-consuming. We develop an algorithm to adapt a fair model to remain fair and generalizable under domain shift using solely new unannotated data points. We recast this learning setting as an unsupervised domain adaptation (UDA) problem. Our algorithm is based on updating the model such that the internal representation of data remains unbiased despite distributional shifts in the input space. We provide empirical validation on three common fairness datasets to show that the challenge exists in practical setting and to demonstrate the effectiveness of our algorithm.

Linear quadratic control and estimation synthesis for multi‐agent systems with application to formation flight

AbstractThis paper concerns the optimality problem of distributed linear quadratic control in a linear stochastic multi‐agent system (MAS). The main challenge stems from MAS network topology that limits access to information from non‐neighbouring agents, imposing structural constraints on the control input space. A distributed control‐estimation synthesis is proposed which circumvents this issue by integrating distributed estimation for each agent into distributed control law. Based on the agents' state estimate information, the distributed control law allows each agent to interact with non‐neighbouring agents, thereby relaxing the structural constraint. Then, the primal optimal distributed control problem is recast to the joint distributed control‐estimation problem whose solution can be obtained through the iterative optimization procedure. The stability of the proposed method is verified and the practical effectiveness is supported by numerical simulations and real‐world experiments with multi‐quadrotor formation flight.

Anomaly Detection Using Complete Cycle Consistent Generative Adversarial Network.

This research presents a robust adversarial method for anomaly detection in real-world scenarios, leveraging the power of generative adversarial neural networks (GANs) through cycle consistency in reconstruction error. Traditional approaches often falter due to high variance in class-wise accuracy, rendering them ineffective across different anomaly types. Our proposed model addresses these challenges by introducing an innovative flow of information in the training procedure and integrating it as a new discriminator into the framework, thereby optimizing the training dynamics. Furthermore, it employs a supplementary distribution in the input space to steer reconstructions toward the normal data distribution. This adjustment distinctly isolates anomalous instances and enhances detection precision. Also, two unique anomaly scoring mechanisms were developed to augment detection capabilities. Comprehensive evaluations on six varied datasets have confirmed that our model outperforms one-class anomaly detection benchmarks. The implementation is openly accessible to the academic community, available on Github.a.

A theory-informed multivariate causal framework for trustworthy short-term urban traffic forecasting

Traffic forecasting using Deep Learning has been a remarkably active and innovative research field during the last decades. However, there are still several barriers to real-world, large-scale implementation of Deep Learning forecasting models, including their data requirements, limited explainability and low efficiency. In this paper, we propose a novel theory-driven framework that is based on a Granger causality-inspired feature selection method and a multitask LSTM to jointly predict two traffic variables. Traffic flow theory intuition is induced in the training process by an enhanced Traffic Flow Theory-Informed loss function (TFTI loss), which includes the divergence of the joint prediction of two traffic variables from the actual fundamental diagram of the corresponding location. The theory-informed, Granger causal, multitask LSTM is trained for one step ahead volume and speed forecasting using loop detector data coming from the extended Athens road network (Greece). Findings indicate that the models trained using the TFTI loss and a reduced input space, which includes only causal information, achieve a significantly improved performance, compared to the models using the classic Mean Squared Error loss function. Moreover, we introduce a dedicated trustworthiness evaluation framework that indicates that the proposed approach enhances the trustworthiness of the predictions, as well as the models’ transparency and resilience to noisy data.

All you need is rotation: Construction of developable strips

We present a novel approach to generate developable strips along a space curve. The key idea of the new method is to use the rotation angle between the Frenet frame of the input space curve, and its Darboux frame of the curve on the resulting developable strip as a free design parameter, thereby revolving the strip around the tangential axis of the input space curve. This angle is not restricted to be constant but it can be any differentiable function defined on the curve, thereby creating a large design space of developable strips that share a common directrix curve. The range of possibilities for choosing the rotation angle is diverse, encompassing constant angles, linearly varying angles, sinusoidal patterns, and even solutions derived from initial value problems involving ordinary differential equations. This enables the potential of the proposed method to be used for a wide range of practical applications, spanning fields such as architectural design, industrial design, and papercraft modeling. In our computational and physical examples, we demonstrate the flexibility of the method by constructing, among others, toroidal and helical windmill blades for papercraft models, curved foldings, triply orthogonal structures, and developable strips featuring a log-aesthetic directrix curve.

Meaning And Humor Comprehension of Chinese-English Pseudo Words Among Chinese EFL Speakers: A Conceptual Blending Perspective

Assuming that Chinese–English pseudo words (CEPWs) are humorous and have multiple meanings, this study, based on the conceptual blending theory (CBT), investigates the comprehension of meaning and humor in CEPWs for Chinese speakers learning English as a foreign language (EFL). Specifically, it aimed to answer the following questions: 1) What is the humorous effect of CEPWs as perceived by Chinese EFL speakers? 2) What is the comprehensive understanding of CEPWs among Chinese EFL speakers according to CBT? and 3) How are Chinese EFL speakers’ understanding of CEPWs and their perception of the humorous effect related? This study adopted a mixed method of quantitative (questionnaire) and qualitative (semi-structured interview) approaches. It surveyed 114 native Chinese EFL speakers and interviewed 12 of them. Quantitative data were collected via a questionnaire disseminated on the online questionnaire platform wenjuan.com, gathering 114 valid responses. After collection, the quantitative data were analyzed using SPSS Statistics 20 software. The findings revealed that CEPWs are more humorous than their Chinese originals. A conceptual blending model of Chinese EFL speakers’ comprehension of CEPWs was proposed, which allows these speakers to generate numerous interpretations of CEPWs. The humor in CEPWs comes from homophony, bilingual speech, and conceptual blending, facilitated by the meaning correspondence of English words in the CEPW and context. This suggests that the incongruity between input spaces and the emergent structure of conceptual blending is involved in creating humor. CEPWs meet the criteria for puns, providing a new perspective on this phenomenon. The findings provided empirical data, based on which a model of Chinese EFL speakers’ comprehension process of CEPWs was devised, providing insights into the cognitive process behind the use of Internet language.

Llasm: Naming Functions in Binaries by Fusing Encoder-only and Decoder-only LLMs

Predicting function names in stripped binaries, which requires succinctly summarizing semantics of binary code in natural languages, is a crucial but challenging task. Recently, many machine learning based solutions have been proposed. However, they have poor generalizability, i.e., fail to handle unseen binaries. To advance the state of the art, we present llasm ( L arge AS sembly L anguage M odel), a novel framework which fuses encoder-only and decoder-only LLMs for function name prediction. It refines encoder-only models to preserve more binary information and learn better binary representations. Then it adopts a novel architecture to project the encoding to the input space of a decoder-only natural language model, which enables it to have better capability of inferring general knowledge and better generalizability. We have evaluated llasm in the BinaryCorp and Debin datasets. llasm outperforms the state-of-the-art function name prediction tools by up to 19.9%, 40.7%, and 36.5% in precision, recall, and F1 score, with significantly better generalizability in unseen binaries. Our case studies further demonstrate the practical use cases of llasm in analyzing real-world malware, showing the usefulness of function name prediction.

Understanding with Toy Surrogate Models in Machine Learning

In the natural and social sciences, it is common to use toy models—extremely simple and highly idealized representations—to understand complex phenomena. Some of the simple surrogate models used to understand opaque machine learning (ML) models, such as rule lists and sparse decision trees, bear some resemblance to scientific toy models. They allow non-experts to understand how an opaque ML model works globally via a much simpler model that highlights the most relevant features of the input space and their effect on the output. The obvious difference is that the common target of a toy and a full-scale model in the sciences is some phenomenon in the world, while the target of a surrogate model is another model. This essential difference makes toy surrogate models (TSMs) a new object of study for theories of understanding, one that is not easily accommodated under current analyses. This paper provides an account of what it means to understand an opaque ML model globally with the aid of such simple models.

Cognitive Analysis of Euphemisms in The Big Bang Theory under CBT

This research delves into the pragmatic function of euphemisms in the context of the American sitcom The Big Bang Theory, utilizing Conceptual Blending Theory (CBT) as the analytical framework. The purpose is to understand how euphemisms, as a form of polite language, are constructed to avoid direct and potentially offensive expressions, thus promoting social harmony. The methodology involves a detailed analysis of euphemistic language within the sitcom, focusing on the integration of mental spaces as proposed by CBT. The study identifies key euphemisms and constructs a conceptual integration network for each, revealing the input spaces, generic space, and the blended space that emerges, offering new meanings. The main results demonstrate that euphemisms are cognitively processed through the blending of spaces, where elements from different domains combine to form a less confrontational meaning. The conclusion underscores the efficacy of CBT in explaining the complexity of euphemistic language, showcasing its role in navigating social norms and cultural sensitivities. This research provides insights into American cultural nuances and the strategic use of language in social interactions, enhancing the understanding of euphemisms as a cross-cultural communication tool.

Strata Design for Variance Reduction in Stochastic Simulation

Stratified sampling is one of the powerful variance reduction methods for analyzing system performance, such as reliability, with stochastic simulation. It divides the input space into disjoint subsets, called strata, to draw samples from each stratum. Partitioning the input space properly and allocating greater computational effort to crucial strata can help accurately estimate system performance with a limited computational budget. How to create strata, however, has yet to be thoroughly examined. Strata design faces the curse of dimensionality and data scarcity as the input dimension increases. We analytically derive the optimal stratification structure that minimizes the estimation variance for univariate problems. Further, reconciling the optimal stratification into decision trees, we devise a robust algorithm for multi-dimensional problems. Numerical experiments and a wind turbine case study demonstrate the superiority of the proposed method in terms of variance reduction, leading to computational efficiency and scalability.

A model of task-level human stepping regulation yields semistable walking.

A simple lateral dynamic walker, with swing leg dynamics and three adjustable input parameters, is used to study how motor regulation affects frontal-plane stepping. Motivated by experimental observations and phenomenological models, we imposed task-level multi-objective regulation targeting the walker's optimal lateral foot placement at each step. The regulator prioritizes achieving step width and lateral body position goals to varying degrees by choosing a mixture parameter. Our model thus integrates a lateral mechanical template, which captures the fundamental mechanics of frontal-plane walking, with a lateral motor regulation template, an empirically verified model of how humans manipulate lateral foot placements in a goal-directed manner. The model captures experimentally observed stepping fluctuation statistics and demonstrates how linear empirical models of stepping dynamics can emerge from first-principles nonlinear mechanics. We find that task-level regulation gives rise to a goal-equivalent manifold in the system's extended state space of mechanical states and inputs, a subset of which contains a continuum of period-1 gaits forming a semistable set: perturbations off of any of its gaits result in transients that return to the set, though typically to different gaits.

RandONets: Shallow networks with random projections for learning linear and nonlinear operators

Deep neural networks have been extensively used for the solution of both the forward and the inverse problem for dynamical systems. However, their implementation necessitates optimizing a high-dimensional space of parameters and hyperparameters. This fact, along with the requirement of substantial computational resources, pose a barrier to achieving high numerical accuracy, but also interpretability. Here, to address the above challenges, we present Random Projection-based Operator Networks (RandONets): shallow networks with random projections and tailor-made numerical analysis methods that learn accurately and fast linear and nonlinear operators. Building on previous works, we prove that RandOnets are universal approximators of linear and nonlinear operators. Due to their simplicity, RandONets provide a one-step transformation of the input space, facilitating interpretability. For the evaluation of their performance, we focus on operators of PDEs. We show, that RandONets outperform by several orders of magnitude, both in terms of numerical approximation accuracy and computational cost, the “vanilla” DeepONets. Hence, we believe that our method will trigger further developments in the field of scientific machine learning, for the development of new ‘’light”schemes that will provide high accuracy while reducing dramatically the computational cost. A MATLAB toolbox for RandONets, including demos, is available on GitHub at https://github.com/GianlucaFabiani/RandONets.

Application of Fuzzy Mamdani Method to Predict the Number of Blood Bags Based on Demand and Supply Data Using Matlab

Fuzzy logic is a control system technique in solving problems and is applied to systems, from basic systems to difficult or complex systems. Fuzzy logic is the proper method to plan an input space into an output space using MATLAB's mathematical theory of fuzzy sets. The reason for using fuzzy logic is because it is related to uncertainty. The unstable demand for blood bags in hospitals makes the supply of blood bags excessive or lacking from demand. The lack of blood supply results in the unfulfilled demand for blood needed by the hospital, while the excess blood supply worsens the quality of blood. In this study, we will predict the number of bags produced using the Mamdani Fuzzy Inference System (FIS) method based on the minimum demand and maximum demand values and the minimum supply and maximum supply that produce output from the defuzzification process. Applying the Mamdani Fuzzy Inference System (FIS) method based on demand and supply data obtains optimal output with MATLAB in predicting the number of blood bags produced. The results of the study showed that the Mean Absolute Percentage Error (MAPE) fuzzy logic Mamdani error value was 24%, the accuracy value of the Fuzzy Inference System (FIS) Mamdani in determining the number of blood bag production was 76%, and the production output generated through the Fuzzy Inference System (FIS) Mamdani was 4,774 blood bags. The number of blood requests at the hospital is 4,443 blood bags, so the amount of blood that must be produced to meet the hospital's demand is 4,774 bags.

Uncertainty quantification of acoustic metamaterial bandgaps with stochastic material properties and geometric defects

Acoustic metamaterials are a subject of increasing study and utility. Through designed combinations of geometries with material properties, acoustic metamaterials can be built to arbitrarily manipulate acoustic waves for various applications. Despite the theoretical advances in this field, however, acoustic metamaterials have seen limited penetration into industry and commercial use. This is largely due to the difficulty of manufacturing the intricate geometries that are integral to their function and the sensitivity of metamaterial designs to material batch variability and manufacturing defects. Capturing the effects of stochastic material properties and geometric defects requires empirical testing of manufactured samples, but this can quickly become prohibitively expensive with higher precision requirements or with an increasing number of input variables. This paper demonstrates how uncertainty quantification techniques, and more specifically the use of polynomial chaos expansions and spectral projections, can be used to greatly reduce sampling needs for characterizing acoustic metamaterial dispersion curves. With a novel method of encoding geometric defects in a 1D, interpretable, resolution-independent way, our uncertainty quantification approach allows for both stochastic material properties and geometric defects to be considered simultaneously. Two to three orders of magnitude sampling reductions down to ∼100 and ∼101 were achieved in 1D and 7D input space scenarios, respectively. Remarkably, this reduction in sampling was possible while preserving accurate output probability distributions of the metamaterial performance characteristics (bandgap size and location).

Order Effects in Second Language Learning

AbstractWe explore how general principles of learning apply to and combine with usage‐based approaches to language learning and teaching, with a focus on the effects of order of exposure to new information in second language (L2) instruction. Although the effects of input spacing and timing on memory and learning have been previously explored (see Rogers, 2020; Shintani, 2017, for an overview), the effects of order of exposure to the input remain understudied. In this study, we tested whether order of exposure to information in the L2 plays a role in L2 learning and whether an optimal order where the most reliable cues are introduced first ensures that foreign language learners are better equipped to form representations that allow flexible, yet accurate generalisations. We analysed data collected through a training study that teaches the English article system to first language (L1) speakers whose language does not have markers of definiteness.

A nested non-intrusive stochastic isogeometric method for nonlinear thermal vibration of FGM plates under random loading

This paper introduces an adaptive nested non-invasive dynamic SIGA method based on RBFNN for the nonlinear thermal vibration analysis of FGM plates under random loading. This method is robust and accurate in high-dimensional input spaces regardless of the sample sequence, addresses the low convergence rate of QMCS, and solves the stochastic dynamic response probability density function under random loading. Firstly, the multidimensional input space for random loading is constructed based on several mutually independent random variables via the stationary Gaussian stochastic process simulation technique. The nonlinear FGM plate model subjected to random loading in the thermal environment is modeled using HSDT with von Kármán strain-displacement relation within the IGA framework. Subsequently, a nested non-invasive dynamic response surface analysis method, SIGA-RBFNN, is proposed based on the RBFNN technique. Analysis of the FGM plate response under random loading demonstrates that SIGA-RBFNN, compared to tensor-product-based SIGA methods, is less affected by sample sequence types. It effectively addresses high-dimensional stochasticity and offers significant advantages in robustness, accuracy, and efficiency. Finally, SIGA-RBFNN compensates for the low convergence speed of QMCS and successfully solves the FGM plate displacement response statistics and probability density function under random loading.

The primacy model and the structure of olfactory space.

Understanding sensory processing relies on the establishment of a consistent relationship between the stimulus space, its neural representation, and perceptual quality. In olfaction, the difficulty in establishing these links lies partly in the complexity of the underlying odor input space and perceptual responses. Based on the recently proposed primacy model for concentration invariant odor identity representation and a few assumptions, we have developed a theoretical framework for mapping the odor input space to the response properties of olfactory receptors. We analyze a geometrical structure containing odor representations in a multidimensional space of receptor affinities and describe its low-dimensional implementation, the primacy hull. We propose the implications of the primacy hull for the structure of feedforward connectivity in early olfactory networks. We test the predictions of our theory by comparing the existing receptor-ligand affinity and connectivity data obtained in the fruit fly olfactory system. We find that the Kenyon cells of the insect mushroom body integrate inputs from the high-affinity (primacy) sets of olfactory receptors in agreement with the primacy theory.

Gaussian Process-Gated Hierarchical Mixtures of Experts.

In this article, we propose novel Gaussian process-gated hierarchical mixtures of experts (GPHMEs). Unlike other mixtures of experts with gating models linear in the input, our model employs gating functions built with Gaussian processes (GPs). These processes are based on random features that are non-linear functions of the inputs. Furthermore, the experts in our model are also constructed with GPs. The optimization of the GPHMEs is performed by variational inference. The proposed GPHMEs have several advantages. They outperform tree-based HME benchmarks that partition the data in the input space, and they achieve good performance with reduced complexity. Another advantage is the interpretability they provide for deep GPs, and more generally, for deep Bayesian neural networks. Our GPHMEs demonstrate excellent performance for large-scale data sets, even with quite modest sizes.

Information based explanation methods for deep learning agents—with applications on large open-source chess models

With large chess-playing neural network models like AlphaZero contesting the state of the art within the world of computerised chess, two challenges present themselves: the question of how to explain the domain knowledge internalised by such models, and the problem that such models are not made openly available. This work presents the re-implementation of the concept detection methodology applied to AlphaZero, by using large, open-source chess models with comparable performance. We obtain results similar to those achieved when applying this methodology to AlphaZero, while relying solely on open-source resources. We also present a novel explainable AI (XAI) method, which is guaranteed to highlight exhaustively and exclusively the information used by the explained model. This method generates visual explanations tailored to domains characterised by discrete input spaces, as is the case for chess. Our presented method has the desirable property of controlling the information flow between any input vector and the given model, which in turn provides strict guarantees regarding what information is used by the trained model during inference. We demonstrate the viability of our method by applying it to standard 8×8\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$8 \ imes 8$$\\end{document} chess, using large open-source chess models.

Bidirectional generative adversarial representation learning for natural stimulus synthesis.

Thousands of species use vocal signals to communicate with one another. Vocalizations carry rich information, yet characterizing and analyzing these complex, high-dimensional signals is difficult and prone to human bias. Moreover, animal vocalizations are ethologically relevant stimuli whose representation by auditory neurons is an important subject of research in sensory neuroscience. A method that can efficiently generate naturalistic vocalization waveforms would offer an unlimited supply of stimuli with which to probe neuronal computations. Although unsupervised learning methods allow for the projection of vocalizations into low-dimensional latent spaces learned from the waveforms themselves, and generative modeling allows for the synthesis of novel vocalizations for use in downstream tasks, we are not aware of any model that combines these tasks to synthesize naturalistic vocalizations in the waveform domain for stimulus playback. In this paper, we demonstrate BiWaveGAN: a bidirectional generative adversarial network (GAN) capable of learning a latent representation of ultrasonic vocalizations (USVs) from mice. We show that BiWaveGAN can be used to generate, and interpolate between, realistic vocalization waveforms. We then use these synthesized stimuli along with natural USVs to probe the sensory input space of mouse auditory cortical neurons. We show that stimuli generated from our method evoke neuronal responses as effectively as real vocalizations, and produce receptive fields with the same predictive power. BiWaveGAN is not restricted to mouse USVs but can be used to synthesize naturalistic vocalizations of any animal species and interpolate between vocalizations of the same or different species, which could be useful for probing categorical boundaries in representations of ethologically relevant auditory signals.NEW & NOTEWORTHY A new type of artificial neural network is presented that can be used to generate animal vocalization waveforms and interpolate between them to create new vocalizations. We find that our synthetic naturalistic stimuli drive auditory cortical neurons in the mouse equally well and produce receptive field features with the same predictive power as those obtained with natural mouse vocalizations, confirming the quality of the stimuli produced by the neural network.