Input Space Research Articles

Progressive Expansion Sampling of Quasi-Static Magnetic Fields in Unconfined Regions

We present a fast and accurate measurement technique for quasi-static magnetic fields by employing a progressive sampling method in an unconfined input space. The proposed machine learning algorithm is tested against uniform sampling on printed circuit board test structures and a buck converter. We prove allocation of multiple, separated regions with predefined lateral field limits at MHz frequencies. The feasibility of equivalent magnetic dipole source modeling based on a small number of samples is demonstrated. Compared to uniform testing, progressive expansion sampling identifies contours of given field limits in less than 3% of the reference measurement time.

TestSGD : Interpretable Testing of Neural Networks against Subtle Group Discrimination

Discrimination has been shown in many machine learning applications, which calls for sufficient fairness testing before their deployment in ethic-relevant domains. One widely concerning type of discrimination, testing against group discrimination, mostly hidden , is much less studied, compared with identifying individual discrimination . In this work, we propose TestSGD , an interpretable testing approach that systematically identifies and measures hidden (which we call “subtle”) group discrimination of a neural network characterized by conditions over combinations of the sensitive attributes . Specifically, given a neural network, TestSGD first automatically generates an interpretable rule set that categorizes the input space into two groups. Alongside, TestSGD also provides an estimated group discrimination score based on sampling the input space to measure the degree of the identified subtle group discrimination, which is guaranteed to be accurate up to an error bound. We evaluate TestSGD on multiple neural network models trained on popular datasets including both structured data and text data. The experiment results show that TestSGD is effective and efficient in identifying and measuring such subtle group discrimination that has never been revealed before. Furthermore, we show that the testing results of TestSGD can be used to mitigate such discrimination through retraining with negligible accuracy drop.

ExpanStick: Augmenting Gamepad Thumbstick using Force at the Limit

We present ExpanStick, a novel thumbstick that augments video game interaction using the force that the user applies at the physical limit of the gamepad. Gamers frequently manipulate a thumbstick leaning on the limit, or physical boundary around it, either to apply maximum input or to secure physical support, which makes the contact force an easily accessible input channel. The ExpanStick preserves the form factor and usability of the thumbstick, while expanding its input space. We built the ExpanStick and inserted it in a commercial gamepad as a proof-of-concept prototype. Using the prototype, we conducted a design workshop with console gamers to design the ExpanStick game interaction. Through demonstration of the interactions, we found the early use experience with the ExpanStick is positive and well accepted by the game players.

In Pursuit of the Exceptional: Research Directions for Machine Learning in Chemical and Materials Science.

Exceptional molecules and materials with one or more extraordinary properties are both technologically valuable and fundamentally interesting, because they often involve new physical phenomena or new compositions that defy expectations. Historically, exceptionality has been achieved through serendipity, but recently, machine learning (ML) and automated experimentation have been widely proposed to accelerate target identification and synthesis planning. In this Perspective, we argue that the data-driven methods commonly used today are well-suited for optimization but not for the realization of new exceptional materials or molecules. Finding such outliers should be possible using ML, but only by shifting away from using traditional ML approaches that tweak the composition, crystal structure, or reaction pathway. We highlight case studies of high-Tc oxide superconductors and superhard materials to demonstrate the challenges of ML-guided discovery and discuss the limitations of automation for this task. We then provide six recommendations for the development of ML methods capable of exceptional materials discovery: (i) Avoid the tyranny of the middle and focus on extrema; (ii) When data are limited, qualitative predictions that provide direction are more valuable than interpolative accuracy; (iii) Sample what can be made and how to make it and defer optimization; (iv) Create room (and look) for the unexpected while pursuing your goal; (v) Try to fill-in-the-blanks of input and output space; (vi) Do not confuse human understanding with model interpretability. We conclude with a description of how these recommendations can be integrated into automated discovery workflows, which should enable the discovery of exceptional molecules and materials.

Uncertainty Quantification for Thermodynamic Simulations with High-Dimensional Input Spaces Using Sparse Polynomial Chaos Expansion: Retrofit of a Large Thermal Power Plant

The assessment of the future thermodynamics performance of a retrofitted heat and power production unit is prone to many uncertainties due to the large number of parameters involved in the modeling of all its components. To carry out uncertainty quantification analysis, alternatives to the traditional Monte Carlo method must be used due to the large stochastic dimension of the problem. In this paper, sparse polynomial chaos expansion (SPCE) is applied to the retrofit of a large coal-fired power plant into a biomass-fired combined heat and power unit to quantify the main drivers and the overall uncertainty on the plant’s performance. The thermodynamic model encompasses over 180 components and 1500 parameters. A methodology combining the use of SPCE and expert judgment is proposed to narrow down the sources of uncertainty and deliver reliable probability distributions for the main key performance indicators (KPIs). The impact of the uncertainties on each input parameter vary with the considered KPI and its assessment through the computation of Sobol’ indices. For both coal and biomass operations, the most impactful input parameters are the composition of the fuel and its heating value. The uncertainty on the performance and steam quality parameters is not much affected by the retrofit. Key furnace parameters exhibit a skewed probability distribution with large uncertainties, which is a strong attention point in terms of boiler operation and maintenance.

Using an Opportunity Matrix to Select Centers for RBF Neural Networks

When designed correctly, radial basis function (RBF) neural networks can approximate mathematical functions to any arbitrary degree of precision. Multilayer perceptron (MLP) neural networks are also universal function approximators, but RBF neural networks can often be trained several orders of magnitude more quickly than an MLP network with an equivalent level of function approximation capability. The primary challenge with designing a high-quality RBF neural network is selecting the best values for the network’s “centers”, which can be thought of as geometric locations within the input space. Traditionally, the locations for the RBF nodes’ centers are chosen either through random sampling of the training data or by using k-means clustering. The current paper proposes a new algorithm for selecting the locations of the centers by relying on a structure known as an “opportunity matrix”. The performance of the proposed algorithm is compared against that of the random sampling and k-means clustering methods using a large set of experiments involving both a real-world dataset from the steel industry and a variety of mathematical and statistical functions. The results indicate that the proposed opportunity matrix algorithm is almost always much better at selecting locations for an RBF network’s centers than either of the two traditional techniques, yielding RBF neural networks with superior function approximation capabilities.

Smart manufacturing under limited and heterogeneous data: a sim-to-real transfer learning with convolutional variational autoencoder in thermoforming

ABSTRACT Data in advanced manufacturing are often sparse and collected from various sensory devices in a heterogeneous and multi-modal fashion. Thus, for such intricate input spaces, learning robust and reliable predictive models for product quality assessments entails implementing complex nonlinear models such as deep learning. However, these ‘data-greedy’ models require massive datasets for training, and they tend to exhibit poor generalization performance otherwise. To address the data paucity and the data heterogeneity in smart manufacturing applications, this paper introduces a sim-to-real transfer-learning framework. Specifically, using a unified wide-and-deep learning approach, the model pre-processes structured sensory data (wide) as well as high-dimensional thermal images (deep) separately, and then passes the respective concatenated features to a regressor for predicting product quality metrics. Convolutional variational autoencoder (ConvVAE) is utilized to learn concise representations of thermal images in an unsupervised fashion. ConvVAE is trained via a sim-to-real transfer learning approach, backed by theory-based heat transfer simulations. The proposed metamodeling framework was evaluated in an industrial thermoforming process case study. The results suggested that ConvVAE outperforms conventional dimensionality reduction methods despite limited data. A model explainability analysis was conducted and the resulting SHAP values demonstrated the agreement between the model’s predictions, theoretical expectations, and data correlation statistics.

An Intelligent Decision Support System for Crop Yield Prediction Using Machine Learning and Deep Learning Algorithms

Agriculture is crucial to economic growth and development. Crop yield forecasting is critical for food production which includes vegetables, fruits, flowers, and cattle. Artificial Intelligence (AI) is rising in agriculture, providing farmers with real-time or long-term insights about their fields. It allows us to identify the areas that require irrigation, fertilization, or pesticide treatment. Statistical models struggle to track complex relationships in crop yields due to numerous factors. Machine Learning (ML) and Deep Learning (DL) algorithms can solve this problem by training themselves in these relationships, enabling accurate predictions in agricultural yield prediction methods. Predicting product performance in agriculture is challenging due to various factors, but profit forecasting improves decision-making, production, economics, and food safety. The present study focuses on the use of ML and DL algorithms to suggest a novel decision support system for crop yield prediction with the objectives to develop a robust, accurate model, investigate algorithm effectiveness, and create a user-friendly system for informed crop production decisions. According to the results, the developed system is capable of making precise predictions, which can support farmers in making better decisions about how to manage their crops. The simulation results demonstrate that the intelligent decision support system proposed for crop yield prediction using ML and DL algorithms is capable of achieving high accuracy and precision. The system can be used to help farmers make better decisions about crop planting and management, which can lead to increased crop yields and profits. The results of our experiment show that our model is better than the others and it achieves an accuracy of 99.82 %. Additionally, we utilized ML to condense the input space while preserving high accuracy.

An unsupervised leak detection method with aggregating prediction and reconstruction along projection pathway for natural gas gathering pipelines

Early detection of leaks in natural gas gathering pipelines makes proactive maintenance and corrective measures take place more effective, thus enhancing the safe and reliable operation of pipelines and mitigating serious threats to the environment and human life. The data-driven leak detection methods have become a natural choice owing to the extensive deployment of sensors in the pipeline network. However, the supervised and semi-supervised leak detection methods need to utilize a certain amount of leak data to train reliable classifiers, and labeled data usually difficult to obtain in real-world applications especially for the gas industry. Besides, most of data-driven methods today only consider the long-term or short-term patterns hidden in the multivariate time series rather than both, which may reduce their effectiveness since temporal data generated in the real-world applications often relates to a mixture of these two patterns. To settle this challenge, we propose a novel unsupervised leak detection method by utilizing a one-dimensional convolutional autoencoder (1D-CAE) to learn short-term dependency patterns from local time series and a long short-term memory (LSTM) to discover evolving trends about long-term patterns from entire temporal data. The proposed method not only calculates the difference between actual samples and their predictions but also compares the inputs and their reconstructions, where comparisons of reconstruction are further extended from input space to hidden spaces. The obtained deviations are then integrated through the developed integration strategy and employed to calculate their global leak scores. Specially, considering that the magnitudes of deviations integrated together are diverse or there may exist correlation among these deviations caused by correlated neurons across layers, the minimum covariance determinant (MCD) method is employed to scale deviations and eliminate correlations along the projection pathway. The effectiveness of the proposed method is verified on real datasets of natural gas gathering pipeline. In addition, analysis of experimental results demonstrates that the proposed method has good accuracy, and is conducive to operating in the context of uncontrolled long-term monitoring.

Analysis of Function Approximation and Stability of General DNNs in Directed Acyclic Graphs Using Un-Rectifying Analysis

A general lack of understanding pertaining to deep feedforward neural networks (DNNs) can be attributed partly to a lack of tools with which to analyze the composition of non-linear functions, and partly to a lack of mathematical models applicable to the diversity of DNN architectures. In this study, we analyze DNNs using directed acyclic graphs (DAGs) under a number of basic assumptions pertaining to activation functions, non-linear transformations, and DNN architectures. DNNs that satisfy these assumptions are referred to as general DNNs. Our construction of an analytic graph was based on an axiomatic method in which DAGs are built from the bottom–up through the application of atomic operations to basic elements in accordance with regulatory rules. This approach allowed us to derive the properties of general DNNs via mathematical induction. We demonstrate that the proposed analysis method enables the derivation of some properties that hold true for all general DNNs, namely that DNNs “divide up” the input space, “conquer” each partition using a simple approximating function, and “sparsify” the weight coefficients to enhance robustness against input perturbations. This analysis provides a systematic approach with which to gain theoretical insights into a wide range of complex DNN architectures.

Model predictive PESQ-ANFIS/FUZZY C-MEANS for image-based speech signal evaluation

This paper presents a new method to evaluate the quality of speech signals through images generated from a psychoacoustic model to estimate PESQ (ITU-T P862) values using a first-order Fuzzy Sugeno approach implemented in the Adaptive Neuro-Fuzzy Inference System - ANFIS. The factors feeding the network were obtained using an image-processing technique from the perceptual model coefficients. All simulations were performed using a database containing clean and corrupted signals by eight types of noises found in everyday situations. The proposal uses the PESQ values of the signals to train the network. The analyses proved that the predictive performance will depend on the choice of a psychoacoustic model, the factor extraction technique, the combination of these factors, the fuzzification algorithm, and the type of membership function in the ANFIS input space. The data sets for training and testing for each signal directory were randomly created and executed fifty times. The proposal achieves the best prediction values for PESQ when the averages of the measurements reach MAPE ≤0.09, RMSE ≤0.20, and R2≥95. In general, the approach provided satisfactory results compared to Multilayer Perceptron networks with their different learning algorithms, compared to another psychoacoustic model, to ITU-T P.563 and other non-intrusive methods that evaluate the quality of voice signals, and it was efficient regardless of the number of signals and the database used.

Forecasting of Flood Flow of Panam River Basin using Adaptive Neuro-Fuzzy Inference System (ANFIS) and ANN with Comparative Study

Flood forecasting is one of the most important issues in the hydrology due to its indispensable contribution in lowering monetary and life losses. In recent years reliability of flood forecasting using various modelling tools has improved to a great extent due to hydrologic modelling, development in expertise and knowledge, and advancement in the collection of data and algorithms for evaluation. The current study presents the use of the Adaptive Neuro Fussy Inference System (ANFIS) in forecasting floods for the Panam River basin system. ANFIS combines neural network algorithms and fuzzy reasoning to map an input space to an output space. This paper includes the development of ANFIS models using various membership function and their comparison for forecasting the inflow rate into the Panam dam, which creates flooding conditions on the downstream side. The different evaluation parameters like Root Mean Square Error (RMSE), Correlation Coefficient (R), Coefficient of Determination (R2) and Discrepancy Ratio (D) are used to evaluate the results of each model. From all the developed ANFIS models, the best ANFIS model is selected, having RMSE as 271.91, R as 0.98, R2 as 0.96, and D as 1.00 for training the model, and RMSE as 2000.74, R as 0.95, R2 as 0.90 and D as 1.12 for validating the model. Artificial Neural Network (ANN) model has also been developed for forecasting flood. In the Neural network there are total 3 types of transfer function in each layer i.e., LOGSIG, TANSIG and PURELIN. All the developed is evaluated with the coefficient of correlation (R) and Mean Squared Error (MSE). In the end, future headings for innovative research work are discussed.

Multi‐fidelity data fusion through parameter space reduction with applications to automotive engineering

AbstractMulti‐fidelity models are of great importance due to their capability of fusing information coming from different numerical simulations, surrogates, and sensors. We focus on the approximation of high‐dimensional scalar functions with low intrinsic dimensionality. By introducing a low dimensional bias we can fight the curse of dimensionality affecting these quantities of interest, especially for many‐query applications. We seek a gradient‐based reduction of the parameter space through linear active subspaces or a nonlinear transformation of the input space. Then we build a low‐fidelity response surface based on such reduction, thus enabling nonlinear autoregressive multi‐fidelity Gaussian process regression without the need of running new simulations with simplified physical models. This has a great potential in the data scarcity regime affecting many engineering applications. In this work we present a new multi‐fidelity approach that involves active subspaces and the nonlinear level‐set learning method, starting from the preliminary analysis previously conducted (Romor F, Tezzele M, Rozza G. Proceedings in Applied Mathematics & Mechanics. Wiley Online Library; 2021). The proposed framework is tested on two high‐dimensional benchmark functions, and on a more complex car aerodynamics problem. We show how a low intrinsic dimensionality bias can increase the accuracy of Gaussian process response surfaces.

Visualising flight regimes using self-organising maps

AbstractThe purpose of this paper is to group the flight data phases based on the sensor readings that are most distinctive and to create a representation of the higher-dimensional input space as a two-dimensional cluster map. The research design includes a self-organising map framework that provides spatially organised representations of flight signal features and abstractions. Flight data are mapped on a topology-preserving organisation that describes the similarity of their content. The findings reveal that there is a significant correlation between monitored flight data signals and given flight data phases. In addition, the clusters of flight regimes can be determined and observed on the maps. This suggests that further flight data processing schemes can use the same data marking and mapping themes regarding flight phases when working on a regime basis. The contribution of the research is the grouping of real data flows produced by in-flight sensors for aircraft monitoring purposes, thus visualising the evolution of the signal monitored on a real aircraft.

Penerapan Metode Segmentasi Gabor Filter Dan Algoritma Support Vector Machine Untuk Pendeteksian Penyakit Daun Tomat

This research discusses about processing a formulation that we can give to diseased tomato leaves. Gabor Filter is a method used to detect textures using frequency and orientation parameters. The Support Vector Machine (SVM) algorithm is an algorithm that can be used classifying tomato leaf diseases. The purpose of this research is to determine the accuracy of the Gabor Filter segmentation and the Support Vector Machine Algorithm for detecting tomato leaf disease to facilitate farmers in analyzing diseases on tomato leaves. The input will go through pre-processing of RGB pixels to Greyscale ones before being processed using Gabor Filter. This Gabor Filter process segments the image to produce a magnitude value. The results of the image magnitude values here will be seen and will enter the classification process using SVM. The SVM algorithm aims to find the best hyperlane on tomato leaves that have been segmented to separate classes in the input space. The application of the SVM method with class classification of tomato leaves by calculating the energy value and entropy of the extraction results, assisted by 12 features, namely: CiriR, Feature G, FeatureB, Standard DeviationR, Standard DeviationG, Standard DeviationB, SkewnessR, SkewnessG, SkewnessB, Mean, Energy, Entropy are used to the simplity classification process with a high degree of accuracy. The process of classification of tomato leaf disease with test data of 600 images managed to get an accuracy value of 74.1667%. In order to facilitate the performance of farmers in predicting tomato leaf disease.

Density-Based Distance Preserving Graph: Theoretical and Practical Analyses.

This brief aims to provide theoretical guarantee and practical guidance on constructing a type of graphs from input data via distance preserving criterion. Unlike the graphs constructed by other methods, the targeted graphs are hidden through estimating a density function of latent variables such that the pairwise distances in both the input space and the latent space are retained, and they have been successfully applied to various learning scenarios. However, previous work heuristically treated the multipliers in the dual as the graph weights, so the interpretation of this graph from a theoretical perspective is still missing. In this brief, we fill up this gap by presenting a detailed interpretation based on optimality conditions and their connections to neighborhood graphs. We further provide a systematic way to set up proper hyperparameters to prevent trivial graphs and achieve varied levels of sparsity. Three extensions are explored to leverage different measure functions, refine/reweigh an initial graph, and reduce computation cost for medium-sized graph. Extensive experiments on both synthetic and real datasets were conducted and experimental results verify our theoretical findings and the showcase of the studied graph in semisupervised learning provides competitive results to those of compared methods with their best graph.

Optimal supervised reduction of high dimensional transcription data.

The plight of navigating high-dimensional transcription datasets remains a persistent problem. This problem is further amplified for complex disorders, such as cancer as these disorders are often multigenic traits with multiple subsets of genes collectively affecting the type, stage, and severity of the trait. We are often faced with a trade off between reducing the dimensionality of our datasets and maintaining the integrity of our data. To accomplish both tasks simultaneously for very high dimensional transcriptome for complex multigenic traits, we propose a new supervised technique, Class Separation Transformation (CST). CST accomplishes both tasks simultaneously by significantly reducing the dimensionality of the input space into a one-dimensional transformed space that provides optimal separation between the differing classes. Furthermore, CST offers an means of explainable ML, as it computes the relative importance of each feature for its contribution to class distinction, which can thus lead to deeper insights and discovery. We compare our method with existing state-of-the-art methods using both real and synthetic datasets, demonstrating that CST is the more accurate, robust, scalable, and computationally advantageous technique relative to existing methods. Code used in this paper is available on https://github.com/richiebailey74/CST.

Levels of Schematicity in Metaphorical Animal Nicknames: The Case Study of Koczkodan Nickname in Polish

The aim of this paper is to recognise the contemporary meaning of the nickname koczkodan in Polish [English: hag, frump; literally: a vervet monkey], based on extracts of real discurse taken from the Polish Corpus. This meaning has been obtained by examining the metaphors underlying the nickname, which are motivated by various contextual factors, e.g. the speaker’s gender and discourse registers in which the phrase is pronounced. Accordingly, the working CMT-CMA hybrid model is applied for the purpose of the study, which derives both from the extended view of Conceptual Metaphor Theory (CMT) and from the discursive and critical view of metaphor within Critical Metaphor Analysis (CMA). The results obtained in the study reveal that the meaning of the koczkodan nickname is based on six metaphorical dynamic structures organised hierarchically, from the most schematic image schema that emerges at the very moment when the speaker needs to use the nickname, through more detailed metaphors at the levels of domains and frames, until the level of mental spaces is reached. This level is of great importance for the new meaning of the koczkodan nickname, as the integration of two incompatible input spaces (animal and human characteristics) are blended to form one single space. This results in the generation of a novel meaning of koczkodan in Polish, which makes reference not only to one’s physical ugliness but also to one’s evil character traits.

Feasible set estimation under functional uncertainty by Gaussian Process modelling

In this paper we deal with the estimation of a feasible set defined by an inequality constraint on the output of a time-consuming black-box simulator. We focus on the setting where the black-box simulator takes as inputs both a set of scalar controlled variables and a functional uncontrolled variable. We then place ourselves in a probabilistic framework, modelling the functional uncontrolled variable by a random process. The inequality constraint is formulated as the expectation of the output of the simulator conditional on the values taken by the set of controlled variables. We propose an original method to solve the above feasibility problem with a reduced number of evaluations of the costly simulator. A Gaussian Process model of the simulator is learned in the joint space of controlled and uncontrolled input variables, on the basis of a set of simulations which is enriched through a sequential procedure. This procedure aims to reduce the estimation error of the feasible set by evaluating the simulator on new points chosen sequentially in the joint input space according to specific enrichment criteria. It involves as a preliminary step the reduction of the dimension of the uncontrolled input space. A variation of this strategy is also proposed, which increases adaptively the dimension of the reduced space, leading to an improvement in terms of number of calls to the simulator. The procedure we propose is compared with other sampling procedures and another modelling approach on analytical examples. Finally our methodology is implemented on an automotive industrial application. For this application, the feasible set to be recovered is the set of values of controlled variables of a gas after-treatment device, leading to the respect of pollutant emission standards of a vehicle under driving profile uncertainties.

Adversarial attacks and defenses using feature-space stochasticity

Recent studies in deep neural networks have shown that injecting random noise in the input layer of the networks contributes towards ℓp-norm-bounded adversarial perturbations. However, to defend against unrestricted adversarial examples, most of which are not ℓp-norm-bounded in the input layer, such input-layer random noise may not be sufficient. In the first part of this study, we generated a novel class of unrestricted adversarial examples termed feature-space adversarial examples. These examples are far from the original data in the input space but adjacent to the original data in a hidden-layer feature space and far again in the output layer. In the second part of this study, we empirically showed that while injecting random noise in the input layer was unable to defend these feature-space adversarial examples, they were defended by injecting random noise in the hidden layer. These results highlight the novel benefit of stochasticity in higher layers, in that it is useful for defending against these feature-space adversarial examples, a class of unrestricted adversarial examples.