Input Space Research Articles

Rolling bearing fault diagnosis under variable working conditions using deep convolutional fuzzy system

This paper addresses the application of a deep convolutional fuzzy system (DCFS) for the fault diagnosis of rolling element bearings. The limitations of the conventional deep convolutional neural network (CNN) are the huge computational load of training the tones of parameters and the lack of interpretability for the corresponding parameters. In this paper, a DCFS-based bearing fault diagnosis method under variable working conditions is proposed. The DCFS on a high dimensional input space is a multilayer connection of many low dimensional fuzzy systems, which can overcome the computational and interpretability problems of the traditional CNN. Moreover, to improve the identification efficiency and diagnosis accuracy, the infinite feature selection (Inf-FS) algorithm is employed to select the most informative fault features. The proposed approach is experimentally demonstrated to be able to identify the different fault types and fault severities of rolling bearings under variable running states.

Improving adversarial robustness of deep neural networks via adaptive margin evolution

Adversarial training is the most popular and general strategy to improve Deep Neural Network (DNN) robustness against adversarial noises. Many adversarial training methods have been proposed in the past few years. However, most of these methods are highly susceptible to hyperparameters, especially the training noise upper bound. Tuning these hyperparameters is expensive and difficult for people not in the adversarial robustness research domain, which prevents adversarial training techniques from being used in many application fields. In this study, we propose a new adversarial training method, named Adaptive Margin Evolution (AME). Besides being hyperparameter-free for the user, our AME method places adversarial training samples into the optimal locations in the input space by gradually expanding the exploration range with self-adaptive and gradient-aware step sizes. We evaluate AME and the other seven well-known adversarial training methods on three common benchmark datasets (CIFAR10, SVHN, and Tiny ImageNet) under the most challenging adversarial attack: AutoAttack. The results show that: (1) On the three datasets, AME has the best overall performance; (2) On the Tiny ImageNet dataset, which is much more challenging, AME has the best performance at every noise level. Our work may pave the way for adopting adversarial training techniques in application domains where hyperparameter-free methods are preferred.

Enhancing diagnostic of stochastic mortality models leveraging contrast trees: an application on Italian data

The rise in longevity in the twentieth century has led to a growing interest in modeling mortality, and new advanced techniques such as machine learning have recently joined to more traditional models, such as the Lee–Carter or the Age Period Cohort. However, the performances of these models, in terms of fitting to the observed data, are difficult to compare in a unified framework. The goodness-of-fit measures summarizing the discrepancy between the estimates from the model and the observed values are different for traditional mortality models and machine learning. We, therefore, employ a new technique, Contrast trees, which, leveraging on decision trees, provides a general approach for evaluating the quality of fit of different kinds of models by detecting the regions in the input space where models work poorly. Once the low-performance regions are detected, we use Contrast boosting to improve the inaccuracies of mortality estimates provided by each model. To verify the ability of this approach, we consider both standard stochastic mortality models and machine learning algorithms in the estimate of the Italian mortality rates from the Human Mortality Database. The results are discussed using both graphical and numerical tools, with particular attention to the high-error regions.

Adaptive Single-Input Recurrent WCMAC-Based Supervisory Control for De-icing Robot Manipulator

The control of any robotic system always faces many great challenges in theory and practice. Because between theory and reality, there is always a huge difference in the uncertainty components in the system. That leads to the accuracy and stability of the system not being guaranteed with the set requirements. This paper presents a novel adaptive single-input recurrent wavelet differentiable cerebellar model articulation controller (S-RWCMAC)-based supervisory control system for an m-link robot manipulator to achieve precision trajectory tracking. This adaptive S-RWCMAC-based supervisory control system consists of a main adaptive S-RWCMAC, a supervisory controller, and an adaptive robust controller. The S-RWCMAC incorporates the advantages of the wavelet decomposition property with a CMAC fast learning ability, dynamic response, and input space dimension of RWCMAC can be simplified; and it is used to control the plant. The supervisory controller is appended to the adaptive S-RWCMAC to force the system states within a predefined constraint set and the adaptive robust controller is developed to dispel the effect of the approximate error. In this scheme, if the adaptive S-RWCMAC can not maintain the system states within the constraint set. Then, the supervisory controller will work to pull the states back to the constraint set and otherwise is idle. The online tuning laws of S-RWCMAC and the robust controller parameters are derived from the gradient-descent learning method and Lyapunov function so that the stability of the system can be guaranteed. The simulation and experimental results of the novel three-link De-icing robot manipulator are provided to verify the effectiveness of the proposed control methodology. The results indicate that the proposed model has superior accuracy compared to that of the Standalone CMAC Controller. The parameters of the average squared error in the S-RWCMAC -based 3 robot joints are lower than those of the Standalone CMAC Controller by 0.023%, 0.029%, and 0.032%, respectively.

Data-Driven Interval Type-2 Fuzzy Inference System Based on the Interval Type-2 Distending Function

Fuzzy type-2 modeling techniques are increasingly being used to model uncertain dynamical systems. However, some challenges arise when applying the existing techniques. These are: 1) A large number of rules are required to complete cover the whole input space; 2) A large of parameters associated with type-2 membership functions have to be determined; 3) The identified fuzzy model is usually difficult to interpret due to the large number of rules; 4) Designing a fuzzy type-2 controller using these models is a computationally expensive task. To overcome these limitations, a procedure is proposed here to identify the fuzzy type-2 model directly from the data. This model is called the Distending Function-based Fuzzy Inference System (DFIS). This model consists of rules and interval Type-2 Distending Functions (T2DFs). First, a few key rules are identified from the data and later more rules are added until the error is less than the threshold. The proposed procedure is used to model the altitude controller of a quadcopter. The DFIS model performance is compared with various fuzzy models. Furthermore a simplified procedure based on the rules is presented to design a computationally low-cost type-2 controller. The effectiveness of the controller is shown by regulating the height of a quadcopter in the presence of noisy sensory data. The performance of this controller is compared with various other controllers. Lastly, the proposed type-2 controller was implemented on a Parrot Mambo quadcopter to demonstrate its real-time performance.

Enhanced universal kriging for transformed input parameter spaces

With computational models becoming more expensive and complex, surrogate models have gained increasing attention in many scientific disciplines and are often necessary to conduct sensitivity studies, parameter optimization etc. In the scientific discipline of uncertainty quantification (UQ), model input quantities are often described by probability distributions. For the construction of surrogate models, space-filling designs are generated in the input space to define training points, and evaluations of the computational model at these points are then conducted. The physical parameter space is often transformed into an i.i.d. uniform input space in order to apply space-filling training procedures in a sensible way. Due to this transformation surrogate modeling techniques tend to suffer with regard to their prediction accuracy. Therefore, a new method is proposed in this paper where input parameter transformations are applied to basis functions for universal kriging. To speed up hyperparameter optimization for universal kriging, suitable expressions for efficient gradient-based optimization are developed. Several benchmark functions are investigated and the proposed method is compared with conventional methods.

Data-driven templates with dictionary learning and sparse representations for photometric redshift estimation

The quality of photometric redshift estimates is fundamental for cosmological applications, as the constraints on the cosmological parameters obtained by the photometric surveys strongly rely on their precision and accuracy. In order to obtain reliable redshift estimates, a large number of studies have proposed different algorithms based on SED template fitting and machine learning methodologies. While template fitting strategies do not need spectroscopic data for training, they also provide consistent estimates in most scenarios. On the other hand, machine learning-based approaches succeed at building a mapping function between the input space (composed by magnitudes and other physical parameters) and the target photometric redshift space. These empirical methods lead to more accurate results as long as the spectroscopic ground truth properly represents the actual galaxy population distribution. Thus, combining template fitting and machine learning techniques would allow to leverage the advantages of both methodologies yielding hybrid methods. The goal is to provide a good estimation accuracy while maximizing the photometric range coverage. This is particularly important in the high-z regime, where the spectroscopic training sample is rarely available. We propose in this article a new method to derive data-driven templates from simulated spectra using dictionary learning. As these representations are obtained directly from the data, they allow to capture the physical properties of the galaxies better than theoretical templates. Inspired by hybrid algorithms, the dictionary is built hierarchically and the features of the different types of galaxies are separated. Once the dictionary has been created, the data-driven templates are artificially redshifted and the observed spectra are sparsely decomposed on this dictionary. The value providing the minimum reconstruction error is selected as the photometric redshift estimate. This new technique, astride template fitting and machine learning, builds representations for the galaxy spectra through unsupervised learning and computes the photometric redshifts by χ2 minimization. The performance of the algorithm has been evaluated on realistic galaxy photometric simulations as well as on real data from the Canada–France–Hawaii Telescope Lensing Survey.

Generative Multiform Bayesian Optimization.

Many real-world problems, such as airfoil design, involve optimizing a black-box expensive objective function over complex-structured input space (e.g., discrete space or non-Euclidean space). By mapping the complex-structured input space into a latent space of dozens of variables, a two-stage procedure labeled as generative model-based optimization (GMO), in this article, shows promise in solving such problems. However, the latent dimension of GMO is hard to determine, which may trigger the conflicting issue between desirable solution accuracy and convergence rate. To address the above issue, we propose a multiform GMO approach, namely, generative multiform optimization (GMFoO), which conducts optimization over multiple latent spaces simultaneously to complement each other. More specifically, we devise a generative model which promotes a positive correlation between latent spaces to facilitate effective knowledge transfer in GMFoO. And furthermore, by using Bayesian optimization (BO) as the optimizer, we propose two strategies to exchange information between these latent spaces continuously. Experimental results are presented on airfoil and corbel design problems and an area maximization problem as well to demonstrate that our proposed GMFoO converges to better designs on a limited computational budget.

Variance-based sensitivity analysis of oil spill predictions in the Red Sea region

To support accidental spill rapid response efforts, oil spill simulations may generally need to account for uncertainties concerning the nature and properties of the spill, which compound those inherent in model parameterizations. A full detailed account of these sources of uncertainty would however require prohibitive resources needed to sample a large dimensional space. In this work, a variance-based sensitivity analysis is conducted to explore the possibility of restricting a priori the set of uncertain parameters, at least in the context of realistic simulations of oil spills in the Red Sea region spanning a two-week period following the oil release. The evolution of the spill is described using the simulation capabilities of Modelo Hidrodinâmico, driven by high-resolution metocean fields of the Red Sea (RS) was adopted to simulate accidental oil spills in the RS. Eight spill scenarios are considered in the analysis, which are carefully selected to account for the diversity of metocean conditions in the region. Polynomial chaos expansions are employed to propagate parametric uncertainties and efficiently estimate variance-based sensitivities. Attention is focused on integral quantities characterizing the transport, deformation, evaporation and dispersion of the spill. The analysis indicates that variability in these quantities may be suitably captured by restricting the set of uncertain inputs parameters, namely the wind coefficient, interfacial tension, API gravity, and viscosity. Thus, forecast variability and confidence intervals may be reasonably estimated in the corresponding four-dimensional input space.

Map-based experience replay: a memory-efficient solution to catastrophic forgetting in reinforcement learning.

Deep reinforcement learning (RL) agents often suffer from catastrophic forgetting, forgetting previously found solutions in parts of the input space when training new data. Replay memories are a common solution to the problem by decorrelating and shuffling old and new training samples. They naively store state transitions as they arrive, without regard for redundancy. We introduce a novel cognitive-inspired replay memory approach based on the Grow-When-Required (GWR) self-organizing network, which resembles a map-based mental model of the world. Our approach organizes stored transitions into a concise environment-model-like network of state nodes and transition edges, merging similar samples to reduce the memory size and increase pair-wise distance among samples, which increases the relevancy of each sample. Overall, our study shows that map-based experience replay allows for significant memory reduction with only small decreases in performance.

Very Fast, Approximate Counterfactual Explanations for Decision Forests

We consider finding a counterfactual explanation for a classification or regression forest, such as a random forest. This requires solving an optimization problem to find the closest input instance to a given instance for which the forest outputs a desired value. Finding an exact solution has a cost that is exponential on the number of leaves in the forest. We propose a simple but very effective approach: we constrain the optimization to input space regions populated by actual data points. The problem reduces to a form of nearest-neighbor search using a certain distance on a certain dataset. This has two advantages: first, the solution can be found very quickly, scaling to large forests and high-dimensional data, and enabling interactive use. Second, the solution found is more likely to be realistic in that it is guided towards high-density areas of input space.

Redactor: A Data-Centric and Individualized Defense against Inference Attacks

Information leakage is becoming a critical problem as various information becomes publicly available by mistake, and machine learning models train on that data to provide services. As a result, one's private information could easily be memorized by such trained models. Unfortunately, deleting information is out of the question as the data is already exposed to the Web or third-party platforms. Moreover, we cannot necessarily control the labeling process and the model trainings by other parties either. In this setting, we study the problem of targeted disinformation generation where the goal is to dilute the data and thus make a model safer and more robust against inference attacks on a specific target (e.g., a person's profile) by only inserting new data. Our method finds the closest points to the target in the input space that will be labeled as a different class. Since we cannot control the labeling process, we instead conservatively estimate the labels probabilistically by combining decision boundaries of multiple classifiers using data programming techniques. Our experiments show that a probabilistic decision boundary can be a good proxy for labelers, and that our approach is effective in defending against inference attacks and can scale to large data.

DARL: Distance-Aware Uncertainty Estimation for Offline Reinforcement Learning

To facilitate offline reinforcement learning, uncertainty estimation is commonly used to detect out-of-distribution data. By inspecting, we show that current explicit uncertainty estimators such as Monte Carlo Dropout and model ensemble are not competent to provide trustworthy uncertainty estimation in offline reinforcement learning. Accordingly, we propose a non-parametric distance-aware uncertainty estimator which is sensitive to the change in the input space for offline reinforcement learning. Based on our new estimator, adaptive truncated quantile critics are proposed to underestimate the out-of-distribution samples. We show that the proposed distance-aware uncertainty estimator is able to offer better uncertainty estimation compared to previous methods. Experimental results demonstrate that our proposed DARL method is competitive to the state-of-the-art methods in offline evaluation tasks.

Application Fuzzy For Measuring Lecturer Performance Using Matlab Software

Research by Lecturer, Graha University Nusantara Padangsidimpuan, Simrittabumas Data still in Guidance Category for Upgrade to Intermediate Category are required to apply. For this, we need an application that can calculate and document the lecturer's performance in the resulting research. The purpose of this study is to apply fuzzy logic with Mamdani method to evaluate the research achievements of lecturers at Graha Nusantara Padangsidimpuan University. This study uses Mamdani fuzzy logic. The fuzzy Mamdani method is a method of mapping the input space to the output space. This method is a mathematical framework for expressing uncertainty, ambiguity, inaccuracy, lack of information, and partial truth. The phase of research using the Mamdani method is to create input variables from Sinta-accredited papers, Simlitabmas Grant papers, and papers from international and national journals. Find the maximum and minimum values for each variable. Create a fuzzy set using the Mamdani method. Building assertions with defuzzification using Matlab.

DeepGemini: Verifying Dependency Fairness for Deep Neural Network

Deep neural networks (DNNs) have been widely adopted in many decision-making industrial applications. Their fairness issues, i.e., whether there exist unintended biases in the DNN, receive much attention and become critical concerns, which can directly cause negative impacts in our daily life and potentially undermine the fairness of our society, especially with their increasing deployment at an unprecedented speed. Recently, some early attempts have been made to provide fairness assurance of DNNs, such as fairness testing, which aims at finding discriminatory samples empirically, and fairness certification, which develops sound but not complete analysis to certify the fairness of DNNs. Nevertheless, how to formally compute discriminatory samples and fairness scores (i.e., the percentage of fair input space), is still largely uninvestigated. In this paper, we propose DeepGemini, a novel fairness formal analysis technique for DNNs, which contains two key components: discriminatory sample discovery and fairness score computation. To uncover discriminatory samples, we encode the fairness of DNNs as safety properties and search for discriminatory samples by means of state-of-the-art verification techniques for DNNs. This reduction enables us to be the first to formally compute discriminatory samples. To compute the fairness score, we develop counterexample guided fairness analysis, which utilizes four heuristics to efficiently approximate a lower bound of fairness score. Extensive experimental evaluations demonstrate the effectiveness and efficiency of DeepGemini on commonly-used benchmarks, and DeepGemini outperforms state-of-the-art DNN fairness certification approaches in terms of both efficiency and scalability.

GTR-GA: Harnessing the power of graph-based neural networks and genetic algorithms for text augmentation

Text augmentation is a popular technique in natural language processing (NLP) that has been shown to improve the performance of various downstream tasks. The goal of text augmentation is to generate additional training data from existing data, thereby increasing the amount of data available for training machine learning models. In this paper, we propose a novel approach to text augmentation that combines the power of graph-based neural networks and genetic algorithms. Our proposed scheme, called GTR-GA, is designed to generate diverse and high-quality augmented text data by exploring the high-dimensional feature space of text data using graph-based neural networks and genetic algorithms. Our approach utilizes a graph attention network (GAT) based model, called HetGAPN, to learn node representations in a heterogeneous graph representing text data. We introduce the concept of node aggregation and bidirectional attention in HetGAPN to better capture the relationships between different features of the input text data. We then use a genetic algorithm to generate a diverse set of candidate embeddings that explore different parts of the input space. Our approach uses perplexity as the objective function to evaluate the quality of the augmented text data generated by each candidate embedding. The most fit candidate embeddings are selected as parents, and crossover and mutation operators are used to generate a new population of candidate embeddings. Our experimental results show that GTR-GA is capable of generating high-quality augmented text data that improves the performance of various downstream NLP tasks, such as sentiment analysis and text classification. The proposed scheme can be applied to a wide range of NLP tasks and can help overcome the data scarcity problem that is often encountered in NLP.

Comparison of KNN and SVM Algorithms in Facial Image Recognition Using Haar Wavelet Feature Extraction

To process all the pixels in the face image, feature extraction can be performed using the Haar Wavelet method so that it processes identifiers with lower dimensions. However, a classification algorithm must separate the distance between classes with minimal data to classify low-dimensional facial images. KNN and SVM algorithms are classifiers that can be used for facial image recognition. When classifying images, SVM creates a hyperplane, divides the input space between classes and classifies based on which side of the hyperplane the unclassified object is placed when it is placed in the input space. KNN uses a voting system to determine which class an unclassified object belongs to, taking into account the nearest neighbor class in the decision space. When classifying, KNN will generally classify accurately, resulting in some minor misclassifications that plagued the final classified image. This study aims to compare the two algorithms on image identifiers with low dimensions resulting from haar wavelet extraction. The research results obtained are facial image classification using the haar wavelet extraction method using the SVM algorithm to obtain an accuracy of 98.8%. Whereas when using the KNN algorithm, the accuracy obtained is 96.6%. The results of this study show that the SVM algorithm produces better accuracy in facial image recognition using haar wavelet feature extraction compared to the KNN algorithm. The SVM algorithm can recognize facial images even though it uses image training data with various face poses and sizes, resulting in higher accuracy.

Bayesian Optimization for Design of Multiscale Biological Circuits.

Recent advances in synthetic biology have enabled the construction of molecular circuits that operate across multiple scales of cellular organization, such as gene regulation, signaling pathways, and cellular metabolism. Computational optimization can effectively aid the design process, but current methods are generally unsuited for systems with multiple temporal or concentration scales, as these are slow to simulate due to their numerical stiffness. Here, we present a machine learning method for the efficient optimization of biological circuits across scales. The method relies on Bayesian optimization, a technique commonly used to fine-tune deep neural networks, to learn the shape of a performance landscape and iteratively navigate the design space toward an optimal circuit. This strategy allows the joint optimization of both circuit architecture and parameters, and provides a feasible approach to solve a highly nonconvex optimization problem in a mixed-integer input space. We illustrate the applicability of the method on several gene circuits for controlling biosynthetic pathways with strong nonlinearities, multiple interacting scales, and using various performance objectives. The method efficiently handles large multiscale problems and enables parametric sweeps to assess circuit robustness to perturbations, serving as an efficient in silico screening method prior to experimental implementation.

Global and pyramid convolutional neural network with hybrid attention mechanism for hyperspectral image classification

Convolutional neural networks (CNNs) have shown impressive results in the hyperspectral image (HSI) classification. However, they still face certain limitations that can impact their effectiveness. The kernels in standard convolution have fixed spatial sizes and spectral depths, which cannot capture global semantic features from HSI and the existing methods for extracting multiscale information are inadequate. Based on these, we propose a global and pyramid convolutional network with a hybrid attention mechanism (GPHANet) for HSI classification. GPHANet adopts a two-branch architecture to extract both local and global hyperspectral features, leveraging the strengths of each to enhance classification performance. The local branch employs dynamic pyramid convolution, which customizes parameters for convolutional kernels of multiple scales based on the input image. This design enables the model to accurately and comprehensively extract multiscale feature information, capturing subtle variations in the HSI. The global branch utilizes circular convolution with a global receptive field, enabling each position in the convolutional layers to gather information from the entire input space. This allows the model to learn global contextual features, enhancing its understanding of the overall structure and semantics of the HSI. After extracting deep global and local features, we apply a compact hybrid attention mechanism (HAM) to capture long-range dependencies along both spatial and spectral dimensions, resulting in a more discriminative feature representation. On the Pavia University, Salinas, and Hong Hu datasets, the proposed model achieved overall accuracies of 99.22%, 98.74%, and 96.80%, respectively, using limited training samples. These results are much better compared to the state-of-the-art methods.

Implementation of the Mamdani Fuzzy Method to Evaluate the Performance of Lecturers in the Research Field

Today, information technology, especially soft computing technology, has grown very rapidly. One of the soft computing technologies that has been widely developed is fuzzy logic, because it can be used to measure various phenomena that are ambiguous, disguised or fuzzy. One of the research topics that uses the application of fuzzy logic is the assessment system in the field of research. Research by Lecturers at the University of Graha Nusantara Padangsidimpuan in Simlitabmas Data Still in the Guidance Category to upgrade to the Middle Category UGN Padangsidimpuan lecturers are challenged to be able to develop, devote, and apply the knowledge needed in research. For that we need an application that can be used to calculate and record the performance of Lecturers on the resulting Research. The purpose of this study is to apply fuzzy logic with the Mamdani method in assessing the research performance of lecturers at the University of Graha Nusantara Padangsidimpuan. This research uses Mamdani Fuzzy Logic. Fuzzy Mamdani method is a way to map an input space into an output space. This method is a mathematical framework used to represent uncertainty, ambiguity, imprecision, lack of information, and partial truth. The stages of research using the Mamdani method are Creating Input Variables taken from Sinta-accredited Articles, Simlitabmas Grant Articles and Articles in International and National Journals. Finding the Max-Min Value of Each Variable. Creating a fuzzy set using the Mamdani method. Creating Assertions with Defuzzification using Matlab.