Regions Of Feature Space Research Articles

Euclid preparation. LI. Forecasting the recovery of galaxy physical properties and their relations with template-fitting and machine-learning methods.

will collect an enormous amount of data during the mission's lifetime, observing billions of galaxies in the extragalactic sky. Along with traditional template-fitting methods, numerous machine learning (ML) algorithms have been presented for computing their photometric redshifts and physical parameters (PPs), requiring significantly less computing effort while producing equivalent performance measures. However, their performance is limited by the quality and amount of input information entering the model (the features), to a level where the recovery of some well-established physical relationships between parameters might not be guaranteed -- for example, the star-forming main sequence (SFMS). To forecast the reliability of photo-$z$s and PPs calculations, we produced two mock catalogs simulating the photometry with the UNIONS $ugriz$ and filters. We simulated the Euclid Wide Survey (EWS) and Euclid Deep Fields (EDF), alongside two auxiliary fields. We tested the performance of a template-fitting algorithm ( and four ML methods in recovering photo-$z$s, PPs (stellar masses and star formation rates), and the SFMS on the simulated fields. To mimic the processing as closely as possible, the models were trained with labels and tested on the simulated ground truth. For the EWS, we found that the best results are achieved with a mixed labels approach, training the models with wide survey features and labels from the results on deeper photometry, that is, with the best possible set of labels for a given photometry. This imposes a prior to the input features, helping the models to better discern cases in degenerate regions of feature space, that is, when galaxies have similar magnitudes and colors but different redshifts and PPs, with performance metrics even better than those found with We found no more than 3<!PCT!> performance degradation using a COSMOS-like reference sample or removing $u$ band data, which will not be available until after data release DR1. The best results are obtained for the EDF, with appropriate recovery of photo-$z$, PPs, and the SFMS.

Identification of High-Reliability Regions of Machine Learning Predictions Based on Materials Chemistry.

Progress in the application of machine learning (ML) methods to materials design is hindered by the lack of understanding of the reliability of ML predictions, in particular, for the application of ML to small data sets often found in materials science. Using ML prediction for transparent conductor oxide formation energy and band gap, dilute solute diffusion, and perovskite formation energy, band gap, and lattice parameter as examples, we demonstrate that (1) construction of a convex hull in feature space that encloses accurately predicted systems can be used to identify regions in feature space for which ML predictions are highly reliable; (2) analysis of the systems enclosed by the convex hull can be used to extract physical understanding; and (3) materials that satisfy all well-known chemical and physical principles that make a material physically reasonable are likely to be similar and show strong relationships between the properties of interest and the standard features used in ML. We also show that similar to the composition-structure-property relationships, inclusion in the ML training data set of materials from classes with different chemical properties will not be beneficial for the accuracy of ML prediction and that reliable results likely will be obtained by ML model for narrow classes of similar materials even in the case where the ML model will show large errors on the data set consisting of several classes of materials.

Category Learning Selectively Enhances Representations of Boundary-Adjacent Exemplars in Early Visual Cortex.

Category learning and visual perception are fundamentally interactive processes, such that successful categorization often depends on the ability to make fine visual discriminations between stimuli that vary on continuously valued dimensions. Research suggests that category learning can improve perceptual discrimination along the stimulus dimensions that predict category membership and that these perceptual enhancements are a byproduct of functional plasticity in the visual system. However, the precise mechanisms underlying learning-dependent sensory modulation in categorization are not well understood. We hypothesized that category learning leads to a representational sharpening of underlying sensory populations tuned to values at or near the category boundary. Furthermore, such sharpening should occur largely during active learning of new categories. These hypotheses were tested using fMRI and a theoretically constrained model of vision to quantify changes in the shape of orientation representations while human adult subjects learned to categorize physically identical stimuli based on either an orientation rule (N = 12) or an orthogonal spatial frequency rule (N = 13). Consistent with our predictions, modeling results revealed relatively enhanced reconstructed representations of stimulus orientation in visual cortex (V1-V3) only for orientation rule learners. Moreover, these reconstructed representations varied as a function of distance from the category boundary, such that representations for challenging stimuli near the boundary were significantly sharper than those for stimuli at the category centers. These results support an efficient model of plasticity wherein only the sensory populations tuned to the most behaviorally relevant regions of feature space are enhanced during category learning.

Category structure and region-specific selective attention.

A fundamental component of human categorization involves learning to attend selectively to relevant dimensions and ignore irrelevant ones. Past research has shown that humans can learn flexible strategies in which the attended dimensions vary depending on the region of feature space in which classification takes place. However, region-specific selective attention (RSA) is often challenging to learn. Here, we test the hypothesis that RSA is facilitated when individual categories are embedded within single regions of stimulus space rather than dispersed across multiple regions. We conduct an experiment that varies across conditions whether categories are embedded within regions, but in which the same RSA strategy would benefit performance across the conditions. To evaluate the hypothesis, we use measures of overall performance accuracy as well as comparisons among formal computational models that do and do not make allowance for RSA. We find strong support for the hypothesis among the upper-median-performing participants in the tested groups. However, even in the condition that promotes the learning of RSA, performance is considerably worse than in comparison conditions in which a single set of dimensions can be attended throughout the entire stimulus space.

Integrated Test Pattern Extraction and Generation for Accurate Lithography Modeling

A set of good test patterns is a key for accurate lithography modeling. We address three sub-problems in test pattern extraction: (1) Each pattern is associated with one or more gauges, where CD is measured and modeling is performed. We perform gauge clustering in feature space, while the goal is to minimize the number of chosen patterns. (2) Sample patterns do not usually guarantee enough coverage, so once coverage of chosen patterns is identified, some synthetic patterns are systematically generated to fill the empty region in feature space. (3) Pattern representation defines the feature space. We propose to use some Gaussian convolutions (or a few of their principal components) together with image parameter set (IPS) to reflect exposure for optical model and both PEB and development for resist model. Experiments with 1xnm DRAM design demonstrate that lithography modeling through our test pattern extraction and generation is performed with 35% smaller number of test patterns while CD error is reduced by 55%, with pattern extraction in feature space of IPS parameters as a reference.

A discrete time-varying greywolf IoT botnet detection system

Internet of things (IoT) is an emerging network that is trending due to advances in computing and networking development. A botnet is an attack that threatens the IoT system because of the increased number of compromised connected IoT devices.The conventional counter measurements are unable to detect this attack. This problem becomes a hot topic for researchers and practitioners who introduced many secured solutions to stop the risks of a botnet attack. The intrusion detection systems are promising solutions to tackle botnet attacks and discover malicious patterns. Recently, many studies investigated the impact of reducing the number of dataset’s attributes (features) on the performance of detecting IoT attacks. Selecting the relevant features in a dataset is a data mining technique that has been efficiently integrated with designing secured systems for detecting botnets.This research paper proposes a new system for discovering botnet attacks in the context of IoT by applying a wrapper feature selection (FS) technique using an improved algorithm inspired by the natural swarming architecture of gray wolves called Gray Wolves Optimization (GWO). The transfer function (TF) maps the standard GWO which is originally developed to work in continuous search space to perform its optimization job in discrete search space. Different types of TFs that belong to S type and V type groups are used to generate eight discrete versions of GWO to optimize the binary feature space.This study contributes by adopting time-variant TFs to identify the best time to switch the global search into local search to achieve trade-offs in the search job of the BGWO and approach to the best-optimized solution. Time-variant TFs facilitate the global search at the beginning of the search process to fetch new solutions in new regions of feature space. In the later phases of the search process, the need is to expose more searches in the local region to get the most optimized solution among the neighborhood solutions.A real IoT traffic that is represented by the N-BaIoT dataset is utilized to evaluate the BGWO and other compared methods. The comparison results of the experiments show that the time-variant TFs enhance the capability of the GWO optimizer in alleviating the premature conversion and finding the best feature subset within a reasonable running time. Therefore, the BGWO-TV-S1 is recommended to be integrated into the IoT network as an intrusion detection algorithm with accuracy 98.97%, fitness value 1.31%, 51.2210 selected features and running time 503.7132 s.

D-MONA: A dilated mixed-order non-local attention network for speaker and language recognition

Attention-based convolutional neural network (CNN) models are increasingly being adopted for speaker and language recognition (SR/LR) tasks. These include time, frequency, spatial and channel attention, which can focus on useful time frames, frequency bands, regions or channels while extracting features. However, these traditional attention methods lack the exploration of complex information and multi-scale long-range speech feature interactions, which can benefit SR/LR tasks. To address these issues, this paper firstly proposes mixed-order attention (MOA) for low frame-level speech features to gain the finest grain multi-order information at higher resolution. We then combine that with a non-local attention (NLA) mechanism and a dilated residual structure to balance fine grained local detail with convolution from multi-scale long-range time/frequency regions in feature space. The proposed dilated mixed-order non-local attention network (D-MONA) exploits the detail available from the first and the second-order feature attention analysis, but achieves this over a much wider context than purely local attention. Experiments are conducted on three datasets, including two SR tasks of Voxceleb and CN-celeb, and one LR task, NIST LRE 07. For SR, D-MONA improves on ResNet-34 results by at least 29% and 15% for Voxceleb1 and CN-celeb respectively. For the LR task, a large improvement is achieved over ResNet-34 of 21% for the challenging 3s utterance condition, 59% for the 10s condition and 67% for the 30s condition. It also outperforms the state-of-the-art deep bottleneck feature-DNN (DBF-DNN) x-vector system at all scales.

Feature Completion for Occluded Person Re-Identification.

Person re-identification (reID) plays an important role in computer vision. However, existing methods suffer from performance degradation in occluded scenes. In this work, we propose an occlusion-robust block, Region Feature Completion (RFC), for occluded reID. Different from most previous works that discard the occluded regions, RFC block can recover the semantics of occluded regions in feature space. First, a Spatial RFC (SRFC) module is developed. SRFC exploits the long-range spatial contexts from non-occluded regions to predict the features of occluded regions. The unit-wise prediction task leads to an encoder/decoder architecture, where the region-encoder models the correlation between non-occluded and occluded region, and the region-decoder utilizes the spatial correlation to recover occluded region features. Second, we introduce Temporal RFC (TRFC) module which captures the long-term temporal contexts to refine the prediction of SRFC. RFC block is lightweight, end-to-end trainable and can be easily plugged into existing CNNs to form RFCnet. Extensive experiments are conducted on occluded and commonly holistic reID benchmarks. Our method significantly outperforms existing methods on the occlusion datasets, while remains top even superior performance on holistic datasets. The source code is available at https://github.com/blue-blue272/OccludedReID-RFCnet.

Detecting Ordinal Subcascades

Ordinal classifier cascades are constrained by a hypothesised order of the semantic class labels of a dataset. This order determines the overall structure of the decision regions in feature space. Assuming the correct order on these class labels will allow a high generalisation performance, while an incorrect one will lead to diminished results. In this way ordinal classifier systems can facilitate explorative data analysis allowing to screen for potential candidate orders of the class labels. Previously, we have shown that screening is possible for total orders of all class labels. However, as datasets might comprise samples of ordinal as well as non-ordinal classes, the assumption of a total ordering might be not appropriate. An analysis of subsets of classes is required to detect such hidden ordinal substructures. In this work, we devise a novel screening procedure for exhaustive evaluations of all order permutations of all subsets of classes by bounding the number of enumerations we have to examine. Experiments with multi-class data from diverse applications revealed ordinal substructures that generate new and support known relations.

SACCOS: A Semi-Supervised Framework for Emerging Class Detection and Concept Drift Adaption Over Data Streams

In this paper, we address the challenges of detecting instances from emerging classes over a non-stationary data stream during classification. In particular, instances from an entirely unknown class may appear in a data stream over time. Existing classification techniques utilize unsupervised clustering to identify the emergence of such data instances. Unfortunately, they make strong assumptions which are typically invalid in practice; (i) Most instances associated with a class are closer to each other in feature space than instances associated with different classes, (ii) Covariates of data are normalized through an oracle to overcome the effect of a few data instances having large feature values, and (iii) Labels of instances from emerging classes are readily available soon after detection. To address the challenges that occur in practice when the above assumptions are weak, we propose a practical semi-supervised emerging class detection framework. Particularly, we aim to identify similar data instances within local regions in feature space by incorporating a mutual graph clustering mechanism. Our empirical evaluation of this framework on real-world datasets demonstrates its superiority of classification performance compared to existing methods while using significantly fewer labeled instances.

A rough-granular approach to the imbalanced data classification problem

More than two decades ago the imbalanced data problem turned out to be one of the most important and challenging problems. Indeed, missing information about the minority class leads to a significant degradation in classifier performance. Moreover, comprehensive research has proved that there are certain factors increasing the problem’s complexity. These additional difficulties are closely related to the data distribution over decision classes. In spite of numerous methods which have been proposed, the flexibility of existing solutions needs further improvement. Therefore, we offer a novel rough–granular computing approach (RGA, in short) to address the mentioned issues. New synthetic examples are generated only in specific regions of feature space. This selective oversampling approach is applied to reduce the number of misclassified minority class examples. A strategy relevant for a given problem is obtained by formation of information granules and an analysis of their degrees of inclusion in the minority class. Potential inconsistencies are eliminated by applying an editing phase based on a similarity relation. The most significant algorithm parameters are tuned in an iterative process. The set of evaluated parameters includes the number of nearest neighbours, complexity threshold, distance threshold and cardinality redundancy. Each data model is built by exploiting different parameters’ values. The results obtained by the experimental study on different datasets from the UCI repository are presented. They prove that the proposed method of inducing the neighbourhoods of examples is crucial in the proper creation of synthetic positive instances. The proposed algorithm outperforms related methods in most of the tested datasets. The set of valid parameters for the Rough–Granular Approach (RGA) technique is established.

Sparse Gradient Based Structured Matrix Decomposition for Salient Object Detection

In the salient object detection, the given image can be decomposed into background regions (low-rank part) and salient regions (sparse part). In this paper, we present a novel sparse gradient-based structured matrix decomposition model for salient object detection. We use the ${l_{1}}$ norm of logistic function on the singular values to approximate the rank function, which avoid over-penalized problem of the nuclear norm. And a group sparsity induced norm regularization is imposed on the salient part to explore the relationship among superpixels. In order to widen the gap between salient regions and background regions in feature space, we suggest a sparse gradient regularization to replace the conventional Laplacian regularization. Finally, the model is solved through an augmented Lagrange multipliers method, and high-level priors are embedded into our model to promote the performance. Experiments indicate that the proposed method performs better in terms of various evaluation metrics than the state-of-the-art methods.

Navigating features: a topologically informed chart of electromyographic features space.

The success of biological signal pattern recognition depends crucially on the selection of relevant features. Across signal and imaging modalities, a large number of features have been proposed, leading to feature redundancy and the need for optimal feature set identification. A further complication is that, due to the inherent biological variability, even the same classification problem on different datasets can display variations in the respective optimal sets, casting doubts on the generalizability of relevant features. Here, we approach this problem by leveraging topological tools to create charts of features spaces. These charts highlight feature sub-groups that encode similar information (and their respective similarities) allowing for a principled and interpretable choice of features for classification and analysis. Using multiple electromyographic (EMG) datasets as a case study, we use this feature chart to identify functional groups among 58 state-of-the-art EMG features, and to show that they generalize across three different forearm EMG datasets obtained from able-bodied subjects during hand and finger contractions. We find that these groups describe meaningful non-redundant information, succinctly recapitulating information about different regions of feature space. We then recommend representative features from each group based on maximum class separability, robustness and minimum complexity.

Error modeling for surrogates of dynamical systems using machine learning

SummaryA machine learning–based framework for modeling the error introduced by surrogate models of parameterized dynamical systems is proposed. The framework entails the use of high‐dimensional regression techniques (eg, random forests, and LASSO) to map a large set of inexpensively computed “error indicators” (ie, features) produced by the surrogate model at a given time instance to a prediction of the surrogate‐model error in a quantity of interest (QoI). This eliminates the need for the user to hand‐select a small number of informative features. The methodology requires a training set of parameter instances at which the time‐dependent surrogate‐model error is computed by simulating both the high‐fidelity and surrogate models. Using these training data, the method first determines regression‐model locality (via classification or clustering) and subsequently constructs a “local” regression model to predict the time‐instantaneous error within each identified region of feature space. We consider 2 uses for the resulting error model: (1) as a correction to the surrogate‐model QoI prediction at each time instance and (2) as a way to statistically model arbitrary functions of the time‐dependent surrogate‐model error (eg, time‐integrated errors). We apply the proposed framework to model errors in reduced‐order models of nonlinear oil‐water subsurface flow simulations, with time‐varying well‐control (bottom‐hole pressure) parameters. The reduced‐order models used in this work entail application of trajectory piecewise linearization in conjunction with proper orthogonal decomposition. When the first use of the method is considered, numerical experiments demonstrate consistent improvement in accuracy in the time‐instantaneous QoI prediction relative to the original surrogate model, across a large number of test cases. When the second use is considered, results show that the proposed method provides accurate statistical predictions of the time‐ and well‐averaged errors.

On-line active learning: A new paradigm to improve practical useability of data stream modeling methods

The central purpose of this survey is to provide readers an insight into the recent advances and challenges in on-line active learning. Active learning has attracted the data mining and machine learning community since around 20 years. This is because it served for important purposes to increase practical applicability of machine learning techniques, such as (i) to reduce annotation and measurement costs for operators and measurement equipments, (ii) to reduce manual labeling effort for experts and (iii) to reduce computation time for model training. Almost all of the current techniques focus on the classical pool-based approach, which is off-line by nature as iterating over a pool of (unlabeled) reference samples a multiple times to choose the most promising ones for improving the performance of the classifiers. This is achieved by (time-intensive) re-training cycles on all labeled samples available so far. For the on-line, stream mining case, the challenge is that the sample selection strategy has to operate in a fast, ideally single-pass manner. Some first approaches have been proposed during the last decade (starting from around 2005) with the usage of machine learning (ML) oriented incremental classifiers, which are able to update their parameters based on selected samples, but not their structures. Since 2012, on-line active learning concepts have been proposed in connection with the paradigm of evolving models, which are able to expand their knowledge into feature space regions so far unexplored. This opened the possibility to address a particular type of uncertainty, namely that one which stems from a significant novelty content in streams, as, e.g., caused by drifts, new operation modes, changing system behaviors or non-stationary environments. We will provide an overview about the concepts and techniques for sample selection and active learning within these two principal major research lines (incremental ML models versus evolving systems), a comparison of their essential characteristics and properties (raising some advantages and disadvantages), and a study on possible evaluation techniques for them. We conclude with an overview of real-world application examples where various on-line AL approaches have been already successfully applied in order to significantly reduce user’s interaction efforts and costs for model updates.

Using Voronoi diagrams to improve classification performances when modeling imbalanced datasets

An over-sampling technique called V-synth is proposed and compared to borderline SMOTE (bSMOTE), a common methodology used to balance an imbalanced dataset for classification purposes. V-synth is a machine learning methodology that allows synthetic minority points to be generated based on the properties of a Voronoi diagram. A Voronoi diagram is a collection of geometric regions that encapsulate classifying points in such a way that any point within the region is closest to the encapsulated classifier than any other adjacent classifiers based on their distance from one another. Because of properties inherent to Voronoi diagrams, V-synth identifies exclusive regions of feature space where it is ideal to create synthetic minority samples. To test the generalization and application of V-synth, six databases from various problem domains were selected from the University of California Irvine's Machine Learning Repository. Though not always guaranteed due to the random nature of synthetic over-sampling, significant evidence is presented that supports the hypothesis that V-synth more consistently leads to the creation of more accurate and better-balanced classification models than bSMOTE when the classification complexity of a dataset is high.

Statistical nonlinear analysis for reliable promotion decision-making

New economic conditions have led to innovations in retail industries, such as more dynamic retail approaches based on flexible strategies. We propose and compare different approaches incorporating nonlinear methods for promotional decision-making using retail aggregated data registered at the point of the sale. Specifically, this paper describes a reliable quantification tool as an effective information system leveraged on recent and historical data that provides managers with an operative vision. Furthermore, a new set of indicators are proposed to evaluate the reliability and stability of the data model in the multidimensional feature space by using nonparametric resampling techniques. This allows the user to make a clearer comparison among linear, nonlinear, static, and dynamic data models, and to identify the uncertainty of different feature space regions, for example, corresponding to the most frequent deal features. This methodology allows retailers to use aggregated data in suitable conditions that will result in acceptable confidence intervals. To test the proposed methodology, we used a database containing the sales history of representative products registered by a Spanish retail chain. The results indicate that: (1) the deal effect curve analysis and the time series linear model do not provide enough expressive capacity, and (2) nonlinear promotional models more accurately follow the actual sales pattern obtained in response to the implemented sales promotions. The quarterly temporal analysis conducted enabled the authors to identify long-term changes in the dynamics of the model for several products, especially during the early stage of most recent economic crisis, consistent with the information provided by the reliability indices in terms of the feature space. We conclude that the proposed method provides a reliable operative tool for decision support, allowing retailers to alter their strategies to accommodate consumer behavior.

Online stimulus optimization rapidly reveals multidimensional selectivity in auditory cortical neurons.

Neurons in sensory brain regions shape our perception of the surrounding environment through two parallel operations: decomposition and integration. For example, auditory neurons decompose sounds by separately encoding their frequency, temporal modulation, intensity, and spatial location. Neurons also integrate across these various features to support a unified perceptual gestalt of an auditory object. At higher levels of a sensory pathway, neurons may select for a restricted region of feature space defined by the intersection of multiple, independent stimulus dimensions. To further characterize how auditory cortical neurons decompose and integrate multiple facets of an isolated sound, we developed an automated procedure that manipulated five fundamental acoustic properties in real time based on single-unit feedback in awake mice. Within several minutes, the online approach converged on regions of the multidimensional stimulus manifold that reliably drove neurons at significantly higher rates than predefined stimuli. Optimized stimuli were cross-validated against pure tone receptive fields and spectrotemporal receptive field estimates in the inferior colliculus and primary auditory cortex. We observed, from midbrain to cortex, increases in both level invariance and frequency selectivity, which may underlie equivalent sparseness of responses in the two areas. We found that onset and steady-state spike rates increased proportionately as the stimulus was tailored to the multidimensional receptive field. By separately evaluating the amount of leverage each sound feature exerted on the overall firing rate, these findings reveal interdependencies between stimulus features as well as hierarchical shifts in selectivity and invariance that may go unnoticed with traditional approaches.

An Online One Class Support Vector Machine-Based Person-Specific Fall Detection System for Monitoring an Elderly Individual in a Room Environment

In this paper, we propose a novel computer vision-based fall detection system for monitoring an elderly person in a home care, assistive living application. Initially, a single camera covering the full view of the room environment is used for the video recording of an elderly person's daily activities for a certain time period. The recorded video is then manually segmented into short video clips containing normal postures, which are used to compose the normal dataset. We use the codebook background subtraction technique to extract the human body silhouettes from the video clips in the normal dataset and information from ellipse fitting and shape description, together with position information, is used to provide features to describe the extracted posture silhouettes. The features are collected and an online one class support vector machine (OCSVM) method is applied to find the region in feature space to distinguish normal daily postures and abnormal postures such as falls. The resultant OCSVM model can also be updated by using the online scheme to adapt to new emerging normal postures and certain rules are added to reduce false alarm rate and thereby improve fall detection performance. From the comprehensive experimental evaluations on datasets for 12 people, we confirm that our proposed person-specific fall detection system can achieve excellent fall detection performance with 100% fall detection rate and only 3% false detection rate with the optimally tuned parameters. This work is a semiunsupervised fall detection system from a system perspective because although an unsupervised-type algorithm (OCSVM) is applied, human intervention is needed for segmenting and selecting of video clips containing normal postures. As such, our research represents a step toward a complete unsupervised fall detection system.

Discriminative Power Feature Selection Method for Motor Imagery EEG Classification in Brain Computer Interface Systems

Motor imagery classification in electroencephalography (EEG)-based brain-computer interface (BCI) systems is an important research area. To simplify the complexity of the classification, selected power bands and electrode channels have been widely used to extract and select features from raw EEG signals, but there is still a loss in classification accuracy in the state-of-the-art approaches. To solve this problem, we propose a discriminative feature extraction algorithm based on power bands with principle component analysis (PCA). First, the raw EEG signals from the motor cortex area were filtered using a bandpass filter with μ and β bands. This research considered the power bands within a 0.4 second epoch to select the optimal feature space region. Next, the total feature dimensions were reduced by PCA and transformed into a final feature vector set. The selected features were classified by applying a support vector machine (SVM). The proposed method was compared with a state-of-art power band feature and shown to improve classification accuracy.