Feature Rescaling Research Articles

On the Cross-Validation Bias due to Unsupervised Preprocessing

AbstractCross-validation is the de facto standard for predictive model evaluation and selection. In proper use, it provides an unbiased estimate of a model's predictive performance. However, data sets often undergo various forms of data-dependent preprocessing, such as mean-centring, rescaling, dimensionality reduction and outlier removal. It is often believed that such preprocessing stages, if done in an unsupervised manner (that does not incorporate the class labels or response values) are generally safe to do prior to cross-validation. In this paper, we study three commonly practised preprocessing procedures prior to a regression analysis: (i) variance-based feature selection; (ii) grouping of rare categorical features; and (iii) feature rescaling. We demonstrate that unsupervised preprocessing can, in fact, introduce a substantial bias into cross-validation estimates and potentially hurt model selection. This bias may be either positive or negative and its exact magnitude depends on all the parameters of the problem in an intricate manner. Further research is needed to understand the real-world impact of this bias across different application domains, particularly when dealing with small sample sizes and high-dimensional data.

Multigenerational inspections of environmental thermal perturbations promote metabolic trade-offs in developmental stages of tropical fish

Global warming both reduces global temperature variance and increases the frequency of extreme weather events. In response to these ambient perturbations, animals may be subject to trans- or intra-generational phenotype modifications that help to maintain homeostasis and fitness. Here, we show how temperature-associated transgenerational plasticity in tilapia affects metabolic trade-offs during developmental stages under a global warming scenario. Tropical tilapia reared at a stable temperature of 27 °C for a decade were divided into two temperature-experience groups for four generations of breeding. Each generation of one group was exposed to a single 15 °C cold-shock experience during its lifetime (cold-experienced CE group), and the other group was kept stably at 27 °C throughout their lifetimes (cold-naïve CN group). The offspring at early life stages from the CE and CN tilapia were then assessed by metabolomics-based profiling, and the results implied that parental cold-experience might affect energy provision during reproduction. Furthermore, at early life stages, progeny may be endowed with metabolic traits that help the animals cope with ambient temperature perturbations. This study also applied the feature rescaling and Uniform Manifold Approximation and Projection (UMAP) to visualize metabolic dynamics, and the result could effectively decompose the complex omic-based datasets to represent the energy trade-off variability. For example, the carbohydrate to free amino acid conversion and enhanced compensatory features appeared to be hypothermic-responsive traits. These multigenerational metabolic effects suggest that the tropical ectothermic tilapia may exhibit transgenerational phenotype plasticity, which could optimize energy allocation under ambient temperature challenges. Knowledge about such metabolism-related transgenerational plasticity effects in ectothermic aquatic species may allow us to better predict how adaptive mechanisms will affect fish populations in a climate with narrow temperature variation and frequent extreme weather events.

Improving cluster recovery with feature rescaling factors

The data preprocessing stage is crucial in clustering. Features may describe entities using different scales. To rectify this, one usually applies feature normalisation aiming at rescaling features so that none of them overpowers the others in the objective function of the selected clustering algorithm. In this paper, we argue that the rescaling procedure should not treat all features identically. Instead, it should favour the features that are more meaningful for clustering. With this in mind, we introduce a feature rescaling method that takes into account the within-cluster degree of relevance of each feature. Our comprehensive simulation study, carried out on real and synthetic data, with and without noise features, clearly demonstrates that clustering methods that use the proposed data normalization strategy clearly outperform those that use traditional data normalization.

Feature Rescaling and Fusion for Tiny Object Detection

Recent years have witnessed rapid developments on computer vision, however, there are still challenges in detecting tiny objects in a large-scale background. The tiny objects knowledge become sparse and weak due to their tiny size, which makes the tiny objects difficult to be detected with the common approaches. In this paper, a new network named Specific Characteristics based Feature Rescaling and Fusion (SFRF) is designed to detect tiny persons in a broad horizon and massive background. Different from the methods in general, a Nonparametric Adaptive Dense Perceiving Algorithm (NADPA) is designed to automatically select and generate a new resized feature map with the high density distribution of tiny objects. Then, a method called Many-For-One strategy is used for feature fusion of the feature pyramid network (FPN) layers to improve the feature representation and detection. Finally, an ensemble model method named hierarchical Coarse-to-fine mechanism is designed based on the proposed methods to further improve the performance. The experiments demonstrate that the proposed approach achieves an obvious performance improvement on tiny object detection than the existing approaches, and our approach has been awarded as the 1st-place in the first large-scale Tiny Object Detection (TOD) challenge.

Reproducible Evaluation of Diffusion MRI Features for Automatic Classification of Patients with Alzheimer's Disease.

Diffusion MRI is the modality of choice to study alterations of white matter. In past years, various works have used diffusion MRI for automatic classification of Alzheimer's disease. However, classification performance obtained with different approaches is difficult to compare because of variations in components such as input data, participant selection, image preprocessing, feature extraction, feature rescaling (FR), feature selection (FS) and cross-validation (CV) procedures. Moreover, these studies are also difficult to reproduce because these different components are not readily available. In a previous work (Samper-González et al. 2018), we propose an open-source framework for the reproducible evaluation of AD classification from T1-weighted (T1w) MRI and PET data. In the present paper, we first extend this framework to diffusion MRI data. Specifically, we add: conversion of diffusion MRI ADNI data into the BIDS standard and pipelines for diffusion MRI preprocessing and feature extraction. We then apply the framework to compare different components. First, FS has a positive impact on classification results: highest balanced accuracy (BA) improved from 0.76 to 0.82 for task CN vs AD. Secondly, voxel-wise features generally gives better performance than regional features. Fractional anisotropy (FA) and mean diffusivity (MD) provided comparable results for voxel-wise features. Moreover, we observe that the poor performance obtained in tasks involving MCI were potentially caused by the small data samples, rather than by the data imbalance. Furthermore, no extensive classification difference exists for different degree of smoothing and registration methods. Besides, we demonstrate that using non-nested validation of FS leads to unreliable and over-optimistic results: 5% up to 40% relative increase in BA. Lastly, with proper FR and FS, the performance of diffusion MRI features is comparable to that of T1w MRI. All the code of the framework and the experiments are publicly available: general-purpose tools have been integrated into the Clinica software package ( www.clinica.run ) and the paper-specific code is available at: https://github.com/aramis-lab/AD-ML .

End‐to‐end double JPEG detection with a 3D convolutional network in the DCT domain

Detection of double JPEG compression is essential in the field of digital image forensics. Although double JPEG compression detection methods have greatly improved with the development of convolutional neural networks (CNNs), they rely on handcrafted features such as discrete cosine transform (DCT) histograms. In this Letter, the authors propose an end-to-end trainable 3D CNN in the DCT domain for double JPEG compression detection. Moreover, they also propose a new type of module, called feature rescaling, to insert the quantisation table into the network suitably. The experiments show that the proposed method outperforms state-of-the-art methods.

A systematic model of stable multilateral automated negotiation in e-market environment

In e-market environment, the participants are usually bilateral such as in Consumer-to-Business or Customer-to-Customer business models. The participant on each side prefers the counterpart from which the concerned issues or profits can be pursued. Hence, the effective matching from a global point of view and the stable matching from an individual point of view become the critical function of the business models. In this paper, a systematic model of Stable Multilateral Automated Negotiation (SMAN) is proposed to facilitate the involved parties’ matching process in two-sided e-market, where confidential mediator agent as well as party agents communicate and make decisions on behalf of their principal parties. To make the matching effective and stable, two optimization models are designed. One is matching points model which makes an effective balance among the proposal value of issues for each possible pair of matching, such that the joint weighted profit measure is optimized with feature rescaling. The other one is matching scheme model which optimizes Social Welfare (SW) subject to the stable constraints, and ensures the engaged individual party satisfies the matching result from its viewpoint. And the optimality of stable matching is proved by mathematical deduction. Finally, numerical experiments are illustrated and show that the designed systematic models can generate effective matchings with individually stable advantages over the traditional Multilateral Automated Negotiation of Two Sides (MANTS).

Quantum Minimum Distance Classifier

We propose a quantum version of the well known minimum distance classification model called Nearest Mean Classifier (NMC). In this regard, we presented our first results in two previous works. First, a quantum counterpart of the NMC for two-dimensional problems was introduced, named Quantum Nearest Mean Classifier (QNMC), together with a possible generalization to any number of dimensions. Secondly, we studied the n-dimensional problem into detail and we showed a new encoding for arbitrary n-feature vectors into density operators. In the present paper, another promising encoding is considered, suggested by recent debates on quantum machine learning. Further, we observe a significant property concerning the non-invariance by feature rescaling of our quantum classifier. This fact, which represents a meaningful difference between the NMC and the respective quantum version, allows us to introduce a free parameter whose variation provides, in some cases, better classification results for the QNMC. The experimental section is devoted: (i) to compare the NMC and QNMC performance on different datasets; and (ii) to study the effects of the non-invariance under uniform rescaling for the QNMC.

Recovering the number of clusters in data sets with noise features using feature rescaling factors

In this paper we introduce three methods for re-scaling data sets aiming at improving the likelihood of clustering validity indexes to return the true number of spherical Gaussian clusters with additional noise features. Our method obtains feature re-scaling factors taking into account the structure of a given data set and the intuitive idea that different features may have different degrees of relevance at different clusters.We experiment with the Silhouette (using squared Euclidean, Manhattan, and the pth power of the Minkowski distance), Dunn’s, Calinski–Harabasz and Hartigan indexes on data sets with spherical Gaussian clusters with and without noise features. We conclude that our methods indeed increase the chances of estimating the true number of clusters in a data set.

Feature Relevance in Ward’s Hierarchical Clustering Using the L p Norm

In this paper we introduce a new hierarchical clustering algorithm called Wardp. Unlike the original Ward, Wardp generates feature weights, which can be seen as feature rescaling factors thanks to the use of the Lp norm. The feature weights are cluster dependent, allowing a feature to have different degrees of relevance at different clusters. We validate our method by performing experiments on a total of 75 real-world and synthetic datasets, with and without added features made of uniformly random noise. Our experiments show that: (i) the use of our feature weighting method produces results that are superior to those produced by the original Ward method on datasets containing noise features; (ii) it is indeed possible to estimate a good exponent p under a totally unsupervised framework. The clusterings produced by Wardp are dependent on p. This makes the estimation of a good value for this exponent a requirement for this algorithm, and indeed for any other also based on the Lp norm.

Image annotation based on feature fusion and semantic similarity

Abstract The present study developed a new algorithm based on multi-feature fusion and semantic similarity for image annotation. To study the relationship between feature distance and semantic similarity, the feature-annotation space is transformed into a distance–similarity space. Therefore, the intrinsic statistical relationship between visual and semantic information can be studied. Re-scaling is necessary to fuse multiple features. The potential multimodal properties of image features mean that traditional feature re-scaling is based on boundary values, which are sensitive to outliers. Our proposed distance re-scaling method overcomes the drawbacks of using statistical information. In the distance space, each distance vector of an image pair is treated as a sample. The anisotropic Gaussian distribution is transformed into an isotropic Gaussian with a mean of zero and standard variance. The nearest-distance images are retrieved from this space. To select features of not only low distance correlations but also a high semantic correlation, the visual and semantic relationship is studied using canonical correlation analysis. The canonical correlation coefficient of the similarity and distance is found to connect closely with the annotation score of the feature. Experiments showed that the proposed multi-feature fusion method removed the effects of scale and the correlations of feature distances, so it could represent the total distance better and find the nearest neighbors. We tested our method using the Corel5K, IAPR-TC12, ESP Game, and VOC PASCAL datasets, which showed that it outperformed existing approaches.

A modular framework for the automatic classification of chromosomes in Q-band images

Abstract The manual analysis of the karyogram is a complex and time-consuming operation, as it requires meticulous attention to details and well-trained personnel. Routine Q-band laboratory images show chromosomes that are randomly rotated, blurred or corrupted by overlapping and dye stains. We address here the problem of robust automatic classification, which is still an open issue. The proposed method starts with an improved estimation of the chromosome medial axis, along which an established set of features is then extracted. The following novel polarization stage estimates the chromosome orientation and makes this feature set independent on the reading direction along the axis. Feature rescaling and normalizing techniques take full advantage of the results of the polarization step, reducing the intra-class and increasing the inter-class variances. After a standard neural network based classification, a novel class reassignment algorithm is employed to maximize the probability of correct classification, by exploiting the constrained composition of the human karyotype. An average 94% of correct classification was achieved by the proposed method on 5474 chromosomes, whose images were acquired during laboratory routine and comprise karyotypes belonging to slightly different prometaphase stages. In order to provide the scientific community with a public dataset, all the data we used are publicly available for download.

Minkowski metric, feature weighting and anomalous cluster initializing in K-Means clustering

This paper represents another step in overcoming a drawback of K-Means, its lack of defense against noisy features, using feature weights in the criterion. The Weighted K-Means method by Huang et al. (2008, 2004, 2005) [5-7] is extended to the corresponding Minkowski metric for measuring distances. Under Minkowski metric the feature weights become intuitively appealing feature rescaling factors in a conventional K-Means criterion. To see how this can be used in addressing another issue of K-Means, the initial setting, a method to initialize K-Means with anomalous clusters is adapted. The Minkowski metric based method is experimentally validated on datasets from the UCI Machine Learning Repository and generated sets of Gaussian clusters, both as they are and with additional uniform random noise features, and appears to be competitive in comparison with other K-Means based feature weighting algorithms.

Feature Rescaling of Support Vector Machines

Support vector machines (SVMs) have widespread use in various classification problems. Although SVMs are often used as an off-the-shelf tool, there are still some important issues which require improvement, such as feature rescaling. Standardization is the most commonly used feature rescaling method. However, standardization does not always improve classification accuracy. This paper describes two feature rescaling methods: multiple kernel learning-based rescaling (MKL-SVM) and kernel-target alignment-based rescaling (KTA-SVM). MKL-SVM makes use of the framework of multiple kernel learning (MKL) and KTA-SVM is built upon the concept of kernel alignment, which measures the similarity between kernels. The proposed methods were compared with three other methods: an SVM method without rescaling, an SVM method with standardization, and SCADSVM. Test results demonstrate that different rescaling methods apply to different situations and that the proposed methods outperform the others in general.