Scale Equivariant Research Articles

Riesz Feature Representation: Scale Equivariant Scattering Network for Classification Tasks

Riesz Feature Representation: Scale Equivariant Scattering Network for Classification Tasks

A Novel Rotation and Scale Equivariant Network for Optical-SAR Image Matching

A Novel Rotation and Scale Equivariant Network for Optical-SAR Image Matching

Equivariant Wavelets: Fast Rotation and Translation Invariant Wavelet Scattering Transforms.

Wavelet scattering networks, which are convolutional neural networks (CNNs) with fixed filters and weights, are promising tools for image analysis. Imposing symmetry on image statistics can improve human interpretability, aid in generalization, and provide dimension reduction. In this work, we introduce a fast-to-compute, translationally invariant and rotationally equivariant wavelet scattering network (EqWS) and filter bank of wavelets (triglets). We demonstrate the interpretability and quantify the invariance/equivariance of the coefficients, briefly commenting on difficulties with implementing scale equivariance. On MNIST, we show that training on a rotationally invariant reduction of the coefficients maintains rotational invariance when generalized to test data and visualize residual symmetry breaking terms. Rotation equivariance is leveraged to estimate the rotation angle of digits and reconstruct the full rotation dependence of each coefficient from a single angle. We benchmark EqWS with linear classifiers on EMNIST and CIFAR-10/100, introducing a new second-order, cross-color channel coupling for the color images. We conclude by comparing the performance of an isotropic reduction of the scattering coefficients and RWST, a previous coefficient reduction, on an isotropic classification of magnetohydrodynamic simulations with astrophysical relevance.

Scale-Aware Network with Scale Equivariance

The convolutional neural network (CNN) has achieved good performance in object classification due to its inherent translation equivariance, but its scale equivariance is poor. A Scale-Aware Network (SA Net) with scale equivariance is proposed to estimate the scale during classification. The SA Net only learns samples of one scale in the training stage; in the testing stage, the unknown-scale testing samples are up-sampled and down-sampled, and a group of image copies with different scales are generated to form the image pyramid. The up-sampling adopts interpolation, and the down-sampling adopts interpolation combined with wavelet transform to avoid spectrum aliasing. The generated test samples with different scales are sent to the Siamese network with weight sharing for inferencing. According to the position of the maximum value of the classification-score matrix, the testing samples can be classified and the scale can be estimated simultaneously. The results on the MNIST and FMNIST datasets show that the SA Net has better performance than the existing methods. When the scale is larger than 4, the SA Net has higher classification accuracy than other methods. In the scale-estimation experiment, the SA Net can achieve low relative RMSE on any scale. The SA Net has potential for effective use in remote sensing, optical image recognition and medical diagnosis in cytohistology.

Equivariant Wavelets: Fast Rotation and Translation Invariant Wavelet Scattering Transforms

Wavelet scattering networks, which are convolutional neural networks (CNNs) with fixed filters and weights, are promising tools for image analysis. Imposing symmetry on image statistics can improve human interpretability, aid in generalization, and provide dimension reduction. In this work, we introduce a fast-to-compute, translationally invariant and rotationally equivariant wavelet scattering network (EqWS) and filter bank of wavelets (triglets). We demonstrate the interpretability and quantify the invariance/equivariance of the coefficients, briefly commenting on difficulties with implementing scale equivariance. On MNIST, we show that training on a rotationally invariant reduction of the coefficients maintains rotational invariance when generalized to test data and visualize residual symmetry breaking terms. Rotation equivariance is leveraged to estimate the rotation angle of digits and reconstruct the full rotation dependence of each coefficient from a single angle. We benchmark EqWS with linear classifiers on EMNIST and CIFAR-10/100, introducing a new second-order, cross-color channel coupling for the color images. We conclude by comparing the performance of an isotropic reduction of the scattering coefficients and RWST, a previous coefficient reduction, on an isotropic classification of magnetohydrodynamic simulations with astrophysical relevance.

The BAB algorithm for computing the total least trimmed squares estimator

Robust estimation in the errors-in-variables (EIV) model remains a difficult problem because of the leverage point and the masking effect and swamping effect. In this contribution, a new robust estimator is introduced for the EIV model. This method is a follow-up to least trimmed squares, which is applied to the Gauss–Markov model when only the observation vector contains outliers. We call this estimator the total least trimmed squares (TLTS) estimator because its criterion function consists of squared orthogonal residuals. The TLTS estimator excludes some large squared orthogonal residuals from the criterion function, thereby allowing the fit to ignore outliers. The TLTS estimator inherits appropriate equivariance properties, namely regression equivariance, scale equivariance and affine equivariance, and the maximal 50% asymptotic breakdown point in terms of observations $$ \varvec{y} $$ within the special cofactor matrix structure. The TLTS estimate can directly be obtained by the exhaustive evaluation method. We further develop another algorithm for the TLTS estimator based on the branch-and-bound method without exhaustive evaluation, but the cofactor matrix of the independent variables needs to have a certain block structure. Finally, two simulation studies provide insights into the robustness and efficiency of the proposed algorithms.

On an order-based multivariate median

Although the order-based definition of the univariate median is ubiquitous in statistics, the same order-based definition is typically abandoned when extending the univariate median to higher dimensions. In this paper, an example of order-based multivariate median based on the use of a linear extension of the product order is brought to the attention. Symmetry and internality properties are fulfilled by such order-based multivariate median, however, it turns out that this function fails to fulfil in general other appealing properties such as equivariance to different geometrical transformations, monotonicity properties and continuity. Interestingly, translation and scale equivariance and some monotonicity properties can be guaranteed if the linear extension of the product order is carefully chosen. Finally, it is proved that, unlike in the univariate setting, the finite-sample breakdown point of the order-based multivariate median is 1/n, thus implying that it is not robust in the presence of outliers.

Estimation after Selection from Uniform Populations under an Asymmetric Loss Function

SYNOPTIC ABSTRACTThe problem of estimation after selection arises in the situations where we wish to select a population among k available populations and estimate the parameter of the selected population. This paper considers estimation of the scale parameter θS of the selected uniform population, under the criterion of an asymmetric scale equivariant (ASE) loss function. For selecting the best uniform population, a class of selection rules proposed by Arshad and Misra (2015b) is used. Three natural estimators of θS based on the maximum likelihood estimators, uniformly minimum variance unbiased estimators, and minimum risk equivariant estimators are considered. The generalized Bayes estimator of θS with respect to a non-informative prior is derived. Under the ASE loss function, a general result for improving a scale-equivariant estimator of θS is provided. A consequence of this result, the estimators better than some of the natural estimators are obtained. Also, under the ASE loss function, a subclass of natural-type estimators is shown to be inadmissible for estimating θS. Finally, the risk functions of the various competing estimators of θS are compared via a simulation study.

Baseline configuration for GNSS attitude determination with an analytical least-squares solution

The GNSS attitude determination using carrier phase measurements with 4 antennas is studied on condition that the integer ambiguities have been resolved. The solution to the nonlinear least-squares is often obtained iteratively, however an analytical solution can exist for specific baseline configurations. The main aim of this work is to design this class of configurations. Both single and double difference measurements are treated which refer to the dedicated and non-dedicated receivers respectively. More realistic error models are employed in which the correlations between different measurements are given full consideration. The desired configurations are worked out. The configurations are rotation and scale equivariant and can be applied to both the dedicated and non-dedicated receivers. For these configurations, the analytical and optimal solution for the attitude is also given together with its error variance–covariance matrix.

Disjoint factor analysis with cross-loadings

Disjoint factor analysis (DFA) is a new latent factor model that we propose here to identify factors that relate to disjoint subsets of variables, thus simplifying the loading matrix structure. Similarly to exploratory factor analysis (EFA), the DFA does not hypothesize prior information on the number of factors and on the relevant relations between variables and factors. In DFA the population variance–covariance structure is hypothesized block diagonal after the proper permutation of variables and estimated by Maximum Likelihood, using an Coordinate Descent type algorithm. Inference on parameters on the number of factors and to confirm the hypothesized simple structure are provided. Properties such as scale equivariance, uniqueness, optimal simplification of loadings are satisfied by DFA. Relevant cross-loadings are also estimated in case they are detected from the best DFA solution. DFA has also the option to constrain a variable to load on a pre-specified factor so that the researcher can assume, a priori, some relations between variables and loadings. A simulation study shows performances of DFA and an application to optimally identify the dimensions of well-being is used to illustrate characteristics of the new methodology. A final discussion concludes the paper.

Improved estimation of the covariance matrix and the generalized variance of a multivariate normal distribution: some unifying results

Suppose that there is a typical (scale equivariant) improved estimator of the variance, σ 2, of a univariate normal distribution with unknown mean at our disposal. Using this estimator, in this work we construct in a very simple way an improved estimator of the covariance matrix, Σ, of a multivariate normal distribution with unknown mean and another improved estimator of the generalized variance, $\lvert \Sigma\rvert$ . The data is a sample of i.i.d. observations from this distribution. The loss function is the entropy loss or the quadratic loss in the case of Σ and a general loss satisfying a certain condition in the case of $\lvert\Sigma\rvert$ . These novel results reduce, in a specific way, the problems of estimating Σ or $\lvert\Sigma\rvert$ to the univariate problem of estimating σ 2. As a consequence, Stein-type, Brewster and Zidek-type, Strawderman-type and Maruyama-type improved estimators for Σ and $\lvert\Sigma\rvert$ are directly constructed from their univariate counterparts. This work unifies and extends previously obtained results on the estimation of Σ and $\lvert\Sigma\rvert$ .

On the uniqueness of distance covariance

Distance covariance and distance correlation are non-negative real numbers that characterize the independence of random vectors in arbitrary dimensions. In this work we prove that distance covariance is unique, starting from a definition of a covariance as a weighted L2 norm that measures the distance between the joint characteristic function of two random vectors and the product of their marginal characteristic functions. Rigid motion invariance and scale equivariance of these weighted L2 norms imply that the weight function of distance covariance is unique.

Simultaneous estimation of hazard rates of several exponential populations

Suppose independent random samples are drawn from k (2) populations with a common location parameter and unequal scale parameters. We consider the problem of estimating simultaneously the hazard rates of these populations. The analogues of the maximum likelihood (ML), uniformly minimum variance unbiased (UMVU) and the best scale equivariant (BSE) estimators for the one population case are improved using Rao‐Blackwellization. The improved version of the BSE estimator is shown to be the best among these estimators. Finally, a class of estimators that dominates this improved estimator is obtained using the differential inequality approach.

Comparison of normal variance estimators under multiple criteria and towards a compromise estimator

For estimating a normal variance under the squared error loss function it is well known that the best affine (location and scale) equivariant estimator, which is better than the maximum likelihood estimator as well as the unbiased estimator, is also inadmissible. The improved estimators, e.g., stein type, brown type and Brewster–Zidek type, are all scale equivariant but not location invariant. Lately, a good amount of research has been done to compare the improved estimators in terms of risk, but comparatively less attention had been paid to compare these estimators in terms of the Pitman nearness criterion (PNC) as well as the stochastic domination criterion (SDC). In this paper, we have undertaken a comprehensive study to compare various variance estimators in terms of the PNC and the SDC, which has been long overdue. Finally, using the results for risk, the PNC and the SDC, we propose a compromise estimator (sort of a robust estimator) which appears to work ‘well’ under all the criteria discussed above.

Admissibility of the Minimum Risk Scale Equivariant Estimator in the Problem of Nile

The decision problem of estimating the parameter in the problem of the Nile is considered. It is shown that the minimum risk scale equivariant estimator is admissible under the entropy loss function .

An exploratory positioning analysis: three-mode multivariate analysis for semantic differential data

The purpose of this study was to propose an exploratory positioning analysis method based on a three-mode multivariate statistical model for analyzing semantic differential (SD) data. This new method can be easily implemented by using a structural equation modeling (SEM) program such as LISREL and CALIS, because the mean structure and covariance structure of this method are expressed as submodels of SEM. Two necessary conditions for checking identifiability of this model are shown. It is also shown that this model is scale equivariant for SD data.

On equivariance and invariance of standard errors in three exploratory factor models

Current practice in factor analysis typically involves analysis of correlation rather than covariance matrices. We study whether the standardz-statistic that evaluates whether a factor loading is statistically necessary is correctly applied in such situations and more generally when the variables being analyzed are arbitrarily rescaled. Effects of rescaling on estimated standard errors of factor loading estimates, and the consequent effect onz-statistics, are studied in three variants of the classical exploratory factor model under canonical, raw varimax, and normal varimax solutions. For models with analytical solutions we find that some of the standard errors as well as their estimates are scale equivariant, while others are invariant. For a model in which an analytical solution does not exist, we use an example to illustrate that neither the factor loading estimates nor the standard error estimates possess scale equivariance or invariance, implying that different conclusions could be obtained with different scalings. Together with the prior findings on parameter estimates, these results provide new guidance for a key statistical aspect of factor analysis.

Scale equivariance and the Box-Cox transformation

The Box-Cox family of transformations has two useful features: first, it includes linear and logarithmic transformations as special cases; and, second, it possesses strong scale equivariance properties, including the property that the transformation parameter is unaffected by the rescaling. Its main disadvantage is that both the domain and the range of the transformation are, in general, bounded. We show that, for a certain class of models, if a model demonstrates these scale equivariance properties for the dependent variable then the transformation on the dependent variable must be a variant of the Box-Cox transformation.

Estimation of quantile density function based on regression quantiles

We propose two estimators of quantile density function in linear regression model. The estimators, either of histogram or of kernel types, are based on regression quantiles and extend the Falk (1986) estimators based on order statistics from the location to the linear regression model. Unlike various other estimators proposed in the literature, our estimators are regression invariant and scale equivariant and hence applicable in estimation, testing, bounded-length confidence interval estimation and other inference based on L 1-norm.

Some modifications of improved estimators of a normal variance

Consider the problem of estimating a normal variance based on a random sample when the mean is unknown. Scale equivariant estimators which improve upon the best scale and translation equivariant one have been proposed by several authors for various loss functions including quadratic loss. However, at least for quadratic loss function, improvement is not much. Herein, some methods are proposed to construct improving estimators which are not scale equivariant and are expected to be considerably better when the true variance value is close to the specified one. The idea behind the methods is to modify improving equivariant shrinkage estimators, so that the resulting ones shrink little when the usual estimate is less than the specified value and shrink much more otherwise. Sufficient conditions are given for the estimators to dominate the best scale and translation equivariant rule under the quadratic loss and the entropy loss. Further, some results of a Monte Carlo experiment are reported which show the significant improvements by the proposed estimators.