Desirable Invariance Research Articles

Supervised Learning for Nonsequential Data: A Canonical Polyadic Decomposition Approach.

Efficient modeling of feature interactions underpins supervised learning for nonsequential tasks, characterized by a lack of inherent ordering of features (variables). The brute force approach of learning a parameter for each interaction of every order comes at an exponential computational and memory cost (curse of dimensionality). To alleviate this issue, it has been proposed to implicitly represent the model parameters as a tensor, the order of which is equal to the number of features; for efficiency, it can be further factorized into a compact tensor train (TT) format. However, both TT and other tensor networks (TNs), such as tensor ring and hierarchical Tucker, are sensitive to the ordering of their indices (and hence to the features). To establish the desired invariance to feature ordering, we propose to represent the weight tensor through the canonical polyadic (CP) decomposition (CPD) and introduce the associated inference and learning algorithms, including suitable regularization and initialization schemes. It is demonstrated that the proposed CP-based predictor significantly outperforms other TN-based predictors on sparse data while exhibiting comparable performance on dense nonsequential tasks. Furthermore, for enhanced expressiveness, we generalize the framework to allow feature mapping to arbitrarily high-dimensional feature vectors. In conjunction with feature vector normalization, this is shown to yield dramatic improvements in performance for dense nonsequential tasks, matching models such as fully connected neural networks.

Color Control Functions for Multiprimary Displays I: Robustness Analysis and Optimization Formulations.

Color management for a multiprimary display requires, as a fundamental step, the determination of a color control function (CCF) that specifies control values for reproducing each color in the display's gamut. Multiprimary displays offer alternative choices of control values for reproducing a color in the interior of the gamut and accordingly alternative choices of CCFs. Under ideal conditions, alternative CCFs render colors identically. However, deviations in the spectral distributions of the primaries and the diversity of cone sensitivities among observers impact alternative CCFs differently, and, in particular, make some CCFs prone to artifacts in rendered images. We develop a framework for analyzing robustness of CCFs for multiprimary displays against primary and observer variations, incorporating a common model of human color perception. Using the framework, we propose analytical and numerical approaches for determining robust CCFs. First, via analytical development, we: (a) demonstrate that linearity of the CCF in tristimulus space endows it with resilience to variations, particularly, linearity can ensure invariance of the gray axis, (b) construct an axially linear CCF that is defined by the property of linearity over constant chromaticity loci, and (c) obtain an analytical form for the axially linear CCF that demonstrates it is continuous but suffers from the limitation that it does not have continuous derivatives. Second, to overcome the limitation of the axially linear CCF, we motivate and develop two variational objective functions for optimization of multiprimary CCFs, the first aims to preserve color transitions in the presence of primary/observer variations and the second combines this objective with desirable invariance along the gray axis, by incorporating the axially linear CCF. A companion Part II paper, presents an algorithmic approach for numerically computing optimal CCFs for the two alternative variational objective functions proposed here and presents results comparing alternative CCFs for several different 4,5, and 6 primary designs.

Sensitivity Analysis of Deep Neural Networks

Deep neural networks (DNNs) have achieved superior performance in various prediction tasks, but can be very vulnerable to adversarial examples or perturbations. Therefore, it is crucial to measure the sensitivity of DNNs to various forms of perturbations in real applications. We introduce a novel perturbation manifold and its associated influence measure to quantify the effects of various perturbations on DNN classifiers. Such perturbations include various external and internal perturbations to input samples and network parameters. The proposed measure is motivated by information geometry and provides desirable invariance properties. We demonstrate that our influence measure is useful for four model building tasks: detecting potential ‘outliers’, analyzing the sensitivity of model architectures, comparing network sensitivity between training and test sets, and locating vulnerable areas. Experiments show reasonably good performance of the proposed measure for the popular DNN models ResNet50 and DenseNet121 on CIFAR10 and MNIST datasets.

Effectively Quantifying the Performance of Lower-Limb Exoskeletons Over a Range of Walking Conditions.

Exoskeletons and other wearable robotic devices have a wide range of potential applications, including assisting patients with walking pathologies, acting as tools for rehabilitation, and enhancing the capabilities of healthy humans. However, applying these devices effectively in a real-world setting can be challenging, as the optimal design features and control commands for an exoskeleton are highly dependent on the current user, task and environment. Consequently, robust metrics and methods for quantifying exoskeleton performance are required. This work presents an analysis of walking data collected for healthy subjects walking with an active pelvis exoskeleton over three assistance scenarios and five walking contexts. Spatial and temporal, kinematic, kinetic and other novel dynamic gait metrics were compared to identify which metrics exhibit desirable invariance properties, and so are good candidates for use as a stability metric over varying walking conditions. Additionally, using a model-based approach, the average metabolic power consumption was calculated for a subset of muscles crossing the hip, knee and ankle joints, and used to analyse how the energy-reducing properties of an exoskeleton are affected by changes in walking context. The results demonstrated that medio-lateral centre of pressure displacement and medio-lateral margin of stability exhibit strong invariance to changes in walking conditions. This suggests that these dynamic gait metrics are optimised in human gait and are potentially suitable metrics for optimising in an exoskeleton control paradigm. The effectiveness of the exoskeleton at reducing human energy expenditure was observed to increase when walking on an incline, where muscles aiding in hip flexion were assisted, but decrease when walking at a slow speed. These results underline the need for adaptive control algorithms for exoskeletons if they are to be used in varied environments.

A Proportional Scale-Invariant Measurement of Heterogeneous Directional Distances to the Technological Frontier

We consider a stochastic multiplicative-directional-distance-function-based formulation of the production technology in the presence of bad outputs. In contrast to the popular additive directional distance function, our approach preserves the highly desired scale invariance property of the proportional radial distance function and can easily assume popular translog specification. Our model can also meaningfully accommodate technological heterogeneity across individual firms by letting the direction in which the distance to frontier is measured be an unknown function of the firm’s idiosyncratic characteristics reflective of the heterogeneous economic environment in which it operates. To mitigate the uncertainty associated with an arbitrary choice of the direction, we rely on a data-driven selection method to determine these varying heterogeneous directional vectors for individual firms. We showcase practical advantages of the proposed model by estimating the production technology of U.S. commercial banks in the presence of undesirable non-performing loans during the 2001–2010 period.

Approximate joint diagonalization and geometric mean of symmetric positive definite matrices.

We explore the connection between two problems that have arisen independently in the signal processing and related fields: the estimation of the geometric mean of a set of symmetric positive definite (SPD) matrices and their approximate joint diagonalization (AJD). Today there is a considerable interest in estimating the geometric mean of a SPD matrix set in the manifold of SPD matrices endowed with the Fisher information metric. The resulting mean has several important invariance properties and has proven very useful in diverse engineering applications such as biomedical and image data processing. While for two SPD matrices the mean has an algebraic closed form solution, for a set of more than two SPD matrices it can only be estimated by iterative algorithms. However, none of the existing iterative algorithms feature at the same time fast convergence, low computational complexity per iteration and guarantee of convergence. For this reason, recently other definitions of geometric mean based on symmetric divergence measures, such as the Bhattacharyya divergence, have been considered. The resulting means, although possibly useful in practice, do not satisfy all desirable invariance properties. In this paper we consider geometric means of covariance matrices estimated on high-dimensional time-series, assuming that the data is generated according to an instantaneous mixing model, which is very common in signal processing. We show that in these circumstances we can approximate the Fisher information geometric mean by employing an efficient AJD algorithm. Our approximation is in general much closer to the Fisher information geometric mean as compared to its competitors and verifies many invariance properties. Furthermore, convergence is guaranteed, the computational complexity is low and the convergence rate is quadratic. The accuracy of this new geometric mean approximation is demonstrated by means of simulations.

Shape Matching Using Multiscale Integral Invariants.

We present a shape descriptor based on integral kernels. Shape is represented in an implicit form and it is characterized by a series of isotropic kernels that provide desirable invariance properties. The shape features are characterized at multiple scales which form a signature that is a compact description of shape over a range of scales. The shape signature is designed to be invariant with respect to group transformations which include translation, rotation, scaling, and reflection. In addition, the integral kernels that characterize local shape geometry enable the shape signature to be robust with respect to undesirable perturbations while retaining discriminative power. Use of our shape signature is demonstrated for shape matching based on a number of synthetic and real examples.

Localization in Urban Environments Using a Panoramic Gist Descriptor

Vision-based topological localization and mapping for autonomous robotic systems have received increased research interest in recent years. The need to map larger environments requires models at different levels of abstraction and additional abilities to deal with large amounts of data efficiently. Most successful approaches for appearance-based localization and mapping with large datasets typically represent locations using local image features. We study the feasibility of performing these tasks in urban environments using global descriptors instead and taking advantage of the increasingly common panoramic datasets. This paper describes how to represent a panorama using the global gist descriptor, while maintaining desirable invariance properties for location recognition and loop detection. We propose different gist similarity measures and algorithms for appearance-based localization and an online loop-closure detection method, where the probability of loop closure is determined in a Bayesian filtering framework using the proposed image representation. The extensive experimental validation in this paper shows that their performance in urban environments is comparable with local-feature-based approaches when using wide field-of-view images.

Sparse Reduced-Rank Regression for Simultaneous Dimension Reduction and Variable Selection

The reduced-rank regression is an effective method in predicting multiple response variables from the same set of predictor variables. It reduces the number of model parameters and takes advantage of interrelations between the response variables and hence improves predictive accuracy. We propose to select relevant variables for reduced-rank regression by using a sparsity-inducing penalty. We apply a group-lasso type penalty that treats each row of the matrix of the regression coefficients as a group and show that this penalty satisfies certain desirable invariance properties. We develop two numerical algorithms to solve the penalized regression problem and establish the asymptotic consistency of the proposed method. In particular, the manifold structure of the reduced-rank regression coefficient matrix is considered and studied in our theoretical analysis. In our simulation study and real data analysis, the new method is compared with several existing variable selection methods for multivariate regression and exhibits competitive performance in prediction and variable selection.

Exact solutions to three-dimensional time-dependent Schrödinger equation

With a view to obtain exact analytic solutions to the time-dependent Schrodinger equation for a few potentials of physical interest in three dimensions, transformation-group method is used. Interestingly, the integrals of motion in the new coordinates turn out to be the desired invariants of the systems.

Frequency dependent heat capacity in the cure of epoxy resins

Different temperature modulated differential scanning calorimetry ( TMDSC) approaches were explored for studying the frequency dependence of the vitrification phenomenon during the cure of a thermosetting epoxy resin. All approaches show the expected decrease in vitrification time with increasing frequency. Light ( heating) ( temperature) modulated DSC ( LMDSC), developed by Saruyama, offers the advantage that the frequency range for direct measurements is extended (from 0.01 to 1 Hz). However, at frequencies above 0.1 Hz, the results contain information about the changing thermal diffusivity (for samples above 100 μm thick). A complex saw-tooth modulation method, developed by Wunderlich, allows multiple frequencies (over almost one decade) to be measured in a single experiment, thus eliminating effects of sample reproducibility. For studying the frequency dependence during (chemical) transformations, such as cure, the desired invariance of the sample during at least one modulation period and thermal diffusivity effects (even for films of only 100 μm thick) limit a further extension of the frequency domain to lower and higher frequencies, respectively.

Word Spotting in Bitmapped Fax Documents

Images and signals may be represented by forms invariant to time shifts, spatial shifts, frequency shifts, and scale changes. Advances in time-frequency analysis and scale transform techniques have made this possible. However, factors such as noise contamination and “style” differences complicate this. An example is found in text, where letters and words may vary in size and position. Examples of complicating variations include the font used, corruption during facsimile (fax) transmission, and printer characteristics. The solution advanced in this paper is to cast the desired invariants into separate subspaces for each extraneous factor or group of factors. The first goal is to have minimal overlap between these subspaces and the second goal is to be able to identify each subspace accurately. Concepts borrowed from high-resolution spectral analysis, but adapted uniquely to this problem have been found to be useful in this context. Once the pertinent subspace is identified, the recognition of a particular invariant form within this subspace is relatively simple using well-known singular value decomposition (SVD) techniques. The basic elements of the approach can be applied to a variety of pattern recognition problems. The specific application covered in this paper is word spotting in bitmapped fax documents.

Development in a biologically inspired spinal neural network for movement control

In two phases, we develop increasingly complex neural network models of spinal circuitry that self-organizes into networks with opponent channels for the control of an antagonistic muscle pair. The self-organization is enabled by a Hebbian learning rule operating during spontaneous activity present in the spinal cord. After the self-organized development, the networks enable independent control of the length and tension of the innervated muscles. This allows higher centers to hold joint angle invariant while varying joint stiffness and vice versa. The first network comprises only spontaneous activity generators, motorneurons, and inhibitory interneurons through which the two channels interact. The inhibitory interneurons enhance reciprocal action, and prevent saturation of the motorneuron pools, which is a necessary condition for independent control. In the second network, the neurons in the motorneuron pools obey the size-principle, which, when added by itself, leads to a loss of the desired invariance property. To restore the desired invariance, the second network further incorporated inhibitory interneurons analogous to Renshaw cells. The results obtained from the two models compare favourably with the FLETE-model for spinal circuitry ( Bullock and Contreras-Vidal, 1993; Bullock et al., 1992; Bullock and Grossberg, 1991) which, although successful in explaining several phenomena related to motor control, did not self-organize its connection weights. Finally, we suggest ways in which this research could be applied in technology.

Pattern recognition using invariants defined from higher order spectra: 2-D image inputs

A new algorithm for extracting features from images for object recognition is described. The algorithm uses higher order spectra to provide desirable invariance properties, to provide noise immunity, and to incorporate nonlinearity into the feature extraction procedure thereby allowing the use of simple classifiers. An image can be reduced to a set of 1D functions via the Radon transform, or alternatively, the Fourier transform of each 1D projection can be obtained from a radial slice of the 2D Fourier transform of the image according to the Fourier slice theorem. A triple product of Fourier coefficients, referred to as the deterministic bispectrum, is computed for each 1D function and is integrated along radial lines in bifrequency space. Phases of the integrated bispectra are shown to be translation- and scale-invariant. Rotation invariance is achieved by a regrouping of these invariants at a constant radius followed by a second stage of invariant extraction. Rotation invariance is thus converted to translation invariance in the second step. Results using synthetic and actual images show that isolated, compact clusters are formed in feature space. These clusters are linearly separable, indicating that the nonlinearity required in the mapping from the input space to the classification space is incorporated well into the feature extraction stage. The use of higher order spectra results in good noise immunity, as verified with synthetic and real images. Classification of images using the higher order spectra-based algorithm compares favorably to classification using the method of moment invariants.

An assumed‐stress hybrid 4‐node shell element with drilling degrees of freedom

AbstractAn assumed‐stress hybrid/mixed 4‐node quadrilateral shell element is introduced that alleviates most of the deficiencies associated with such elements. The formulation of the element is based on the assumed‐stress hybrid/mixed method using the Hellinger‐Reissner variational principle. The membrane part of the element has 12 degrees of freedom including rotational or ‘drilling’ degrees of freedom at the nodes. The bending part of the element also has 12 degrees of freedom. The bending part of the element uses the Reissner‐Mindlin plate theory which takes into account the transverse shear contributions. The element formulation is derived from an 8‐node isoparametric element by expressing the midside displacement degrees of freedom in terms of displacement and rotational degrees of freedom at corner nodes. The element passes the patch test, is nearly insensitive to mesh distortion, does not ‘lock’, possesses the desirable invariance properties, has no hidden spurious modes, and for the majority of test cases used in this paper produces more accurate results than the other elements employed herein for comparison.

A Distribution-Free Test for Symmetry Based on a Runs Statistic

Abstract I present a simple test, based on a runs statistic, for symmetry of a continuous distribution about a known median. The statistic has a binomial sampling distribution and desirable invariance properties. Monte Carlo studies demonstrate that, for a wide variety of alternative asymmetric distributions, the test is more powerful than tests proposed by Butler (1969), Rothman and Woodroofe (1972), or Hill and Rao (1977).

Invariant surface characteristics for 3D object recognition in range images

In recent years there has been a tremendous increase in computer vision research using range images (or depth maps) as sensor input data. The most attractive feature of range images is the explicitness of the surface information. Many industrial and navigational robotic tasks will be more easily accomplished if such explicit depth information can be efficiently obtained and interpreted. Intensity image understanding research has shown that the early processing of sensor data should be data-driven. The goal of early processing is to generate a rich description for later processing. Classical differential geometry provides a complete local description of smooth surfaces. The first and second fundamental forms of surfaces provide a set of differential-geometric shape descriptors that capture domain-independent surface information. Mean curvature and Gaussian curvature are the fundamental second-order surface characteristics that possess desirable invariance properties and represent extrinsic and intrinsic surface geometry respectively. The signs of these surface curvatures are used to classify range image regions into one of eight basic viewpoint-independent surface types. Experimental results for real and synthetic range images show the properties, usefulness, and importance of differential-geometric surface characteristics.

Some Considerations About Bond Indices in Non-orthogonal Bases and the MO Calculation of Valence and Oxidation Number

In order to guarantee the desired invariance properties of bond indices, we show the importance of expliciting the tensor character of the matrices concerned, so as to deal with a contraction in the tensor sense between a covariant index and a contravariant one. A straightforward definition of oxidation numbers is proposed. IEH calculations of valence and oxidation numbers for some appropriate examples are performed: they emphasize the role of "secondary" bonds in N and C-containing compounds; the hydrogen behaviour in half-bonds and strong H-bonds is satisfactorily accounted for; valence and oxidation number values are assigned to Fe, Co and Ni in a few complexes.

Partial differential approximants for multivariable power series. I. Definitions and faithfulness

The approximation of functions of two (or more) variables given in terms of their power series is discussed. Partial differential approximants, and their multipoint and constrained extensions, are defined systematically and their theory is developed. The questions of existence and uniqueness are addressed and broad criteria are established under which the approxi­mants are faithful in that their expansions match the originally given series to appropriate order. The problem is raised of identifying classes of approximants possessing desirable invariance properties: part II (in preparation) attacks this generally and provides explicit criteria for invariance under Euler transformations and under homogeneous linear transformations of the variables.

The Role of Randomization in Model‐Based Inference

SummaryA common justification for randomization in sampling and experimental design is that it creates certain desirable invariance properties, e.g., exchangeability, in the distribution of the sample responses, and this is so regardless of the distribution of the population responses. This paper argues that for model‐based inference randomization's role is much more limited, viz., to preserve invariance properties already embodied in the model for the population responses.