Smoothness Classes Research Articles

Approximation bounds for norm constrained neural networks with applications to regression and GANs

This paper studies the approximation capacity of ReLU neural networks with norm constraint on the weights. We prove upper and lower bounds on the approximation error of these networks for smooth function classes. The lower bound is derived through the Rademacher complexity of neural networks, which may be of independent interest. We apply these approximation bounds to analyze the convergences of regression using norm constrained neural networks and distribution estimation by GANs. In particular, we obtain convergence rates for over-parameterized neural networks. It is also shown that GANs can achieve optimal rate of learning probability distributions, when the discriminator is a properly chosen norm constrained neural network.

Lecturers’ attitude towards the use of e-learning tools in higher education: A case of Portugal

Abstract This study aims to assess the lecturers’ opinions about the use of e-learning tools to support distance and blended learning in higher education in Portugal, evidently reinforced by the COVID-19 pandemic. This research was based on a qualitative methodology, specifically, a focus group with professors from five higher education institutions from different geographical areas in Portugal. The obtained results were analysed along four main dimensions: (1) the level of knowledge of e-learning tools, (2) the reasons for using or (3) not using them, and, finally, (4) the opinion of lecturers on the student assessment process using these tools. The results showed that in addition to the concerns with smooth running classes and the appropriate delivery of the syllabus, the lecturers considered the transition to the e-learning context to have been easy. They noted a high level of literacy in the used tools, believed in the continued use of e-learning in the post-pandemic context, indicated several advantages for those involved in the e-learning context and a majority of limitations related to the time required for the adoption of more tools; and, finally, underlined the student assessment issue, which was pointed out as the most sensitive topic in the whole e-learning context. The study informed on the lecturers’ perspective on e-learning and the used tools and provided insight into their perceived usefulness and benefits for lecturers and students. An especially strong concern was verified on the part of lecturers to optimise e-learning tools to provide better knowledge delivery to students.

Nonlinear model reduction to fractional and mixed-mode spectral submanifolds.

A primary spectral submanifold (SSM) is the unique smoothest nonlinear continuation of a nonresonant spectral subspace E of a dynamical system linearized at a fixed point. Passing from the full nonlinear dynamics to the flow on an attracting primary SSM provides a mathematically precise reduction of the full system dynamics to a very low-dimensional, smooth model in polynomial form. A limitation of this model reduction approach has been, however, that the spectral subspace yielding the SSM must be spanned by eigenvectors of the same stability type. A further limitation has been that in some problems, the nonlinear behavior of interest may be far away from the smoothest nonlinear continuation of the invariant subspace E. Here, we remove both of these limitations by constructing a significantly extended class of SSMs that also contains invariant manifolds with mixed internal stability types and of lower smoothness class arising from fractional powers in their parametrization. We show on examples how fractional and mixed-mode SSMs extend the power of data-driven SSM reduction to transitions in shear flows, dynamic buckling of beams, and periodically forced nonlinear oscillatory systems. More generally, our results reveal the general function library that should be used beyond integer-powered polynomials in fitting nonlinear reduced-order models to data.

Insulator OOD state identification algorithm based on distribution calibration with smooth classification boundaries

Insulator OOD state identification algorithm based on distribution calibration with smooth classification boundaries

FCR-TrackNet: Towards high-performance 6D pose tracking with multi-level features fusion and joint classification-regression

Tracking an object’s 6D pose is significant in various real-world applications, such as robotic grasping, virtual reality, and self-driving. Still the existing methods suffer from following challenges: i) geometric shapes, such as rotational symmetry shapes, and ii) complex scenes, for instance, cluttered backgrounds and occluded scenes. To tackle these problems, we propose FCR-TrackNet, a novel tracking network that utilizes a residual iterative framework with low-and high-level feature fusion and joint classification-regression. FCR-TrackNet inputs the rendered RGB-D image from the previous frame pose and the current RGB-D image to extract low-level features. Then, high-level features are also obtained by the convolutional network and fused with low-level one to capture subtle variations in target object features across adjacent frames. To reduce computational complexity and ensure high tracking speed, we adopt decoupled branches to estimate the translation and rotation of the pose independently. At last, a joint classification-regression is designed to address the boundary problem of the rotation angle. We introduce a smooth classification label that effectively enhances the accuracy of rotation vector classification. We evaluate the performance of FCR-TrackNet on two well-known datasets, YCB-Video and YCBInEOAT. FCR-TrackNet achieves state-of-the-art ADD values of 94.5% and 93.2%, and ADD-S values of 96.7% and 96.0%, respectively, with a tracking speed of 89.6 Hz. It also outperforms competing algorithms in target pose tracking in occlusion and rotational symmetry shapes. The quantitative and qualitative results validate the high performance of FCR-TrackNet on 6D pose tracking.

Learning Block Structured Graphs in Gaussian Graphical Models

A prior distribution for the underlying graph is introduced in the framework of Gaussian graphical models. Such a prior distribution induces a block structure in the graph’s adjacency matrix, allowing learning relationships between fixed groups of variables. A novel sampling strategy named Double Reversible Jumps Markov chain Monte Carlo is developed for learning block structured graphs under the conjugate G-Wishart prior. The algorithm proposes moves that add or remove not just a single edge of the graph but an entire group of edges. The method is then applied to smooth functional data. The classical smoothing procedure is improved by placing a graphical model on the basis expansion coefficients, providing an estimate of their conditional dependence structure. Since the elements of a B-Spline basis have compact support, the conditional dependence structure is reflected on well-defined portions of the domain. A known partition of the functional domain is exploited to investigate relationships among portions of the domain and improve the interpretability of the results. Supplementary materials for this article are available online.

Local approximation of operators

Many applications, such as system identification, classification of time series, direct and inverse problems in partial differential equations, and uncertainty quantification lead to the question of approximation of a non-linear operator between metric spaces X and Y. We study the problem of determining the degree of approximation of such operators on a compact subset KX⊂X using a finite amount of information. If F:KX→KY, a well established strategy to approximate F(F) for some F∈KX is to encode F (respectively, F(F)) in terms of a finite number d (respectively m) of real numbers. Together with appropriate reconstruction algorithms (decoders), the problem reduces to the approximation of m functions on a compact subset of a high dimensional Euclidean space Rd, equivalently, the unit sphere Sd embedded in Rd+1. The problem is challenging because d, m, as well as the complexity of the approximation on Sd are all large, and it is necessary to estimate the accuracy keeping track of the inter-dependence of all the approximations involved. In this paper, we establish constructive methods to do this efficiently; i.e., with the constants involved in the estimates on the approximation on Sd being O(d1/6). We study different smoothness classes for the operators, and also propose a method for approximation of F(F) using only information in a small neighborhood of F, resulting in an effective reduction in the number of parameters involved. To further mitigate the problem of large number of parameters, we propose prefabricated networks, resulting in a substantially smaller number of effective parameters. The problem is studied in both deterministic and probabilistic settings.

Region feature smoothness assumption for weakly semi‐supervised crowd counting

AbstractCrowd counting is a hot issue in visual data processing. It also plays an important role in the field of video surveillance, social security, and traffic control. However, most of the existing crowd counting methods always adopt a mount of training data or point‐level annotation to learn the mapping relationships between images and density maps, which would cost much human labor. In this paper, we propose a new weakly semi‐supervised crowd counting method which uses less count‐level data for data training. In particular, we extend the classical smoothness assumption and design a many‐to‐many Region Feature Smoothness Assumption to deal with the uneven density distribution problem within crowd region. Further, we adopt hypergraph representation to explore the complex high‐order relationship for different crowd regions. Besides, we design a multi‐scale dynamic hypergraph convolutional module and hyperedge contrastive loss. Extensive experiments have been conducted on five public datasets. The experimental results show that the proposed method outperforms the state‐of‐the‐art ones.

KaRIn Noise Reduction Using a Convolutional Neural Network for the SWOT Ocean Products

The SWOT (Surface Water Ocean Topography) mission will provide high-resolution and two-dimensional measurements of sea surface height (SSH). However, despite its unprecedented precision, SWOT’s Ka-band Radar Interferometer (KaRIn) still exhibits a substantial amount of random noise. In turn, the random noise limits the ability of SWOT to capture the smallest scales of the ocean’s topography and its derivatives. In that context, this paper explores the feasibility, strengths and limits of a noise-reduction algorithm based on a convolutional neural network. The model is based on a U-Net architecture and is trained and tested with simulated data from the North Atlantic. Our results are compared to classical smoothing methods: a median filter, a Lanczos kernel smoother and the SWOT de-noising algorithm developed by Gomez-Navarro et al. Our U-Net model yields better results for all the evaluation metrics: 2 mm root mean square error, sub-millimetric bias, variance reduction by factor of 44 (16 dB) and an accurate power spectral density down to 10–20 km wavelengths. We also tested various scenarios to infer the robustness and the stability of the U-Net. The U-Net always exhibits good performance and can be further improved with retraining if necessary. This robustness in simulation is very encouraging: our findings show that the U-Net architecture is likely one of the best candidates to reduce the noise of flight data from KaRIn.

A calibrated SVM based on weighted smooth [formula omitted] for Alzheimer’s disease prediction

Alzheimer’s disease (AD) is currently one of the mainstream senile diseases in the world. It is a key problem predicting the early stage of AD. Low accuracy recognition of AD and high redundancy brain lesions are the main obstacles. Traditionally, Group Lasso method can achieve good sparseness. But, redundancy inside group is ignored. This paper proposes an improved smooth classification framework which combines the weighted smooth GL1/2 (wSGL1/2) as feature selection method and a calibrated support vector machine (cSVM) as the classifier. wSGL1/2 can make intra-group and inner-group features sparse, in which the group weights can further improve the efficiency of the model. cSVM can enhance the speed and stability of model by adding calibrated hinge function. Before feature selecting, an anatomical boundary-based clustering, called as ac-SLIC-AAL, is designed to make adjacent similar voxels into one group for accommodating the overall differences of all data. The cSVM model is fast convergence speed, high accuracy and good interpretability on AD classification, AD early diagnosis and MCI transition prediction. In experiments, all steps are tested respectively, including classifiers’ comparison, feature selection verification, generalization verification and comparing with state-of-the-art methods. The results are supportive and satisfactory. The superior of the proposed model are verified globally. At the same time, the algorithm can point out the important brain areas in the MRI, which has important reference value for the doctor’s predictive work. The source code and data is available at http://github.com/Hu-s-h/c-SVMForMRI.

Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions

We study the approximation by tensor networks (TNs) of functions from classical smoothness classes. The considered approximation tool combines a tensorization of functions in $L^p([0,1))$, which allows to identify a univariate function with a multivariate function (or tensor), and the use of tree tensor networks (the tensor train format) for exploiting low-rank structures of multivariate functions. The resulting tool can be interpreted as a feed-forward neural network, with first layers implementing the tensorization, interpreted as a particular featuring step, followed by a sum-product network with sparse architecture. In part I of this work, we presented several approximation classes associated with different measures of complexity of tensor networks and studied their properties. In this work (part II), we show how classical approximation tools, such as polynomials or splines (with fixed or free knots), can be encoded as a tensor network with controlled complexity. We use this to derive direct (Jackson) inequalities for the approximation spaces of tensor networks. This is then utilized to show that Besov spaces are continuously embedded into these approximation spaces. In other words, we show that arbitrary Besov functions can be approximated with optimal or near to optimal rate. We also show that an arbitrary function in the approximation class possesses no Besov smoothness, unless one limits the depth of the tensor network.

Periodic points of self-maps of products of lens spaces $L(3)\times L(3)$

Let $f\colon M\to M$ be a self-map of a compact manifold and $n\in \mathbb{N}$. The least number of $n$-periodic points in the smooth homotopy class of $f$ may be smaller than in the continuous homotopy class. We ask: for which self-maps $f\colon M\to M$ the two minima are the same, for each prescribed multiplicity? In the study of self-maps of tori and compact Lie groups a necessary condition appears. Here we give a criterion which helps to decide whether the necessary condition is also sufficient. We apply this result to show that for self-maps of the product of the lens space $M=L(3)\times L(3)$ the necessary condition is also sufficient.

Beyond the Concept of Method: Up-to-date School Language Teachers

Prospective teachers are crammed with a good number of teaching methods while doing their courses in colleges or universities. All these methods theoretically valid. However, once teachers commence their journeys and get a room in practical classrooms, one can observe that no particular method is executed wholly, and in some circumstances these methods are not functioning well in one particular classroom or on a particular learner as every classroom is unique and every learner is unique. Teachers need to develop their own theories to be practised in classrooms as most of the proposed widespread theories and methods of teaching have been developed by professionals in their labs or universities settings, and who might never pay a visit to a classroom, especially at school level. The present paper is in response to the current affairs caused by the pandemic, Covid-19 and urges teachers in their classrooms to adopt a holistic approach to successfully manage what is happening in their classrooms or online teaching. The Covid-19 (January through September 2020) was an evaluative test to many teachers and institutes. Upon the announcement of spread of this particular pandemic, almost all schools and universities in many countries felt like paralysed, shut their classrooms and halls and even suspended the exams. Technology as a medium of teaching should have been employed from the very beginning of Covid-19 period, but teachers and institutes were not well-prepared for such a case and many could not dare to teach using online platforms. Succeeding to deliver smooth and fruitful classes in this period of time will have a positive effect in the post-covid-19 time in which a perfect blend between classroom and online learning-teaching will be practiced with ease and maximise the outcomes among learners.

Hyperspectral Image Classification Using a Superpixel–Pixel–Subpixel Multilevel Network

Hyperspectral images (HSIs) often contain irregular ground cover with mixed spectral features and noise, which makes it challenging to identify the ground cover using only pixel features, superpixel features, or a combination of both. To alleviate the above problem, this paper proposes a superpixel-pixel-subpixel multilevel network (SPSM), which compensates for the insufficiencies of the different levels and decrease the information loss. For arbitrary irregular regions, superpixel features are simulated as network nodes using a graph convolutional network (GCN) to capture the spatial texture structure of the HSI, which improves the smooth classification results of local regions while facilitating the identification of different vegetation features in the region. Additionally, the global attention module (GAM) learns local regular regions based on pixel-level features to extend the global interactive representation capability and reduce information loss. To overcome spectral mixing and enhance material discrimination, the normalized attention module (NAM) is used to suppress unimportant subpixel information and identify and remove irrelevant details, thereby improving the identification of critical features that differentiate different materials. Finally, the three features are fused to build a SPSM classification framework to improve robustness to overfitting, reduce computational complexity, and facilitate target recognition. Experimental results on four HSI datasets demonstrate that the method is more capable of recognizing detailed features than other advanced comparison methods.

The classification of linked 3–manifolds in 6–space

Let $M_1$ and $M_2$ be closed connected orientable $3$-manifolds. We classify the sets of smooth and piecewise linear isotopy classes of embeddings $M_1\sqcup M_2\rightarrow S^6$.

Anomalies in (2+1)D Fermionic Topological Phases and (3+1)D Path Integral State Sums for Fermionic SPTs

Given a (2+1)D fermionic topological order and a symmetry fractionalization class for a global symmetry group $G$, we show how to construct a (3+1)D topologically invariant path integral for a fermionic $G$ symmetry-protected topological state ($G$-FSPT) in terms of an exact combinatorial state sum. This provides a general way to compute anomalies in (2+1)D fermionic symmetry-enriched topological states of matter. Equivalently, our construction provides an exact (3+1)D combinatorial state sum for a path integral of any FSPT that admits a symmetry-preserving gapped boundary, including the (3+1)D topological insulators and superconductors in class AII, AIII, DIII, and CII that arise in the free fermion classification. Our construction uses the fermionic topological order (characterized by a super-modular tensor category) and symmetry fractionalization data to define a (3+1)D path integral for a bosonic theory that hosts a non-trivial emergent fermionic particle, and then condenses the fermion by summing over closed 3-form $\mathbb{Z}_2$ background gauge fields. This procedure involves a number of non-trivial higher-form anomalies associated with Fermi statistics and fractional quantum numbers that need to be appropriately canceled off with a Grassmann integral that depends on a generalized spin structure. We show how our construction reproduces the $\mathbb{Z}_{16}$ anomaly indicator for time-reversal symmetric topological superconductors with ${\bf T}^2 = (-1)^F$. Mathematically, with standard technical assumptions, this implies that our construction gives a combinatorial state sum on a triangulated 4-manifold that can distinguish all $\mathbb{Z}_{16}$ $\mathrm{Pin}^+$ smooth bordism classes. As such, it contains the topological information encoded in the eta invariant of the pin$^+$ Dirac operator, thus giving an example of a state sum TQFT that can distinguish exotic smooth structure.

A Note on Equivalence Classes of SO0(p,q)-actions on Sp+q-1 whose Restricted SO(p)×SO(q)-action is the Standard Action

In [5] we gave examples of triples (g,h1,h2) of smooth functions to construct smooth SU(p,q)-actions on the projective space Pℂp+q-1 (p,q≥3) whose restricted S(U(p)×U(q))-action is the standard action. By using these examples of triples (g,h1,h2), we shall construct smooth (resp. analytic) SO0(p,q)-actions on Sp+q-1 (p,q≥3) whose restricted SO(p)×SO(q)-action is the standard action. Consequently, we shall show that for each positive integer m there exist uncountably infinite smooth conjugacy classes of smooth SO0(p,q)-actions on Sp+q-1 (p,q≥3) with m closed and m+1 open orbits whose restricted SO(p)×SO(q)-action is the standard action. On the other hand, we shall show that there exist exactly countably many analytic conjugacy classes of analytic SO0(p,q)-actions on Sp+q-1 (p,q≥3) with one closed and two open orbits whose restricted SO(p)×SO(q)-action is the standard action.

A class of high-order improved fast weighted essentially non-oscillatory schemes for achieving optimal order at any critical points

The weighted essentially non-oscillatory (WENO) scheme is one of the most popular shock-capturing schemes, and constructing a more efficient and higher-order WENO scheme has always been an intention of optimization design. In the general WENO reconstruction framework, the smoothness indicator plays an important role in identifying whether the sub-stencils are in discontinuous or smooth regions. However, the classical smoothness indicator is the most expensive one in the whole reconstruction algorithm, and its computational complexity increases sharply with the improvement of the accuracy order. Therefore, a class of efficient and superior WENO schemes called improved fast WENO (IFWENO) are proposed based on the fast WENO (FWENO). To improve efficiency, the smoothness indicator of the IFWENO scheme is simplified from the traditional version, and the nonlinear weight calculation method is modified. The parameter ε is carefully designed to obtain the superior property that the accuracy of the spatial derivatives will not degrade at any order critical point in smooth regions. The reason for the instability occurring in the high-order FWENO is revealed, and the parameter p is likewise specifically selected to improve robustness at discontinuities. The excellent multi-scale resolution of the proposed IFWENO scheme is proven by theoretical analyses and numerical experiments. Through several typical examples, the consistently high accuracy and efficiency of the designed scheme in both smooth and discontinuous regions are verified.

Nonparametric Bayesian inference for reversible multidimensional diffusions

We study nonparametric Bayesian models for reversible multidimensional diffusions with periodic drift. For continuous observation paths, reversibility is exploited to prove a general posterior contraction rate theorem for the drift gradient vector field under approximation-theoretic conditions on the induced prior for the invariant measure. The general theorem is applied to Gaussian priors and p-exponential priors, which are shown to converge to the truth at the optimal nonparametric rate over Sobolev smoothness classes in any dimension.

Braid loops with infinite monodromy on the Legendrian contact DGA

We present the first examples of elements in the fundamental group of the space of Legendrian links in ( S 3 , ξ st ) $(\mathbb {S}^3,\xi _{\text{st}})$ whose action on the Legendrian contact DGA is of infinite order. This allows us to construct the first families of Legendrian links that can be shown to admit infinitely many Lagrangian fillings by Floer-theoretic techniques. These new families include the first-known Legendrian links with infinitely many fillings that are not rainbow closures of positive braids, and the smallest Legendrian link with infinitely many fillings known to date. We discuss how to use our examples to construct other links with infinitely many fillings, and in particular give the first Floer-theoretic proof that Legendrian ( n , m ) $(n,m)$ torus links have infinitely many Lagrangian fillings if n ⩾ 3 , m ⩾ 6 $n\geqslant 3,m\geqslant 6$ or ( n , m ) = ( 4 , 4 ) , ( 4 , 5 ) $(n,m)=(4,4),(4,5)$ . In addition, for any given higher genus, we construct a Weinstein 4-manifold homotopic to the 2-sphere whose wrapped Fukaya category can distinguish infinitely many exact closed Lagrangian surfaces of that genus in the same smooth isotopy class, but distinct Hamiltonian isotopy classes. A key technical ingredient behind our results is a new combinatorial formula for decomposable cobordism maps between Legendrian contact DGAs with integer (group ring) coefficients.