Normative Weight Research Articles

The underlying mechanisms of alignment in error backpropagation through arbitrary weights

Understanding the mechanisms by which plasticity in millions of synapses in the brain is orchestrated to achieve behavioral and cognitive goals is a fundamental question in neuroscience. In this regard, insights from learning methods in artificial neural networks (ANNs) and in particular supervised learning using backpropagation (BP) seem inspiring. However, the implementation of BP requires exact matching between forward and backward weights, which is unrealistic given the known connectivity pattern in the brain (known as the “weight transport problem”). Notably, it has been shown that under certain conditions, error BackPropagation Through Arbitrary Weights (BP-TAW) can lead to a partial alignment between forward and backward weights (weight alignment or WA). This learning algorithm, which is also known as feedback alignment (FA), can result in surprisingly good degrees of accuracy in simple classification tasks. However, the underlying mechanisms and mathematical basis of WA are not thoroughly understood. In this work, we demonstrate the mathematical basis of WA and answer the question of why and in what conditions WA occurs. We show that the occurrence of WA in ANNs is induced by statistical properties of the output and error signals of neurons, such as autocorrelation and cross-correlation, and can happen even in the absence of learning or reduction of the loss function. Moreover, we show that WA can be improved significantly by limiting the norm of input weights to neurons and that such a weight normalization (WN) method can improve the classification accuracy of BP-TAW. The findings presented can be used to further improve the performance of BP-TAW and open new ways for exploring possible learning mechanisms in biological neural networks without exact matching between forward and backward weights.

Representations and generalization in artificial and brain neural networks

Humans and animals excel at generalizing from limited data, a capability yet to be fully replicated in artificial intelligence. This perspective investigates generalization in biological and artificial deep neural networks (DNNs), in both in-distribution and out-of-distribution contexts. We introduce two hypotheses: First, the geometric properties of the neural manifolds associated with discrete cognitive entities, such as objects, words, and concepts, are powerful order parameters. They link the neural substrate to the generalization capabilities and provide a unified methodology bridging gaps between neuroscience, machine learning, and cognitive science. We overview recent progress in studying the geometry of neural manifolds, particularly in visual object recognition, and discuss theories connecting manifold dimension and radius to generalization capacity. Second, we suggest that the theory of learning in wide DNNs, especially in the thermodynamic limit, provides mechanistic insights into the learning processes generating desired neural representational geometries and generalization. This includes the role of weight norm regularization, network architecture, and hyper-parameters. We will explore recent advances in this theory and ongoing challenges. We also discuss the dynamics of learning and its relevance to the issue of representational drift in the brain.

Normative evidence weighting and accumulation in correlated environments.

The brain forms certain deliberative decisions following normative principles related to how sensory observations are weighed and accumulated over time. Previously we showed that these principles can account for how people adapt their decisions to the temporal dynamics of the observations (Glaze et al., 2015). Here we show that this adaptability extends to accounting for correlations in the observations, which can have a dramatic impact on the weight of evidence provided by those observations. We tested online human participants on a novel visual-discrimination task with pairwise-correlated observations. With minimal training, the participants adapted to uncued, trial-by-trial changes in the correlations and produced decisions based on an approximately normative weighting and accumulation of evidence. The results highlight the robustness of our brain's ability to process sensory observations with respect to not just their physical features but also the weight of evidence they provide for a given decision.

Body mass assessment of male soldiers of the Armed Forces of Ukraine

Body weight control of male soldiers of the Armed Forces of Ukraine is not only about ensuring their somatic (physical) health. It is also the choice of proper nutrition, neat appearance, successful passing physical training tests, high work capacity and timely performance of military duties. Moreover, it is also the presence of special criteria – norms (standards), the development of which has unreasonably little attention in the past in the Armed Forces of Ukraine, but in the current conditions they are extremely relevant. Excessive body weight, and especially obesity, reduces a person’s work capacity, complicates intellectual and physical activity, causes disability, premature aging and shortened life expectancy. Body weight standards should be universal enough to take into account as much as possible departmental interests, in particular the military, and to cover individual characteristics of human development, such as height, age, and body constitution. The purpose of the study is to improve body weight assessment system of male servicemen of the Armed Forces of Ukraine with different height, age and body constitution. Methods of the study are as following: theoretical analysis and generalization; extrapolation; method of least squares; regression analysis. Material: according to the authors’ formula of optimal body weight for men of different height, age and different body constitution, the limits of body weight norms for nine age groups were obtained in accordance with the requirements of physical training standards. Results: a table of minimum and maximum body weight for servicemen of different heights depending on the age group was developed. A methodology for assessing the body weight of military personnel, taking into account their body constitution, has been developed. Conclusions. To estimate the body weight of male servicemen of the Armed Forces of Ukraine, a table has been developed within the height range from 1.50 to 2.06 m in nine age groups from 18 to 60 years and older for soldiers with different body constitutions. In the case when body weight of male servicemen of the Armed Forces of Ukraine exceeds the table requirements, it is necessary to obtain the percentage of its deviation from the optimal body weight for men of different height, age and different body constitution, which is calculated according to the formula: Optimal body weight for men = (4.7619 × wrist circumference – 90.4762) × (0/08135×age+21.6916) × L2 / 100+(0.08135 × age + 21.6916) × L2 . If the body weight differs from the optimal body weight for men by no more than ±10%, then it meets the requirements of the norm.

Weight Decay With Tailored Adam on Scale-Invariant Weights for Better Generalization.

Weight decay (WD) is a fundamental and practical regularization technique in improving generalization of current deep learning models. However, it is observed that the WD does not work effectively for an adaptive optimization algorithm (such as Adam), as it works for SGD. Specifically, the solution found by Adam with the WD often generalizes unsatisfactorily. Though efforts have been made to mitigate this issue, the reason for such deficiency is still vague. In this article, we first show that when using the Adam optimizer, the weight norm increases very fast along with the training procedure, which is in contrast to SGD where the weight norm increases relatively slower and tends to converge. The fast increase of weight norm is adverse to WD; in consequence, the Adam optimizer will lose efficacy in finding solution that generalizes well. To resolve this problem, we propose to tailor Adam by introducing a regularization term on the adaptive learning rate, such that it is friendly to WD. Meanwhile, we introduce first moment on the WD to further enhance the regularization effect. We show that the proposed method is able to find solution with small norm and generalizes better than SGD. We test the proposed method on general image classification and fine-grained image classification tasks with different networks. Experimental results on all these cases substantiate the effectiveness of the proposed method in help improving the generalization. Specifically, the proposed method improves the test accuracy of Adam by a large margin and even improves the performance of SGD by 0.84% on CIFAR 10 and 1.03 % on CIFAR 100 with ResNet-50. The code of this article is public available at xxx.

Class-Balanced Regularization for Long-Tailed Recognition

Long-tailed recognition performs poorly on minority classes. The extremely imbalanced distribution of classifier weight norms leads to a decision boundary biased toward majority classes. To address this issue, we propose Class-Balanced Regularization to balance the distribution of classifier weight norms so that the model can make more balanced and reasonable classification decisions. In detail, CBR separately adjusts the regularization factors based on L2 regularization to be correlated with the class sample frequency positively, rather than using a fixed regularization factor. CBR trains balanced classifiers by increasing the L2 norm penalty for majority classes and reducing the penalty for minority classes. Since CBR is mainly used for classification adjustment instead of feature extraction, we adopt a two-stage training algorithm. In the first stage, the network with the traditional empirical risk minimization is trained, and in the second stage, CBR for classifier adjustment is applied. To validate the effectiveness of CBR, we perform extensive experiments on CIFAR10-LT, CIFAR100-LT, and ImageNet-LT datasets. The results demonstrate that CBR significantly improves performance by effectively balancing the distribution of classifier weight norms.

Should gestational weight gain charts exclude individuals with excess postpartum weight retention?

High gestational weight gain is associated with excess postpartum weight retention, yet excess postpartum weight retention is not an exclusion criterion for current gestational weight gain charts. We aimed to assess the impact of excluding individuals with high interpregnancy weight change (a proxy for excess postpartum weight retention) on gestational weight gain distributions. We included individuals with an index birth from 2008 to 2014 and a subsequent birth before 2019, in the population-based Stockholm-Gotland Perinatal Cohort. We estimated gestational weight gain (kg) at 25 and 37 weeks, using weight at first prenatal visit (<14 weeks) as the reference. We calculated high interpregnancy weight change (≥10 kg and ≥5 kg) using the difference between weight at the start of an index and subsequent pregnancy. We compared gestational weight gain distributions and percentiles (stratified by early-pregnancy body mass index) before and after excluding participants with high interpregnancy weight change. Among 55,723 participants, 17% had ≥10 kg and 34% had ≥5 kg interpregnancy weight change. The third, tenth, 50th, 90th and 97th percentiles of gestational weight gain were similar (largely within 1 kg) before versus after excluding participants with high interpregnancy weight change, at both 25 and 37 weeks. For example, among normal weight participants at 37 weeks, the 50th and 97th percentiles were 14 kg and 23 kg including versus 13 kg and 23 kg excluding participants with ≥5 kg interpregnancy weight change. Excluding individuals with excess postpartum weight retention from normative gestational weight gain charts may not meaningfully impact the charts' percentiles.

Territorial Impact Assessment for Coal Sites in Transition

This article describes a part of the POTENTIALS project promoted by the Research Fund for Coal and Steel (RFCS) of the EU, which, in general, has aimed to develop business models for the reuse of former industrial and mining sites in different European regions. The goal of this project focused on the creation of so-called eco-industrial parks to enable sustainable energy production and reduce waste and pollution on coal sites in transition. A key aspect was the development of a suitable territorial impact assessment (TIA), a new and complex policy tool for the assessment of the territorial impacts of EU policies and projects on territorial cohesion. Therefore, the special TEQUILA (Territorial Efficiency, Quality and Identity Layer Assessment) approach is used to describe the TIA for this case and emphasizes its application in assessing ex ante the impacts of the transition from a coal site to an eco-industrial park. It underlines the need for a differentiated understanding of the regional characteristics and potential impacts of transition policies or projects. Furthermore, the process and results of applying the TEQUILA methodology, a multicriteria analysis, in the context of regionalized impact models has shown how important it is to select well-defined, expert-based criteria, but at the same time, to establish a system that is flexible and adaptable to the needs of political decision makers and stakeholders considering the normative weights of the criteria. This has been illustrated by some examples.

Eating Disorders Risk Assessment and Body Esteem among Amateur and Professional Football Players.

The purpose of this study was to assess the risk of eating disorders and attitudes toward one's own body among football players at amateur and professional levels. The study included 90 players from football clubs located in the Upper Silesia Metropolitan Area, participating in the 4th and 5th leagues in Poland. A survey questionnaire was used to conduct the study, which consisted of a metric section, an Eating Attitudes Test, and Body Esteem Scale questionnaires. The players were divided into two groups according to their sports level. Results showed that 24.4% of players were overweight, while 75.6% had a normative body weight. Approximately 16.7% met the criteria indicating susceptibility to an eating disorder. Body Esteem Scale interpretations revealed moderate body appraisal among players. Both amateur and professional athletes showed no significant difference in eating disorder risk, but professionals rated their bodies higher. Social media use, particularly on Twitter and Instagram, is correlated with eating disorders, with longer daily use associated with lower body ratings.

SWRM: Similarity Window Reweighting and Margin for Long-Tailed Recognition

Real-world data usually obeys a long-tailed distribution, where a few classes have higher number of samples compared to the other classes. Recent studies have been proposed to alleviate the extreme data imbalance from different perspectives. In this article, we experimentally find that due to the easily confusing visual features between some head- and tail classes, the cross-entropy model is prone to misclassify tail samples to similar head classes. Therefore, to alleviate the influence of the confusion on model performance and improve the classification of tail classes, we propose a Similarity Window Reweighting and Margin (SWRM) algorithm, where the SWRM consists of Similarity Window Reweighting (SWR) and Similarity Window Margin (SWM) algorithms. For the confusable head- and tail classes, SWR assigns larger weights to tail classes and smaller weights to head classes. Therefore, the model can enlarge the importance of tail classes and effectively improve their classification. Moreover, SWR considers the difference in label frequency and the impact of category similarity simultaneously, so that the weight coefficients are more reasonable and efficacious. SWM generates adaptive margins that are proportional to the ratio of the classifier’s weight norm, thus promoting the learning of tail classifier with small weight norm. Our SWRM effectively eliminates the confusion between head- and tail classes and alleviates the misclassification issues. Extensive experiments on three long-tailed datasets, i.e., CIFAR100-LT, ImageNet-LT, and Places-LT, verify our proposed method’s effectiveness and superiority over comparative methods.

Improving extreme learning machine model using deep learning feature extraction and grey wolf optimizer: Application to image classification

The Extreme Learning Machine (ELM) is a highly efficient model for real-time network retraining due to its fast learning speed, unlike traditional machine learning methods. However, the performance of ELM can be negatively impacted by the random initialization of weights and biases. Moreover, poor input feature quality can further degrade performance, particularly with complex visual data. To overcome these issues, this paper proposes optimizing the input features as well as the initial weights and biases. We combine both Convolutional Neural Network (CNN) and Convolutional AutoEncoder (CAE) extracted features to optimize the quality of the input features. And we use our hybrid Grey Wolf Optimizer-Multi-Verse Optimizer (GWO-MVO) metaheuristic for initializing weights and biases by applying four fitness functions based on: the norm of the output weights, the error rate on the training set, and the error rate on the validation set. Our method is evaluated on image classification tasks using two benchmark datasets: CIFAR-10 and CIFAR-100. Since image quality may vary in real-world applications, we trained and tested our models on the dataset’s original and noisy versions. The results demonstrate that our method provides a robust and efficient alternative for image classification tasks, offering improved accuracy and reduced overfitting.

A Sparse Model for Electricity Price Forecasting with LASSO-GRBFN and Brain Storm Optimization

This paper proposes a new method for predicting electricity prices. The method is based on the Generalized Radial Basis Function Network (GRBFN), an extension of the RBFN of Artificial Neural Network (ANN) used for one-step-ahead forecasting. The GRBFN is superior to the RBFN because it evaluates the Gaussian function parameters during learning. Conventional ANN methods consider overfitting with the weight decay method, based on the L2 norm of weights between neurons, but there is still room for improvement. Therefore, the paper proposes using the Least Absolute Shrinkage and Selection Operator (LASSO) to improve the model’s performance. Furthermore, the paper presents the Brain Storm Optimization (BSO) of evolutionary computation to evaluate the cost function with the term of the L1 norm. Conventional methods with the gradient are not suitable for the L1 norm. The proposed method is successfully applied to actual data of ISO New England in the USA.

The Norming Sets of L(2d*(1, w)2)

Let 𝑛 ∈ ℕ. An element (x1, … , x𝑛) ∈ En is called a norming point of T ∈ (nE) if ‖x1‖ = ⋯ = ‖xn‖ = 1 and |T (x1, … , xn)| = ‖T‖, where (nE) denotes the space of all continuous n-linear forms on E. For T ∈ (nE), we defineNorm(T) = {(x1, … , xn) ∈ En ∶ (x1, … , xn) is a norming point of T}.Norm(T) is called the norming set of T. We classify Norm(T) for every T ∈ (2𝑑∗(1, w)2), where 𝑑∗(1, w)2 = ℝ2 with the octagonal norm of weight 0 &lt; w &lt; 1 endowed with .

Visual Servoing Control of Robotics With A Neural Network Estimator Based on Spectral Adaptive Law

In recent years, with the development of machine vision and other relative techniques, visual servoing control of robotics has been wildly applied. A complex calibration process is usually required to get the accurate parameters of the camera and the robot, so that getting the projection relationship between changes of images and movement of robot joints usually takes much effort. In order to solve this problem, a ReLU activating neural network (NN) estimator is proposed to estimate the compound Jacobian matrix of the system in this paper. The weight of the NN is updated online by a project algorithm with a novel spectral adaptive law which can effectively improve the generalization ability of the NN and the robustness of the system. By constructing a new Lyapunov function with the spectral norm of weight of NN, the stability of the proposed adaptive algorithm and the controller can be proved. Simulations and experimental results validate the effectiveness of the proposed controller.

A Unified Multi-Class Feature Selection Framework for Microarray Data.

In feature selection research, simultaneous multi-class feature selection technologies are popular because they simultaneously select informative features for all classes. Recursive feature elimination (RFE) methods are state-of-the-art binary feature selection algorithms. However, extending existing RFE algorithms to multi-class tasks may increase the computational cost and lead to performance degradation. With this motivation, we introduce a unified multi-class feature selection (UFS) framework for randomization-based neural networks to address these challenges. First, we propose a new multi-class feature ranking criterion using the output weights of neural networks. The heuristic underlying this criterion is that "the importance of a feature should be related to the magnitude of the output weights of a neural network". Subsequently, the UFS framework utilizes the original features to construct a training model based on a randomization-based neural network, ranks these features by the criterion of the norm of the output weights, and recursively removes a feature with the lowest ranking score. Extensive experiments on 15 real-world datasets suggest that our proposed framework outperforms state-of-the-art algorithms. The code of UFS is available at https://github.com/SVMrelated/UFS.git.

Motivations for and interests in health-enhancing exercise of adolescent girls with different body weight

The article discusses the issue of development of motives and interests in adolescent children with different body weights (normal and underweight) when involving them in programs of physical activity based on the use of health fitness means. The aim of the study was to assess the motives and interests of adolescent girls with different body weights for participation in health-enhancing physical activities in order to increase the level of their involvement in regular physical activity classes. Methods: analysis of theoretical material, comparative analysis (comparison and comparison), sociological methods (questionnaires), and methods of mathematical statistics. The study involved 138 girls aged 12-13 years. Results. Priority types of physical activity of 12-13-year-old girls during their leisure time were: walks, dance fitness, roller skating, mental fitness, functional training, and aqua fitness. The choice of types of health-enhancing recreational physical activity did not differ significantly among normal weight and underweight adolescent girls aged 12-13 years. When choosing health-enhancing physical activities, the main objectives were: aesthetic motives (increasing physical attractiveness and body weight management), achievement of prestigious motives (desire to increase prestige, to be accepted in society), recreational motives (active leisure time, emotionally attractive organization of free time, satisfaction from activity). For most adolescent girls, health reasons (improvement of health status and functional capabilities of the body) were not a priority. The difference in answers between normal weight and underweight girls was found to be significant. Conclusion. In the rating of motives of underweight girls, the first places were occupied by aesthetic, prestigious, recreational, and personal motives, whereas in girls with a normative body weight, psychological, aesthetic, and health-recreational motives prevailed.

The Art of Not Being Sexed Quite So Much: A Feminist Reading of Roland Barthes

This article offers an underexplored resistance strategy to gender norms, based on a feminist and queer reading of the work of French thinker Roland Barthes. Building on Barthes’s peculiar conception of what he calls “the Neutral” and revisiting his work in light of feminist and queer scholarship on sexual (in)difference, my main goal is to reshape our understanding of what it means to be gender neutral. In opposition to classical conceptions of neutrality associated with passivity, indifference, and blandness, I show that Barthes’s Neutral can be conceived as an active gesture, which alters common systems of meanings and power, including the gender binary. But this conception of the Neutral, I argue, neither refers to a call for an androgynous, queer, or nonbinary gender experience. It does not target gender embodiments but gender regulations—that is, the social enforcement of gender categories (by way, for instance, of administrative forms, single-sex bathrooms, or normalizing attitudes toward others). A gender-neutral arrangement, therefore, refers to an ethical, administrative, social, or spatial relation in which subjects are not assigned based on gender categories. This practice of gender suspension, I show, has two main transformative effects. First, it opens a space of freedom and livability for gender-nonconforming subjects. Second, it contributes to lessening the significance of sexual difference in social life and therefore alleviating its symbolic and normative weight for everyone.

The Normative Value of Human Rights Treaty Bodies’ Interpretations at the International Court of Justice

Although the International Court of Justice (ICJ) is not a human rights court, it has recently addressed human rights issues, which has prompted interaction with the opinions of human rights mechanisms. This article will analyze the normative value of human rights treaty monitoring bodies’ interpretations recognized by the ICJ in three cases of its jurisprudence: the Wall Advisory Opinion, the case of Ahmadou Sadio Diallo, and the case of Qatar v. the United Arab Emirates. This analysis indicates that the ICJ has ascribed great normative weight to these interpretations, but it has been reluctant to adopt their views without conducting its own interpretative assessment of the norms. In its most recent case, the ICJ took an approach that completely departs from the interpretation adopted by the relevant treaty body. This article argues that such a position must be considered in light of the growing criticism that treaty bodies are facing about the quality of their reasonings and command of general international law. Accordingly, treaty bodies could enhance their legitimacy by learning from the ICJ’s approach to treaty interpretation, and in addition, could take advantage of the positive aspects of their non-binding character to develop meta-juridical discussions.

Current insights into the possible role of laboratory monitoring of effectiveness and safety of direct oral anticoagulants

The article is devoted to the discussion of modern approaches to the use of laboratory methods to improve the tactics of using direct oral anticoagulants (DOACs) therapy. Clinical situations in which it may be reasonable to use data on the blood concentration of DOACs are given, including very old age, a marked deviation from the norm of body weight or impaired renal function. Data on the role of measuring the blood level of DOACs are considered in cases of the development of diseases or complications in which information is required on the preservation of the anticoagulant effect of DOACs, for example, in the development of severe bleeding or the need for urgent surgical intervention. The advantages and limitations of modern laboratory methods for assessing the blood concentration of DOACs are discussed. It is emphasized that one of the main advantages of using DOACs is the absence of the need to monitor laboratory parameters in most patients. Data from pharmacological studies are presented that may be useful in explaining the mechanisms that determine the higher safety of some DOACs compared to others. Promising methods for assessing the blood level of DOACs, as well as the possibility of using less specific reagents for assessing the concentration of DOACs, are considered. The possibility of using less specific, but more accessible methods for assessing the blood concentration of factor Xa inhibitors, in particular, a reagent for assessing the level of the antifactor, which is used to determine the blood level of heparin, is being considered. The opinions of experts on the role of assessing the blood level of DOACs and the possibility of tactics for selecting doses of DOACs based on laboratory analysis data are given.

Better Generalized Few-Shot Learning Even without Base Data

This paper introduces and studies zero-base generalized few-shot learning (zero-base GFSL), which is an extreme yet practical version of few-shot learning problem. Motivated by the cases where base data is not available due to privacy or ethical issues, the goal of zero-base GFSL is to newly incorporate the knowledge of few samples of novel classes into a pretrained model without any samples of base classes. According to our analysis, we discover the fact that both mean and variance of the weight distribution of novel classes are not properly established, compared to those of base classes. The existing GFSL methods attempt to make the weight norms balanced, which we find help only the variance part, but discard the importance of mean of weights particularly for novel classes, leading to the limited performance in the GFSL problem even with base data. In this paper, we overcome this limitation by proposing a simple yet effective normalization method that can effectively control both mean and variance of the weight distribution of novel classes without using any base samples and thereby achieve a satisfactory performance on both novel and base classes. Our experimental results somewhat surprisingly show that the proposed zero-base GFSL method that does not utilize any base samples even outperforms the existing GFSL methods that make the best use of base data. Our implementation is available at: https://github.com/bigdata-inha/Zero-Base-GFSL.