International Conference On Machine Learning Research Articles

Achieving fairness with a simple ridge penalty

In this paper, we present a general framework for estimating regression models subject to a user-defined level of fairness. We enforce fairness as a model selection step in which we choose the value of a ridge penalty to control the effect of sensitive attributes. We then estimate the parameters of the model conditional on the chosen penalty value. Our proposal is mathematically simple, with a solution that is partly in closed form and produces estimates of the regression coefficients that are intuitive to interpret as a function of the level of fairness. Furthermore, it is easily extended to generalised linear models, kernelised regression models and other penalties, and it can accommodate multiple definitions of fairness. We compare our approach with the regression model from Komiyama et al. (in: Proceedings of machine learning research. 35th international conference on machine learning (ICML), vol 80, pp 2737–2746, 2018), which implements a provably optimal linear regression model and with the fair models from Zafar et al. (J Mach Learn Res 20:1–42, 2019). We evaluate these approaches empirically on six different data sets, and we find that our proposal provides better goodness of fit and better predictive accuracy for the same level of fairness. In addition, we highlight a source of bias in the original experimental evaluation in Komiyama et al. (in: Proceedings of machine learning research. 35th international conference on machine learning (ICML), vol 80, pp 2737–2746, 2018).

Align, then memorise: the dynamics of learning with feedback alignment* *This is a lightly edited version of an article that appeared previously in the proceedings of the 38th International Conference on Machine Learning (ICML) in 2021 [

Direct feedback alignment (DFA) is emerging as an efficient and biologically plausible alternative to backpropagation for training deep neural networks. Despite relying on random feedback weights for the backward pass, DFA successfully trains state-of-the-art models such as transformers. On the other hand, it notoriously fails to train convolutional networks. An understanding of the inner workings of DFA to explain these diverging results remains elusive. Here, we propose a theory of feedback alignment algorithms. We first show that learning in shallow networks proceeds in two steps: an alignment phase, where the model adapts its weights to align the approximate gradient with the true gradient of the loss function, is followed by a memorisation phase, where the model focuses on fitting the data. This two-step process has a degeneracy breaking effect: out of all the low-loss solutions in the landscape, a network trained with DFA naturally converges to the solution which maximises gradient alignment. We also identify a key quantity underlying alignment in deep linear networks: the conditioning of the alignment matrices. The latter enables a detailed understanding of the impact of data structure on alignment, and suggests a simple explanation for the well-known failure of DFA to train convolutional neural networks. Numerical experiments on MNIST and CIFAR10 clearly demonstrate degeneracy breaking in deep non-linear networks and show that the align-then-memorize process occurs sequentially from the bottom layers of the network to the top.

МЕЖДУНАРОДНАЯ НАУЧНАЯ КОНФЕРЕНЦИЯ «ИСКУССТВО И МАШИННАЯ ЦИВИЛИЗАЦИЯ»

The text reviews the Arts and Machine Civilization International Scientific Conference. The conference took place on March 30—April 2, 2021, and was organized by the State Institute for Art Studies, GITR Film and Television School, and the Saint Petersburg State University. SIAS has been hosting conferences on contemporary culture, screen art and television for 17 years. This year, for the first time in the history of such forums, the researchers were tasked with analyzing the new things that machines have brought to the arts and, in general, to human life. The conference took its special place among the forums held over the past year in Russia and abroad in the following areas: artificial intelligence (Artificial Intelligence Journey, Moscow,Russia); machine learning (International Conferenceon Machine Learning, Vienna, Austria; 3rd International Conference on Machine Learning and Machine Intelligence, Hangzhou, China); civilization of knowledge (Civilization of Knowledge: Russian Realities, Moscow, Russia), etc. The novelty of the conference lies in the unification of the seemingly incompatible phenomena: art and machine civilization. As is commonly known, art was traditionally opposed to technology as something alien, sometimes hostile, although the both were born in human mind and created by human hands. Until now, the expression “machine civilization” in art has been used mainly in the genre of fantasy and with an emphasis on its negative connotations. The purpose of the conference was to comprehend the artistic practices in the era of machine civilization, get acquainted with current hypotheses, publish new facts and discuss modern terminologies (law of spontaneity, law of semantic uncertainty, algorithmic apophenia, post-opera, artificial life and new vitality). Along with the study of new challenges, old issues were raised, which became in demand in the machine civilization: originals and copies of artworks, the boundaries of conventionality and overcoming distrust in new media, narratives and poetics in serious and entertaining screen genres. The conference reports were divided into six blocks: Theoretical Models, ScreenArts—Cinema, Fine Arts, Music, PC Games, and Digitalization.

Grassmann multimodal implicit feature selection

In pattern recognition field, objects are usually represented by multiple features (multimodal features). For example, to characterize a natural scene image, it is essential to extract a set of visual features representing its color, texture, and shape information. However, integrating multimodal features for recognition is challenging because: (1) each feature has its specific statistical property and physical interpretation, (2) huge number of features may result in the curse of dimensionality (When data dimension is high, the distances between pairwise objects in the feature space become increasingly similar due to the central limit theory. This phenomenon influences negatively to the recognition performance), and (3) some features may be unavailable. To solve these problems, a new multimodal feature selection algorithm, termed Grassmann manifold feature selection (GMFS), is proposed. In particular, by defining a clustering criterion, the multimodal features are transformed into a matrix, and further treated as a point on the Grassmann manifold in Hamm and Lee (Grassmann discriminant analysis: a unifying view on subspace-based learning. In: Proceedings of the 25th international conference on machine learning (ICML), pp. 376---383, Helsinki, Finland [2008]). To deal with the unavailable features, L2-Hausdorff distance, a metric between different-sized matrices, is computed and the kernel is obtained accordingly. Based on the kernel, we propose supervised/unsupervised feature selection algorithms to achieve a physically meaningful embedding of the multimodal features. Experimental results on eight data sets validate the effectiveness the proposed approach.

Learning a Reachable Space Map in a Gaze Centered Reference Frame

Learning a Reachable Space Map in a Gaze Centered Reference Frame

Introduction to the special issue on mining and learning with graphs

Driven by application areas ranging from biology to the World Wide Web, research in Data Mining and Machine Learning is nowadays increasingly focusing on the analysis of structured data. Of particular interest are data that consist of interrelated parts or data that are characterized by collections of interrelated objects, linked together into complex graphs and structures. Dealing with such interrelated data is one of the major research challenges that we are facing. The aim of this special issue is to bring together papers from different sub-disciplines within Machine Learning and Data Mining that focus on the analysis of structured data. This special issue was first conceived as a forum for publishing extended versions of papers presented at the annual workshop on Machine Learning and Graphs (MLG) 2008 colocated with the International Conference on Machine Learning (ICML) 2008 in Helsinki, Finland. Later, the scope was extended and we solicited relevant papers from researchers in all areas of machine learning and data mining, working on mining and learning with graphs

The Reinforcement Learning Competitions

This article reports on the reinforcement learning competitions, which have been held annually since 2006. In these events, researchers from around the world developed reinforcement learning agents to compete in domains of various complexity and difficulty. We focus on the 2008 competition, which employed fundamentally redesigned evaluation frameworks that aimed systematically to encourage the submission of robust learning methods. We describe the unique challenges of empirical evaluation in reinforcement learning and briefly review the history of the previous competitions and the evaluation frameworks they employed. We describe the novel frameworks developed for the 2008 competition as well as the software infrastructure on which they rely. Furthermore, we describe the six competition domains, present selected competition results, and discuss the implications of these results. Finally, we summarize the 2009 competition, which used the same evaluation framework but different events, and outline ideas for the future of the competition.

The Ninth International Conference on Machine Learning

The Ninth International Conference on Machine Learning was held in Aberdeen, Scotland, from 1-3 July 1992, with 198 participants in attendance. The conference covered a broad range of topics drawn from the general area of machine learning, including concept-learning algorithms, clustering, speedup learning, formal analysis of learning systems, neural networks, genetic algorithms, and applications of machine learning. This article briefly touches on six selected talks that were of exceptional interest.