Multi-Attribute Vision Transformers are Efficient and Robust Learners

The presence of noise is common in any real-world dataset and may adversely affect the accuracy, construction time and complexity of the classifiers in this context. Traditionally, many algorithms have incorporated mechanisms to deal with noisy problems and reduce noise's effects on performance; they are called robust learners. The C4.5 crisp algorithm is a well-known example of this group of methods. On the other hand, models built by Fuzzy Rule Based Classification Systems are widely recognized for their robustness to imperfect data, but also for their interpretability. The aim of this contribution is to analyze the good behavior and robustness of Fuzzy Rule Based Classification Systems when noise is present in the examples' class labels, especially versus robust learners. In order to accomplish this study, a large number of datasets are created by introducing different levels of noise into the class labels in the training sets. We compare a Fuzzy Rule Based Classification System, the Fuzzy Unordered Rule Induction Algorithm, with respect to the C4.5 classic robust learner which is considered tolerant to noise. From the results obtained it is possible to observe that Fuzzy Rule Based Classification Systems have a good tolerance, in comparison to the C4.5 algorithm, to class noise.

The experiences of an instructor and teaching assistant who employed online communication strategies in a graduate seminar are examined in this paper. This paper expands on the findings reported in an earlier article on virtual learning communities founded on social constructivist pedagogy (Schwier & Balbar, 2002). We examine how the instructors constructed and refined structured discussions of content with synchronous and asynchronous communication at the graduate level. The instructors offer several observations and principles that are organized into categories that illustrate the source, message, channel and receiver in the communication system. The critical reflections of the instructors are compared with data from interviews with students about learning experienced in the online discussions (Dykes, 2003). Findings include the realization that instructors may fundamentally misinterpret or overlook important elements of communication, but that students are robust learners who can transcend the limitations of the medium and the instructor if given the authority in a social constructivist learning environment.

This work presents a novel procedure to detect and correct noise in a continuous dependent variable. The presence of noise in a dataset represents a significant challenge to data mining algorithms, as incorrect values in both the independent and dependent variables can severely corrupt the results of even robust learners. The problem of noise is especially severe when it is located in the dependent variable. In the worst case, severe noise in one of the independent variables can be handled by eliminating that attribute from the dataset, provided that the practitioner knows that noise is present. In the setting of supervised learning, the dependent variable is the most critical attribute in the dataset and therefore cannot be eliminated even if significant noise is present. Noise handling procedures in relation to the dependent variable are therefore absolutely critical to the success of a supervised learning initiative. In contrast to a binary dependent variable or class, noise in a continuous dependent variable presents many additional difficulties. Our procedure to detect and correct noise in a continuous dependent variable uses Bayesian multiple imputation, which was initially developed to combat the problem of missing data. Our case study considers a real-world software measurement dataset called CCCS, which has a numeric dependent variable with inherent noise. The results of our experiments show very encouraging results and clearly demonstrate the utility of our procedure

Forecasting the strength of micro/nano silica in cementitious matrix by machine learning approaches

In this paper, we test some of the most commonly used classifiers to identify which ones are the most robust to changing environments. The environment may change over time due to some contextual or definitional changes. The environment may change with location. It would be surprising if the performance of common classifiers did not degrade with these changes. The question, we address here, is whether or not some types of classifier are inherently more immune than others to these effects. In this study, we simulate the changing of environment by reducing the influence on the class of the most significant attributes. Based on our analysis, K-Nearest Neighbor and Artificial Neural Networks are the most robust learners, ensemble algorithms are somewhat robust, whereas Naive Bayes, Logistic Regression and particularly Decision Trees are the most affected.KeywordsClassifier evaluationchanging environmentsclassifier robustness

Variable behavior and repeated learning in two mouse strains: Developmental and genetic contributions

Detailed landslide inventories are required for multiple purposes including disaster damage assessments, susceptibility mapping for spatial planning, and disaster risk reduction. Active learning is an artificial intelligence strategy that can achieve good performances in landslide mapping by training a machine-learning model with a reduced number of landslide/non-landslide observations, which can save time and effort in labeling training instances. Nevertheless, active-learning models are unstable at the beginning of sample selection due to the limited initial knowledge of landslide distribution. Transfer learning can help make the learner robust by transferring a landslide model trained on an existing landslide inventory from a different, but geographically similar source area, to the unseen target area. In order to adjust a transferred machine-learning model to the possibly unique environmental characteristics of the unseen area, we proposed a new framework called Unsupervised Active-Transfer Learning (UATL). This framework used a weight function to combine the landslide model transferred from the source area, with a model trained on a small, but increasing number of landslide/non-landslide observations from the target area to efficiently build a more robust learner. We examined two methods, adaptive UATL and regular UATL, which differed in the way they assign weights to the combined learners. We evaluated our proposed new methods by comparing them with three benchmark methods (active learning only, model transfer only, and the model trained in the unseen area itself) by means of the partial area under the receiver operating characteristic (ROC) curve (AUROC) as the evaluation criterion. The results showed that the new methods, and especially adaptive UATL, can achieve good predictive performances. With only about 235 training instances from the target area, the partial AUROC obtained from adaptive UATL was only 2% lower than that obtained from the model trained in the target area itself, and consistently outperformed the other two benchmarks. Overall, we suggest that the framework proposed can be applied to the natural hazards management workflow for assisting in emergency response, especially in data-scarce regions (e.g., mountainous areas and developing countries).

Transformers, composed of multiple self-attention layers, hold strong promises toward a generic learning primitive applicable to different data modalities, including the recent breakthroughs in computer vision achieving state-of-the-art (SOTA) standard accuracy. What remains largely unexplored is their robustness evaluation and attribution. In this work, we study the robustness of the Vision Transformer (ViT) (Dosovitskiy et al. 2021) against common corruptions and perturbations, distribution shifts, and natural adversarial examples. We use six different diverse ImageNet datasets concerning robust classification to conduct a comprehensive performance comparison of ViT(Dosovitskiy et al. 2021) models and SOTA convolutional neural networks (CNNs), Big-Transfer (Kolesnikov et al. 2020). Through a series of six systematically designed experiments, we then present analyses that provide both quantitative andqualitative indications to explain why ViTs are indeed more robust learners. For example, with fewer parameters and similar dataset and pre-training combinations, ViT gives a top-1accuracy of 28.10% on ImageNet-A which is 4.3x higher than a comparable variant of BiT. Our analyses on image masking, Fourier spectrum sensitivity, and spread on discrete cosine energy spectrum reveal intriguing properties of ViT attributing to improved robustness. Code for reproducing our experiments is available at https://git.io/J3VO0.

In target tracking with an active sonar, a major problem is the association of new echoes with track data previously collected. This paper reports work on a method for associating a sonar target's echoes with an existing track by assigning a degree of association by means of gates, the sizes of which are determined by a genetic algorithm using a simulator. The resultant gates encapsulate the uncertainties and the imperfections imposed by the system and the environment, and also are able to take into account unexpected changes in the target's dynamic behaviour. The results presented here illustrate the effective role of the GA as a robust learner in this application.

Artificial Intelligence (AI) can support diagnostic workflows in oncology by aiding diagnosis and providing biomarkers directly from routine pathology slides. However, AI applications are vulnerable to adversarial attacks. Hence, it is essential to quantify and mitigate this risk before widespread clinical use. Here, we show that convolutional neural networks (CNNs) are highly susceptible to white- and black-box adversarial attacks in clinically relevant weakly-supervised classification tasks. Adversarially robust training and dual batch normalization (DBN) are possible mitigation strategies but require precise knowledge of the type of attack used in the inference. We demonstrate that vision transformers (ViTs) perform equally well compared to CNNs at baseline, but are orders of magnitude more robust to white- and black-box attacks. At a mechanistic level, we show that this is associated with a more robust latent representation of clinically relevant categories in ViTs compared to CNNs. Our results are in line with previous theoretical studies and provide empirical evidence that ViTs are robust learners in computational pathology. This implies that large-scale rollout of AI models in computational pathology should rely on ViTs rather than CNN-based classifiers to provide inherent protection against perturbation of the input data, especially adversarial attacks.

In the past decade the field of neural interface systems has enjoyed an increase in attention from the scientific community and the general public, in part due to the enormous potential that such systems have to increase the quality of life for paralyzed patients. While significant progress has been made, serious challenges remain to be addressed from both biological and engineering perspectives. A key issue is how to optimize the decoding of neural information, such that neural signals are correctly mapped to effectors that interact with the outside world - like robotic hands and limbs or the patient's own muscles. Here we present some recent progress on tackling this problem by applying the latest developments in machine learning. Neural data was collected from macaque monkeys performing a real-time hand grasp decoding task. Signals were recorded via chronically implanted electrodes in the anterior intraparietal cortex (AIP) and ventral premotor cortex (F5), brain areas that are known to be involved in the transformation of visual signals into hand grasping instructions. We present a comparative study of different classical machine learning methods with an application of decoding of hand postures, as well as a new approach for more robust decoding. Results suggests that combining data-driven algorithmic approaches with well-known parametric methods could lead to better performing and more robust learners, which may have direct implications for future clinical devices.

In this paper, a method of polarimetric SAR image classification based on polarimetric decomposition and AdaBoost algorithm is proposed. The proposed method improves classification accuracy and speed. AdaBoost algorithm, as a robust learner with high accuracy, can fully utilize the polarimetric features to achieve image classification. In simulated tests, the proposed method is observed to produce improved classification accuracy and speed, compared with H /α classification algorithm.

Prior findings suggest that investors learn with experience. We study the complementary channel of learning from data, particularly the effects of making additional predictive signals available to investors. We analyse a panel of systematic traders' investment outcomes, sourced from a FinTech platform that organises trading contests under highly-controlled conditions that allow us to identify learning effects. Investor outcomes improve with experience, and this is also apparent when counterfactually assessing their trading decisions on historical data, suggesting that they make use of historical data to attain their objectives. Importantly, when additional predictive variables are added to the common part of investors' information sets, the individual-level dispersions of investors' performance outcomes narrow, while their relative performance outcomes improve at higher experience levels. To explain why this widening of their common dataset benefits experienced investors, we model an investor as choosing a portfolio by learning from historical data while also taking model uncertainty into account. The robust learner therefore ignores predictive signals with historical predictive contributions below a subjective model uncertainty threshold; we conjecture this threshold varies with experience.

Using Individualized Treatment Effects to Assess Treatment Effect Heterogeneity.

Estimating how crop yield responds to site-specific nitrogen (N) fertilization is essential for maximizing yield potential under variable field conditions. However, classical Machine Learning (ML) approaches applied to observational farm data primarily focus on yield prediction and often fail to recover causal N response due to confounding arising from non-random fertilizer application. In this study, we develop and evaluate a Causal Machine Learning (CML) framework to estimate heterogeneous N treatment effects under real commercial rice-farming conditions in the Axios River Plain, Greece. The proposed approach combines Double Machine Learning (DML) with remote sensing, soil, climatic, and management data to adjust for confounding and identify causal relationships between N inputs, Leaf Nitrogen Concentration (LNC), and yield. A doubly robust (DR) learner is used to estimate yield sensitivity to N at key agronomic thresholds, while a Causal Forest model leverages LNC to assess crop physiological N status. Results demonstrate that the CML-based framework outperforms conventional XGBoost predictive models in identifying field plots that are responsive to additional N. By integrating causal effect estimation with plant-status information, the proposed decision support system identifies zones where yield gains can be achieved through targeted N increases while avoiding overfertilization in non-responsive areas.