Performance In Learning Tasks Research Articles

A fine-grained self-adapting prompt learning approach for few-shot learning with pre-trained language models

Pre-trained language models have demonstrated remarkable performance in few-shot learning through the emergence of “prompt-based learning” methods, where the performance of these tasks highly rely on the quality of prompts. Existing prompt learning methods typically customize a single prompt to each few-shot learning task and all the examples in the task share the universal prompt. However, a fine-grained prompt design can enhance the performance of few-shot learning task by leveraging more diverse information hidden in the set of examples. In light of this motivation, this paper introduce an example-specific prompt learning method to embody fine-grained self-adapting prompts for few-shot learning with pre-trained models. Specifically, we introduce the concept of the “weak consistency assumption”, to trade-off the task-specific consistent and example-specific diversity. Based on this assumption, a novel method called Self-adapting Continuous Prompt Learning (SP-learning) to learn example-specific prompts is proposed. It employs a cross-attention prompt generator that considers the characteristics of input samples and utilizes a diversity calibration technique to adjust the prompt generator accordingly. By personalizing prompts for each example, SP-learning aims to improve few-shot learning performance. We perform a systematic evaluation on 10 public benchmark tasks and our method outperforms 8 of those tasks. Our research sheds light on the importance of personalized prompts and opens up new possibilities for improving few-shot learning tasks.

Comparing hybrid models for recognising objects in thermal images at nighttime

This research aims to revolutionize urban object recognition by developing cloud-based Python programs using intelligent algorithms. Unlike current models that focus on colour enhancement in nighttime thermal images, this work addresses the critical challenge of accurate object detection in urban landscapes. The proposed method incorporates a binary generative adversarial network (GAN) generator that can switch bidirectionally between daytime colour (DC) and nighttime infrared (NTIR) images. memory-based visual image memory (MVAM), system extracts important descriptive information from urban landscape images, reducing problems related to small sample sizes. This discussion presents a comprehensive improvement and evaluation of a deep learning image classification pipeline using Google Colab, demonstrating advanced image processing. Using TensorFlow, Keres and scikit image libraries combined with advanced algorithms such as DenseNet121 and MobileNetV2 presents a clear approach. We created a Bidirectional GAN + MVAM for object recognition in this work. Our method performed well, with an accuracy of 81.43%, precision of 51.16, recall of 50.11, and F-score of 46.37. The systematic presentation of the code presents a careful strategy to ensure optimal performance, stability, and efficiency of deep learning and image processing tasks.

From land to water: “Sunken” T-maze for associated learning in cichlid fish

The study introduced and evaluated learning paradigms for Maylandia callainos cichlids using a modified version of the rodent T-maze, filled with tank water (the "sunken" modification). Both male and female fish underwent training in two distinct conditioning paradigms. Firstly, simple operant conditioning involved placing a food reward in either the right or left compartment. Cichlids demonstrated the ability to purposefully find the bait within 6 days of training, with a persistent place preference lasting up to 6 days. Additionally, the learning dynamics varied with sex: female cichlids exhibited reduction in latency to visit the target compartment and consume the bait, along with a decrease in the number of errors 3 and 4 days earlier than males, respectively. Secondly, visually-cued operant conditioning was conducted, with a food reward exclusively placed in the yellow compartment, randomly positioned on the left or right side of the maze during each training session. Visual learning persisted for 10 days until reaction time improvement plateaued. Color preference disappeared after 4 consecutive check-ups, with no sex-related interference. For further validation of visually-cued operant conditioning paradigm, drugs MK-801 (dizocilpine) and caffeine, known to affect performance in learning tasks, were administered intraperitoneally. Chronic MK-801 (0.17 mg/kg) impaired maze learning, resulting in no color preference development. Conversely, caffeine administration enhanced test performance, increasing precision in fish. This developed paradigm offers a viable approach for studying learning and memory and presents an effective alternative to rodent-based drug screening tools, exhibiting good face and predictive validity.

GDF11 improves hippocampal neurogenesis and cognitive abilities in diabetic mice by reducing neural inflammation

BackgroundThe cognitive decline associated with type 2 diabetes (T2D) is often attributed to compromised hippocampal neurogenesis and exacerbated neural inflammation. This study investigates the therapeutic potential of growth differentiation factor 11 (GDF11) in reversing these neurodegenerative processes in diabetic mice. ResultWe utilized a murine model of T2D and examined the effects of GDF11 on learning, memory, neurogenesis, and neuroinflammatory markers. Our results indicate that diabetic mice exhibit significant deficits in cognitive function, mirrored by reduced hippocampal neurogenesis and increased neuroinflammation. Chronic administration of GDF11 was observed to significantly enhance cognitive abilities, as evidenced by improved performance in learning and memory tasks. Concurrently, GDF11 treatment restored neural activity and promoted the regeneration of new neurons within the hippocampus. Inflammatory profiling revealed a reduction in neuroinflammatory markers, which was further supported by reduced microglia numbers. To delineate the role of neuroinflammation, we pharmacologically depleted microglia, leading to a restoration of neurogenesis and cognitive functions in diabetic mice. ConclusionThese findings endorse the hypothesis that GDF11 exerts its beneficial effects by modulating neuroinflammatory pathways. Consequently, GDF11 represents a promising intervention to ameliorate diabetes-induced cognitive impairments and neural degeneration through its anti-inflammatory properties.

Semi-supervised PolSAR Image Change Detection using Similarity Matching

Abstract. The lack of precisely labeled data limits the development of supervised polarimetric synthetic aperture radar (PolSAR) image change detection. Therefore, semi-supervised deep learning methods have recently demonstrated their significant capability for PolSAR image change detection. Similarity Matching (SimMatch) improves the performance of semi-supervised learning tasks across different benchmark datasets and different settings. Introducing SimMatch into the field of PolSAR image change detection can improve the performance of semi-supervised PolSAR image change detection under limited labeled data conditions. Usually, semi-supervision solves the problem of insufficient labeled data by generating pseudo-labels. However, when the pseudo-label method is simply applied, the model will fit on the confident but wrong pseudo-labels, resulting in poor performance. SimMatch offers a solution by requiring the strongly augmented view to share the same semantic similarity (i.e. label prediction) and instance characteristics (i.e. similarity between instances) with a weak augmented view for more intrinsic feature matching. Besides, by using a labeled memory buffer, the two similarities can be isomorphically transformed with each other by introducing the aggregating and unfolding techniques. Therefore, the semantic and instance pseudo-labels can be mutually propagated, and then, the detection performance of the PolSAR image change detection is improved. Experimental results on real PolSAR datasets demonstrated that SimMatch is an effective semi-supervised PolSAR change detection method and its performance surpasses some well-known change detection methods. Compared to the fully-supervised algorithm CWNN, the semi-supervised SimMatch algorithm can improve accuracy by up to 14.4%.

Limitations to optimal search in naturalistic active learning.

Optimality in active learning is under intense debate in numerous disciplines. We introduce a new empirical paradigm for studying naturalistic active learning, as well as new computational tools for jointly modeling algorithmic and rational theories of information search. Participants in our task can ask questions and learn about hundreds of everyday items but must retrieve queried items from memory. To maximize information gain, participants need to retrieve sequences of dissimilar items. In eight experiments (N = 795), we find that participants are unable to do this. Instead, associative memory mechanisms lead to the successive retrieval of similar items, an established memory effect known as semantic congruence. The extent of semantic congruence (and thus suboptimality in question asking) is unaffected by task instructions and incentives, though participants can identify efficient query sequences when given a choice between query sequences. Overall, our results indicate that participants can distinguish between optimal and suboptimal search if explicitly asked to do so, but have difficulty implementing optimal search from memory. We conclude that associative memory processes may place critical restrictions on people's ability to ask good questions in naturalistic active learning tasks. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

과학 영재 특성 구인과 과제 수행 단계의 연관성

In this study, it was assumed that it was necessary to separately investigate the characteristics of gifted students in science at each stage in the process of performing a task, rather than a single characteristic. For this purpose, the affective characteristics of gifted children, which are traditionally emphasized, such as motivation, passion, and task concentration, were assumed to be characteristics that are uniquely emphasized at each stage of task performance, such as beginning, progress, and completion. Afterwards, each category was matched to the task performance stage, and then the relationships within and between categories were investigated. These results showed that the affective characteristics of the gifted students in science had correlations within categories. However, the correlation of questions between categories was relatively high between the motivation category and the passion category, but the correlation between the motivation and task concentration categories was low. This means that each characteristic in the task concentration category is more unique than those in other categories. This pattern appeared similarly in all grades of middle school, with relatively more distinct results in the third grade. The correlation between the characteristics of each stage investigated in this study indicates that it continues meaningfully while students perform learning tasks regardless of grade in middle school.

SSGAN: A Semantic Similarity-Based GAN for Small-Sample Image Augmentation

Image sample augmentation refers to strategies for increasing sample size by modifying current data or synthesizing new data based on existing data. This technique is of vital significance in enhancing the performance of downstream learning tasks in widespread small-sample scenarios. In recent years, GAN-based image augmentation methods have gained significant attention and research focus. They have achieved remarkable generation results on large-scale datasets. However, their performance tends to be unsatisfactory when applied to datasets with limited samples. Therefore, this paper proposes a semantic similarity-based small-sample image augmentation method named SSGAN. Firstly, a relatively shallow pyramid-structured GAN-based backbone network was designed, aiming to enhance the model’s feature extraction capabilities to adapt to small sample sizes. Secondly, a feature selection module based on high-dimensional semantics was designed to optimize the loss function, thereby improving the model’s learning capacity. Lastly, extensive comparative experiments and comprehensive ablation experiments were carried out on the “Flower” and “Animal” datasets. The results indicate that the proposed method outperforms other classical GANs methods in well-established evaluation metrics such as FID and IS, with improvements of 18.6 and 1.4, respectively. The dataset augmented by SSGAN significantly enhances the performance of the classifier, achieving a 2.2% accuracy improvement compared to the best-known method. Furthermore, SSGAN demonstrates excellent generalization and robustness.

Dual-channel deep graph convolutional neural networks.

The dual-channel graph convolutional neural networks based on hybrid features jointly model the different features of networks, so that the features can learn each other and improve the performance of various subsequent machine learning tasks. However, current dual-channel graph convolutional neural networks are limited by the number of convolution layers, which hinders the performance improvement of the models. Graph convolutional neural networks superimpose multi-layer graph convolution operations, which would occur in smoothing phenomena, resulting in performance decreasing as the increasing number of graph convolutional layers. Inspired by the success of residual connections on convolutional neural networks, this paper applies residual connections to dual-channel graph convolutional neural networks, and increases the depth of dual-channel graph convolutional neural networks. Thus, a dual-channel deep graph convolutional neural network (D2GCN) is proposed, which can effectively avoid over-smoothing and improve model performance. D2GCN is verified on CiteSeer, DBLP, and SDBLP datasets, the results show that D2GCN performs better than the comparison algorithms used in node classification tasks.

Time-Aware Knowledge Representations of Dynamic Objects with Multidimensional Persistence

Learning time-evolving objects such as multivariate time series and dynamic networks requires the development of novel knowledge representation mechanisms and neural network architectures, which allow for capturing implicit time-dependent information contained in the data. Such information is typically not directly observed but plays a key role in the learning task performance. In turn, lack of time dimension in knowledge encoding mechanisms for time-dependent data leads to frequent model updates, poor learning performance, and, as a result, subpar decision-making. Here we propose a new approach to a time-aware knowledge representation mechanism that notably focuses on implicit time-dependent topological information along multiple geometric dimensions. In particular, we propose a new approach, named Temporal MultiPersistence (TMP), which produces multidimensional topological fingerprints of the data by using the existing single parameter topological summaries. The main idea behind TMP is to merge the two newest directions in topological representation learning, that is, multi-persistence which simultaneously describes data shape evolution along multiple key parameters, and zigzag persistence to enable us to extract the most salient data shape information over time. We derive theoretical guarantees of TMP vectorizations and show its utility, in application to forecasting on benchmark traffic flow, Ethereum blockchain, and electrocardiogram datasets, demonstrating the competitive performance, especially, in scenarios of limited data records. In addition, our TMP method improves the computational efficiency of the state-of-the-art multipersistence summaries up to 59.5 times.

University students’ academic vocabulary development through mobile-assisted learning: Exploring the impacts on receptive and productive knowledge

This study explored the effectiveness of mobile-assisted vocabulary learning through digital flashcards on receptive and productive aspects of academic vocabulary knowledge among Iranian EFL university students. In a quasi-experimental design, 86 participants were divided into three groups: a digital flashcard group (DFs), a traditional paper flashcard (PFs) group, and a control group using word lists, to assess the impact of these methods on vocabulary acquisition over five weeks. The findings revealed that students utilizing DFs exhibited significant improvements in both receptive and productive vocabulary knowledge compared to those using PFs and the control method. Notably, the increase in receptive vocabulary was more substantial than in productive vocabulary, highlighting the differential effects of DFs on various aspects of vocabulary learning. This finding underscores the need for targeted strategies to enhance productive aspects of academic vocabulary specifically. The study supports the integration of DFs into English for Academic Purposes (EAP) programs to leverage their potential in boosting vocabulary acquisition effectively. However, the lesser gains in productive vocabulary suggest the necessity for complementary instructional methods, which focus on more active vocabulary learning tasks. Based on these findings, the study argues that mobile-assisted vocabulary learning should be considered a practical strategy for supporting academic vocabulary development among university students.

A Procedural Constructive Learning Mechanism with Deep Reinforcement Learning for Cognitive Agents

Recent advancements in AI and deep learning have created a growing demand for artificial agents capable of performing tasks within increasingly complex environments. To address the challenges associated with continuous learning constraints and knowledge capacity in this context, cognitive architectures inspired by human cognition have gained significance. This study contributes to existing research by introducing a cognitive-attentional system employing a constructive neural network-based learning approach for continuous acquisition of procedural knowledge. We replace an incremental tabular Reinforcement Learning algorithm with a constructive neural network deep reinforcement learning mechanism for continuous sensorimotor knowledge acquisition, thereby enhancing the overall learning capacity. The primary emphasis of this modification centers on optimizing memory utilization and reducing training time. Our study presents a learning strategy that amalgamates deep reinforcement learning with procedural learning, mirroring the incremental learning process observed in human sensorimotor development. This approach is embedded within the CONAIM cognitive-attentional architecture, leveraging the cognitive tools of CST. The proposed learning mechanism allows the model to dynamically create and modify elements in its procedural memory, facilitating the reuse of previously acquired functions and procedures. Additionally, it equips the model with the capability to combine learned elements to effectively adapt to complex scenarios. A constructive neural network was employed, initiating with an initial hidden layer comprising one neuron. However, it possesses the capacity to adapt its internal architecture in response to its performance in procedural and sensorimotor learning tasks, inserting new hidden layers or neurons. Experimentation conducted through simulations involving a humanoid robot demonstrates the successful resolution of tasks that were previously unsolved through incremental knowledge acquisition. Throughout the training phase, the constructive agent achieved a minimum of 40% greater rewards and executed 8% more actions when compared to other agents. In the subsequent testing phase, the constructive agent exhibited a 15% increase in the number of actions performed in contrast to its counterparts.

MFRLMO: Model-free reinforcement learning for multi-objective optimization of apache spark

Hyperparameter optimization (HO) is a must to figure out to what extent can a specific configuration of hyperparameters contribute to the performance of a machine learning task. The hardware and MLlib library of Apache Spark have the potential to improve big data processing performance when a tuning operation is combined with the exploitation of hyperparameters. To the best of our knowledge, the most of existing studies employ a black-box approach that results in misleading results due to ignoring the interior dynamics of big data processing. They suffer from one or more drawbacks including high computational cost, large search space, and sensitivity to the dimension of multi-objective functions. To address the issues above, this work proposes a new model-free reinforcement learning for multi-objective optimization of Apache Spark, thereby leveraging reinforcement learning (RL) agents to uncover the internal dynamics of Apache Spark in HO. To bridge the gap between multi-objective optimization and interior constraints of Apache Spark, our method runs a lot of iterations to update each cell of the RL grid. The proposed model-free learning mechanism achieves a tradeoff between three objective functions comprising time, memory, and accuracy. To this end, optimal values of the hyperparameters are obtained via an ensemble technique that analyzes the individual results yielded by each objective function. The results of the experiments show that the number of cores has not a direct effect on $speedup$. Further, although grid size has an impact on the time passed between two adjoining iterations, it is negligible in the computational burden. Dispersion and risk values of model-free RL differ when the size of the data is small. On average, MFRLMO produced $speedup$ that is 37% better than those of the competitors. Last, our approach is very competitive in terms of converging to a high accuracy when optimizing Convolutional Neural networks (CNN).

The Active Place Avoidance (APA) Test, an Effective, Versatile and Repeatable Spatial Learning Task for Mice.

Hippocampus-dependent spatial learning in rodents has been tested using a variety of methods. These include the Morris water maze (MWM), Y-maze, and novel object location (NOL) tasks. More recently, the active place avoidance (APA) task has been developed as an alternative to these more traditional approaches. In the APA task, mice must use spatial cues placed around a rotating arena to avoid a stationary shock zone. Due to the multiple parameters that can be adjusted, the APA task has been demonstrated to be a very versatile approach. It lends itself to being used longitudinally and repeatedly for the same cohort of mice. Here, we provide a detailed protocol to successfully conduct the APA task. We also highlight alternative APA approaches that can be used to examine different components of spatial learning. We describe the data collection and analysis processes. Critical steps during the APA task are discussed to increase the likelihood of successfully conducting the test. The APA task has several advantages over more traditional spatial navigation tests. It is appropriate to use with aged mice or those with disease phenotypes such as Alzheimer's disease. The complexity of the task can be easily altered, allowing a wide range of mouse strains to be tested. Further, the APA task is suitable for testing animals that have undergone surgery or experimental interventions that may have affected motor or neural function, such as stroke or traumatic brain injury.

Generative model for learning quantum ensemble with optimal transport loss

Generative modeling is an unsupervised machine learning framework, that exhibits strong performance in various machine learning tasks. Recently, we find several quantum versions of generative model, some of which are even proven to have quantum advantage. However, those methods are not directly applicable to construct a generative model for learning a set of quantum states, i.e., ensemble. In this paper, we propose a quantum generative model that can learn quantum ensemble, in an unsupervised machine learning framework. The key idea is to introduce a new loss function calculated based on optimal transport loss, which have been widely used in classical machine learning due to its good properties; e.g., no need to ensure the common support of two ensembles. We then give in-depth analysis on this measure, such as the scaling property of the approximation error. We also demonstrate the generative modeling with the application to quantum anomaly detection problem, that cannot be handled via existing methods. The proposed model paves the way for a wide application such as the health check of quantum devices and efficient initialization of quantum computation.

Multi-task twin support vector machine with Universum data

Multi-task learning (MTL) has emerged as a promising topic of machine learning in recent years, aiming to enhance the performance of numerous related learning tasks by exploiting beneficial information. Traditionally, during the training phase, existing multi-task learning models focused solely on the data related to the target task. In our approach, Universum data, which does not belong to any class in the classification problem but belongs to the same domain as the target data, is incorporated into classifier training as prior knowledge. This study looks at the challenge of multi-task learning using Universum data to employ non-target task data, which leads to better performance. It proposes a multi-task twin support vector machine with Universum data (UMTSVM) and provides two approaches to its solution. The first approach takes into account the dual formulation of UMTSVM and tries to solve a quadratic programming problem. The second approach formulates a least-squares version of UMTSVM and refers to it as LS-UMTSVM to further increase the generalization performance. The solution of the two primal problems in LS-UMTSVM is simplified to solving just two systems of linear equations, resulting in an incredibly simple and quick approach. Numerical experiments on several popular multi-task data sets and medical data sets demonstrate the efficiency of the proposed methods.

A search for effective reinforcers in appetitive conditioning for adult zebrafish: Ecologically relevant unconditioned stimuli

Learning and memory related brain disorders represent a large unmet medical need. Laboratory studies with animals may model brain disorders and facilitate uncovering their mechanisms. The zebrafish has been proposed for such studies. However, numerous factors that influence performance in learning tasks have yet to be understood in zebrafish. One such factor is what motivates zebrafish. Here we introduce a novel reinforcer, an ecologically relevant unconditioned stimulus (US). We placed a photograph of gravel underneath quarter of the bottom of an experimental tank on one side and also positioned artificial plants there, the “natural” US. First, we showed that this stimulus was preferred by zebrafish. Next, we investigated whether this stimulus could serve as US for associative learning. We marked the walls of the tank on the side where the US was presented with red paper, the conditioned stimulus (CS+) we found neutral before, and we also marked the walls on the other side of the tank where no US was placed with blue paper (CS-). In addition to fish receiving this “paired” training, we also ran unpaired training with another group of zebrafish, in which the fish saw the US associated with blue and red in a random manner. After having trained the fish in this manner, we tested the performance of the paired and unpaired group of zebrafish in a memory probe trial during which no US was present, and only the CSs (blue and red walls) were shown. We found the paired group of zebrafish to show significant preference for the CS+, as they spent more time and swam closer to the red side compared to the unpaired group and compared to chance. We conclude that ecologically relevant stimuli can serve as efficient US in appetitive conditioning of zebrafish.

Deep Learning-Based Assessment Model for Real-Time Identification of Visual Learners Using Raw EEG.

Automatic identification of visual learning style in real time using raw electroencephalogram (EEG) is challenging. In this work, inspired by the powerful abilities of deep learning techniques, deep learning-based models are proposed to learn high-level feature representation for EEG visual learning identification. Existing computer-aided systems that use electroencephalograms and machine learning can reasonably assess learning styles. Despite their potential, offline processing is often necessary to eliminate artifacts and extract features, making these methods unsuitable for real-time applications. The dataset was chosen with 34 healthy subjects to measure their EEG signals during resting states (eyes open and eyes closed) and while performing learning tasks. The subjects displayed no prior knowledge of the animated educational content presented in video format. The paper presents an analysis of EEG signals measured during a resting state with closed eyes using three deep learning techniques: Long-term, short-term memory (LSTM), Long-term, short-term memory-convolutional neural network (LSTM-CNN), and Long-term, short-term memory-Fully convolutional neural network (LSTM-FCNN). The chosen techniques were based on their suitability for real-time applications with varying data lengths and the need for less computational time. The optimization of hypertuning parameters has enabled the identification of visual learners through the implementation of three techniques. LSTM-CNN technique has the highest average accuracy of 94%, a sensitivity of 80%, a specificity of 92%, and an F1 score of 94% when identifying the visual learning style of the student out of all three techniques. This research has shown that the most effective method is the deep learning-based LSTM-CNN technique, which accurately identifies a student's visual learning style.

Improved Test Input Prioritization Using Verification Monitors with False Prediction Cluster Centroids

Deep learning (DL) systems have been remarkably successful in various applications, but they could have critical misbehaviors. To identify the weakness of a trained model and overcome it with new data collection(s), one needs to figure out the corner cases of a trained model. Constructing new datasets to retrain a DL model requires extra budget and time. Test input prioritization (TIP) techniques have been proposed to identify corner cases more effectively. The state-of-the-art TIP approach adopts a monitoring method to TIP and prioritizes based on Gini impurity; one estimates the similarity between a DL prediction probability and uniform distribution. This letter proposes a new TIP method that uses a distance between false prediction cluster (FPC) centroids in a training set and a test instance in the last-layer feature space to prioritize error-inducing instances among an unlabeled test set. We refer to the proposed method as DeepFPC. Our numerical experiments show that the proposed DeepFPC method achieves significantly improved TIP performance in several image classification and active learning tasks.

Edge Learning for Large-Scale Internet of Things With Task-Oriented Efficient Communication

In the Internet of Things (IoT) networks, edge learning for data-driven tasks provides intelligent applications and services. As the network size becomes large, different users may generate distinct datasets. Thus, to suit multiple edge learning tasks for large-scale IoT networks, this paper performs efficient communication under the task-oriented principle by using the collaborative design of wireless resource allocation and edge learning error prediction. In particular, we start with multi-user scheduling to alleviate co-channel interference in dense networks. Then, we perform optimal power allocation in parallel for different learning tasks. Thanks to the high parallelization of the designed algorithm, extensive experimental results corroborate that the multi-user scheduling and task-oriented power allocation improve the performance of distinct edge learning tasks efficiently compared with the state-of-the-art benchmark algorithms.