Curriculum Learning Strategy Research Articles

Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformer.

Since labeled samples are typically scarce in real-world scenarios, self-supervised representation learning in time series is critical. Existing approaches mainly employ the contrastive learning framework, which automatically learns to understand similar and dissimilar data pairs. However, they are constrained by the request for cumbersome sampling policies and prior knowledge of constructing pairs. Also, few works have focused on effectively modeling temporal-spectral correlations to improve the capacity of representations. In this article, we propose the cross reconstruction transformer (CRT) to solve the aforementioned issues. CRT achieves time series representation learning through a cross-domain dropping-reconstruction task. Specifically, we obtain the frequency domain of the time series via the fast Fourier transform (FFT) and randomly drop certain patches in both time and frequency domains. Dropping is employed to maximally preserve the global context while masking leads to the distribution shift. Then a Transformer architecture is utilized to adequately discover the cross-domain correlations between temporal and spectral information through reconstructing data in both domains, which is called Dropped Temporal-Spectral Modeling. To discriminate the representations in global latent space, we propose instance discrimination constraint (IDC) to reduce the mutual information between different time series samples and sharpen the decision boundaries. Additionally, a specified curriculum learning (CL) strategy is employed to improve the robustness during the pretraining phase, which progressively increases the dropping ratio in the training process. We conduct extensive experiments to evaluate the effectiveness of the proposed method on multiple real-world datasets. Results show that CRT consistently achieves the best performance over existing methods by 2%-9%. The code is publicly available at https://github.com/BobZwr/Cross-Reconstruction-Transformer.

Robust speech command recognition in challenging industrial environments

Speech is among the main forms of communication between humans and robots in industrial settings, being the most natural way for a human worker to issue commands. However, the presence of pervasive and loud environmental noise poses significant challenges to the adoption of Speech-Command Recognition systems onboard manufacturing robots; indeed, they are expected to perform in real time on hardware with limited computational capabilities and also to be robust and accurate in such complex environments. In this paper, we propose an innovative system based on an End-to-End architecture with a Conformer backbone. Our system is specifically designed to achieve high accuracy in noisy industrial environments and to guarantee a minimal computational burden to meet stringent real-time requirements while running on computing devices that are embedded in robots. In order to increase the generalization capability of the system, the training procedure is driven by a Curriculum Learning strategy combined with dynamic data augmentation techniques, that progressively increase the complexity of input samples by increasing the noise during the training phase. We have conducted extensive experimentation to assess the effectiveness of our system, using a dataset composed of more than 50,000 samples, of which about 2,000 have been acquired during the daily operations of a Stellantis Italian factory. The results confirm the suitability of the proposed approach to be adopted in a real industrial environment; indeed, it is able to achieve, on both English and Italian commands, an accuracy higher than 90%, maintaining a compact model size (the network is 1.81 MB) and running in real-time on an industrial embedded device (namely 41ms over an NVIDIA Xavier NX).

Joint Imbalance Adaptation for Radiology Report Generation.

Radiology report generation, translating radiological images into precise and clinically relevant description, may face the data imbalance challenge - medical tokens appear less frequently than regular tokens; and normal entries are significantly more than abnormal ones. However, very few studies consider the imbalance issues, not even with conjugate imbalance factors. In this study, we propose a Joint Imbalance Adaptation (JIMA) model to promote task robustness by leveraging token and label imbalance. JIMA predicts entity distributions from images and generates reports based on these distributions and image features. We employ a hard-to-easy learning strategy that mitigates overfitting to frequent labels and tokens, thereby encouraging the model to focus more on rare labels and clinical tokens. JIMA shows notable improvements (16.75% - 50.50% on average) across evaluation metrics on IU X-ray and MIMIC-CXR datasets. Our ablation analysis proves that JIMA's enhanced handling of infrequent tokens and abnormal labels counts the major contribution. Human evaluation and case study experiments further validate that JIMA can generate more clinically accurate reports. Data imbalance (e.g., infrequent tokens and abnormal labels) leads to the underperformance of radiology report generation. Our curriculum learning strategy successfully reduce data imbalance impacts by reducing overfitting on frequent patterns and underfitting on infrequent patterns. While data imbalance remains challenging, our approach opens new directions for the generation task.

Enhancing Temporal Knowledge Graph Representation with Curriculum Learning

Temporal knowledge graph representation approaches encounter significant challenges in handling the complex dynamic relations among entities, relations, and time. These challenges include the high difficulty of training and poor generalization performance, particularly with large datasets. To address these issues, this paper introduces curriculum learning strategies from machine learning, aiming to improve learning efficiency through effective curriculum planning. The proposed framework constructs a high-dimensional filtering model based on graph-based high-order receptive fields and employs a scoring model that uses a curriculum temperature strategy to evaluate the difficulty of temporal knowledge graph data quadruples at each stage. By progressively expanding the receptive field and dynamically adjusting the difficulty of learning samples, the model can better understand and capture multi-level information within the graph structure, thereby improving its generalization capabilities. Additionally, a temperature factor is introduced during model training to optimize parameter gradients, alongside a gradually increasing training strategy to reduce training difficulty. Experiments on the benchmark datasets ICEWS14 and ICEWS05-15 demonstrate that this framework not only significantly enhances model performance on these datasets but also substantially reduces training convergence time.

An innovative Doctor of Nursing Practice scholarly project curriculum model: Strategies for supporting learners at various entry levels

The Doctor of Nursing Practice (DNP) Program curriculum must address the challenge of developing Competency Based Education (CBE) curriculum for DNP Scholarly Projects tailored to meet variations in nursing learner preparation and practice, from novice bachelor's entry level and master's advanced nurse practice expert entry level pathways. The recent Future of Nursing 2020–2030: Charting a Path to Achieve Health Equity report by the National Academy of Medicine advocates that competency-based education (CBE) approaches in nursing schools should focus upon the advanced practice nursing population within collaborative academic practice partnerships. This article describes an innovative DNP Scholarly Project Curriculum model at an academic health science center that integrates academic partnerships and CBE strategies that have been developed for second degree Bachelor's direct entry to nursing (direct entry), post BS in nursing to DNP (BS to DNP), Post Master's to DNP (PM DNP) entry levels of preparation. The DNP Scholarly Project Curriculum utilizes an innovative, scaffolded approach to fostering CBE for the assessment of learning activities at the doctoral level. The DNP Scholarly Project Curriculum includes domains of quality improvement, critical appraisal of the literature, evidence-based practice models, implementation practice models, policy analysis, and process/outcome evaluations. Preliminary alumni and stakeholder feedback suggests that these projects are not only impacting practice but improving learner competency in quality improvement science. Implementation of this model has led to enhanced CBE-driven curriculum teaching and learning strategies, and sustainable collaborative academic partnerships.

Link prediction for hypothesis generation: an active curriculum learning infused temporal graph-based approach

Over the last few years Literature-based Discovery (LBD) has regained popularity as a means to enhance the scientific research process. The resurgent interest has spurred the development of supervised and semi-supervised machine learning models aimed at making previously implicit connections between scientific concepts/entities within often extensive bodies of literature explicit—i.e., suggesting novel scientific hypotheses. In doing so, understanding the temporally evolving interactions between these entities can provide valuable information for predicting the future development of entity relationships. However, existing methods often underutilize the latent information embedded in the temporal aspects of the interaction data. Motivated by applications in the food domain—where we aim to connect nutritional information with health-related benefits—we address the hypothesis-generation problem using a temporal graph-based approach. Given that hypothesis generation involves predicting future (i.e., still to be discovered) entity connections, in our view the ability to capture the dynamic evolution of connections over time is pivotal for a robust model. To address this, we introduce THiGER, a novel batch contrastive temporal node-pair embedding method. THiGER excels in providing a more expressive node-pair encoding by effectively harnessing node-pair relationships. Furthermore, we present THiGER-A, an incremental training approach that incorporates an active curriculum learning strategy to mitigate label bias arising from unobserved connections. By progressively training on increasingly challenging and high-utility samples, our approach significantly enhances the performance of the embedding model. Empirical validation of our proposed method demonstrates its effectiveness on established temporal-graph benchmark datasets, as well as on real-world datasets within the food domain.

Dual Contrastive Learning Network for Graph Clustering.

Graph representation is an important part of graph clustering. Recently, contrastive learning, which maximizes the mutual information between augmented graph views that share the same semantics, has become a popular and powerful paradigm for graph representation. However, in the process of patch contrasting, existing literature tends to learn all features into similar variables, i.e., representation collapse, leading to less discriminative graph representations. To tackle this problem, we propose a novel self-supervised learning method called dual contrastive learning network (DCLN), which aims to reduce the redundant information of learned latent variables in a dual manner. Specifically, the dual curriculum contrastive module (DCCM) is proposed, which approximates the node similarity matrix and feature similarity matrix to a high-order adjacency matrix and an identity matrix, respectively. By doing this, the informative information in high-order neighbors could be well collected and preserved while the irrelevant redundant features among representations could be eliminated, hence improving the discriminative capacity of the graph representation. Moreover, to alleviate the problem of sample imbalance during the contrastive process, we design a curriculum learning strategy, which enables the network to simultaneously learn reliable information from two levels. Extensive experiments on six benchmark datasets have demonstrated the effectiveness and superiority of the proposed algorithm compared with state-of-the-art methods.

Comparison and analysis of new curriculum criteria for end-to-end ASR

Traditionally, teaching a human and a Machine Learning (ML) model is quite different, but organized and structured learning has the ability to enable faster and better understanding of the underlying concepts. For example, when humans learn to speak, they first learn how to utter basic phones and then slowly move towards more complex structures such as words and sentences. Motivated by this observation, researchers have started to adapt this approach for training ML models. Since the main concept, the gradual increase in difficulty, resembles the notion of the curriculum in education, the methodology became known as Curriculum Learning (CL). In this work, we design and test new CL approaches to train Automatic Speech Recognition systems, specifically focusing on the so-called end-to-end models. These models consist of a single, large-scale neural network that performs the recognition task, in contrast to the traditional way of having several specialized components focusing on different subtasks (e.g., acoustic and language modeling). We demonstrate that end-to-end models can achieve better performances if they are provided with an organized training set consisting of examples that exhibit an increasing level of difficulty. To impose structure on the training set and to define the notion of an easy example, we explored multiple solutions that use either external, static scoring methods or incorporate feedback from the model itself. In addition, we examined the effect of pacing functions that control how much data is presented to the network during each training epoch. Our proposed curriculum learning strategies were tested on the task of speech recognition on two data sets, one containing spontaneous Finnish speech where volunteers were asked to speak about a given topic, and one containing planned English speech. Empirical results showed that a good curriculum strategy can yield performance improvements and speed-up convergence. After a given number of epochs, our best strategy achieved a 5.6% and 3.4% decrease in terms of test set word error rate for the Finnish and English data sets, respectively.

Dual-stage feedback network for lightweight color image compression artifact reduction

Lossy image coding techniques usually result in various undesirable compression artifacts. Recently, deep convolutional neural networks have seen encouraging advances in compression artifact reduction. However, most of them focus on the restoration of the luma channel without considering the chroma components. Besides, most deep convolutional neural networks are hard to deploy in practical applications because of their high model complexity. In this article, we propose a dual-stage feedback network (DSFN) for lightweight color image compression artifact reduction. Specifically, we propose a novel curriculum learning strategy to drive a DSFN to reduce color image compression artifacts in a luma-to-RGB manner. In the first stage, the DSFN is dedicated to reconstructing the luma channel, whose high-level features containing rich structural information are then rerouted to the second stage by a feedback connection to guide the RGB image restoration. Furthermore, we present a novel enhanced feedback block for efficient high-level feature extraction, in which an adaptive iterative self-refinement module is carefully designed to refine the low-level features progressively, and an enhanced separable convolution is advanced to exploit multiscale image information fully. Extensive experiments show the notable advantage of our DSFN over several state-of-the-art methods in both quantitative indices and visual effects with lower model complexity.

FISTNet: FusIon of STyle-path generative Networks for facial style transfer

With the surge in emerging technologies such as Metaverse, spatial computing, and generative AI, the application of facial style transfer has gained much interest from researchers and startups enthusiasts alike. StyleGAN methods have paved the way for transfer-learning strategies that could reduce the dependency on the vast data available for the training process. However, StyleGAN methods tend to need to be more balanced, resulting in the introduction of artifacts in the facial images. Studies such as DualStyleGAN proposed multipath networks but required the networks to be trained for a specific style rather than simultaneously generating a fusion of facial styles. In this paper, we propose a Fusion of STyles (FIST) network for facial images that leverages pretrained multipath style transfer networks to eliminate the problem associated with the lack of enormous data volume in the training phase and the fusion of multiple styles at the output. We leverage pretrained styleGAN networks with an external style pass that uses a residual modulation block instead of a transform coding block. The method also preserves facial structure, identity, and details via the gated mapping unit introduced in this study. The aforementioned components enable us to train the network with minimal data while generating high-quality stylized images, opening up new possibilities for facial style transfer in emerging technologies. Our training process adapts curriculum learning strategy to perform efficient, flexible style, and model fusion in the generative space. We perform extensive experiments to show the superiority of the proposed FISTNet compared to existing state-of-the-art methods.

Interpretable multi-task neural network modeling and particle swarm optimization of process parameters in laser welding

The process parameters directly affect the quality and performance of laser welding (LW) by shaping the molten pool appearance. However, with the increase in the dimensions of process parameters and the number of samples, the relationship between process parameters and molten pool often exhibits high levels of uncertainty and nonlinearity. Machine learning (ML) techniques have been used for complex LW systems modeling, but they may be limited and unable to fully capture the complex relationships when faced with complex systems. Additionally, traditional ML techniques may lack interpretability within the model, making it difficult to understand the input features and their impact on the output. To address these issues, a method that combines an interpretable multi-task neural network (MTNN) and particle swarm optimization (PSO) is proposed. Firstly, an LW experimental platform is constructed, and the orthogonal design is conducted for data collection. In the data preparation stage, the curriculum learning strategy is employed to reorganize the raw experimental data. Subsequently, the MTNN is constructed to automatically learn feature representations among input process parameters, capture nonlinear relationships between data, and share weights across multiple related tasks to improve efficiency and performance. Furthermore, the PSO is designed to determine the constraints and objective functions. In addition, the shapley additive explanation (SHAP) method is utilized to calculate the importance of process parameters and enhance the interpretability. Finally, the optimization results are validated, and the results demonstrate that the optimized results of the proposed method have achieved satisfactory performance compared to the traditional ML methods.

Age of information minimization in UAV-assisted data harvesting networks by multi-agent deep reinforcement curriculum learning

Unmanned Aerial Vehicles (UAVs) are increasingly being used for data harvesting from Wireless Sensor Nodes (SNs). This study aims to minimize the Age of Information (AoI) during data collection, while also considering the energy sustainability of the UAVs. Addressing the challenge of optimizing performance in Multi-Agent Reinforcement Learning (MARL) systems as the number of agents increases, our study introduces a MARL approach combined with curriculum learning and evolutionary strategy. This innovation specifically targets the performance issue in traditional MARL setups when scaling up the number of agents. By applying curriculum learning and evolutionary strategies, our method not only enhances MARL scalability but also integrates energy-efficient charging mechanisms, effectively enhancing system performance in large-scale deployments. Numerical results show that our proposed algorithm outperforms baselines in terms of AoI and charging, proving its effectiveness in managing the complexities of large-scale MARL systems.

An Attention-Guided Complex-Valued Transformer for Intra-Pulse Retransmission Interference Suppression

With the maturation of digital radio frequency memory (DRFM) technology, various intra-pulse retransmission interference methods have emerged. These flexible and changeable retransmission interference methods pose significant challenges to radar detection tasks, particularly in modern battlefields. This paper proposes an attention-guided complex-valued transformer (AGCT) as a solution. First, the encoder maps the received signal contaminated by interference and noise into a high-dimensional space. Then, the dilated convolution block (DCB) group and attention block (AB) group in the mask estimator extract the delicate multi-scale features and large-scale features of the interference, respectively, to obtain a multidimensional space mask. Finally, the decoder restores interference to the time domain and outputs the estimated target echo using residual learning. Considering the characteristics of intra-pulse interference, we introduced the energy attention block (EAB) at the end of the DCBs and the ABs within our network. This addition ensures a heightened focus on extracting interference features. Furthermore, we implemented a curriculum learning strategy during the network training. This approach gradually acclimates the network to fit different types of retransmission interference, starting from simpler to more complex scenarios. Our extensive experiments, conducted under various conditions, have provided compelling evidence of the AGCT’s superior performance. Compared to the comparative network, the AGCT’s advantages are particularly pronounced under more harsh conditions, demonstrating its robustness and effectiveness.

Literasi Digital, Pemanfaatan Media Pembelajaran dan Kemandirian Belajar: Kontribusinya terhadap Prestasi Akademik di SMA Intensif Taruna Pembangunan Surabaya

Education in the digital era requires the development of learning methods that are innovative and responsive to technological advances. Digital literacy and the use of learning media have an important role in improving student learning achievement. The aim of this research is to determine the impact of digital literacy and the use of learning media on student learning achievement at Taruna Pembangunan Intensive High School Surabaya. Digital literacy is considered the key to opening students' access to digital learning resources, improving understanding and providing a competitive advantage. The use of learning media with various approaches is also considered to have a positive impact on student learning achievement. Using quantitative methods, data was collected from 148 class XI students via questionnaire. Regression analysis is used to evaluate the relationship between variables. The research results show that the two independent variables have a significant influence on student learning achievement. Therefore, it is recommended to increase digital literacy and utilize learning media in curriculum planning and learning strategies.

Distributional Fairness-aware Recommendation

Fairness has been gradually recognized as a significant problem in the recommendation domain. Previous models usually achieve fairness by reducing the average performance gap between different user groups. However, the average performance may not sufficiently represent all the characteristics of the performances in a user group. Thus, equivalent average performance may not mean the recommender model is fair, for example, the variance of the performances can be different. To alleviate this problem, in this article, we define a novel type of fairness, where we require that the performance distributions across different user groups should be similar. We prove that with the same performance distribution, the numerical characteristics of the group performance, including the expectation, variance, and any higher-order moment, are also the same. To achieve distributional fairness, we propose a generative and adversarial training framework. Specifically, we regard the recommender model as the generator to compute the performance for each user in different groups, and then we deploy a discriminator to judge which group the performance is drawn from. By iteratively optimizing the generator and the discriminator, we can theoretically prove that the optimal generator (the recommender model) can indeed lead to the equivalent performance distributions. To smooth the adversarial training process, we propose a novel dual curriculum learning strategy for optimal scheduling of training samples. Additionally, we tailor our framework to better suit top-N recommendation tasks by incorporating softened ranking metrics as measures of performance discrepancies. We conduct extensive experiments based on real-world datasets to demonstrate the effectiveness of our model.

Exploring the Potential of Pre-Trained Language Models of Code for Automated Program Repair

In the realm of software development, automated program repair (APR) emerges as a pivotal technique, autonomously debugging faulty code to boost productivity. Despite the notable advancements of large pre-trained language models of code (PLMCs) in code generation, their efficacy in complex tasks like APR remains suboptimal. This limitation is attributed to the generic development of PLMCs, whose specialized potential for APR is yet be to fully explored. In this paper, we propose a novel approach designed to enhance PLMCs’ APR performance through source code augmentation and curriculum learning. Our approach employs code augmentation operators to generate a spectrum of syntactically varied yet semantically congruent bug-fixing programs, thus enriching the dataset’s diversity. Furthermore, we design a curriculum learning strategy, enabling PLMCs to develop a deep understanding of program semantics from these enriched code variants, thereby refining their APR fine-tuning prowess. We apply our approach across different PLMCs and systematically evaluate it on three benchmarks: BFP-small, BFP-medium, and Defects4J. The experimental results show that our approach outperforms both original models and existing baseline methods, demonstrating the promising future of adapting PLMCs for code debugging in practice.

Implementasi Strategi Pembelajaran Kurikulum 2013 Saintifik pada Mata Pelajaran Agama di Kelas 5 SD Kartika I-2 Medan

This research aims to describe the implementation of the 2013/Scientific curriculum learning strategy and describe the advantages and disadvantages of applying scientific implementation to learning. The implementation of learning through the teaching and learning process is basically the core of education as a whole. Teachers as educators who participate directly in the learning process are obliged to ensure that students are active in the learning process by using appropriate learning models. The type of research used in this study is descriptive qualitative to obtain complete and in-depth descriptions in the form of written and spoken words of people and observable behavior. Information is analyzed by reducing data, describing data and drawing conclusions. This research is to identify how strategies are implemented in religious subjects at SD Kartika I- 2. This research examines teachers' teaching strategies in religious subjects using the 2013/Scientific curriculum learning strategies.

Curriculum learning-based domain generalization for cross-domain fault diagnosis with category shift

Intelligent fault diagnosis has witnessed significant advancements in the preceding years. Domain generalization-based methods can effectively alleviate the domain shift problem and be employ for fault diagnosis in unknown domains. Apart from the problem of domain shift, another challenge arises from the incomplete label space of each source domain due to the difficulty of data acquisition. Category shift can have a significant impact on the subsequent application of intelligent algorithms. To confront this more challenging and practical problem, we begin by formulating the setting of domain generalization with category shift. This paper proposes a Curriculum Learning-based Domain Generalization method (CLDG) to tackle with the intricate problem. The basic network consists of a feature extractor, a mixup-based reciprocal point learning classifier for tackling the category shift between the source and target domains, and a conditional domain discriminator for addressing the domain shift. In addition, we construct a curriculum learning strategy that uses the knowledge of categories with high observation degree to assist in extracting domain invariant features of lower ones, dealing with the category shift between the source domains and improving the generalization ability of the categorical information. Extensive experimental results on two datasets provide evidence for the effectiveness and superiority of the proposed algorithm in classifying known and missing classes in each source domain, as well as identifying unobserved failure modes in unknown target domains.

Combination of concept maps and case-based learning in a flipped classroom: A mixed-methods study

AimThis study aimed to evaluate the impact of a hybrid learning method on the academic achievements, learning skills and experiences of undergraduate nursing students. BackgroundCase-based learning is a student-centered approach that is commonly integrated into the flipped classroom model in nursing education. However, effectively combining the flipped classroom model and case-based learning into a hybrid learning format is challenging. To address this gap, this study integrated concept maps with case-based learning and the flipped classroom and evaluated the impact of this hybrid learning method on the academic achievements, learning skills and experiences of undergraduate nursing students. DesignA mixed-methods approach, including a quasi-experimental study and semi-structured interviews, was employed. MethodsA total of 277 undergraduate nursing students participated, with 136 students in the control group undergoing case-based learning and 141 students in the experimental group taught using the hybrid learning approach, which combined concept maps, case-based learning and a flipped classroom. The study assessed self-directed learning, critical thinking, learning strategy and curriculum grades in both groups before and after the intervention. Additionally, students in the experimental group participated in semi-structured interviews. ResultsThe quantitative findings indicated that both case-based learning and the hybrid learning method (combined concept maps, case-based learning and a flipped classroom) had similar impacts on the curriculum grades, self-directed learning, cognitive maturity and learning strategy of nursing students. The qualitative results further demonstrated how the hybrid approach facilitated integrated learning, promoted self-evaluation, aided adaptation to the flipped classroom and enhanced teacher-student interaction. ConclusionsCombining case-based learning, a flipped classroom and concept maps is an effective learning approach for undergraduate nursing students. It may improve students' self-directed learning, cognitive maturity and learning strategy. Additionally, concept maps are a beneficial supplement to case-based learning and a flipped classroom in terms of guiding integrated learning, promoting self-evaluation, enhancing adaptation to a flipped classroom and increasing interaction between teachers and students and between classmates.

ERNetCL: A novel emotion recognition network in textual conversation based on curriculum learning strategy

Emotion recognition in conversation (ERC) has emerged as a research hotspot in domains such as conversational robots and question-answer systems. How to efficiently and adequately retrieve contextual emotional cues has been one of the key challenges in the ERC task. Existing efforts do not fully model the context and employ complex network structures, resulting in limited performance gains. In this paper, we propose a novel emotion recognition network based on curriculum learning strategy (ERNetCL). The proposed ERNetCL primarily consists of temporal encoder (TE), spatial encoder (SE), and curriculum learning (CL) loss. We utilize TE and SE to combine the strengths of previous methods in a simplistic manner to efficiently capture temporal and spatial contextual information in the conversation. To ease the harmful influence resulting from emotion shift and simulate the way humans learn curriculum from easy to hard, we apply the idea of CL to the ERC task to progressively optimize the network parameters. At the beginning of training, we assign lower learning weights to difficult samples. As the epoch increases, the learning weights for these samples are gradually raised. Extensive experiments on four datasets exhibit that our proposed method is effective and dramatically beats other baseline models.