Curriculum Learning Strategy Research Articles

Deep Reinforcement Learning for UAV Routing in the Presence of Multiple Charging Stations

Deploying Unmanned Aerial Vehicles (UAVs) for traffic monitoring has been a hotspot given their flexibility and broader view. However, a UAV is usually constrained by battery capacity due to limited payload. On the other hand, the development of wireless charging technology has allowed UAVs to replenish energy from charging stations. In this paper, we study a UAV routing problem in the presence of multiple charging stations (URPMCS) with the objective of minimizing the total distance traveled by the UAV during traffic monitoring. We present a deep reinforcement learning based method, where a multi-head heterogeneous attention mechanism is designed to facilitate learning a policy that automatically and sequentially constructs the route, while taking the energy consumption into account. In our method, two types of attentions are leveraged to learn the relations between monitoring targets and charging station nodes, adopting an encoder-decoder-like policy network. Moreover, we also employ a curriculum learning strategy to enhance generalization to different numbers of charging stations. Computational results show that our method outperforms conventional algorithms with higher solution quality (except for exact methods such as Gurobi) and shorter runtime in general, and also exhibits strong generalized performance on problem instances with different distributions and sizes.

Multimedia Technology Based Interactive Translation Learning for Students

Multimedia technology incorporates into the educational arena to translate traditional educational material into interactive digital mode. This has permitted teachers into the learning environment to design and integrate interactive multimedia learning. Many problems towards giving more attention to students facing teacher's multimedia levels are uneven, and their integration significance, bringing enormous curriculum learning strategies, is not entirely apparent. This research introduces multimedia network interpretation teaching using machine learning (MNIT-ML)in multimedia education design to enhance and strengthen the traditional teaching process and promote various modern approaches to transmit knowledge towards students.Allocation of learning resources (ALR) framework is mainly to enlarge the use and utilization of materials for learning activities from impressed resources into recordings, video content, motion graphics, and other forms of resources. The purpose of the Allocation of Learning Resources (ALR) framework is to aid teachers in making sound decisions about how to distribute available educational materials. Its goal is to guide teachers in making smart choices about how to use limited resources like time, money, and technology in order to improve students' educational achievements. The ALR framework was picked because it offers a methodical strategy for selecting choices that is founded in educational research and best practices. Resource allocation is a key aspect in influencing student outcomes, and efficient allocation can help guarantee that all students have access to the tools they need to succeed. There are a number of alternative frameworks that might direct studies on the distribution of educational resources. The Technological Acceptance Model (TAM) is a common tool for analysing what factors lead to widespread implementation of educational technology. The goal of other frameworks like the Universal Design for Learning (UDL) framework is to help educators create lessons and methods that are inclusive of students of all backgrounds and abilities. The final decision on which framework to use will be determined by the nature of the research issues and the setting in which they are being investigated. In order to make educated decisions, it is crucial to pick a framework that is appropriate for the research issues at hand and that gives a systematic approach based on established educational best practices. The digital promoting resources for teaching (DPRT) method creates a real environment for students to learn, focusing on enhancing training to make students feel the standard language translation skills. The simulation analysis is performed based on security 94.6%, the performance of 95.9%, and privacy, proving the proposed framework's reliability overall ratio of 93.4%.

Healthy living character-building strategies: A systematic literature review

The healthy living character needs to be intervened and habituated in elementary school students because it positively impacts children’s healthy food intake (fruits and vegetables) and several psychosocial factors (attitudes, knowledge, and self-efficacy). Improving the character of a healthy life needs to be supported by determining an accurate strategy. The study aimed to identify healthy living character-building strategies of elementary school students. This study used systematic review with Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) guidelines. Searches for articles related to the topic used Open Knowledge Maps, Google Scholar and Litmaps and found 60 primary articles and supported by secondary sources such as books and proceedings. The review results in the descriptive analysis include geographical distribution and research design. The review results show that current research is still dominated by social cognitive theory and social learning theory, and treatment lengths are categorized into 40 weeks, six months and six years. The findings show that eight dominant strategies are used in shaping healthy living characters through healthy foods for the immune formation of elementary school students. The most effective strategies to facilitate healthy food for primary school students are the strategy of a mix of enhanced curriculum, cross-curriculum and experiential learning and other strategies such as parental involvement. This research provides insight into healthy living character-building strategies in elementary school students and can guide policymakers, principals, and teachers in the development of mixed strategies to improve children’s healthy living character.

MSdocTr-Lite: A lite transformer for full page multi-script handwriting recognition

The Transformer has quickly become the dominant architecture for various pattern recognition tasks due to its capacity for long-range representation. However, transformers are data-hungry models and need large datasets for training. In Handwritten Text Recognition (HTR), collecting a massive amount of labeled data is a complicated and expensive task. In this paper, we propose a lite transformer architecture for full-page multi-script handwriting recognition. The proposed model comes with three advantages: First, to solve the common problem of data scarcity, we propose a lite transformer model that can be trained on a reasonable amount of data, which is the case of most HTR public datasets, without the need for external data. Second, it can learn the reading order at page-level thanks to a curriculum learning strategy, allowing it to avoid line segmentation errors, exploit a larger context and reduce the need for costly segmentation annotations. Third, it can be easily adapted to other scripts by applying a simple transfer-learning process using only page-level labeled images. Extensive experiments on different datasets with different scripts (French, English, Spanish, and Arabic) show the effectiveness of the proposed model.

On exploring weakly supervised domain adaptation strategies for semantic segmentation using synthetic data

Pixel-wise image segmentation is key for many Computer Vision applications. The training of deep neural networks for this task has expensive pixel-level annotation requirements, thus, motivating a growing interest on synthetic data to provide unlimited data and its annotations. In this paper, we focus on the generation and application of synthetic data as representative training corpuses for semantic segmentation of urban scenes. First, we propose a synthetic data generation protocol, which identifies key features affecting performance and provides datasets with variable complexity. Second, we adapt two popular weakly supervised domain adaptation approaches (combined training, fine-tuning) to employ synthetic and real data. Moreover, we analyze several backbone models, real/synthetic datasets and their proportions when combined. Third, we propose a new curriculum learning strategy to employ several synthetic and real datasets. Our major findings suggest the high performance impact of pace and order of synthetic and real data presentation, achieving state of the art results for well-known models. The results by training with the proposed dataset outperform popular alternatives, thus demonstrating the effectiveness of the proposed protocol. Our code and dataset are available at http://www-vpu.eps.uam.es/publications/WSDA_semantic/

Curriculum learning for data-driven modeling of dynamical systems.

The reliable prediction of the temporal behavior of complex systems is key in numerous scientific fields. This strong interest is however hindered by modeling issues: Often, the governing equations describing the physics of the system under consideration are not accessible or, when known, their solution might require a computational time incompatible with the prediction time constraints. Not surprisingly, approximating complex systems in a generic functional format and informing it ex-nihilo from available observations has become common practice in the age of machine learning, as illustrated by the numerous successful examples based on deep neural networks. However, generalizability of the models, margins of guarantee and the impact of data are often overlooked or examined mainly by relying on prior knowledge of the physics. We tackle these issues from a different viewpoint, by adopting a curriculum learning strategy. In curriculum learning, the dataset is structured such that the training process starts from simple samples toward more complex ones in order to favor convergence and generalization. The concept has been developed and successfully applied in robotics and control of systems. Here, we apply this concept for the learning of complex dynamical systems in a systematic way. First, leveraging insights from the ergodic theory, we assess the amount of data sufficient for a-priori guaranteeing a faithful model of the physical system and thoroughly investigate the impact of the training set and its structure on the quality of long-term predictions. Based on that, we consider entropy as a metric of complexity of the dataset; we show how an informed design of the training set based on the analysis of the entropy significantly improves the resulting models in terms of generalizability and provide insights on the amount and the choice of data required for an effective data-driven modeling.

PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning.

The predictive learning of spatiotemporal sequences aims to generate future images by learning from the historical context, where the visual dynamics are believed to have modular structures that can be learned with compositional subsystems. This paper models these structures by presenting PredRNN, a new recurrent network, in which a pair of memory cells are explicitly decoupled, operate in nearly independent transition manners, and finally form unified representations of the complex environment. Concretely, besides the original memory cell of LSTM, this network is featured by a zigzag memory flow that propagates in both bottom-up and top-down directions across all layers, enabling the learned visual dynamics at different levels of RNNs to communicate. It also leverages a memory decoupling loss to keep the memory cells from learning redundant features. We further propose a new curriculum learning strategy to force PredRNN to learn long-term dynamics from context frames, which can be generalized to most sequence-to-sequence models. We provide detailed ablation studies to verify the effectiveness of each component. Our approach is shown to obtain highly competitive results on five datasets for both action-free and action-conditioned predictive learning scenarios.

Feedback Network for Compact Thin Cloud Removal

The thin cloud removal (CR) technique has great practical value for the application of remote sensing images. Existing deep-learning-based methods have attained remarkable achievements. However, most of them neglect the inherent feature correlations in deeper layers due to learning in a successive manner. In this letter, we propose a compact thin cloud removal network based on the feedback (FB) mechanism, called CRFB-Net, which leverages the high-level features as feedback information to modulate shallow representations. CRFB-Net employs the recurrent architecture to achieve such a feedback scheme. Specifically, the restoration process does not terminate after obtaining an output. In this case, the output of intermediate iterations will flow into the next iteration as feedback. For better utilization of feedback, a multiscale feature fusion block (MFFB) is designed to refine the low-level representations from three scales. Furthermore, we introduce a curriculum learning strategy to train the CRFB-Net by gradually increasing the complexity of restoration, through which a sharper result is produced step by step. Extensive experiments demonstrate the superiority of our CRFB-Net, outperforming state-of-the-art.

Scheduling in Multiagent Systems Using Reinforcement Learning

The paper is devoted to scheduling in multiagent systems in the framework of the Flatland 3 competition. The main aim of this competition is to develop an algorithm for the effective control of dense traffic in complex railroad networks according to a given schedule. The proposed solution is based on reinforcement learning. To adapt this method to the particular scheduling problem, a novel approach based on structuring the reward function that stimulates an agent to adhere to its schedule was developed. The architecture of the proposed model is based on a multiagent version of centralized critic with proximal policy optimization (PPO) learning. In addition, a curriculum learning strategy was developed and implemented. This allowed the agent to cope with each level of complexity on time and train the model in more difficult conditions. The proposed solution won first place in the Flatland 3 competition in the reinforcement learning track.

Su-MICL: Severity-Guided Multiple Instance Curriculum Learning for Histopathology Image Interpretable Classification.

Histopathology image classification plays a critical role in clinical diagnosis. However, due to the absence of clinical interpretability, most existing image-level classifiers remain impractical. To acquire the essential interpretability, lesion-level diagnosis is also provided, relying on detailed lesion-level annotations. Although the multiple-instance learning (MIL)-based approach can identify lesions by only utilizing image-level annotations, it requires overly strict prior information and has limited accuracy in lesion-level tasks. Here, we present a novel severity-guided multiple instance curriculum learning (Su-MICL) strategy to avoid tedious labeling. The proposed Su-MICL is under a MIL framework with a neglected prior: disease severity to define the learning difficulty of training images. Based on the difficulty degree, a curriculum is developed to train a model utilizing images from easy to hard. The experimental results for two histopathology image datasets demonstrate that Su-MICL achieves comparable performance to the state-of-the-art weakly supervised methods for image-level classification, and its performance for identifying lesions is closest to the supervised learning method. Without tedious lesion labeling, the Su-MICL approach can provide an interpretable diagnosis, as well as an effective insight to aid histopathology image diagnosis.

The National Alliance for Pharmacy Education (NAPE): Leading pharmacy intern training in Australia

Four of Australia’s leading pharmacy schools formed the National Alliance for Pharmacy Education (NAPE) to support the advancement of the pharmacy profession as a critical member of the healthcare team. NAPE builds capacity to design, implement, and evaluate an accredited curriculum framework, learning and assessment strategy, and workplace-integrated learning program for intern pharmacists through a commitment to shared values, collaboration, and engagement with key stakeholders. This article discusses NAPE’s strategies for delivering a consistent, outcomes-driven, accredited pharmacy intern training program (ITP) across Australia. Since the program's commencement 12 years ago, enrolments in the program have increased, accompanied by high completion rates and excellent outcomes. The strategies which underpin the collaboration have been successful in ensuring the program’s quality, consistency, and sustainability.

Blind Restoration of Atmospheric Turbulence-Degraded Images Based on Curriculum Learning

Atmospheric turbulence-degraded images in typical practical application scenarios are always disturbed by severe additive noise. Severe additive noise corrupts the prior assumptions of most baseline deconvolution methods. Existing methods either ignore the additive noise term during optimization or perform denoising and deblurring completely independently. However, their performances are not high because they do not conform to the prior that multiple degradation factors are tightly coupled. This paper proposes a Noise Suppression-based Restoration Network (NSRN) for turbulence-degraded images, in which the noise suppression module is designed to learn low-rank subspaces from turbulence-degraded images, the attention-based asymmetric U-NET module is designed for blurred-image deconvolution, and the Fine Deep Back-Projection (FDBP) module is used for multi-level feature fusion to reconstruct a sharp image. Furthermore, an improved curriculum learning strategy is proposed, which trains the network gradually to achieve superior performance through a local-to-global, easy-to-difficult learning method. Based on NSRN, we achieve state-of-the-art performance with PSNR of 30.1 dB and SSIM of 0.9 on the simulated dataset and better visual results on the real images.

Comparison of Malaysia's Bachelor of Dental Technology curriculum with three other countries: Proposing a basic curriculum framework.

A dental technologist is one of the most essential allied dental health professionals and the dental technology curriculum should be comprehensively reviewed on a regular basis. This study aims to compare the only existing Bachelor of Dental Technology (BDT) curriculum in Malaysia with BDT programmes offered by other well-established universities, and map out the similarities and differences, as well as to explore future recommendations and propose a new curriculum framework. A descriptive analysis was carried out using Laurie Brady's four-stage strategy. First, available curriculum materials were collected from four different institutions' electronic webpage: AIMST (Malaysia), GU (Australia), UO (New Zealand) and CMU (United Kingdom), and then compared based on three key domains: curriculum contents, teaching and learning strategies and assessments. Following that, the similarities and differences between various curricula were identified. Future recommendations and a curriculum framework were then proposed. The core BDT curriculum content is concurred upon by all four universities, with an emphasis on basic sciences, laboratory materials, practical sessions and research projects. However, the credit weightage for each course or module varied across the four institutions, with some offering unique subjects and implementing different teaching methods and assessments. A simple BDT curriculum framework with a proposed syllabus was designed based on the three key domains and future recommendations for curriculum improvement were explored. The present study identified several areas for Malaysian BDT curriculum development and improvement. The proposed framework can be a guide for Malaysian dental schools in designing a comprehensive dental technology programme.

Boosting Unsupervised Domain Adaptation with Soft Pseudo-Label and Curriculum Learning

By leveraging data from a fully labeled source domain, unsupervised domain adaptation (UDA) improves classification performance on an unlabeled target domain through explicit discrepancy minimization of data distribution or adversarial learning. As an enhancement, category alignment is involved during adaptation to reinforce target feature discrimination by utilizing model prediction. However, there remain unexplored problems about pseudo-label inaccuracy incurred by wrong category predictions on target domain, and distribution deviation caused by overfitting on source domain. In this paper, we propose a model-agnostic two-stage learning framework, which greatly reduces flawed model predictions using soft pseudo-label strategy and avoids overfitting on source domain with a curriculum learning strategy. Theoretically, it successfully decreases the combined risk in the upper bound of expected error on the target domain. In the first stage, we train a model with distribution alignment-based UDA method to obtain soft semantic label on target domain with rather high confidence. To avoid overfitting on source domain, in the second stage, we propose a curriculum learning strategy to adaptively control the weighting between losses from the two domains so that the focus of the training stage is gradually shifted from source distribution to target distribution with prediction confidence boosted on the target domain. Extensive experiments on two well-known benchmark datasets validate the universal effectiveness of our proposed framework on promoting the performance of the top-ranked UDA algorithms and demonstrate its consistent superior performance.

Curriculum learning for early Alzheimer's Disease diagnosis.

The early and asymptomatic stages of Alzheimer's Disease (AD), such as mild cognitive impairment (MCI), are hard to classify, even by experienced physicians. Deep learning approaches, such as convolutional neural networks (CNNs), have been shown to help, achieving similar or even better results. Although these methods have the advantage that features are automatically extracted from images rather than handcrafted, they do not allow for incorporating medical knowledge. In this paper we propose curriculum learning (CL) strategies for CNNs designed to diagnose healthy subjects, MCI and AD, as a way to incorporate medical knowledge to boost the performance of the networks for early AD diagnosis. CL is a training strategy of the networks that tries to mimic the way humans, in this case doctors, learn. Several CL strategies were implemented and compared to commonly used baseline methods. The results show that they improve the performance, particularly that of MCI. Clinical relevance- This work shows that the use of CL strategies improve the diagnosis of AD, particularly at an early stage.

Coach-assisted multi-agent reinforcement learning framework for unexpected crashed agents

Multi-agent reinforcement learning is difficult to apply in practice, partially because of the gap between simulated and real-world scenarios. One reason for the gap is that simulated systems always assume that agents can work normally all the time, while in practice, one or more agents may unexpectedly “crash” during the coordination process due to inevitable hardware or software failures. Such crashes destroy the cooperation among agents and lead to performance degradation. In this work, we present a formal conceptualization of a cooperative multi-agent reinforcement learning system with unexpected crashes. To enhance the robustness of the system to crashes, we propose a coach-assisted multi-agent reinforcement learning framework that introduces a virtual coach agent to adjust the crash rate during training. We have designed three coaching strategies (fixed crash rate, curriculum learning, and adaptive crash rate) and a re-sampling strategy for our coach agent. To our knowledge, this work is the first to study unexpected crashes in a multi-agent system. Extensive experiments on grid-world and StarCraft II micromanagement tasks demonstrate the efficacy of the adaptive strategy compared with the fixed crash rate strategy and curriculum learning strategy. The ablation study further illustrates the effectiveness of our re-sampling strategy.

Hybrid Curriculum Learning for Emotion Recognition in Conversation

Emotion recognition in conversation (ERC) aims to detect the emotion label for each utterance. Motivated by recent studies which have proven that feeding training examples in a meaningful order rather than considering them randomly can boost the performance of models, we propose an ERC-oriented hybrid curriculum learning framework. Our framework consists of two curricula: (1) conversation-level curriculum (CC); and (2) utterance-level curriculum (UC). In CC, we construct a difficulty measurer based on ``emotion shift'' frequency within a conversation, then the conversations are scheduled in an ``easy to hard" schema according to the difficulty score returned by the difficulty measurer. For UC, it is implemented from an emotion-similarity perspective, which progressively strengthens the model’s ability in identifying the confusing emotions. With the proposed model-agnostic hybrid curriculum learning strategy, we observe significant performance boosts over a wide range of existing ERC models and we are able to achieve new state-of-the-art results on four public ERC datasets.

Interpretable Low-Resource Legal Decision Making

Over the past several years, legal applications of deep learning have been on the rise. However, as with other high-stakes decision making areas, the requirement for interpretability is of crucial importance. Current models utilized by legal practitioners are more of the conventional machine learning type, wherein they are inherently interpretable, yet unable to harness the performance capabilities of data-driven deep learning models. In this work, we utilize deep learning models in the area of trademark law to shed light on the issue of likelihood of confusion between trademarks. Specifically, we introduce a model-agnostic interpretable intermediate layer, a technique which proves to be effective for legal documents. Furthermore, we utilize weakly supervised learning by means of a curriculum learning strategy, effectively demonstrating the improved performance of a deep learning model. This is in contrast to the conventional models which are only able to utilize the limited number of expensive manually-annotated samples by legal experts. Although the methods presented in this work tackles the task of risk of confusion for trademarks, it is straightforward to extend them to other fields of law, or more generally, to other similar high-stakes application scenarios.

Curriculum Learning: A Survey

Training machine learning models in a meaningful order, from the easy samples to the hard ones, using curriculum learning can provide performance improvements over the standard training approach based on random data shuffling, without any additional computational costs. Curriculum learning strategies have been successfully employed in all areas of machine learning, in a wide range of tasks. However, the necessity of finding a way to rank the samples from easy to hard, as well as the right pacing function for introducing more difficult data can limit the usage of the curriculum approaches. In this survey, we show how these limits have been tackled in the literature, and we present different curriculum learning instantiations for various tasks in machine learning. We construct a multi-perspective taxonomy of curriculum learning approaches by hand, considering various classification criteria. We further build a hierarchical tree of curriculum learning methods using an agglomerative clustering algorithm, linking the discovered clusters with our taxonomy. At the end, we provide some interesting directions for future work.

The multi-mode operation decision of cleaning robot based on curriculum learning strategy and feedback network

The multi-mode operation decision of cleaning robot based on curriculum learning strategy and feedback network