Diverse Learning Tasks Research Articles

Implementing an Educational Design Based on Hypermedia to Enhance Creative Imagination Among Fifth-Grade Literary Students

The research delves into the intricate design and effectiveness of hypermedia-based educational systems, emphasizing their pivotal role in enhancing learning processes through spatially induced learning formats and creative tools. Hypermedia, a sophisticated form of computer-based media technology, facilitates non-linear, feature-rich media interfaces that significantly augment educational content delivery. This technology transcends traditional instructional tools by embedding navigational methods tailored to diverse learning tasks and environments, thereby fostering deeper understanding and engagement among learners. In this context, the ADDIE model a structured approach comprising Analysis, Design, Development, Implementation, and Evaluation—is crucial for the systematic development of educational programs. Each phase of ADDIE builds upon the insights gained from the previous, starting with a comprehensive analysis of training needs, followed by the design and development of instructional materials that incorporate effective strategies and media. The implementation phase involves deploying these materials through learning management systems, ensuring accessibility and alignment with educational goals. Finally, the evaluation phase seeks feedback to refine and enhance the learning modules, demonstrating ADDIE’s iterative nature and its focus on continuous improvement. Moreover, the study engages with a broader theoretical and methodological discourse, integrating perspectives from instructional design and creative imagination within the framework of hypermedia technology. This integration supports the construction of a learning environment that not only addresses theoretical knowledge but also stimulates creativity and innovation among students. The effectiveness of these instructional strategies is evidenced through rigorous assessments and evaluations, highlighting the significant impact of hypermedia on student creativity, problem-solving skills, and overall academic performance. This comprehensive approach not only validates the use of hypermedia in educational settings but also outlines potential improvements and future directions for integrating technology in learning processes, thereby contributing to the evolving landscape of educational technology and pedagogy.

AdaGCR: An improved method for optimizing machine learning training process

In contemporary machine learning, training datasets are typically divided into batches, and models are updated incrementally through batch iterations to save memory and reduce overfitting. However, determining the optimal hyperparameters like learning rate, batch size and number of epochs remains a challenge which often relying on empirical insights. This paper explores a novel method called Adaptive Gradient Conflict Rate (AdaGCR) to optimize the training process. It leverages the idea of gradient conflict rate, which reflects the models position within a batch model set and accordingly adjusts the global learning rate. This proposed method is tested by training a Deep Neural Network (DNN) model with MNIST dataset which represents simple tasks and a ResNet-18 model with CIFAR-10 dataset which represents more complicated tasks for solving real world problems. Experiments conducted on DNN demonstrates the proposed methods effectiveness in reducing overfitting and enhancing convergence, particularly with a well-suited initial learning rate. However, its applicability to more complex models like ResNet-18 may require further refinements, such as layer-specific learning rate adjustments. Future research should focus on fine-tuning AdaGCR and extending its utility across diverse machine learning models and tasks.

CoRe optimizer: an all-in-one solution for machine learning

The optimization algorithm and its hyperparameters can significantly affect the training speed and resulting model accuracy in machine learning (ML) applications. The wish list for an ideal optimizer includes fast and smooth convergence to low error, low computational demand, and general applicability. Our recently introduced continual resilient (CoRe) optimizer has shown superior performance compared to other state-of-the-art first-order gradient-based optimizers for training lifelong ML potentials. In this work we provide an extensive performance comparison of the CoRe optimizer and nine other optimization algorithms including the Adam optimizer and resilient backpropagation (RPROP) for diverse ML tasks. We analyze the influence of different hyperparameters and provide generally applicable values. The CoRe optimizer yields best or competitive performance in every investigated application, while only one hyperparameter needs to be changed depending on mini-batch or batch learning.

Hierarchical Optimization-Derived Learning.

In recent years, by utilizing optimization techniques to formulate the propagation of deep model, a variety of so-called Optimization-Derived Learning (ODL) approaches have been proposed to address diverse learning and vision tasks. Although having achieved relatively satisfying practical performance, there still exist fundamental issues in existing ODL methods. In particular, current ODL methods tend to consider model constructing and learning as two separate phases, and thus fail to formulate their underlying coupling and depending relationship. In this work, we first establish a new framework, named Hierarchical ODL (HODL), to simultaneously investigate the intrinsic behaviors of optimization-derived model construction and its corresponding learning process. Then we rigorously prove the joint convergence of these two sub-tasks, from the perspectives of both approximation quality and stationary analysis. To our best knowledge, this is the first theoretical guarantee for these two coupled ODL components: optimization and learning. We further demonstrate the flexibility of our framework by applying HODL to challenging learning tasks, which have not been properly addressed by existing ODL methods. Finally, we conduct extensive experiments on both synthetic data and real applications in vision and other learning tasks to verify the theoretical properties and practical performance of HODL in various application scenarios.

A universal framework for accurate and efficient geometric deep learning of molecular systems

Molecular sciences address a wide range of problems involving molecules of different types and sizes and their complexes. Recently, geometric deep learning, especially Graph Neural Networks, has shown promising performance in molecular science applications. However, most existing works often impose targeted inductive biases to a specific molecular system, and are inefficient when applied to macromolecules or large-scale tasks, thereby limiting their applications to many real-world problems. To address these challenges, we present PAMNet, a universal framework for accurately and efficiently learning the representations of three-dimensional (3D) molecules of varying sizes and types in any molecular system. Inspired by molecular mechanics, PAMNet induces a physics-informed bias to explicitly model local and non-local interactions and their combined effects. As a result, PAMNet can reduce expensive operations, making it time and memory efficient. In extensive benchmark studies, PAMNet outperforms state-of-the-art baselines regarding both accuracy and efficiency in three diverse learning tasks: small molecule properties, RNA 3D structures, and protein-ligand binding affinities. Our results highlight the potential for PAMNet in a broad range of molecular science applications.

THF: 3-Way Hierarchical Framework for Efficient Client Selection and Resource Management in Federated Learning

Federated learning (FL) is a promising technique for collaboratively training machine-learning models on massively distributed clients data under privacy constraints. However, the existing FL literature focuses on speeding up the learning process and ignores minimizing the communication cost which is critical for resource-constrained clients. To this end, in this article, we propose a novel 3-way hierarchical framework (THF) to promote communication efficiency in FL. Using the proposed framework, only a cluster head (CH) communicates with the cloud server through edge aggregation in order to minimize the communication cost of clients. In particular, the clients upload their local models to their respective CHs, which are responsible to forward them to the corresponding edge server. The edge server averages the local models and iterates until it achieves the edge accuracy. Afterward, each edge server uploads the edge models to the cloud server for global aggregation. In this way, model downloading and uploading requires less bandwidth due to the short distance from source to destination that makes an efficient 3-way hierarchical network structure. In addition, we formulate a joint communication and computation resource management scheme through efficient client selection in order to achieve global cost minimization in FL. We conduct extensive empirical evaluations on diverse data learning tasks on multiple data sets to signify that THF achieves global cost savings and converges within fewer communication rounds compared to other FL approaches.

FedMarket: A Cryptocurrency Driven Marketplace for Mobile Federated Learning Services

Federated learning (FL) enables the training of a shared collaborative machine learning model while keeping all the confidential training data on distributed devices. The FL state-of-the-art considers a monopolist FL task publisher. However, we present a FL marketplace where multiple FL task publishers and mobile devices co-exist for a set of diverse and varying learning tasks. Mobile devices participating in the training of FL models provides pay-as-you-go (i.e. using blockchain-based cryptocurrencies) FL training services to the FL task publishers. In the proposed framework, multiple FL task publishers may compete with each other and the participating workers (i.e. mobile devices) can choose one FL task publisher over another for participation in the training of a global model. We utilize code offloading for enabling customized FL pipelines in mobile devices and mitigating the model heterogeneity inherent in varying and changing FL tasks published by the task publishers. Experimental results indicate the efficacy of the proposed framework.

PARROT is a flexible recurrent neural network framework for analysis of large protein datasets.

The rise of high-throughput experiments has transformed how scientists approach biological questions. The ubiquity of large-scale assays that can test thousands of samples in a day has necessitated the development of new computational approaches to interpret this data. Among these tools, machine learning approaches are increasingly being utilized due to their ability to infer complex nonlinear patterns from high-dimensional data. Despite their effectiveness, machine learning (and in particular deep learning) approaches are not always accessible or easy to implement for those with limited computational expertise. Here we present PARROT, a general framework for training and applying deep learning-based predictors on large protein datasets. Using an internal recurrent neural network architecture, PARROT is capable of tackling both classification and regression tasks while only requiring raw protein sequences as input. We showcase the potential uses of PARROT on three diverse machine learning tasks: predicting phosphorylation sites, predicting transcriptional activation function of peptides generated by high-throughput reporter assays, and predicting the fibrillization propensity of amyloid beta with data generated by deep mutational scanning. Through these examples, we demonstrate that PARROT is easy to use, performs comparably to state-of-the-art computational tools, and is applicable for a wide array of biological problems.

Accurate cancer phenotype prediction with AKLIMATE, a stacked kernel learner integrating multimodal genomic data and pathway knowledge.

Advancements in sequencing have led to the proliferation of multi-omic profiles of human cells under different conditions and perturbations. In addition, many databases have amassed information about pathways and gene “signatures”—patterns of gene expression associated with specific cellular and phenotypic contexts. An important current challenge in systems biology is to leverage such knowledge about gene coordination to maximize the predictive power and generalization of models applied to high-throughput datasets. However, few such integrative approaches exist that also provide interpretable results quantifying the importance of individual genes and pathways to model accuracy. We introduce AKLIMATE, a first kernel-based stacked learner that seamlessly incorporates multi-omics feature data with prior information in the form of pathways for either regression or classification tasks. AKLIMATE uses a novel multiple-kernel learning framework where individual kernels capture the prediction propensities recorded in random forests, each built from a specific pathway gene set that integrates all omics data for its member genes. AKLIMATE has comparable or improved performance relative to state-of-the-art methods on diverse phenotype learning tasks, including predicting microsatellite instability in endometrial and colorectal cancer, survival in breast cancer, and cell line response to gene knockdowns. We show how AKLIMATE is able to connect feature data across data platforms through their common pathways to identify examples of several known and novel contributors of cancer and synthetic lethality.

Potential Deep Learning Solutions to Persistent and Emerging Big Data Challenges—A Practitioners’ Cookbook

The phenomenon of Big Data continues to present moving targets for the scientific and technological state-of-the-art. This work demonstrates that the solution space of these challenges has expanded with deep learning now moving beyond traditional applications in computer vision and natural language processing to diverse and core machine learning tasks such as learning with streaming and non-iid-data, partial supervision, and large volumes of distributed data while preserving privacy. We present a framework coalescing multiple deep methods and corresponding models as responses to specific Big Data challenges. First, we perform a detailed per-challenge review of existing techniques, with benchmarks and usage advice, and subsequently synthesize them together into one organic construct that we discover principally uses extensions of one underlying model, the autoencoder. This work therefore provides a synthesis where challenges at scale across the Vs of Big Data could be addressed by new algorithms and architectures being proposed in the deep learning community. The value being proposed to the reader from either community in terms of nomenclature, concepts, and techniques of the other would advance the cause of multi-disciplinary, transversal research and accelerate the advance of technology in both domains.

Exploring the English Learning Strategies of an Indigenous Papuan Student of Indonesia

Language learning strategies are crucial for learning English as a foreign language (EFL). This study sought to reveal language learning strategies used by Daniela, an Indigenous Papuan student, in developing her knowledge and skills in English. To gain this objective, a case study (Kohlbacher, 2006) using an in-depth interview was employed. Collected data were qualitatively analyzed using descriptive analysis techniques. Results of data analysis revealed two main categories of learning strategies used by Daniela in solving her learning problems, namely direct and indirect learning strategies. In direct learning strategies Daniela dealt with memory, cognition, and compensation strategies, while in indirect learning strategies she dealt more with metacognitive, affective, and social strategies. Apart from these findings, this study also revealed Daniela’s performance in English is mostly influenced by her reading hobby, internal motivation, positive attitude towards English, her personality (diligent and independent), and her dream to become a vocational English teacher. Results of this study may be useful for the head of school to encourage English teachers to provide learners with diverse learning tasks to elicit the use of various learning strategies.

Few-Shot Induction of Generalized Logical Concepts via Human Guidance.

We consider the problem of learning generalized first-order representations of concepts from a small number of examples. We augment an inductive logic programming learner with 2 novel contributions. First, we define a distance measure between candidate concept representations that improves the efficiency of search for target concept and generalization. Second, we leverage richer human inputs in the form of advice to improve the sample efficiency of learning. We prove that the proposed distance measure is semantically valid and use that to derive a PAC bound. Our experiments on diverse learning tasks demonstrate both the effectiveness and efficiency of our approach.

Signal Processing on Signed Graphs: Fundamentals and Potentials

A wide range of data science problems can be modeled in terms of a graph (or network), e.g., social, sensor, communication, infrastructure, and biological networks. The nodes in a graph/network represent the entities of interest, and the edges reflect relations between these entities, such as geographic proximity (e.g., wireless networks), social relations (e.g., Facebook), biological mechanisms (e.g., brain networks), similarity (e.g., texts by identical authors), and statistical dependencies (e.g., gene expression data). Graph signal processing (GSP) advocates the use of graphs as combined data and computation models and has become a groundbreaking and powerful paradigm for solving diverse learning and inference tasks in the area of data science. GSP uses tools from linear algebra, (spectral) graph theory, computational harmonic analysis, and optimization theory to extend conventional signal processing concepts to data located on irregular domains that are characterized by graphs (networks). For entry-level expositions of GSP fundamentals and the relevant state of the art, we refer to [1]-[4] and the May 2018 Proceedings of the IEEE.

Gated Value Network for Multilabel Classification.

We introduce a gated value network (GVN) for general multilabel classification (MLC) tasks. GVN was motivated by deep value network (DVN) that directly exploits the "compatibility" metric as the learning pursuit for MLC. Meanwhile, it further improves traditional DVN on twofold. First, GVN relaxes the complex variable optimization steps in DVN inference by incorporating a feedforward predictor for straightforward multilabel prediction. Second, GVN also introduces the gating mechanism to block confounding factors from the input data that allows more precise compatibility evaluations for data and their potential multilabels. The whole GVN framework is trained in an end-to-end manner with policy gradient approaches. We show the effectiveness and generalization of GVN on diverse learning tasks, including document classification, audio tagging, and image attribute prediction.

Brain graph super-resolution for boosting neurological disorder diagnosis using unsupervised multi-topology connectional brain template learning.

Existing graph analysis techniques generally focus on decreasing the dimensionality of graph data (i.e., removing nodes, edges, or both) in diverse predictive learning tasks in pattern recognition, computer vision, and medical data analysis such as dimensionality reduction, filtering and embedding techniques. However, graph super-resolution is strikingly lacking, i.e., the concept of super-resolving low-resolution (LR) graphs with nr nodes into high-resolution graphs (HR) with [Formula: see text] nodes. Particularly, learning how to automatically generate HR brain connectomes, without resorting to the computationally expensive MRI processing steps such as image registration and parcellation, remains unexplored. To fill this gap, we propose the first technique to super-resolve undirected fully connected graphs with application to brain connectomes. First, we root our brain graph super-resolution (BGSR) framework in learning how to estimate a centered LR population-based brain graph representation, coined as connectional brain template (CBT), acting as a proxy in the target BGSR task. Specifically, we hypothesize that the estimation of a well-representative and centered CBT would help better capture the individuality of each LR brain graph via its residual distance from the population-based CBT. This will eventually allow an accurate identification of the most similar individual graphs to a new testing graph in the LR domain for the target prediction task. Second, we leverage the estimated LR CBT (i.e., population mean) to derive residual LR brain graphs, capturing the deviation of all subjects from the estimated CBT. Third, we learn multi-topology LR graph manifolds using different graph topological measurements (e.g., degree, closeness, betweenness) by estimating residual LR similarity matrices modeling the relationship between pairs of residual LR graphs. These are then fused so we can effectively identify for each testing LR subject its most K similar training LR graphs. Last, the missing testing HR graph is predicted by averaging the HR graphs of the K selected training subjects. Predicted HR from LR functional brain graphs boosted classification results for autistic subjects by 16.48% compared with LR functional graphs.

Teaching towards Design-Based Learning in Manufacturing Technology Course: Sino–Australia Joint Undergraduate Program

The internationalized higher education in Engineering has made the sustainable future of Chinese regional universities prosper in the recent decade. The teaching practices of engineering courses pose many challenges in Sino–foreign joint undergraduate programs. The design-based learning (DBL) approach addresses students learning challenges in the joint undergraduate program facilitated by the Wuhan University of Science and Technology (WUST) in central China along with Deakin University (DU), Australia. Following the seven general principles of DBL, a project of process planning was performed for teaching and learning in the Manufacturing Technology course. An implicit meta-cognitive competence was developed through performing the engineering project tutoring, diverse learning tasks and normative assessment criteria. The DBL pedagogy succeeds in bridging the diverse knowledge systems in the specialized courses of Manufacturing Technology between Chinese and Australian programs in Mechanical Engineering. Many achievements and awards won by the students demonstrate a satisfactory result in the case study on the teaching practice towards DBL. The pedagogy towards DBL truly improves the teaching quality of the courses in joint programs and further strengthens the internationalized engineering education for the sustainable development of regional universities in China.

SampleCNN: End-to-End Deep Convolutional Neural Networks Using Very Small Filters for Music Classification

Convolutional Neural Networks (CNN) have been applied to diverse machine learning tasks for different modalities of raw data in an end-to-end fashion. In the audio domain, a raw waveform-based approach has been explored to directly learn hierarchical characteristics of audio. However, the majority of previous studies have limited their model capacity by taking a frame-level structure similar to short-time Fourier transforms. We previously proposed a CNN architecture which learns representations using sample-level filters beyond typical frame-level input representations. The architecture showed comparable performance to the spectrogram-based CNN model in music auto-tagging. In this paper, we extend the previous work in three ways. First, considering the sample-level model requires much longer training time, we progressively downsample the input signals and examine how it affects the performance. Second, we extend the model using multi-level and multi-scale feature aggregation technique and subsequently conduct transfer learning for several music classification tasks. Finally, we visualize filters learned by the sample-level CNN in each layer to identify hierarchically learned features and show that they are sensitive to log-scaled frequency.

Timing Rules for Synaptic Plasticity Matched to Behavioral Function

It is widely assumed that the complexity of neural circuits enables them to implement diverse learning tasks using just a few generic forms of synaptic plasticity. In contrast, we report that synaptic plasticity can itself be precisely tuned to the requirements of a learning task. We found that the rules for induction of long-term and single-trial plasticity at parallel fiber-to-Purkinje cell synapses vary across cerebellar regions. In the flocculus, associative plasticity invitro and invivo is narrowly tuned for an interval of ∼120ms, which compensates for the specific processing delay for error signals to reach the flocculus during oculomotor learning. In the vermis, which supports a range of behavioral functions, plasticity is induced by a range of intervals, with individual cells tuned for different intervals. Thus, plasticity at a single, anatomically defined type of synapse can have properties that vary in a way that is precisely matched to function.

An ℓ2/ℓ1 regularization framework for diverse learning tasks

Regularization plays an important role in learning tasks, to incorporate prior knowledge about a problem and thus improve learning performance. Well known regularization methods, including ℓ2 and ℓ1 regularization, have shown great success in a variety of conventional learning tasks, and new types of regularization have also been developed to deal with modern problems, such as multi-task learning. In this paper, we introduce the ℓ2/ℓ1 regularization for diverse learning tasks. The ℓ2/ℓ1 regularization is a mixed norm defined over the parameters of the diverse learning tasks. It adaptively encourages the diversity of features among diverse learning tasks, i.e., when a feature is responsible for some tasks it is unlikely to be responsible for the rest of the tasks. We consider two applications of the ℓ2/ℓ1 regularization framework, i.e., learning sparse self-representation of a dataset for clustering and learning one-vs.-rest binary classifiers for multi-class classification, both of which confirm the effectiveness of the new regularization framework over benchmark datasets.

PeerWise provides significant academic benefits to biological science students across diverse learning tasks, but with minimal instructor intervention.

We demonstrate that student engagement with PeerWise, an online tool that allows students to author and answer multiple-choice questions (MCQs), is associated with enhanced academic performance across diverse assessment types on a second year Genetics course. Benefits were consistent over three course deliveries, with differential benefits bestowed on groups of different prior ability. A rating scheme, to assess the educational quality of students' questions, is presented and demonstrates that our students are able intuitively to make such quality assessments, and that the process of authoring high quality questions alone does not explain the academic benefits. We further test the benefits of providing additional PeerWise support and conclude that PeerWise works efficiently with minimal intervention, and can be reliably assessed using automatically generated PeerWise scores.