Knowledge Transfer Framework Research Articles

FedKT: Federated learning with knowledge transfer for non-IID data

Federated Learning enables clients to train a joint model collaboratively without disclosing raw data. However, learning over non-IID data may raise performance degeneration, which has become a fundamental bottleneck. Despite numerous efforts to address this issue, challenges such as excessive local computational burdens and reliance on shared data persist, rendering them impractical in real-world scenarios. In this paper, we propose a novel federated knowledge transfer framework to overcome data heterogeneity issues. Specifically, a model segmentation distillation method and a learnable aggregation network are developed for server-side knowledge ensemble and transfer, while a client-side consistency-constrained loss is devised to rectify local updates, thereby enhancing both global and client models. The framework considers both diversity and consistency among clients and can serve as a general solution for extracting knowledge from distributed nodes. Extensive experiments on four datasets demonstrate our framework’s effectiveness, achieving superior performance compared to advanced competitors in high-heterogeneity settings.

Needs Assessment for Primary Care for Best Practice Implementation: The CIRCE Joint Action

Abstract Introduction CIRCE is a Joint Action (JA) supporting the transfer of six best practices (BPs) to new primary care settings in Europe. High-quality needs assessment can set the foundation for strong planning and implementation of the CIRCE-JA best practices in these new primary care settings. Methods A needs assessment tool was developed based on the TO-REACH knowledge transfer framework and learnings from a scoping review of international knowledge transfer frameworks. The needs assessment provides structured tool for local sites to self-assess the alignment of the BP with local priorities, readiness of the local site to successfully implement the BP, the availability of necessary leadership, resources and partnerships, and an examination of barriers to local BP implementation and mitigation strategies. Results Twenty-six CIRCE-JA teams completed the needs assessments. Local teams identified the CIRCE-JA BPs were value-adding and aligned with local priorities. Cross-cutting barriers were identified which were common among the six CIRCE-JA best practices across five thematic areas: (1) provider level, (2) intervention level, (3) health system/policy level, (4) partnership and physical resources and (5) information technology. The needs assessment supported primary care sites with a structured approach to planning for transfer of BPs to their local setting that is based on established international best practices and knowledge transfer frameworks including early identification of barriers and mitigating strategies. Conclusions The needs assessment has supported teams with conducting a self-assessment and informing initial planning for adoption of the BP in their local primary care practice. Given the importance of the local needs assessment to high quality implementation of BP the CIRCE-JA experience with local needs assessment may provide learnings for future BP transfer initiatives. Key messages • Local needs assessment supports the transfer of best practices to primary care settings, which is a critical component of service transformation during the post-COVID-19 period. • For successful transfer of best practices in primary care, it is vital to give to examine such as provider capacity, available human resources, health policy, and information technology.

AdaGen: Adaptive Generalized Knowledge Transfer Framework for Sensor-Based Surface Classification for Wheelchair Routing

AdaGen: Adaptive Generalized Knowledge Transfer Framework for Sensor-Based Surface Classification for Wheelchair Routing

An evolutionary knowledge transfer framework for solving workgroup reconfiguration problem in aircraft final assembly

Final assembly is a crucial aspect of aircraft manufacturing. The introduction of reconfigurable workgroups to aircraft final assembly lines (AFALs) significantly increases assembly flexibility and efficiency. However, the number of reconfigurations and their corresponding assignments in the constraint-complex AFAL must be considered jointly during the assignment of reconfigurable workgroups. This presents a great difficulty when optimizing the reconfiguration strategy. In this study, an optimization framework based on evolutionary knowledge transfer (EKT) is proposed to efficiently search the metameric-encoded search space. EKT extracts and transfers valuable knowledge in different reconfiguration sub-problems according to the temporal relations between encodings. Meanwhile, the scope of the EKT is constrained based on the historical individual changes of different sub-problems and different regions of the objective space. The experimental results show that the proposed framework outperforms state-of-the-art algorithms on real-world production data. The behavior and mechanism of the EKT approach during the optimization process are also investigated in detail.

A framework for trust-related knowledge transfer in human–robot interaction

Trustworthy human–robot interaction (HRI) during activities of daily living (ADL) presents an interesting and challenging domain for assistive robots, particularly since methods for estimating the trust level of a human participant towards the assistive robot are still in their infancy. Trust is a multifaced concept which is affected by the interactions between the robot and the human, and depends, among other factors, on the history of the robot’s functionality, the task and the environmental state. In this paper, we are concerned with the challenge of trust transfer, i.e. whether experiences from interactions on a previous collaborative task can be taken into consideration in the trust level inference for a new collaborative task. This has the potential of avoiding re-computing trust levels from scratch for every new situation. The key challenge here is to automatically evaluate the similarity between the original and the novel situation, then adapt the robot’s behaviour to the novel situation using previous experience with various objects and tasks. To achieve this, we measure the semantic similarity between concepts in knowledge graphs (KGs) and adapt the robot’s actions towards a specific user based on personalised interaction histories. These actions are grounded and then verified before execution using a geometric motion planner to generate feasible trajectories in novel situations. This framework has been experimentally tested in human–robot handover tasks in different kitchen scene contexts. We conclude that trust-related knowledge positively influences and improves collaboration in both performance and time aspects.

Zero-shot knowledge transfer for seismic damage diagnosis through multi-channel 1D CNN integrated with autoencoder-based domain adaptation

Accurate and timely structural damage diagnosis is crucial to efficient disaster response and city renovation in post-earthquake events. The scarcity of labeled data hinders the powerful deep learning techniques from in-domain damage detection on target structures. Cross-domain transfer learning has emerged as a captivating strategy through digging knowledge from the abundant source domain to detect the damage in the target domain. However, the heterogeneity among multi-domain structures poses the challenge in seismic damage diagnosis. This study proposes a novel zero-shot knowledge transfer approach for seismic damage diagnosis through multi-channel one-dimensional convolutional neural networks (1D CNN) integrated with deep autoencoder (DAE)-based domain adaptation (DA). The framework consists of three modules, namely, data preprocessor adaptive to seismic vibration signals, DAE-based DA module, and damage diagnosis via multi-channel 1D CNN. The DA module is customized to seamlessly translate the unseen target-domain data to mimic latent representation via a DAE pretrained on the source data, thus realizing rigorous zero-shot learning. Imbalanced data distribution is also considered during the network training and testing. Two representative phases of knowledge transfer are performed to substantiate the knowledge transferability of the proposed method. The first phase involves multi-class damage quantification on two ASCE benchmark models from the simplified model to the refined one, and the second phase conducts binary damage detection on a three-story reinforced frame structure from the finite element numerical model to the laboratory-tested physical model. Both examples show that the proposed method exhibits high prediction accuracy and a lower false negative rate in achieving zero-shot knowledge transfer for cross-domain structural damage diagnosis. With a delicate network design for diverse data, the proposed knowledge transfer framework can be further extended from the present zero-shot approach to the few-shot learning paradigm, thus suggesting a feasible algorithm adaptability and promising engineering applicability.

Using the Theoretical-Experiential Binomial for Educating AI-Literate Students

In the dynamic landscape of modern education, characterized by an increasingly active involvement of IT technologies in learning, the imperative to transfer to university students the skills necessary to integrate Artificial Intelligence (AI) into the process represents an important goal. This paper presents a novel framework for knowledge transfer, diverging from traditional programming language-centric approaches by integrating PSoC 6 microcontroller technology. This framework proposes a cyclical learning cycle encompassing theoretical fundamentals and practical experimentation, fostering AI literacy at the edge. Through a structured combination of theoretical instruction and hands-on experimentation, students develop proficiency in understanding and harnessing AI capabilities. Emphasizing critical thinking, problem-solving, and creativity, this approach equips students with the tools to navigate the complexities of real-world AI applications effectively. By leveraging PSoC 6 as an educational tool, a new generation of individuals is efficiently cultivated with essential AI skills. These individuals are adept at leveraging AI technologies to address societal challenges and drive innovation, thereby contributing to long-term sustainability initiatives. Specific strategies for experiential learning, curriculum recommendations, and the results of knowledge application are presented, aimed at preparing university students to excel in a future where AI will be omnipresent and indispensable.

Multi-task federated learning-based system anomaly detection and multi-classification for microservices architecture

The microservices architecture is extensively utilized in cloud-based application development, characterized by the construction of applications through a series of functionally independent, small, autonomous services. This architectural approach is renowned for its attributes such as high cohesion, availability, low coupling, and exceptional scalability. The detection of runtime system point anomalies in microservices architectures is crucial for enhancing the Quality of Service(QoS). Furthermore, identifying the classes of detected anomalies is critical in practical applications. However, given the highly dynamic nature of microservices systems as a distributed computing architecture, conducting real-time system anomaly detection on distributed independent microservices poses a challenging task. To address these challenges, we propose the System Anomaly Detection and Multi-Classification based on Multi-Task Feature Fusion Federated Learning (SADMC-MT-FF-FL) framework. Initially, we introduce a distributed learning framework based on Multi-task Federated Learning (MT-FL) to construct multi-classification anomaly detection models for each microservice. Secondly, to identify complex system anomaly patterns and features during the runtime of microservices, we develop a feature extractor based on External Attention Mechanism and Multi-channel Residual Structure (EA-MRS). Finally, we design a Local–Global Feature-based Parallel Knowledge Transfer (LGF-PKT) framework, utilizing parallel knowledge transfer to parallelize weight updates for local and global features. To validate the effectiveness of our approach, we conducted comprehensive comparative experiments on the microservices benchmark platforms Sock-Shop and Train-Ticket. The experimental results on anomaly detection for multiclassification systems demonstrate that SADMC-MT-FF-FL outperforms the best baseline method by 28.3% and 27.8% for Macro F1 and Micro F1 on Train-Ticket, and by 8.8% and 8.6% on Sock-Shop, respectively. Additionally, we conducted comparison experiments on three public datasets, SWaT, SMD, and SKAB. The F1 scores were 0.5% higher than those of the centralized methods on SMD, respectively, 6% and 2.8% higher than those of the federated learning based method on SWaT and SKAB. Source codes are available at: https://github.com/icc-lab-xhu1/SADMC-MT-FF-FL.

Knowledge Transfer Framework for PVT Robustness in Analog Integrated Circuits

Knowledge Transfer Framework for PVT Robustness in Analog Integrated Circuits

Collaborative Knowledge Distillation via Multiknowledge Transfer.

Knowledge distillation (KD), as an efficient and effective model compression technique, has received considerable attention in deep learning. The key to its success is about transferring knowledge from a large teacher network to a small student network. However, most existing KD methods consider only one type of knowledge learned from either instance features or relations via a specific distillation strategy, failing to explore the idea of transferring different types of knowledge with different distillation strategies. Moreover, the widely used offline distillation also suffers from a limited learning capacity due to the fixed large-to-small teacher-student architecture. In this article, we devise a collaborative KD via multiknowledge transfer (CKD-MKT) that prompts both self-learning and collaborative learning in a unified framework. Specifically, CKD-MKT utilizes a multiple knowledge transfer framework that assembles self and online distillation strategies to effectively: 1) fuse different kinds of knowledge, which allows multiple students to learn knowledge from both individual instances and instance relations, and 2) guide each other by learning from themselves using collaborative and self-learning. Experiments and ablation studies on six image datasets demonstrate that the proposed CKD-MKT significantly outperforms recent state-of-the-art methods for KD.

Toward an Integrated Framework of Knowledge Transfer: When Non-Profit Organizations Become International

Existing literature lacks an in-depth exploration of Non-Profit Organizations (NPOs) studying transnational religious rituals and scripture dissemination, leaving a notable research gap. The study integrated the SECI model with an institutional distance framework to create a new path analysis, this research aims to reveal interorganizational knowledge transfer dynamics within such religious groups, focusing on Tiandi Teachings (TT) in Taiwan and its Branches of the Japan Diocese (BJD). Employing extended case methodology with participant observation and semi-structured interviews, the study analyzed data using grounded theory and NVivo 12 software. It identified insights into knowledge transfer processes within international religious organizations, elucidating mechanisms for establishing local legitimacy and transferring religious knowledge across borders. This research contributes to understanding how religious organizations globalize, enriches institutional theory, and underscores the importance of integrating institutional concepts with knowledge transfer models. Its findings offer practical benefits to religious leaders, policymakers, and academics.

Multi-Layer Machine Learning Knowledge Transfer Framework for Kinect Skeleton Joints Movements Classification

Multi-Layer Machine Learning Knowledge Transfer Framework for Kinect Skeleton Joints Movements Classification

X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-Modal Knowledge Transfer

The field of 4D point cloud understanding is rapidly developing with the goal of analyzing dynamic 3D point cloud sequences. However, it remains a challenging task due to the sparsity and lack of texture in point clouds. Moreover, the irregularity of point cloud poses a difficulty in aligning temporal information within video sequences. To address these issues, we propose a novel cross-modal knowledge transfer framework, called X4D-SceneFormer. This framework enhances 4D-Scene understanding by transferring texture priors from RGB sequences using a Transformer architecture with temporal relationship mining. Specifically, the framework is designed with a dual-branch architecture, consisting of an 4D point cloud transformer and a Gradient-aware Image Transformer (GIT). The GIT combines visual texture and temporal correlation features to offer rich semantics and dynamics for better point cloud representation. During training, we employ multiple knowledge transfer techniques, including temporal consistency losses and masked self-attention, to strengthen the knowledge transfer between modalities. This leads to enhanced performance during inference using single-modal 4D point cloud inputs. Extensive experiments demonstrate the superior performance of our framework on various 4D point cloud video understanding tasks, including action recognition, action segmentation and semantic segmentation. The results achieve 1st places, i.e., 85.3% (+7.9%) accuracy and 47.3% (+5.0%) mIoU for 4D action segmentation and semantic segmentation, on the HOI4D challenge, outperforming previous state-of-the-art by a large margin. We release the code at https://github.com/jinglinglingling/X4D.

Complementary Knowledge Distillation for Robust and Privacy-Preserving Model Serving in Vertical Federated Learning

Vertical Federated Learning (VFL) enables an active party with labeled data to enhance model performance (utility) by collaborating with multiple passive parties that possess auxiliary features corresponding to the same sample identifiers (IDs). Model serving in VFL is vital for real-world, delay-sensitive applications, and it faces two major challenges: 1) robustness against arbitrarily-aligned data and stragglers; and 2) privacy protection, ensuring minimal label leakage to passive parties. Existing methods fail to transfer knowledge among parties to improve robustness in a privacy-preserving way. In this paper, we introduce a privacy-preserving knowledge transfer framework, Complementary Knowledge Distillation (CKD), designed to enhance the robustness and privacy of multi-party VFL systems. Specifically, we formulate a Complementary Label Coding (CLC) objective to encode only complementary label information of the active party's local model for passive parties to learn. Then, CKD selectively transfers the CLC-encoded complementary knowledge 1) from the passive parties to the active party, and 2) among the passive parties themselves. Experimental results on four real-world datasets demonstrate that CKD outperforms existing approaches in terms of robustness against arbitrarily-aligned data, while also minimizing label privacy leakage.

BEV-CFKT: A LiDAR-camera cross-modality-interaction fusion and knowledge transfer framework with transformer for BEV 3D object detection

The BEV-CFKT proposed in this paper leverages knowledge transfer through transformers for LiDAR-Camera fusion in the Bird’s-Eye-View (BEV) space, aiming to achieve accurate and robust 3D object detection. BEV-CFKT comprises three main components, which include the generation of BEV features from images and point clouds, cross-modality interaction, and hybrid object queries using a monocular detection head. By unifying features from both point clouds and images into the BEV space, we simplify modal interaction, facilitate knowledge transfer, and extract richer structural and semantic information from multimodal data. This effectively enhances the network’s performance. To further improve detection performance, BEV-CFKT incorporates a temporal fusion module. Additionally, a hybrid object queries module based on a monocular detection head accelerates the convergence of our model. We demonstrate the effectiveness of our approach through an extensive set of experiments.

Block-Level Knowledge Transfer for Evolutionary Multitask Optimization.

Evolutionary multitask optimization is an emerging research topic that aims to solve multiple tasks simultaneously. A general challenge in solving multitask optimization problems (MTOPs) is how to effectively transfer common knowledge between/among tasks. However, knowledge transfer in existing algorithms generally has two limitations. First, knowledge is only transferred between the aligned dimensions of different tasks rather than between similar or related dimensions. Second, the knowledge transfer among the related dimensions belonging to the same task is ignored. To overcome these two limitations, this article proposes an interesting and efficient idea that divides individuals into multiple blocks and transfers knowledge at the block-level, called the block-level knowledge transfer (BLKT) framework. BLKT divides the individuals of all the tasks into multiple blocks to obtain a block-based population, where each block corresponds to several consecutive dimensions. Similar blocks coming from either the same task or different tasks are grouped into the same cluster to evolve. In this way, BLKT enables the transfer of knowledge between similar dimensions that are originally either aligned or unaligned or belong to either the same task or different tasks, which is more rational. Extensive experiments conducted on CEC17 and CEC22 MTOP benchmarks, a new and more challenging compositive MTOP test suite, and real-world MTOPs all show that the performance of BLKT-based differential evolution (BLKT-DE) is superior to the compared state-of-the-art algorithms. In addition, another interesting finding is that the BLKT-DE is also promising in solving single-task global optimization problems, achieving competitive performance with some state-of-the-art algorithms.

Privacy-Enhanced and Efficient Federated Knowledge Transfer Framework in IoT

Privacy-Enhanced and Efficient Federated Knowledge Transfer Framework in IoT

A Knowledge Transfer Framework Based on Deep-Reinforcement Learning for Multistage Construction Projects

A Knowledge Transfer Framework Based on Deep-Reinforcement Learning for Multistage Construction Projects

Step Back, Translate, and Extend: An Instructional Framework for Enhancing Knowledge Transfer and Self-Efficacy Across Chemistry Courses

Step Back, Translate, and Extend: An Instructional Framework for Enhancing Knowledge Transfer and Self-Efficacy Across Chemistry Courses

Boosting fine-tuning via Conditional Online Knowledge Transfer

Fine-tuning is an effective technique to enhance network performance in scenarios with limited labeled data. To achieve this, recent methods exploit the knowledge mined in the source model (e.g., feature maps) to construct an extra regularization signal (RS), collaboratively supervising the target model along with target labels. However, these RSs are generated independently from the target information or are generated from the rough assistance of the target information, resulting in biased supervision different from the target task. In this paper, we propose a Conditional Online Knowledge Transfer (COKT) framework that finely utilizes the target information to construct robust and target-related RS. Specifically, we train a target-dominant RS branch that online supervises the target model in a knowledge distillation manner. The target information dominates the RS branch from three aspects: sample-wise conditional attention, residual feature fusion, and target task loss. With such a target-oriented framework, we can effectively exploit target-related prior knowledge of the source model. Extensive experiments demonstrate that COKT significantly outperforms the fine-tuning baselines, especially for dissimilar target tasks and small datasets. Moreover, different from most of the fine-tuning methods that are restricted to the vanilla fine-tuning scenario, COKT can be easily extended to cross-model and multi-model fine-tuning scenarios.