Knowledge Transfer Network Research Articles

ERKT-Net: Implementing Efficient and Robust Knowledge Distillation for Remote Sensing Image Classification

The classification of Remote Sensing Images (RSIs) poses a significant challenge due to the presence of clustered ground objects and noisy backgrounds. While many approaches rely on scaling models to enhance accuracy, the deployment of RSI classifiers often requires substantial computational and storage resources, thus necessitating the use of lightweight algorithms. In this paper, we present an efficient and robust knowledge transfer network named ERKT-Net, which is designed to provide a lightweight yet accurate Convolutional Neural Network (CNN) classifier. This method utilizes innovative yet simple concepts to better accommodate the inherent nature of RSIs, thereby significantly improving the efficiency and robustness of traditional Knowledge Distillation (KD) techniques developed on ImageNet-1K. We evaluated ERKT-Net on three benchmark RSI datasets and found that it demonstrated superior accuracy and a very compact volume compared to 40 other advanced methods published between 2020 and 2023. On the most challenging NWPU45 dataset, ERKT-Net outperformed other KD-based methods with a maximum Overall Accuracy (OA) value of 22.4%. Using the same criterion, it also surpassed the first-ranked multi-model method with a minimum OA value of 0.7 but presented at least an 82% reduction in parameters. Furthermore, ablation experiments indicated that our training approach has significantly improved the efficiency and robustness of classic DA techniques. Notably, it can reduce the time expenditure in the distillation phase by at least 80%, with a slight sacrifice in accuracy. This study confirmed that a logit-based KD technique can be more efficient and effective in developing lightweight yet accurate classifiers, especially when the method is tailored to the inherent characteristics of RSIs.

Business model development for installation of rubber floor mats to support productivity in dairy farming

Purpose. Ribbed smoked sheet (RSS) rubber gradually loses its quality as it is stored after production. Rubber floor mats are recommended as a marketable upstream production in the Thailand rubber industry to help add economic value to low-graded RSS. However, limited attention in research has been paid to business development, particularly on practical models focusing on the demand perspectives. This study helped fill the knowledge gap by analysing factors influencing labour production in dairy farming and proposing a business model based on productivity analysis. The aim of the project was to create a practical business model for cattle rubber floor mats made of low-graded RSS rubber used for nourishing and housing management to promote labour productivity in dairy farming. Methodology / approach. The study used mixed-methods research. A theoretical framework was developed concerning rubber floor mats used in dairy farming, which helped improve the nourishing and housing management of dairy cattle and promote labour productivity. This research adopted Osterwalder’s business model canvas to examine the demand and supply elements of dairy cattle floor mats made of low-graded rubber. It also used the knowledge-based entrepreneurship concept to scrutinise knowledge-based business development. A questionnaire survey was used to collect data required for multiple regression analysis of labour productivity using the Ordinary Least Square (OLS) method. It also used semi-structured interviews to scrutinise the subjective experiences of farmers. A focus group discussion was conducted to investigate academics and rubber promotional authorities concerning potential technology and the possibility of transfer of production technology to promote knowledge-based business development. Results. The empirical evidence presented in this paper confirmed that rubber floor mats could help enhance cattle nourishment, dairy housing management and labour productivity. Regional knowledge transfer network was revealed as a driver of knowledge-based development for assisting local upstream rubber farmers in pursuing eco-innovation in rubber production, aligning with the national initiatives of Industry 4.0 and Bio-Circular-Green economy (BCG) concepts. Originality / scientific novelty. The originality of this paper is that it presents the results of an empirical study using mixed methods for the first time aiming at clarifying practical elements of the supply and demand sides’ components of the Osterwalder`s Business Model Canvas for developing a business model delivered as a knowledge-based business model for the production of dairy cattle floor mats made of low-graded RSS rubber to help guide practical downstream rubber business development. The qualitative results, together with the quantitative analysis of labour productivity, are based on the perceptions of local farmers in Thailand regarding the use of rubber floor mats on the farm; previous studies scientifically prove its positive relation to an increase in dairy cattle nourishing, hence the daily productivity. Practical value / implications. The novel business model canvas could promote the practical cattle floor mats business development. University engagement and public support were included as the critical drivers in this knowledge-based business model. The model implementation could help encourage rubber farmers and smallholders in upstream rubber production to move towards downstream business development and become knowledge-based entrepreneurs in value-added rubber floor mats production. It could also guide the agricultural policymakers to benchmark and evaluate their policy targeting innovative knowledge-based business development for rubber extension.

Multi-scale Knowledge Transfer Vision Transformer for 3D vessel shape segmentation

In order to facilitate the robust and precise 3D vessel shape extraction and quantification from in-vivo Magnetic Resonance Imaging (MRI), this paper presents a novel multi-scale Knowledge Transfer Vision Transformer (i.e., KT-ViT) for 3D vessel shape segmentation. First, it uniquely integrates convolutional embeddings with transformer in a U-net architecture, which simultaneously responds to local receptive fields with convolution layers and global contexts with transformer encoders in a multi-scale fashion. Therefore, it intrinsically enriches local vessel feature and simultaneously promotes global connectivity and continuity for a more accurate and reliable vessel shape segmentation. Furthermore, to enable using relatively low-resolution (LR) images to segment fine scale vessel shapes, a novel knowledge transfer network is designed to explore the inter-dependencies of data and automatically transfer the knowledge gained from high-resolution (HR) data to the low-resolution handling network at multiple levels, including the multi-scale feature levels and the decision level, through an integration of multi-level loss functions. The modeling capability of fine-scale vessel shape data distribution, possessed by the HR image transformer network, can be transferred to the LR image transformer to enhance its knowledge for fine vessel shape segmentation. Extensive experimental results on public image datasets have demonstrated that our method outperforms all other state-of-the-art deep learning methods.

MKTN: Adversarial-Based Multifarious Knowledge Transfer Network from Complementary Teachers

With the demands for light deep networks models in various front-end devices, network compression has attracted increasing interest for reducing model sizes yet without sacrificing much model accuracy. This paper presents a multifarious knowledge transfer network (MKTN) that aims to produce a compact yet powerful student network from two complementary teacher networks. Instead of learning homogeneous features, the idea is to pre-train one teacher to capture generative and low-level image features under a reconstruction objective, and another teacher to capture discriminative and task-specific features under the same objective as the student network. During knowledge transfer, the student learns multifarious and complementary knowledge from the two teacher networks under the guidance of the proposed adversarial loss and feature loss respectively. Experimental results indicate that the proposed training losses can effectively guide the student to learn spatial-level and pixel-level information as distilled from teacher networks. On the other hand, our study over a number of widely used datasets shows that transferring multifarious features from complementary teachers equipped with different types of knowledge helps to teach a compact yet powerful student effectively.

A Unified Knowledge Transfer Network for Generalized Category Discovery

Generalized Category Discovery (GCD) aims to recognize both known and novel categories in an unlabeled dataset by leveraging another labeled dataset with only known categories. Without considering knowledge transfer from known to novel categories, current methods usually perform poorly on novel categories due to the lack of corresponding supervision. To mitigate this issue, we propose a unified Knowledge Transfer Network (KTN), which solves two obstacles to knowledge transfer in GCD. First, the mixture of known and novel categories in unlabeled data makes it difficult to identify transfer candidates (i.e., samples with novel categories). For this, we propose an entropy-based method that leverages knowledge in the pre-trained classifier to differentiate known and novel categories without requiring extra data or parameters. Second, the lack of prior knowledge of novel categories presents challenges in quantifying semantic relationships between categories to decide the transfer weights. For this, we model different categories with prototypes and treat their similarities as transfer weights to measure the semantic similarities between categories. On the basis of two treatments, we transfer knowledge from known to novel categories by conducting pre-adjustment of logits and post-adjustment of labels for transfer candidates based on the transfer weights between different categories. With the weighted adjustment, KTN can generate more accurate pseudo-labels for unlabeled data, which helps to learn more discriminative features and boost model performance on novel categories. Extensive experiments show that our method outperforms state-of-the-art models on all evaluation metrics across multiple benchmark datasets. Furthermore, different from previous clustering-based methods that can only work offline with abundant data, KTN can be deployed online conveniently with faster inference speed. Code and data are available at https://github.com/yibai-shi/KTN.

CKTN: Commonsense knowledge transfer network for human activity understanding

In most human activity understanding research, various deep learning networks are well designed to fit the character distribution of different human activities in images. After being fed with enough training data, the networks can perceive related patterns passively. Such abilities originate from the stimulation of training data, but these networks lack the ability to perform subjective judgment. When lacking sufficient training data, networks have difficulty perceiving valuable information, which leads to poor performance. However, human beings can infer human activities in complex situations through subjective judgment with their knowledge. On this point, we propose the commonsense knowledge transfer network (CKTN), which is empowered with human cognitive mechanisms and is able to understand human activities by using prior commonsense knowledge. More specifically, we study the structural combination rules of human bodies and explore the co-occurrence relationship between different body parts and objects. Then, we construct the part action cooccurrence probability matrix and human skeleton knowledge matrix, which work as prior knowledge, guiding our network to absorb and reason what the human activities in images are. Compared with traditional deep learning methods, our network realizes the unification of both subjective cognition and passive perception abilities. Experiments show the effectiveness of this method.

Research on the evolution and determinants of knowledge transfer network in patent-intensive industries

PurposeThis study aims to investigate the structural characteristics, spatial evolution paths and internal driving mechanisms of the knowledge transfer (KT) network in China’s patent-intensive industries (PIIs). The authors' goal is to provide valuable insights to inform policy-making that fosters the development of relevant industries. The authors also aim to offer a fresh perspective for future spatiotemporal studies on industrial KT and innovation networks.Design/methodology/approachIn this study, the authors analyze the patent transfer (PT) data of listed companies in China’s information and communication technology (ICT) industry, spanning from 2010 to 2021. The authors use social network analysis and the quadratic assignment procedure (QAP) method to explore the problem of China’s PIIs KT from the perspectives of technical characteristics evolution, network and spatial evolution and internal driving mechanisms.FindingsThe results indicate that the knowledge fields involved in the PT of China’s ICT industry primarily focus on digital information transmission technology. From 2010 to 2021, the scale of the ICT industry’s KT network expanded rapidly. However, the polarization of industrial knowledge distribution is becoming more serious. QAP regression analysis shows that economic proximity and geographical proximity do not affect KT activities. The similarity of knowledge application capacity, innovation capacity and technology demand categories in various regions has a certain degree of impact on KT in the ICT industry.Originality/valueThe current research on PIIs mainly focuses on measuring economic contributions and innovation efficiency, but less on KT in PIIs. This study explores KT in PIIs from the perspectives of technological characteristics, network and spatial evolution. The authors propose a theoretical framework to understand the internal driving mechanisms of industrial KT networks.

Knowledge transfer to aid social coding: The case of Stack Overflow

Focused online question and answer (Q&A) communities aid social coding. Despite the growing importance of social coding, knowledge transfer in this context remains under-researched. Our primary objective is to understand the knowledge transfer process in this context. We conceptualize knowledge transfer as a process that is impacted by the prior knowledge transfer interactions (network) among participants and is augmented by gamification. We argue that social capital resulting from prior knowledge transfer network interactions impact answer quality. Moreover, we also argue that the relationship between social capital and answer quality is moderated by the complexity of the knowledge transferred. Hence, our models draw from multiple related research streams: online communities, knowledge transfer, social capital, and gamification. These models are empirically tested using data from Stack Overflow (SO), a popular online Q&A community that aids social coding. Our results help to understand knowledge transfer in Q&A communities that aid social coding. Moreover, our results have implications for research on other types of Q&A communities and can inform development of platforms to support online communities.

Knowledge-Guided Semantic Transfer Network for Few-Shot Image Recognition.

Deep learning-based models have been shown to outperform human beings in many computer vision tasks with massive available labeled training data in learning. However, humans have an amazing ability to easily recognize images of novel categories by browsing only a few examples of these categories. In this case, few-shot learning comes into being to make machines learn from extremely limited labeled examples. One possible reason why human beings can well learn novel concepts quickly and efficiently is that they have sufficient visual and semantic prior knowledge. Toward this end, this work proposes a novel knowledge-guided semantic transfer network (KSTNet) for few-shot image recognition from a supplementary perspective by introducing auxiliary prior knowledge. The proposed network jointly incorporates vision inferring, knowledge transferring, and classifier learning into one unified framework for optimal compatibility. A category-guided visual learning module is developed in which a visual classifier is learned based on the feature extractor along with the cosine similarity and contrastive loss optimization. To fully explore prior knowledge of category correlations, a knowledge transfer network is then developed to propagate knowledge information among all categories to learn the semantic-visual mapping, thus inferring a knowledge-based classifier for novel categories from base categories. Finally, we design an adaptive fusion scheme to infer the desired classifiers by effectively integrating the above knowledge and visual information. Extensive experiments are conducted on two widely used Mini-ImageNet and Tiered-ImageNet benchmarks to validate the effectiveness of KSTNet. Compared with the state of the art, the results show that the proposed method achieves favorable performance with minimal bells and whistles, especially in the case of one-shot learning.

PAMK: Prototype Augmented Multi-Teacher Knowledge Transfer Network for Continual Zero-Shot Learning.

Continual zero-shot learning (CZSL) aims to develop a model that accumulates historical knowledge to recognize unseen tasks, while eliminating catastrophic forgetting for seen tasks when learning new tasks. However, existing CZSL methods, while mitigating catastrophic forgetting for old tasks, often lead to negative transfer problem for new tasks by over-focusing on accumulating old knowledge and neglecting the plasticity of the model for learning new tasks. To tackle these problems, we propose PAMK, a prototype augmented multi-teacher knowledge transfer network that strikes a trade-off between recognition stability for old tasks and generalization plasticity for new tasks. PAMK consists of a prototype augmented contrastive generation (PACG) module and a multi-teacher knowledge transfer (MKT) module. To reduce the cumulative semantic decay of the class representation embedding and mitigate catastrophic forgetting, we propose a continual prototype augmentation strategy based on relevance scores in PACG. Furthermore, by introducing the prototype augmented semantic-visual contrastive loss, PACG promotes intra-class compactness for all classes across all tasks. MKT effectively accumulates semantic knowledge learned from old tasks to recognize new tasks via the proposed multi-teacher knowledge transfer, eliminating the negative transfer problem. Extensive experiments on various CZSL settings demonstrate the superior performance of PAMK compared to state-of-the-art methods. In particular, in the practical task-free CZSL setting, PAMK achieves impressive gains of 3.28%, 3.09% and 3.71% in mean harmonic accuracy on the CUB, AWA1, and AWA2 datasets, respectively.

Proposed synergies between oceanography and metrology

Accurate and traceable measurements are required to understand ocean processes, to address pressing societal challenges, such as climate change and to sustainably manage marine resources. Although scientific and engineering research has resulted in advanced methods to measure Essential Ocean Variables (EOVs) there is a need for cross comparison of the techniques and traceability to recognized standards. Metrological laboratories are experienced in accredited methods and assessment of methodology. An EU INFRAIA-02-2020: Integrating Activities for Starting Communities project MINKE (Metrology for Integrated marine maNagement and Knowledge-transfer nEtwork https://minke.eu) brings European marine science and metrology Research Infrastructures together to identify synergies and create an innovative approach to Quality Assurance of oceanographic data. Quality depends both on the accuracy (that can be provided through the metrology component) and the completeness of the data sets. The collaboration between different Marine Research Infrastructures (RIs) places a fundamental role on assuring the completeness of the datasets, particularly at global scales. The MINKE project encourages enhancement through collaboration of national metrology laboratories and the oceanographic community. Metrological assessment of the accuracy and uncertainties within multidisciplinary ocean observations will provide data that are key to delivering policy information. Objectives across all the RIs are to facilitate ocean observation and build wider synergies. MINKE will investigate these synergies, then introduce metrology to the core of various EOV measurements. Currently the marine RIs cover laboratory and field operations, from the surface seafloor, coastal waters to deep sea, fixed ocean stations to ship and autonomous vehicle operations to ships of opportunity, and flux stations focusing on carbonate system variables. The nexus of these operations is the focal point for coordinated improvement of ocean observing methods. Measurement intercomparisons, traceability and uncertainty assessments should be at the core of the scientific observations. Specifically, MINKE will work with RIs and Metrology Institutes to improve the quality of dissolved oxygen, carbonate system, chlorophyll-fluorescence, ocean sound and current meter measurements, through access to metrology laboratories, Transnational Access and intercomparison studies across existing marine consortia and RIs. MINKE will also promote the development of absolute salinity observation, and improvements in marine litter measurements.

PKAMNet: A Transmission Line Insulator Parallel- Gap Fault Detection Network Based on Prior Knowledge Transfer and Attention Mechanism

Insulator parallel gaps in a transmission line protect the insulator from arc burn and provide protection from lightning. Detection of insulator parallel-gap faults based on aerial inspection images of transmission lines may be hindered by various factors, such as complex background, scale distortion, and occlusion. In this study, we propose a prior knowledge transfer and attention mechanism network (PKAMNet) to detect faulty insulator parallel gaps. First, the capability of PKAMNet to learn the features of different types of parallel-gap faults is improved by constructing a prior knowledge transfer model based on visual saliency. Owing to the difficulty of effectively expressing the features of the fault target due to the deviation of the shooting angle and complex background of the inspection image, we enhance the feature expression ability of the fault area by embedding the coordinate attention block. Additionally, the parameters of the detection network are simplified, and the modules are optimized to improve detection efficiency. Comparison results on 3-year aerial inspection videos of 500 kV high-voltage transmission lines show that PKAMNet has a high detection accuracy and can ameliorate the phenomenon of missed and erroneous detection caused by insufficient expression of parallel-gap faults.

Drivers of green economy in an emerging market: Generic and sector-specific insights

The study examines the drivers of the green economy generally and with a focus on sectors regarded as environmental-sensitive hotspots like waste, agriculture, forestry, energy, and transport sectors. The study was grounded on both the natural resource-based view and institutional theories. Creswell's (2014) thematic analysis and the graphic framework of Miles and Huberman (1984, 1994) were used to address the respective objectives of the study. A total of 10 interviewees were selected purposively based on their expertise and the institutions in question who provided responses that were analysed. The results show that the three generic drivers of the green economy include pollution prevention revealed by the impact of climate change, and environmental degradation cost, environmental stewardship portrayed by inefficient resource utilization and global demand for change, as well as, sustainable development manifesting from a futuristic drive, a focus on people-planet-profit, and rejection of environmentally unfriendly development models. The findings further reveal the differences and interrelatedness of the drivers of the green economy within the 5 hotspot sectors under investigation. The results of the study reveal that there are different and sector-specific strategies that policymakers and practitioners have to apply in their quest to drive a green economy at the central government, local government and grassroots levels. The study proposes the development of a comprehensive national green economy policy and also the promotion of peer-to-peer knowledge transfer networks structured as Communities of Practice. This study is one of the few that explores the concept of the green economy within the macroeconomic level and is novel in attaching theoretical lenses.

Research on the Internet Capital and Its Promotion of Rural Teachers in Western China from the Perspective of New Digital Divide

Teachers are the key to education formalization. The study found that the Internet capital of rural teachers in western China is obviously insufficient. Compared with teachers in developed areas, there are significant differences in network facilities, information technology knowledge, online learning and information literacy. The backwardness of network facilities and the lack of information technology knowledge are the important reasons for the lack of Internet capital of rural teachers in western China. Therefore, it is necessary to improve the Internet capital of rural teachers in western China from the aspects of network facilities construction, network knowledge transfer, online learning support and online learning reflection. The proposal of these measures has certain reference value for promoting the improvement of rural teachers’ Internet ability in western China and narrowing the gap with teachers in developed areas.

Domain adaptive object detection with model-agnostic knowledge transferring

The development of deep learning techniques has greatly benefited CNN-based object detectors, leading to unprecedented progress in recent years. However, the distribution variance between training and testing domains causes significant performance degradation. Labeling data for new scenarios is costly and time-consuming, so most existing domain adaptation methods perform feature alignment through adversarial training. While this can improve the accuracy of detectors in unlabeled target domains, the unconstrained domain alignment also negatively transfers the feature distribution, which compromises the recognition ability of the model. To address this problem, we propose the Knowledge Transfer Network (KTNet), which consists of object intrinsic knowledge mining and category relational knowledge constraint modules. Specifically, a binary classifier shared by the source and target domains is designed to extract common attribute knowledge of objects, which can align foreground and background features from different data domains adaptively. Then, we construct relational knowledge graphs to explicitly constrain the category correlations in the source, target, and cross-domain settings. These two modules guide the detector to learn object-related and domain-invariant representations, enabling the proposed KTNet to perform well in four commonly-used cross-domain scenarios. Furthermore, the ablation experiments show that our method is scalable to more complex backbone networks and different detection architectures.

Addressing recruitment challenges in food and drink

Addressing recruitment challenges in food and drink

Networking to reduce microbial risk in foods

Networking to reduce microbial risk in foods

KTN: Knowledge Transfer Network for Learning Multiperson 2D-3D Correspondences

Human densepose estimation, aiming at establishing dense correspondences between 2D pixels of human body and 3D human body template, is a key technique in enabling machines to have an understanding of people in images. It still poses several challenges due to practical scenarios where real-world scenes are complex and only partial annotations are available, leading to incompelete or false estimations. In this work, we present a novel framework to detect the densepose of multiple people in an image. The proposed method, which we refer to Knowledge Transfer Network (KTN), tackles two main problems: 1) how to refine image representation for alleviating incomplete estimations, and 2) how to reduce false estimation caused by the low-quality training labels (i.e., limited annotations and class-imbalance labels). Unlike existing works directly propagating the pyramidal features of regions for densepose estimation, the KTN uses a refinement of pyramidal representation, where it simultaneously maintains feature resolution and suppresses background pixels, and this strategy results in a substantial increase in accuracy. Moreover, the KTN enhances the ability of 3D based body parsing with external knowledges, where it casts 2D based body parsers trained from sufficient annotations as a 3D based body parser through a structural body knowledge graph. In this way, it significantly reduces the adverse effects caused by the low-quality annotations. The effectiveness of KTN is demonstrated by its superior performance to the state-of-the-art methods on DensePose-COCO dataset. Extensive ablation studies and experimental results on representative tasks (e.g., human body segmentation, human part segmentation and keypoints detection) and two popular densepose estimation pipelines (i.e., RCNN and fully-convolutional frameworks), further indicate the generalizability of the proposed method.

A Summary of the Seventh Annual Review of the BCPC's Pests and Beneficials Group

The purpose of the Pests and Beneficials Group is to provide a forum for discussion of relevant research results and new legislation and impact on integrated pest management. The objective of this meeting was to understand better how to deliver practical IPM solutions to farmers. This starts with the policy context, followed by research that can be done and finally the delivery of tools and information to famers and the successful implementation of IPM in the field. The meeting also sought to highlight the need to integrate the thinking about the implications of combining IPM management decisions for insect, disease and weed pests and the trade-offs that may have to be made as a result. Cathryn Lambourne, Knowledge Transfer Manager from Agrifood section of the KTN (Knowledge Transfer Network), UKRI provided an account of the work of the KTN, particularly in the Agrifood sector, providing support for those seeking information, funding, and connections within the industry followed by a short outline of the principles of IPM.

University Outreach in the Triple Helix Model of Collaboration for Entrepreneurial Development

This paper presents an empirical study of the role of university outreach in the triple helix collaboration model adopted for the development of knowledge-based entrepreneurship in a community enterprise in Thailand. The triple helix model is a hybrid collaboration of university-industry-government networks for knowledge transfer and innovation development. This research project was a case study using a qualitative ground theory approach, semi-structured interviews, and documentary analysis for its data collection tools. This study investigates how a university outreach project performed by academic members of a regional university from the Northeast region of Thailand plays a role in promoting a local community enterprise to become knowledge-based entrepreneurship. The study chose a local organic rice farming community enterprise as its case study. The study discovered that the primary role of the university outreach is a source of creative knowledge required for innovative development. The knowledge delivery is via academic service financially supported by funding from a government agency. The empirical evidence showed that the participating academics delivered their knowledge service with sufficient entrepreneurial skills. Moreover, the academic service that promoted the enterprise’s business innovation significantly added economic value to its production. In addition to identifying success factors, this paper also discusses a challenge for further development for the university outreach sustaining its contribution to the triple helix collaboration by constructing systematic support from the university and promoting academic integrity for interdisciplinary research-based innovation development.