Teacher Student Learning Research Articles

Teacher-student learning of generative adversarial network-guided diffractive neural networks for visual tracking and imaging

Teacher-student learning of generative adversarial network-guided diffractive neural networks for visual tracking and imaging

Research and Model Library Construction in Teacher-Student Learning Architectures for Knowledge Transfer

This paper summarizes and replicates multiple classical and cutting-edge knowledge transfer methods, including Factor Transfer (FT), Knowledge Distillation (KD), Deep Mutual Learning (DML), Contrastive Representation Distillation (CRD), and Born-Again Self-Distillation (BSS). Additionally, we studied three advanced knowledge transfer methods: Relational Knowledge Distillation (RKD), Similarity-Preserving (SP), and Attention-based Feature Distillation (AFD), successfully replicating an optimized version of KD, namely RKD. Based on these methods, a flexible model library was constructed in Pycharm, allowing the quick integration of multiple knowledge transfer strategies. The experimental results are visualized through a user-friendly interface, enabling intuitive comparisons of model training speed and performance across different methods. This research provides valuable insights into the challenge of building a reusable framework that efficiently integrates various knowledge transfer strategies into deep neural networks.

Knowledge Distillation and Student-Teacher Learning for Weed Detection in Turf

Abstract Machine vision-based herbicide applications relying on object detection or image classification deep convolutional neural network (DCNN) demand high memory and computational resources, resulting in lengthy inference times. To tackle these challenges, this study assessed the effectiveness of three teacher models, each trained on datasets of varying sizes, including D-20k (comprising 10,000 true positive and true negative images) and D-10k (comprising 5,000 true positive and true negative images). Additionally, knowledge distillation was performed on their corresponding student models across a range of temperature settings. After the process of student-teacher learning, the parameters of all student models were reduced. ResNet18 not only achieved higher accuracy (ACC≥0.989) but also maintained higher frames per second (FPS≥742.9) under its optimal temperature condition (T=1). Overall, the results suggest that employing knowledge distillation on the machine vision models enabled accurate and reliable weed detection in turf while reducing the need for extensive computational resources, thereby facilitating real-time weed detection and contributing to the development of smart, machine vision-based sprayers.

Foundational Segmentation Models and Clinical Data Mining Enable Accurate Computer Vision for Lung Cancer.

This study aims to assess the effectiveness of integrating Segment Anything Model (SAM) and its variant MedSAM into the automated mining, object detection, and segmentation (MODS) methodology for developing robust lung cancer detection and segmentation models without post hoc labeling of training images. In a retrospective analysis, 10,000 chest computed tomography scans from patients with lung cancer were mined. Line measurement annotations were converted to bounding boxes, excluding boxes < 1cm or > 7cm. The You Only Look Once object detection architecture was used for teacher-student learning to label unannotated lesions on the training images. Subsequently, a final tumor detection model was trained and employed with SAM and MedSAM for tumor segmentation. Model performance was assessed on a manually annotated test dataset, with additional evaluations conducted on an external lung cancer dataset before and after detection model fine-tuning. Bootstrap resampling was used to calculate 95% confidence intervals. Data mining yielded 10,789 line annotations, resulting in 5403 training boxes. The baseline detection model achieved an internal F1 score of 0.847, improving to 0.860 after self-labeling. Tumor segmentation using the final detection model attained internal Dice similarity coefficients (DSCs) of 0.842 (SAM) and 0.822 (MedSAM). After fine-tuning, external validation showed an F1 of 0.832 and DSCs of 0.802 (SAM) and 0.804 (MedSAM). Integrating foundational segmentation models into the MODS framework results in high-performing lung cancer detection and segmentation models using only mined clinical data. Both SAM and MedSAM hold promise as foundational segmentation models for radiology images.

English as a Second Language teacher training practicum – exploring Polish university students’ experiences and perceptions – in search of supportive strategies

This study examines the experiences of Polish university students during their English as a Second Language (ESL) Teacher Training Practicum. By examining the challenges and transformations they undergo, we aim to understand their journey towards becoming educators. Beyond students’ reflections, we explore strategies to improve their practicum experiences. Using the Inventory of Experiences and Perceptions at Teaching Practice (IEPTP), we evaluate key aspects: learning and professional growth, socialisation in professional settings, socio-emotional facets, support and supervision, and vocational development. The research highlights shifts in perceptions, emphasising areas like vocational and professional development. It suggests strategies to enhance learning and support for student teachers, emphasising the crucial role of supervisors. The findings guide institutions in refining teacher training and ensuring students become adept ESL educators.

E-formative feedback in emergency remote learning: First-year student teachers' experiences and challenges

This study examined the role of e-formative feedback in emergency remote teaching and learning for first-year student teachers. Numerous scholars have acknowledged the pedagogical value of formative feedback in teacher education. However, there is a pressing need for significant improvement in the quality and accessibility of e-formative feedback provided to student teachers in online learning environments. This study explored the nature of e-formative feedback, technological challenges, and pedagogical practices employed by teacher educators when implementing formative assessment in emergency remote learning contexts. The study is underpinned by Interactive Tutoring Feedback model as the theoretical framework. We utilised the interpretive paradigm and a qualitative approach, specifically employing a case study methodology as a means of inquiry. The empirical investigation involved 20 first-year student teachers selected purposively from a large public university in South Africa. Data were collected through semi-structured interviews and analysis of learning management system logs, with thematic analysis employed to analyse the data. The findings revealed that meaningful e-formative feedback could have been enhanced by addressing contextual factors such as limited internet connectivity and a lack of familiarity with online learning platforms. While student teachers valued the quality of e-formative feedback, they expressed concern regarding their ability to consistently access and engage with feedback to improve their learning in the isolated online environment. This paper recommends providing both student teachers and teacher educators with the necessary technological support and training to effectively utilise e-formative feedback and enhance student teachers' academic achievement and professional development during emergency remote learning.

Development Of Physics Teaching Materials On The Subject Of Light Interference Based On Discovery-inquiry For Senior High School Students

Abstract Efforts to improve student’s concept understanding can be used teaching materials that emphasize process skills. Researchers tried to make teaching materials using discovery-inquiry learning which is learning that emphasizes investigation (process skills) carried out by students. The purpose of this study was to determine the feasibility of Physics Teaching Materials and Learning Guides on the subject of Light Interference in High School. This research is a development research that uses the first five steps of the Borg and Gall method, namely according to Borg and Gall. The results showed that the feasibility of content, language feasibility and content presentation of student teaching materials and teacher learning guides (teacher teaching materials) based on discovery-inquiry were categorized as feasible.

A sound event detection support system for smart home based on “two-to-one” teacher–student learning

Sound event detection (SED) is a core technology in smart home projects that rely on detected sound events to trigger specific actions. SED systems face two major challenges: high labeling costs and complex acoustic environments. To reduce labeling costs, some semi-supervised systems extract both global and local features for classification. However, these methods treat global and local features equally, not accounting for their varying importance when recognizing different types of sound events. Furthermore, to address complex acoustic environments, some studies use multitask learning frameworks to introduce SED-related tasks as auxiliaries to improve detection performance. However, these methods fail to align tasks within the framework, leading to conflicting outputs that may limit system performance. To address these issues, in this paper we propose a “two-to-one” teacher-student learning based semi-supervised SED system. This system employs a gating mechanism to selectively enhance global and local features, improving adaptability to different types of sound events, and incorporates a cross-task alignment module to interact SED with related tasks, reducing the risk of performance degradation caused by conflicting outputs. Experimental results on two datasets demonstrate that our system achieves the best performance in all metrics, with EB-F1 scores of 48.1 % and 64.7 %, representing improvements of 15.3 % and 10.6 % over the baseline ConformerSED system, respectively. Our work offers an effective SED solution for smart home projects by providing a semi-supervised SED system that performs well while reducing labeling costs.

The Impact of Artificial Intelligence in Educational System

The way we teach and learn could be completely transformed by artificial intelligence (AI), which could make the process more efficient, personalized, and interesting. Artificial intelligence (AI) in education is the application of AI technologies, like natural language processing and machine learning, to improve the educational process. Through the use of algorithms, teachers are able to customize learning for each student by analyzing data, finding trends, and making predictions. Artificial intelligence (AI) and its potential impact on education have gained widespread attention because of ChatGPT's impressive performance on standardized academic assessments. For the development and implementation of AI-driven technologies in schools, colleges, and universities to be sustainable, a thorough knowledge of their effects on the current educational system is needed. The application of AI in education has a lot of potential advantages. One of the biggest benefits of AI in education is personalized learning, which allows students to learn at their own pace and in a fashion that best fits their learning preferences. This can improve student results. Chatbots, automated grading and evaluation, and intelligent tutoring systems can boost productivity, free up teachers' time, and deliver more precise and consistent feedback. Some of the challenges that need to be resolved are potential bias, cost, lack of confidence, and privacy and security concerns. AI can improve data analysis, empowering teachers to make facts-based decisions. There are some impacts of artificial intelligence in educational system that are covered in this review. These are the applications of artificial intelligence in automated assessment, intelligent tutoring systems, personalized learning, and collaborative teacher-student learning.

Single/Multi-Source Black-Box Domain Adaption for Sensor Time Series Data.

Unsupervised domain adaption (UDA), which transfers knowledge from a labeled source domain to an unlabeled target domain, has attracted tremendous attention in many machine learning applications. Recently, there have been attempts to apply domain adaption for sensor time series data, such as human activity recognition and gesture recognition. However, existing methods suffer from some drawbacks that hinder further performance improvement. They often require access to source data or source models during training, which is unavailable in some fields because of privacy protection and storage limit. Typically, the source domains may only provide an application programming interface (API) for the target domain to call. On the other hand, current UDA methods have not considered the temporal consistency and low-signal-to-noise ratio (SNR) of sensor time series. To address the challenges, this article presents a black-box domain adaption framework for sensor time series data (B2TSDA). First, we propose a single/multi-source teacher-student learning framework to distill the knowledge from the source domains to a customized target model. Then we design a new temporal consistency loss by combining an adaptive mask method and dynamic threshold method to maintain consistent temporal information and balance the learning difficulties of different classes. For the multisource black-box domain adaption, we further propose a Shapley-enhanced method to determine the contribution of each source domain. Experimental results on both single-source and multisource domain adaption show that our framework has superior performance compared to other black-box UDA methods.

Empowering Educational Leaders: Nurturing Strong Teaching and Learning Environments in Kenyan Schools.

Purpose: Kenya’s paradigm shifts in curriculum from content based to competence-based curriculum is geared towards improving the quality of education to meet the global demands. This prompts the need to build a firm foundation from early childhood education through appropriate classroom learning experiences and good leadership. This study explored educational leaders' practices in their management of primary schools in Kenya. Methodology: This article draws evidence from the Longitudinal Study which was designed among other things to establish the best practices that teacher educators can adopt to inform the professional learning of Student Teachers to become effective champion teachers, and leaders. The study adopted a mixed method approach with a sample size of 29, comprising 4 education officers, 25 head teachers in Mombasa and Kilifi counties in Kenya. Findings: The findings revealed that most education leaders have embraced transformative leadership styles and policies to motivate their subordinates through action research, portfolio, mentorship, coaching, collaboration, and networking. Findings revealed further that mentorship is key in educational leadership and thus should be regarded as a vital aspect in teacher education and in professional learning. Data also indicated that leadership styles differ in relation to the context. Unique Contribution to Theory, Policy and Practice: The leader’s role is deeply involved with setting the school’s direction; thus, teacher leadership preparation is essential for successful implementation of education processes and programs. Hence, we recommend that school leaders should be adequately prepared to face the tasks and the accompanying contextual challenges they face once assigned the responsibilities to lead institutions.

Machine Learned Classification of Ligand Intrinsic Activities at Human μ-Opioid Receptor.

Opioids are small-molecule agonists of μ-opioid receptor (μOR), while reversal agents such as naloxone are antagonists of μOR. Here, we developed machine learning (ML) models to classify the intrinsic activities of ligands at the human μOR based on the SMILES strings and two-dimensional molecular descriptors. We first manually curated a database of 983 small molecules with measured Emax values at the human μOR. Analysis of the chemical space allowed identification of dominant scaffolds and structurally similar agonists and antagonists. Decision tree models and directed message passing neural networks (MPNNs) were then trained to classify agonistic and antagonistic ligands. The hold-out test AUCs (areas under the receiver operator curves) of the extra-tree (ET) and MPNN models are 91.5 ± 3.9% and 91.8 ± 4.4%, respectively. To overcome the challenge of a small data set, a student-teacher learning method called tritraining with disagreement was tested using an unlabeled data set comprised of 15,816 ligands of human, mouse, and rat μOR, κOR, and δOR. We found that the tritraining scheme was able to increase the hold-out AUC of MPNN models to as high as 95.7%. Our work demonstrates the feasibility of developing ML models to accurately predict the intrinsic activities of μOR ligands, even with limited data. We envisage potential applications of these models in evaluating uncharacterized substances for public safety risks and discovering new therapeutic agents to counteract opioid overdoses.

An innovative predictive method based on supervised teacher-student learning for forecasting limited structural responses of long-span bridges from satellite images

The prediction of structural responses of large-scale civil structures offers an effective and efficient alternative to field measurement. Recently, spaceborne remote sensing technology has been increasingly adopted to monitor complex and large bridges in long-term monitoring programs by extracting structural displacements from synthetic aperture radar images. To overcome certain limitations of such images, machine learning-aided prediction of bridge displacement responses has become the best solution. For this purpose, it is necessary to consider main environmental and operational factors that influence bridge responses. Accordingly, various contact-based sensors are installed on bridges to measure influential predictors, including ambient temperature, relative humidity, wind characteristics, and traffic volume. However, in some cases, the installation of these sensors or the preparation of their data may be impractical, leading to unmeasured and unknown conditions that can affect structural responses and are not incorporated into the prediction process. To circumvent poor performance and erroneous prediction outputs, this paper proposes an innovative This method comprises two parts: an elaborate regression model acting as a 'teacher' and a simpler model serving as a 'student.' Initially, measured predictor and response data are fed into the teacher model, which is developed from a long-short-term memory neural network to reconstruct the response data. Subsequently, the residual data between the measured and reconstructed responses is computed to serve as a new predictor. This new predictor, along with the original measured response data, is then used to train a student model developed from a single-hidden-layer feedforward neural network, which finally predicts unseen response data. The effectiveness and reliability of the proposed predictive method are validated using limited information from long-span bridges. Results indicate that this method successfully predicts bridge responses using ambient temperature as the sole main predictor.

Machine Learned Classification of Ligand Intrinsic Activities at Human μ-Opioid Receptor.

Opioids are small-molecule agonists of μ-opioid receptor (μOR), while reversal agents such as naloxone are antagonists of μOR. Here we developed machine learning (ML) models to classify the intrinsic activities of ligands at the human μOR based on the SMILE strings and two-dimensional molecular descriptors. We first manually curated a database of 983 small molecules with measured E max values at the human μOR. Analysis of the chemical space allowed identification of dominant scaffolds and structurally similar agonists and antagonists. Decision tree models and directed message passing neural networks (MPNNs) were then trained to classify agonistic and antagonistic ligands. The hold-out test AUCs (areas under the receiver operator curves) of the extra-tree (ET) and MPNN models are 91.5±3.9% and 91.8± 4.4%, respectively. To overcome the challenge of small dataset, a student-teacher learning method called tri-training with disagreement was tested using an unlabeled dataset comprised of 15,816 ligands of human, mouse, or rat μOR, κOR, or δOR. We found that the tri-training scheme was able to increase the hold-out AUC of MPNN to as high as 95.7%. Our work demonstrates the feasibility of developing ML models to accurately predict the intrinsic activities of μOR ligands, even with limited data. We envisage potential applications of these models in evaluating uncharacterized substances for public safety risks and discovering new therapeutic agents to counteract opioid overdoses.

User-agnostic adaptation of human locomotion intent: Leveraging Teacher-Student-Learning and ensemble modeling

Wearable robot control in complicated contexts requires an understanding of human locomotion intent and behaviors. However, signals from human–robot interfaces are typically reliant on the user, leading to poor performance of the model trained on training user when applied to new end-point users. This study aims to address this problem by developing a novel Teacher-Student-Learning (TSL) approach using Heterogeneous Ensemble Hypotheses (HEH) to achieve unsupervised user-agnostic adaptation. The motivation behind using the Teacher-Student-Learning architecture is to leverage the diverse features extracted by HEH while maintaining computational efficiency. HEH reduces the difference between labeled training users and unlabeled end-point users by incorporating multiple diverse feature generators to extract a wide range of features, thereby increasing classification accuracy and reducing variance. However, this approach sacrifices efficiency due to the ensemble nature of HEH. To address this trade-off, the knowledge from HEH is distilled into a single network using TSL, ensuring precision and efficiency. The proposed approach is evaluated on two publicly available human locomotion datasets and a 2D moon dataset. Experimental results demonstrate the effectiveness of the proposed method, outperforming all other methods. Tested with three datasets, the proposed method can classify end-point users’ data with an average accuracy of 98.9%, 96.7%, and 96.9% while yielding a low processing time (1 ms). Compared to a benchmark method, the suggested strategy improves the average accuracy by 1.5% and 7.2% for categorizing the target users’ locomotor modes and stabilizes the learning curves. The proposed method represents a significant contribution to the field of human activity recognition and human intent prediction for human–robot interaction systems, enhancing the efficiency and generalization capacity of these systems.

Predicting EFL Learners’ Self-Regulated Learning through Technology Acceptance Model

The purpose of this study is to assess the perceived usefulness of technology (PUT), internet self-efficacy (ISE), and perceived ease of use of technology (PEUT), and self-regulated learning (SRL) of EFL student teachers who participated in technology-mediated English learning environment. After obtaining and validating the questionnaire adapted from several relevant sources, an online survey was conducted to 363 third- and fourth-year student teachers of the English education department in Indonesian universities who met the required criteria. SEM was performed to test three hypotheses about the causal relationship between variables. Due to the hypotheses tested, it is revealed that ISE and PEUT have a partially positive and significant effect on SRL, while PUT has a positive but insignificant effect on SRL. Additionally, it is determined that the exogenous variables (PEUT) is the most influential variable on the endogenous variables (SRL). These findings are expected to add to a body of knowledge, particularly in the development of learning autonomy in teacher education, and that ISE and PEUT, in particular, should be considered as important predictors of SRL in technological English learning setting.

Multi-resolution Twinned Residual Auto-Encoders (MR-TRAE)—A Novel DL Model for Image Multi-resolution

In this paper, we design and evaluate the performance of the Multi-resolution Twinned Residual Auto-Encoders (MR-TRAE) model, a deep learning (DL)-based architecture specifically designed for achieving multi-resolution super-resolved images from low-resolution (LR) inputs at various scaling factors. For this purpose, we expand on the recently introduced Twinned Residual Auto-Encoders (TRAE) paradigm for single-image super-resolution (SISR) to extend it to the multi-resolution (MR) domain. The main contributions of this work include (i) the architecture of the MR-TRAE model, which utilizes cascaded trainable up-sampling modules for progressively increasing the spatial resolution of low-resolution (LR) input images at multiple scaling factors; (ii) a novel loss function designed for the joint and semi-blind training of all MR-TRAE model components; and (iii) a comprehensive analysis of the MR-TRAE trade-off between model complexity and performance. Furthermore, we thoroughly explore the connections between the MR-TRAE architecture and broader cognitive paradigms, including knowledge distillation, the teacher-student learning model, and hierarchical cognition. Performance evaluations of the MR-TRAE benchmarked against state-of-the-art models (such as U-Net, generative adversarial network (GAN)-based, and single-resolution baselines) were conducted using publicly available datasets. These datasets consist of LR computer tomography (CT) scans from patients with COVID-19. Our tests, which explored multi-resolutions at scaling factors ×(2,4,8)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ imes (2,4,8)$$\\end{document}, showed a significant finding: the MR-TRAE model can reduce training times by up to 60%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$60\\%$$\\end{document} compared to those of the baselines, without a noticeable impact on achieved performance.

Music cooperating teachers’ perceptions of the university-cooperating teacher partnership

The purpose of this study was to examine three music cooperating teachers’ perceptions of the university-cooperating teacher partnership. Using a phenomenological interview design, data included two focus group interviews (framing the student teaching semester) and an individual interview in the middle of the term. Participants were experienced cooperating teachers who were working with a student teacher in the research study term and had worked with student teachers in the past. Findings include cooperating teachers’ perceptions of their roles, the balance of P–12 student and student teacher learning, and university and cooperating teacher interactions. Conclusions support the notions that cooperating teachers embrace their multi-faceted role in teacher education, prioritize P–12 student learning, and desire deeper connections with the university.

BRAIxDet: Learning to detect malignant breast lesion with incomplete annotations

Methods to detect malignant lesions from screening mammograms are usually trained with fully annotated datasets, where images are labelled with the localisation and classification of cancerous lesions. However, real-world screening mammogram datasets commonly have a subset that is fully annotated and another subset that is weakly annotated with just the global classification (i.e., without lesion localisation). Given the large size of such datasets, researchers usually face a dilemma with the weakly annotated subset: to not use it or to fully annotate it. The first option will reduce detection accuracy because it does not use the whole dataset, and the second option is too expensive given that the annotation needs to be done by expert radiologists. In this paper, we propose a middle-ground solution for the dilemma, which is to formulate the training as a weakly- and semi-supervised learning problem that we refer to as malignant breast lesion detection with incomplete annotations. To address this problem, our new method comprises two stages, namely: (1) pre-training a multi-view mammogram classifier with weak supervision from the whole dataset, and (2) extending the trained classifier to become a multi-view detector that is trained with semi-supervised student–teacher learning, where the training set contains fully and weakly-annotated mammograms. We provide extensive detection results on two real-world screening mammogram datasets containing incomplete annotations and show that our proposed approach achieves state-of-the-art results in the detection of malignant breast lesions with incomplete annotations.

When the Student Becomes the Master: Boosting Intracranial Hemorrhage Detection Generalizability with Teacher-Student Learning.

When the Student Becomes the Master: Boosting Intracranial Hemorrhage Detection Generalizability with Teacher-Student Learning.