Learning System Research Articles

Harnessing Traditional Business Culture Resources: Scalable Parameter Server Architecture with Distributed Machine Learning

This paper explores the development and utilization of Traditional business culture resources, focusing on Yan'an's rich historical significance and its potential political, cultural, educational, and economic value. Leveraging distributed machine learning systems within a parameter server architecture, the research addresses challenges related to scalability and robustness, particularly in the face of random node downtime and network interruptions. Through intelligent simulation and experimentation, the study demonstrates the effectiveness of employing machine learning techniques to achieve high accuracy in modeling Traditional business culture's development and utilization.

The microfoundations of a lean leadership: Monozukuri, Hitozukuri, Kotozukuri

This paper sets out to investigate the microfoundations of lean leadership. To do so, we explore the Japanese philosophical concepts of Monozukuri, Hitozukuri, and Kotozukuri, representing that passion and craftsmanship for making things, developing people, and making things happen. The paper delves into how to foster lean leadership as a pivotal success factor in adopting lean thinking and practices in a manufacturing firm. By addressing the research gap concerning leadership traits contributing to successful lean deployment, the study explores the microfoundations of lean leadership. The paper is based on a single, in-depth longitudinal case study that offers new insights into the microfoundations underpinning lean leadership at VELUX, a Danish rooftop window manufacturer. The research contributes valuable evidence to the existing lean leadership research discourse, where lean should be seen as a human learning system, illustrating how lean leaders can cultivate lean and how the Japanese philosophical concepts of Monozukuri, Hitozukuri, and Kotozukuri resonate in a Western context.

Multi-task Information Enhancement Recommendation model for educational Self-Directed Learning System

In Self-Directed Learning Systems (SDLS), suggesting specific exercises to students after viewing educational videos is crucial, which promotes the integration of learning resources and improves learning efficiency. Given that exercises and educational videos are distinct learning resources, suggesting exercises centered on educational videos containing knowledge concepts is referred to as cross-type recommendations. Traditional cross-type recommendation methods usually separate data representation and recommendation tasks, which hampers effective knowledge transfer and information fusion. Furthermore, educational resources frequently feature an irregular distribution of information, posing a significant challenge to existing models in extracting crucial information. Besides, due to the inadequate data, the cross-type recommendation system needs help in developing a robust model. To address these challenges, we propose a Multi-task Information Enhancement Recommendation (MIER) Model in this paper. This model provides a comprehensive recommendation capability by integrating resource representation and merging recommendations. Multi-task learning improves the capacity to integrate information and transfer knowledge. Then, we propose an Information Enhancement Encoder (IEE) module that utilizes the attention mechanism to enhance the model’s ability to extract essential information from educational videos and exercises. Finally, to address the problem of inadequate data, we design a fusion structure that incorporates the concept knowledge graph as expert information into the model. To evaluate the effectiveness of the MIER, we compare it with other models on both predicting concepts and recommending exercises task. The corresponding results demonstrate that the MIER outperforms all the other models. We comprehensively evaluated the MIER model’s performance across a range of datasets varying in labeled data quantity. The findings demonstrate that the model consistently achieves outstanding performance and exhibits robustness, even under conditions of limited label availability.

Tutor Competence in Online-Learning Service within Distance Education

The online learning landscape in the Elementary Teacher Education (PGSD) program at Universitas Terbuka Makassar poses significant challenges for tutors, requiring them to adapt to new pedagogical approaches and technologies. This study investigates the competence of tutors in facilitating synchronous online-learning tutoring (Tuweb UT) and asynchronous Learning System Management (LMS) through Silayar UT, which integrates an independent learning program into online platforms. The study examines the tutors' social, strategic, and ethical competence in managing online classes, with a focus on their preparation, planning, and interaction with students. The study uses an ex post facto design with descriptive statistics, analyzing data from 185 tutors in 340 online classes within the undergraduate PGSD program during the 2022 registration term. The findings highlight the tutors' ability to prepare and plan for online tutoring, deploy the tutoring unit (SAT) as a learning service scheme, and incorporate relevant competencies and module characteristics into their tutoring mode. The study also shows that tutoring assignments are given in accordance with competence and that active engagement in Webinar UT and LMS (Silayar UT) is high, with attendance averaging 93.03%. The results indicate that tutors are competent in preparing and planning for online tutoring, incorporating online platforms into their experiences, and fostering active interaction with students, which in turn supports the completion of students' academic trajectories. The study's implications for online learning practices are significant, highlighting the importance of tutor competence in navigating the challenges of online education. Recommendations for further research include exploring the impact of tutor competence on student outcomes and investigating the role of technology in enhancing tutor-student interaction.

A novel life prediction method of RF circuits based on the improved recurrent broad learning system

Life prediction of RF circuits can greatly improve the reliability of RF systems. But most literature has been focused on the life prediction of RF devices, which is not applicable to RF circuits. Therefore, a novel RF circuit life prediction method is proposed based on an improved recurrent broad learning system (RBLS). First, a new feature matrix is proposed to characterize a RF circuit at each moment. Then, an improved RBLS model is used to predict the life of the circuit, in which the RBLS model is used to predict the feature vector and then the extreme learning machine (ELM) is used to expand the feature vector into a feature matrix. The method is validated in a low-noise amplifier with classical GRU, BLS-ELM, BLS, RBLS, ELM, and LSTM as a control group. The analysis results show that RBLS-ELM has the highest prediction accuracy with an RMSE of only 2.5959, the smallest prediction uncertainty of 1.2767, and a very short prediction time of 0.4251 s.

Learning with Asynchronous Labels

Learning with data streams has attracted much attention in recent decades. Conventional approaches typically assume that the feature and label of a data item can be timely observed at each round. In many real-world tasks, however, it often occurs that either the feature or the label is observed firstly while the other arrives with delay. For instance, in distributed learning systems, a central processor collects training data from different sub-processors to train a learning model, whereas the feature and label of certain data items can arrive asynchronously due to network latency. The problem of learning with asynchronous feature or label in streams encompasses many applications but still lacks sound solutions. In this paper, we formulate the problem and propose a new approach to alleviate the negative effect of asynchronicity and mining asynchronous data streams. Our approach carefully exploits the timely arrived information and builds an online ensemble structure to adaptively reuse historical models and instances. We provide the theoretical guarantees of our approach and conduct extensive experiments to validate its effectiveness.

BFL-SA: Blockchain-based federated learning via enhanced secure aggregation

Federated learning, involving a central server and multiple clients, aims to keep data local but raises privacy concerns like data exposure and participation privacy. Secure aggregation, especially with pairwise masking, preserves privacy without accuracy loss. Yet, issues persist like security against malicious models, central server fault tolerance, and trust in decryption keys. Resolving these challenges is vital for advancing secure federated learning systems. In this paper, we present BFL-SA, a blockchain-based federated learning scheme via enhanced secure aggregation, which addresses key challenges by integrating blockchain consensus, publicly verifiable secret sharing, and an overdue gradients aggregation module. These enhancements significantly boost security and fault tolerance while improving the efficiency of data utilization in the secure aggregation process. After security analysis, we have demonstrated that BFL-SA achieves secure aggregation even in malicious models. Through experimental comparative analysis, BFL-SA exhibits rapid secure aggregation speed and achieves 100% model aggregation accuracy.

Abstract PO4-07-03: Machine learning breast cancer risk prediction using sequential past mammograms - a pilot study

Abstract Background and Aim Some studies have reported the use of mammogram AI deep learning algorithms to accurately predict the risk of future breast cancer development in women. In these studies, random and unrelated mammograms were independently used for training of the AI model. We propose a novel deep learning system using sequential past mammograms instead of random studies and aimed to determine if it can further improve risk prediction in a pilot study. Methods Pre-training of the backbone of a Swin-transformer machine learning model was performed by applying region-of-interest lesional masks of benign and malignant on a training set of 1644 mammograms derived from the open-access Curated Breast Imaging Subset of Digital Database for Screening Mammography dataset. To further develop the model, deep learning training and validation were performed on the Cohort of Screen-age Women (CSAW) open-access screening mammogram set which was derived from the long term data of 873 women who developed breast cancer for the first time and 7850 non-cancer women. We then compared the diagnostic accuracy of a model using sequential past mammograms for training against the standard, non-sequential model. For the sequential model, imaging segmentation of the potential cancer site on mammography was performed on all past, pre-cancer mammograms obtained within 60 to 2555 days of cancer diagnosis in the pre-processing stage for breast cancer cases. AI training was then carried out by combining the pre-cancer mammograms belonging to the same woman in a temporal sequence using a transformer encoder. The learning rate started from 1e-6 and decayed 10 times every 10 epochs. For the non-sequential model, we removed the temporal encoder and used random, unrelated mammogram studies for training. Validation was performed on a dataset consisting of 75 pre-cancer mammograms from breast cancer cases and 69 mammograms from non-cancer cases. Results Sensitivity, specificity and overall accuracy on the validation set were 77.3% (58/75), 68.1% (47/69) and (105/144) 72.9% respectively for the sequential model compared to 69.3% (52/75), 69.6% (48/69) and 69.4% (100/144) respectively for the non-sequential model. Conclusion The study showed that deep learning breast cancer risk prediction can be further improved by using sequential past mammograms instead of random mammograms for AI model training. This can also potentially enhance other AI risk prediction models that employ combined mammogram and traditional breast cancer clinical risk factors. A larger validation study will be useful. Contact information: email Lester Leong at lester.leong.c.h@singhealth.com.sg ROC curve for breast cancer risk prediction using AI deep learning training with sequential past mammograms. Citation Format: Lester Leong, Mingjie Xu, Weimin Huang, Erli Zhang, Engracia Loh, Sze Yiun Teo, Geok Hoon Lim, Veronique Kiak Mien Tan, Yirong Sim, Fredrik Strand, Ryan Tan. Machine learning breast cancer risk prediction using sequential past mammograms - a pilot study [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO4-07-03.

Deep learning system for screening AIDS-related cytomegalovirus retinitis with ultra-wide-field fundus images

BackgroundOphthalmological screening for cytomegalovirus retinitis (CMVR) for HIV/AIDS patients is important to prevent lifelong blindness. Previous studies have shown good properties of automated CMVR screening using digital fundus images. However, the application of a deep learning (DL) system to CMVR with ultra-wide-field (UWF) fundus images has not been studied, and the feasibility and efficiency of this method are uncertain. MethodsIn this study, we developed, internally validated, externally validated, and prospectively validated a DL system to detect AIDS-related from UWF fundus images from different clinical datasets. We independently used the InceptionResnetV2 network to develop and internally validate a DL system for identifying active CMVR, inactive CMVR, and non-CMVR in 6960 UWF fundus images from 862 AIDS patients and validated the system in a prospective and an external validation data set using the area under the curve (AUC), accuracy, sensitivity, and specificity. A heat map identified the most important area (lesions) used by the DL system for differentiating CMVR. ResultsThe DL system showed AUCs of 0.945 (95 % confidence interval [CI]: 0.929, 0.962), 0.964 (95 % CI: 0.870, 0.999) and 0.968 (95 % CI: 0.860, 1.000) for detecting active CMVR from non-CMVR and 0.923 (95 % CI: 0.908, 0.938), 0.902 (0.857, 0.948) and 0.884 (0.851, 0.917) for detecting active CMVR from non-CMVR in the internal cross-validation, external validation, and prospective validation, respectively. Deep learning performed promisingly in screening CMVR. It also showed the ability to differentiate active CMVR from non-CMVR and inactive CMVR as well as to identify active CMVR and inactive CMVR from non-CMVR (all AUCs in the three independent data sets >0.900). The heat maps successfully highlighted lesion locations. ConclusionsOur UWF fundus image-based DL system showed reliable performance for screening AIDS-related CMVR showing its potential for screening CMVR in HIV/AIDS patients, especially in the absence of ophthalmic resources.

Practical training of future teachers as prevention of psychological barriers in inclusive education

Introduction. Currently, international organizations pay special attention to the development of inclusive education system, which, taking into account the special educational needs of everyone, provides quality education at the place of residence and full socialization of people with disabilities. Inclusiveness has become one of the main components of the modern educational system, but at the moment there is a problem of occurrence of psychological barriers in teachers, which is poorly studied and requires detailed research. The aim of the article is to consider the problem of preventing the emergence of psychological barriers in inclusive education in students of pedagogical directions through practical professional training, including the development of their communicative and organizational abilities. Materials and Methods. The study was conducted on the basis of the North-Eastern Federal University named after M.K. Ammosov (NEFU) among students studying in the following fields of training: "Special (defectology) education", "Pedagogical education", "Psychological and pedagogical education", in the number of 122 people, aged from 18 to 23 years. The following methods were used: analysis of sources, regulatory and educational-methodical documents of the North-Eastern Federal University on practical training of students, observation and surveys using the method of E.S. Slyusareva "Psychological barriers of teachers in the process of interaction in an inclusive educational environment" and the method "Assessment of communicative and organizational abilities" (V.V. Sinyavsky, V.A. Fedoroshin), statistical processing of the results by U-Manna-Whitney test. Results of the research. The system of practical training of higher education students for pedagogical activity in inclusive education of future teachers, including practice-oriented educational process, socially significant public activity is developed. A separate aspect of this training is the prevention of psychological barriers in inclusive education in future teachers. The application of the developed system of training contributed to the effective development of communicative and organizational abilities of students, and also demonstrated reliably significant reductions in the criteria "barrier of mental tension (anxiety, inability to initiate contact with a child with disabilities, etc.)" Uemp=178,5; p<0,001, and "difficulties of choice and transfer of knowledge about the psychological characteristics of children with disabilities in the process of inclusive education", Uemp=248; p<0,043, which indicates that the conducted preventive and developmental work contributed to a decrease in the expression of the presence of difficulties of choice and transfer of knowledge about the psychological characteristics of children with disabilities in the process of inclusive education. Conclusion. The results of the study indicate the need to rethink the organization and content of the practical component of higher pedagogical education in the aspect of preventing the emergence of psychological barriers in inclusive education in future teachers through the development of their communicative and organizational abilities in the system of practice-oriented learning.

Incorporating mitigation strategies in machine learning for landslide susceptibility prediction

This study proposes an approach that considers mitigation strategies in predicting landslide susceptibility through machine learning (ML) and geographic information system (GIS) techniques. ML models, such as random forest (RF), logistic regression (LR), and support vector classification (SVC) are incorporated into GIS to predict landslide susceptibilities in Hong Kong. To consider the effect of mitigation strategies on landslide susceptibility, non-landslide samples were produced in the upgraded area and added to randomly created samples to serve as ML models in training datasets. Two scenarios were created to compare and demonstrate the efficiency of the proposed approach; Scenario I does not considering landslide control while Scenario II considers mitigation strategies for landslide control. The largest landslide susceptibilities are 0.967 (from RF), followed by 0.936 (from LR) and 0.902 (from SVC) in Scenario II; in Scenario I, they are 0.986 (from RF), 0.955 (from LR) and 0.947 (from SVC). This proves that the ML models considering mitigation strategies can decrease the current landslide susceptibilities. The comparison between the different ML models shows that RF performed better than LR and SVC, and provides the best prediction of the spatial distribution of landslide susceptibilities.

A novel approach for identification of zoonotic trypanosome utilizing deep metric learning and vector database-based image retrieval system

Trypanosomiasis, a significant health concern in South America, South Asia, and Southeast Asia, requires active surveys to effectively control the disease. To address this, we have developed a hybrid model that combines deep metric learning (DML) and image retrieval. This model is proficient at identifying Trypanosoma species in microscopic images of thin-blood film examinations. Utilizing the ResNet50 backbone neural network, a trained-model has demonstrated outstanding performance, achieving an accuracy exceeding 99.71 % and up to 96 % in recall. Acknowledging the necessity for automated tools in field scenarios, we demonstrated the potential of our model as an autonomous screening approach. This was achieved by using prevailing convolutional neural network (CNN) applications, and vector database based-images returned by the KNN algorithm. This achievement is primarily attributed to the implementation of the Triplet Margin Loss function as 98 % of precision. The robustness of the model demonstrated in five-fold cross-validation highlights the ResNet50 neural network, based on DML, as a state-of-the-art CNN model as AUC >98 %. The adoption of DML significantly improves the performance of the model, remaining unaffected by variations in the dataset and rendering it a useful tool for fieldwork studies. DML offers several advantages over conventional classification model to manage large-scale datasets with a high volume of classes, enhancing scalability. The model has the capacity to generalize to novel classes that were not encountered during training, proving particularly advantageous in scenarios where new classes may consistently emerge. It is also well suited for applications requiring precise recognition, especially in discriminating between closely related classes. Furthermore, the DML exhibits greater resilience to issues related to class imbalance, as it concentrates on learning distances or similarities, which are more tolerant to such imbalances. These contributions significantly make the effectiveness and practicality of DML model, particularly in in fieldwork research.

Soft-Sensing of Burn-Through Point Based on Weighted Kernel Just-in-Time Learning and Fuzzy Broad-Learning System in Sintering Process

Soft-Sensing of Burn-Through Point Based on Weighted Kernel Just-in-Time Learning and Fuzzy Broad-Learning System in Sintering Process

Machine learning for network defense: automated DDoS detection with telegram notification integration

As the prevalence and sophistication of distributed denial of service (DDoS) attacks escalate, the imperative for advanced defense mechanisms becomes paramount, especially in rapidly growing digital landscapes like Indonesia. This research presents the development of an innovative intrusion detection system (IDS) that harnesses machine learning (ML) algorithms to automate the detection of DDoS attacks in real time. By monitoring TCP streams, the system utilizes ML-enhanced IDS components to identify malicious traffic patterns indicative of DDoS activities. An automatic alert is dispatched to network administrators via Telegram upon detection, ensuring immediate awareness and facilitating swift countermeasures. Additionally, the system embodies a self-improving architecture by retraining its ML model with newly encountered attack data, thus continuously refining its detection capabilities. The system's efficacy, marked by its adaptive learning and proactive notification system, not only contributes to the fortification of network security but also underscores the potential for ML in cybersecurity within Indonesia’s expanding digital domain. The deployment of this system is anticipated to significantly bolster cybersecurity infrastructure by addressing the urgent need for advanced and responsive defense strategies against the evolving landscape of cyber threats.

Comparative analysis of real issues in open-source machine learning projects

ContextIn the last decade of data-driven decision-making, Machine Learning (ML) systems reign supreme. Because of the different characteristics between ML and traditional Software Engineering systems, we do not know to what extent the issue-reporting needs are different, and to what extent these differences impact the issue resolution process.ObjectiveWe aim to compare the differences between ML and non-ML issues in open-source applied AI projects in terms of resolution time and size of fix. This research aims to enhance the predictability of maintenance tasks by providing valuable insights for issue reporting and task scheduling activities.MethodWe collect issue reports from Github repositories of open-source ML projects using an automatic approach, filter them using ML keywords and libraries, manually categorize them using an adapted deep learning bug taxonomy, and compare resolution time and fix size for ML and non-ML issues in a controlled sample.Result147 ML issues and 147 non-ML issues are collected for analysis. We found that ML issues take more time to resolve than non-ML issues, the median difference is 14 days. There is no significant difference in terms of size of fix between ML and non-ML issues. No significant differences are found between different ML issue categories in terms of resolution time and size of fix.ConclusionOur study provided evidence that the life cycle for ML issues is stretched, and thus further work is required to identify the reason. The results also highlighted the need for future work to design custom tooling to support faster resolution of ML issues.

Anomaly Detection in the Automotive Stamping Process: An Unsupervised Machine Learning Approach

In metal forming, such as stamping of automotive parts, unsupervised machine learning models offer a transformative approach to real-time quality control, especially when labelled data are scarce. Leveraging clustering algorithms and autoencoders, we develop a machine learning system capable of autonomously monitoring sensor data and identifying deviations suggestive of potential defects. The system offers multiple benefits including rapid intervention, reduced part defects and lower stoppages required to rectify defects. The use of unsupervised machine learning models also adds a layer of adaptability, allowing the system to continually refine its understanding of what constitutes a ‘normal’ operation. Empirical evaluation demonstrates the potential of the developed system in detecting anomalies in production data collected from dynamic automotive manufacturing environments.

Dynamic Model Interpretation-Guided Online Active Learning Scheme for Real-Time Safety Assessment.

Chunk-level real-time safety assessment of dynamic systems is a critical component of industrial processes, which is essential to prevent hazards and reduce the risk of injury or damage to equipment and facilities, especially in nonstationary environments. In this context, multiple real and complex concept drifts are inevitable in industrial settings, making it crucial to understand their detection and adaptation processes. The incremental learning scheme should also be well considered. However, existing methods have certain limitations in dealing with such issues. In this article, a dynamic model interpretation-guided online active learning scheme, termed a dynamic model interpretation-guided learning scheme (DMI-LS), is proposed. Specifically, the model update strategy with chunk data is designed based on the implementation of the broad learning system. A novel query strategy is then investigated to consider the ranking preference difference, which relies on the interpretation generated by the explainable artificial intelligence method. Several experiments based on the JiaoLong deep-sea manned submersible data are conducted to verify the effects of the proposed DMI-LS. The results show that it outperforms the other advanced existing approaches with different settings in most scenarios.

Integration of eye-tracking and object detection in a deep learning system for quality inspection analysis

Abstract During quality inspection in manufacturing, the gaze of a worker provides pivotal information for identifying surface defects of a product. However, it is challenging to digitize the gaze information of workers in a dynamic environment where the positions and postures of the products and workers are not fixed. A robust, deep learning-based system, ISGOD (Integrated System with worker’s Gaze and Object Detection), is proposed, which analyzes data to determine which part of the object is observed by integrating object detection and eye-tracking information in dynamic environments. The ISGOD employs a six-dimensional pose estimation algorithm for object detection, considering the location, orientation, and rotation of the object. Eye-tracking data were obtained from Tobii Glasses, which enable real-time video transmission and eye-movement tracking. A latency reduction method is proposed to overcome the time delays between object detection and eye-tracking information. Three evaluation indices, namely, gaze score, accuracy score, and concentration index are suggested for comprehensive analysis. Two experiments were conducted: a robustness test to confirm the suitability for real-time object detection and eye-tracking, and a trend test to analyze the difference in gaze movement between experts and novices. In the future, the proposed method and system can transfer the expertise of experts to enhance defect detection efficiency significantly.

IEEE Transactions on Neural Networks and Learning Systems Publication Information

IEEE Transactions on Neural Networks and Learning Systems Publication Information

Harnessing TME depicted by histological images to improve cancer prognosis through a deep learning system

Spatial transcriptomics (ST) provides insights into the tumor microenvironment (TME), which is closely associated with cancer prognosis, but ST has limited clinical availability. In this study, we provide a powerful deep learning system to augment TME information based on histological images for patients without ST data, thereby empowering precise cancer prognosis. The system provides two connections to bridge existing gaps. The first is the integrated graph and image deep learning (IGI-DL) model, which predicts ST expression based on histological images with a 0.171 increase in mean correlation across three cancer types compared with five existing methods. The second connection is the cancer prognosis prediction model, based on TME depicted by spatial gene expression. Our survival model, using graphs with predicted ST features, achieves superior accuracy with a concordance index of 0.747 and 0.725 for The Cancer Genome Atlas breast cancer and colorectal cancer cohorts, outperforming other survival models. For the external Molecular and Cellular Oncologycolorectal cancer cohort, our survival model maintains a stable advantage.