Learning Paradigm Research Articles

A review on adversarial–based deep transfer learning mechanical fault diagnosis

Mechanical equipment is a vital foundational support for promoting national economic development and is widely utilized in key sectors such as aerospace, shipping, construction machinery, energy, petrochemicals, and robotics. With the advancement of artificial intelligence and industrial intelligence, industrial big data and its intelligent analysis provide robust support for fault prediction and health management of equipment. Building on existing research, intelligent diagnostics for mechanical equipment based on deep learning have gained significant attention and application. However, the success of big data relies on comprehensive fault data, which is challenging to obtain in practical applications where continuous equipment operation is essential. Moreover, mechanical equipment often operates under varying conditions, leading to different data distributions for training and testing. This discrepancy can result in low diagnostic accuracy or even failure of deep learning methods. Deep Transfer Learning (DTL) is an emerging machine learning paradigm that not only leverages the advantages of deep learning (DL) in feature representation but also harnesses the strengths of transfer learning (TL) in knowledge transfer. Consequently, DTL techniques can make deep learning-based fault diagnosis methods more reliable, robust, and applicable, leading to extensive development and research in the field of intelligent fault diagnosis. This paper primarily introduces adversarial-based deep transfer learning (ADTL) models, which are fundamentally based on Generative Adversarial Network (GAN). We provide a detailed discussion of the main applications of ADTL and its recent developments in intelligent fault diagnosis, along with some future challenges and prospects.

Increasing Student Learning Activities Through Model Application Problem Based Learning (PBL) Material on the History of Colonialism and Imperialism in Class XI Asteroid at SMA Negeri 2 Medan

This study seeks to detail how class XI Asteroids students at SMA Negeri 2 Medan were able to increase their learning activities by using the Problem Based Learning (PBL) paradigm to content related to the history of colonialism and imperialism. The classroom action research (PTK) approach is used in this study to address a range of issues that transpire inside the educational setting. The four steps of this PTK are preparation, execution, evaluation, and review. Increasing students' historical learning activities is the primary goal of this study, and these four steps repeat themselves in a cycle to achieve this goal. The study was place in July and August of 2024 at SMA N 2 Medan and included students from class XI ASTEROID. Each of the two cycles of this study included two sessions. The study shows that with each cycle, pupils get more engaged in learning about history. The average proportion of student learning activities, which was 48.91% in cycle I and 75.78% in cycle II, shows this growth. Based on these findings, it can be concluded that the research success indicator has been met, with an average proportion of student learning activity reaching or above 70%, during cycle II.

Neural Network Architectures and Magnetic Hysteresis: Overview and Comparisons

The development of innovative materials, based on the modern technologies and processes, is the key factor to improve the energetic sustainability and reduce the environmental impact of electrical equipment. In particular, the modeling of magnetic hysteresis is crucial for the design and construction of electrical and electronic devices. In recent years, additive manufacturing techniques are playing a decisive role in the project and production of magnetic elements and circuits for applications in various engineering fields. To this aim, the use of the deep learning paradigm, integrated with the most common models of the magnetic hysteresis process, has become increasingly present in recent years. The intent of this paper is to provide the features of a wide range of deep learning tools to be applied to magnetic hysteresis context and beyond. The possibilities of building neural networks in hybrid form are innumerable, so it is not plausible to illustrate them in a single paper, but in the present context, several neural networks used in the scientific literature, integrated with various hysteretic mathematical models, including the well-known Preisach model, are compared. It is shown that this hybrid approach not only improves the modeling of hysteresis by significantly reducing computational time and efforts, but also offers new perspectives for the analysis and prediction of the behavior of magnetic materials, with significant implications for the production of advanced devices.

Learning differentiable logic programs for abstract visual reasoning

AbstractVisual reasoning is essential for building intelligent agents that understand the world and perform problem-solving beyond perception. Differentiable forward reasoning has been developed to integrate reasoning with gradient-based machine learning paradigms. However, due to the memory intensity, most existing approaches do not bring the best of the expressivity of first-order logic, excluding a crucial ability to solve abstract visual reasoning, where agents need to perform reasoning by using analogies on abstract concepts in different scenarios. To overcome this problem, we propose NEUro-symbolic Message-pAssiNg reasoNer (NEUMANN), which is a graph-based differentiable forward reasoner, passing messages in a memory-efficient manner and handling structured programs with functors. Moreover, we propose a computationally-efficient structure learning algorithm to perform explanatory program induction on complex visual scenes. To evaluate, in addition to conventional visual reasoning tasks, we propose a new task, visual reasoning behind-the-scenes, where agents need to learn abstract programs and then answer queries by imagining scenes that are not observed. We empirically demonstrate that NEUMANN solves visual reasoning tasks efficiently, outperforming neural, symbolic, and neuro-symbolic baselines.

A Deep Unfolding Network for Multispectral and Hyperspectral Image Fusion

Multispectral and hyperspectral image fusion (MS/HS fusion) aims to generate a high-resolution hyperspectral (HRHS) image by fusing a high-resolution multispectral (HRMS) and a low-resolution hyperspectral (LRHS) images. The deep unfolding-based MS/HS fusion method is a representative deep learning paradigm due to its excellent performance and sufficient interpretability. However, existing deep unfolding-based MS/HS fusion methods only rely on a fixed linear degradation model, which focuses on modeling the relationships between HRHS and HRMS, as well as HRHS and LRHS. In this paper, we break free from this observation model framework and propose a new observation model. Firstly, the proposed observation model is built based on the convolutional sparse coding (CSC) technique, and then a proximal gradient algorithm is designed to solve this model. Secondly, we unfold the iterative algorithm into a deep network, dubbed as MHF-CSCNet, where the proximal operators are learned using convolutional neural networks. Finally, all trainable parameters can be automatically learned end-to-end from the training pairs. Experimental evaluations conducted on various benchmark datasets demonstrate the superiority of our method both quantitatively and qualitatively compared to other state-of-the-art methods.

Functional reorganization of brain regions supporting artificial grammar learning across the first half year of life.

Pre-babbling infants can track nonadjacent dependencies (NADs) in the auditory domain. While this forms a crucial prerequisite for language acquisition, the neurodevelopmental origins of this ability remain unknown. We applied functional near-infrared spectroscopy in neonates and 6- to 7-month-old infants to investigate the neural substrate supporting NAD learning and detection using tone sequences in an artificial grammar learning paradigm. Detection of NADs was indicated by left prefrontal activation in neonates while by left supramarginal gyrus (SMG), superior temporal gyrus (STG), and inferior frontal gyrus activation in 6- to 7-month-olds. Functional connectivity analyses further indicated that the neonate activation pattern during the test phase benefited from a brain network consisting of prefrontal regions, left SMG and STG during the rest and learning phases. These findings suggest a left-hemispheric learning-related functional brain network may emerge at birth and serve as the foundation for the later engagement of these regions for NAD detection, thus, providing a neural basis for language acquisition.

Modelling the properties of aerated concrete on the basis of raw materials and ash-and-slag wastes using machine learning paradigm

The thermal power industry, as a major consumer of hard coal, significantly contributes to harmful emissions, affecting both air quality and soil health during the operation and transportation of ash and slag waste. This study presents the modeling of aerated concrete using local raw materials and ash-and-slag waste in seismic areas through machine learning techniques. A comprehensive literature review and comparative analysis of normative documentation underscore the relevance and feasibility of employing non-autoclaved aerated concrete blocks in such regions. Machine learning methods are particularly effective for disjointed datasets, with neural networks demonstrating superior performance in modeling complex relationships for predicting concrete strength and density. The results reveal that neural networks, especially those with Bayesian Regularisation, consistently outperformed decision trees, achieving higher regression values (Rstrength = 0.9587 and Rdensity = 0.91997) and lower error metrics (MSE, RMSE, RIE, MAE). This indicates their advanced capability to capture intricate non-linear patterns. The study concludes that artificial neural networks are a robust tool for predicting concrete properties, crucial for producing non-autoclaved curing wall blocks suitable for earthquake-resistant construction. Future research should focus on optimizing the balance between density and strength of blocks by enhancing the properties of aerated concrete and utilizing reliable models.

Deep transfer learning in machinery remaining useful life prediction: A systematic review

Abstract As a novel paradigm in machine learning, deep transfer learning (DTL) can harness the strengths of deep learning for feature representation, while also capitalizing on the advantages of transfer learning for knowledge transfer. Hence, DTL can effectively enhance the robustness and applicability of the data-driven RUL prediction methods, and has garnered extensive development and research attention in machinery RUL prediction. Although there are numerous systematic review articles published on the topic of the DTL-based approaches, a comprehensive overview of the application of DTL in the RUL prediction for different mechanical equipment has yet to be systematically conducted. Therefore, it is imperative to further review the pertinent literature on DTL-based approaches. This will facilitate researchers in comprehending the latest technological advancements and devising efficient solutions to address the cross-domain RUL prediction challenge. In this review, a brief overview of the theoretical background of DTL and its application in RUL prediction tasks are provided at first. Then, a detailed discussion of the primary DTL methods and their recent advancements in cross-domain RUL prediction is presented. Next, the practical application of the current research is discussed in relation to the research object and its open-source data. More importantly, several challenges and further trend are further presented to conclude this paper in the end. We have reason to hope this work can offer convenience and inspiration to researchers seeking to advance in the field of RUL prediction.

Graph neural networks for multi-view learning: a taxonomic review

With the explosive growth of user-generated content, multi-view learning has become a rapidly growing direction in pattern recognition and data analysis areas. Due to the significant application value of multi-view learning, there has been a continuous emergence of research based on machine learning methods and traditional deep learning paradigms. The core challenge in multi-view learning lies in harnessing both consistent and complementary information to forge a unified, comprehensive representation. However, many multi-view learning tasks are based on graph-structured data, making existing methods unable to effectively mine the information contained in the input multiple data sources. Recently, graph neural networks (GNN) techniques have been widely utilized to deal with non-Euclidean data, such as graphs or manifolds. Thus, it is essential to combine the advantages of the powerful learning capability of GNN models and multi-view data. In this paper, we aim to provide a comprehensive survey of recent research works on GNN-based multi-view learning. In detail, we first provide a taxonomy of GNN-based multi-view learning methods according to the input form of models: multi-relation, multi-attribute and mixed. Then, we introduce the applications of multi-view learning, including recommendation systems, computer vision and so on. Moreover, several public datasets and open-source codes are introduced for implementation. Finally, we analyze the challenges of applying GNN models on various multi-view learning tasks and state new future directions in this field.

Fisher regularized discriminative broad learning system for visual classification

The Broad Learning System (BLS) is an innovative learning paradigm with significant success in image classification. However, the 0–1 labeling matrix employed in BLS struggles to align with the true data distribution, limiting the flexibility of the regression objective. The ℓ2-norm-based Broad Learning System (L2DBLS) introduces the ɛ-dragging technique to enhance labeling diversity, but the randomness inherent in ɛ-dragging weakens the label correlation within the same category. This paper proposes the Fisher Regularized Discriminative Broad Learning System (FRBLS) to tackle these issues and aims to achieve the following objectives: Firstly, the generated label matrix offers sufficient flexibility to maintain intra-class compactness and inter-class separability. Secondly, the constraints of Fisher Regularization on the same class of features ensure better alignment between samples and labels. Finally, the overall solution process is optimized using an ADAM-based alternating multiplier method, which ensures closed-form solutions at each iteration. Experimental results demonstrate that FRBLS achieves up to 98% accuracy on various face and object datasets, offering superior time efficiency and a 1% improvement in classification performance compared to recent state-of-the-art methods.

Quantifying population-level neural tuning functions using Ricker wavelets and the Bayesian bootstrap

BackgroundExperience changes visuo-cortical tuning. In humans, re-tuning has been studied during aversive generalization learning, in which the similarity of generalization stimuli (GSs) with a conditioned threat cue (CS+) is used to quantify tuning functions. Previous work utilized pre-defined tuning shapes (generalization and sharpening patterns). This approach may constrain the ways in which re-tuning can be characterized since the tuning patterns may not match the prototypical functions. New methodThe present study proposes a flexible and data-driven method for precisely quantifying changes in tuning based on the Ricker wavelet function and the Bayesian bootstrap. This method was applied to EEG and psychophysics data from an aversive generalization learning paradigm. ResultsThe Ricker wavelet model fitted the steady-state visual event potentials (ssVEP), alpha-band power, and detection accuracy data well. A Morlet wavelet function was used for comparison and fit the data better in some situations, but was more challenging to interpret. The pattern of re-tuning in the EEG data, predicted by the Ricker model, resembled the shapes of the best fitting a-priori patterns. Comparison with existing methodsAlthough the re-tuning shape modeled by the Ricker function resembled the pre-defined shapes, the Ricker approach led to greater Bayes factors and more interpretable results compared to a-priori models. The Ricker approach was more easily fit and led to more interpretable results than a Morlet wavelet model. ConclusionThis work highlights the promise of the current method for capturing the precise nature of visuo-cortical tuning, unconstrained by the implementation of a-priori models.

A multicenter bladder cancer MRI dataset and baseline evaluation of federated learning in clinical application

Bladder cancer (BCa), as the most common malignant tumor of the urinary system, has received significant attention in research on the clinical application of artificial intelligence algorithms. Nevertheless, it has been observed that certain investigations use data from various medical facilities to train models for BCa, which may pose a privacy risk. Given this concern, protecting patient privacy during machine learning algorithm training is a crucial aspect that requires substantial attention. One emerging machine learning paradigm that addresses this concern is federated learning (FL). FL enables multiple entities to collaboratively build machine learning models while preserving data privacy and security. In this study, we present a multicenter BCa magnetic resonance imaging (MRI) dataset. The dataset comprises 275 three-dimensional bladder T2-weighted MRI scans collected from four medical centers, and each scan includes diagnostic pathological labels for muscle invasion and pixel-level annotations of tumor contours. Four FL methods are used to assess the baseline of the dataset for both the task of diagnosing muscle-invasive bladder cancer and automatic bladder tumor lesion segmentation.

Enhancing Cybersecurity Awareness through Gamification: Design an Interactive Cybersecurity Learning Platform for Multimedia University Students

Cybersecurity has emerged as a critical imperative in contemporary digital landscapes, necessitating heightened awareness and proficiency across all demographic segments. Accordingly, this research has been meticulously crafted to delve into the complexities of cybersecurity awareness, with a specific focus on university students. The study embarks on an exhaustive analysis encompassing the evaluation of cybersecurity awareness levels within the targeted groups, the identification of prevailing issues and practices, and an exploration of novel methodologies, notably gamification, to fortify cybersecurity knowledge and skills among diverse user cohorts. Central to this investigation is the efficacy of gamified learning environments tailored expressly for augmenting cybersecurity awareness among university students. Through a comprehensive examination of existing platforms, methodological frameworks, and user interactions, this research outlines critical trends, challenges, and latent opportunities within the cybersecurity awareness domain, with a specific emphasis on gamification's transformative potential. The study not only identifies key areas for improvement but also proposes innovative solutions rooted in gamified learning paradigms, with the overarching goal of fostering engaging, effective, and sustainable cybersecurity awareness initiatives among students. Drawing upon a synthesis of theoretical constructs, empirical insights, and pragmatic recommendations, this research significantly contributes to the evolving discourse on cybersecurity education. By underscoring the transformative efficacy of gamification as a pivotal tool in cybersecurity awareness initiatives, this study overlays the way for substantial advancements in cybersecurity education paradigms, offering a roadmap for enhancing cybersecurity awareness levels among university students and beyond.

A Survey of Security Strategies in Federated Learning: Defending Models, Data, and Privacy

Federated Learning (FL) has emerged as a transformative paradigm in machine learning, enabling decentralized model training across multiple devices while preserving data privacy. However, the decentralized nature of FL introduces significant security challenges, making it vulnerable to various attacks targeting models, data, and privacy. This survey provides a comprehensive overview of the defense strategies against these attacks, categorizing them into data and model defenses and privacy attacks. We explore pre-aggregation, in-aggregation, and post-aggregation defenses, highlighting their methodologies and effectiveness. Additionally, the survey delves into advanced techniques such as homomorphic encryption and differential privacy to safeguard sensitive information. The integration of blockchain technology for enhancing security in FL environments is also discussed, along with incentive mechanisms to promote active participation among clients. Through this detailed examination, the survey aims to inform and guide future research in developing robust defense frameworks for FL systems.

Teaching and Learning in an AI-powered world

In a world where artificial intelligence (AI) is increasingly used, education is undergoing profound transformations, as AI inevitably changes the paradigm of teaching and learning. Teachers have already begun to use AI for student-centred learning, data analysis, or improving assessment through standardized systems provided by AI. Although these new technologies promise efficiency and personalized education, AI is not yet developed enough to truly understand the mechanisms underlying learning. Excessive dependence on technology can reduce the teacher’s role and the importance of human interaction, while certain ethical issues may arise regarding data confidentiality and security. To address these challenges, teachers and decision-makers must carefully integrate AI, emphasizing transparency and ethics, to ensure that technology will indeed have a positive impact on society. This article explores the issues and concerns related to the use of AI in education, also examining the potential disadvantages and fears associated with it and providing an overview of the challenges that teachers face when integrating AI into education.

Word learning tasks as a window into the triggering problem for presuppositions

In this paper, we show that native speakers spontaneously divide the complex meaning of a new word into a presuppositional component and an assertive component. These results argue for the existence of a productive triggering algorithm for presuppositions, one that is not based on alternative lexical items nor on contextual salience. On a methodological level, the proposed learning paradigm can be used to test further theories concerned with the interaction of lexical properties and conceptual biases.

Promoting Machine Abilities of Discovering and Utilizing Knowledge in a Unified Zero-shot Learning Paradigm

Knowledge discovery and utilization are two essential cognitive processes that enable humans to understand the world and extract new insights from their surroundings. These processes have motivated machine learning studies, particularly zero-shot learning (ZS), which seeks to identify unseen concepts through the use of side information. Previous ZS studies primarily focused on utilizing existing knowledge to infer unseen events, yet they overlook the crucial process of knowledge discovery and the integrated modeling of these knowledge-aware processes. In this study, we present a comprehensive ZS learning approach that explores and evaluates the machine's abilities of discovering and utilizing knowledge. More specifically, to emulate human-like knowledge discovery and utilization processes, we propose a novel visual-aware ZS knowledge graph completion task for evaluation, incorporating a traditional ZS image classification task. Technically, we develop a unified ZS learning paradigm named Cognitive Learner (CoLa) to foster the two knowledge-aware abilities. Including a knowledge representation learning (KRL) module and a knowledge adaptation (KA) module, CoLa adapts well to the two specified tasks with the corresponding data. Extensive experiments on large-scale datasets demonstrate CoLa models’ outstanding performance over compared methods in the two ZS tasks, illustrating their superior ability of discovering and utilizing knowledge.

Effective Hard Negative Mining for Contrastive Learning-based Code Search

Background. Code search aims to find the most relevant code snippet in a large codebase based on a given natural language query. An accurate code search engine can increase code reuse and improve programming efficiency. The focus of code search is how to represent the semantic similarity of code and query. With the development of code pre-trained models, the pattern of using numeric feature vectors (embeddings) to represent code semantics and using vector distance to represent semantic similarity has replaced traditional string matching methods. The quality of semantic representations is critical to the effectiveness of downstream tasks such as code search. Currently, the state-of-the-art (SOTA) learning method uses the contrastive learning paradigm. The objective of contrastive learning is to maximize the similarity between matching code and query (positive samples) and minimize the similarity between mismatched pairs (negative samples). To increase the reusing of negative samples, prior contrastive learning approaches use a large queue (memory bank) to store embeddings. Problem. However, there is still a lot of room for improvement in using negative examples for code search: ① Due to the random selection of negative samples, semantic representations learned by existing models cannot distinguish similar codes well. ② Since semantic vectors in the memory bank are reused from previous inference results and then directly used for loss function calculation without gradient descent, the model cannot effectively learn the negative sample semantic information. Method. To solve the above problems, we propose a contrastive learning code search model with hard negative mining called CoCoHaNeRe: ❶ To enable the model to distinguish similar codes, we introduce hard negative examples into contrastive training, which are negative examples in the codebase that are most similar to positive examples. As a result, hard negative examples are most likely to make the model make mistakes. ❷ To improve the learning efficiency of negative samples during training, we add all hard negative examples to the model's gradient descent process. Result. To verify the effectiveness of CoCoHaNeRe, we conducted experiments on large code search datasets with six programming languages, as well as similar retrieval tasks code clone detection and code question answering. Experimental results show that our model achieves SOTA performance. In the code search task, the average MRR score of CoCoHaNeRe exceeds CodeBERT, GraphCodeBERT, and UniXcoder by 11.25%, 8.13%, and 7.38%, respectively. It has also made great progress in code clone detection and code question answering. In addition, our method performs well in different programming languages and code pre-training models. Furthermore, qualitative analysis shows that our model effectively distinguishes high-order semantic differences between similar codes.

Physical information-guided multidirectional gated recurrent unit network fusing attention to solve the Black-Scholes equation

Reasonable option pricing is crucial in the financial derivatives market. Finding analytical solutions for the Black-Scholes (BS) equation, particularly for American options or with fluctuating volatility and interest rates, is challenging. BS equations exhibit strong time-series characteristics, with asset prices typically adhering to geometric Brownian motion. To address the BS equations, we propose a sequence-to-sequence model guided by physical information (PI), called PiMGA. The PiMGA fuses a multidirectional gated recurrent unit (GRU) network with an attention module, where multidirectional GRU enhances the coding performance of the input sequences and the attention module balances the feature weights of the hidden variables. Prior physical knowledge in BS equations is jointly used as a constraint, forming the penalty function for objective optimization. This allows PiMGA to serve as an efficient approximation function in the learning paradigm of physically informed machine learning to solve BS equations. BS equations with various complexities illustrate the accuracy and feasibility of PiMGA for numerical solutions. Furthermore, the out-of-distribution generalization ability of PiMGA is verified by predicting the Nasdaq 100 index.

Better early than late: the temporal dynamics of pointing cues during cross-situational word learning

Abstract Learning the meaning of a word is a difficult task due to the variety of possible referents present in the environment. Visual cues such as gestures frequently accompany speech and have the potential to reduce referential uncertainty and promote learning, but the dynamics of pointing cues and speech integration are not yet known. If word learning is influenced by when, as well as whether, a learner is directed correctly to a target, then this would suggest temporal integration of visual and speech information can affect the strength of association of word–referent mappings. Across two pre-registered studies, we tested the conditions under which pointing cues promote learning. In a cross-situational word learning paradigm, we showed that the benefit of a pointing cue was greatest when the cue preceded the speech label, rather than following the label (Study 1). In an eye-tracking study (Study 2), the early cue advantage was due to participants’ attention being directed to the referent during label utterance, and this advantage was apparent even at initial exposures of word–referent pairs. Pointing cues promote time-coupled integration of visual and auditory information that aids encoding of word–referent pairs, demonstrating the cognitive benefits of pointing cues occurring prior to speech.