Learning Paradigm Research Articles

Guided Inquiry Method and Self-Efficacy on High School Students' Physics Learning Outcomes

A change in the learning paradigm is needed to develop a scientific attitude to form students who are creative, critical, open, innovative, and competitive. This study aims to analyze the effect of the guided inquiry learning method and self-efficacy on students' physics learning outcomes. This research is true experimental research that compares the application of the guided and free inquiry methods. This study involved 70 students from 2 classes selected by simple random sampling. The instruments used in this study were self-efficacy questionnaires and student learning outcomes. The data collected were then analyzed descriptively and inferentially using SPSS 20. The results showed that applying the inquiry method and self-efficacy influenced students' physics learning outcomes. Applying the guided inquiry method and high student self-efficacy resulted in high learning outcomes. Conversely, they applied the free inquiry method with low self-efficacy, which resulted in low learning outcomes. However, there was no interaction between the inquiry method and self-efficacy.

Selective Trimmed Average: A Resilient Federated Learning Algorithm With Deterministic Guarantees on the Optimality Approximation.

The federated learning (FL) paradigm aims to distribute the computational burden of the training process among several computation units, usually called agents or workers, while preserving private local training datasets. This is generally achieved by resorting to a server-worker architecture where agents iteratively update local models and communicate local parameters to a server that aggregates and returns them to the agents. However, the presence of adversarial agents, which may intentionally exchange malicious parameters or may have corrupted local datasets, can jeopardize the FL process. Therefore, we propose selective trimmed average (SETA), which is a resilient algorithm to cope with the undesirable effects of a number of misbehaving agents in the global model. SETA is based on properly filtering and combining the exchanged parameters. We mathematically prove that the proposed algorithm is resilient against data and local model poisoning attacks. Most resilient methods presented so far in the literature assume that a trusted server is in hand. In contrast, our algorithm works both in server-worker and shared memory architectures, where the latter excludes the necessity of a trusted server. The theoretical findings are corroborated through numerical results on MNIST dataset and on multiclass weather dataset (MWD).

FedCLCC: A personalized federated learning algorithm for edge cloud collaboration based on contrastive learning and conditional computing

Federated learning (FL) is a distributed machine learning paradigm for edge cloud computing. FL can facilitate data-driven decision-making in tactical scenarios, effectively addressing both data volume and infrastructure challenges in edge environments. However, the diversity of clients in edge cloud computing presents significant challenges for FL. Personalized federated learning (pFL) received considerable attention in recent years. One example of pFL involves exploiting the global and local information in the local model. Current pFL algorithms experience limitations such as slow convergence speed, catastrophic forgetting, and poor performance in complex tasks, which still have significant shortcomings compared to the centralized learning. To achieve high pFL performance, we propose FedCLCC: Federated Contrastive Learning and Conditional Computing. The core of FedCLCC is the use of contrastive learning and conditional computing. Contrastive learning determines the feature representation similarity to adjust the local model. Conditional computing separates the global and local information and feeds it to their corresponding heads for global and local handling. Our comprehensive experiments demonstrate that FedCLCC outperforms other state-of-the-art FL algorithms.

Probabilistic Attention Based on Gaussian Processes for Deep Multiple Instance Learning.

Multiple instance learning (MIL) is a weakly supervised learning paradigm that is becoming increasingly popular because it requires less labeling effort than fully supervised methods. This is especially interesting for areas where the creation of large annotated datasets remains challenging, as in medicine. Although recent deep learning MIL approaches have obtained state-of-the-art results, they are fully deterministic and do not provide uncertainty estimations for the predictions. In this work, we introduce the attention Gaussian process (AGP) model, a novel probabilistic attention mechanism based on Gaussian processes (GPs) for deep MIL. AGP provides accurate bag-level predictions as well as instance-level explainability and can be trained end-to-end. Moreover, its probabilistic nature guarantees robustness to overfit on small datasets and uncertainty estimations for the predictions. The latter is especially important in medical applications, where decisions have a direct impact on the patient's health. The proposed model is validated experimentally as follows. First, its behavior is illustrated in two synthetic MIL experiments based on the well-known MNIST and CIFAR-10 datasets, respectively. Then, it is evaluated in three different real-world cancer detection experiments. AGP outperforms state-of-the-art MIL approaches, including deterministic deep learning ones. It shows a strong performance even on a small dataset with less than 100 labels and generalizes better than competing methods on an external test set. Moreover, we experimentally show that predictive uncertainty correlates with the risk of wrong predictions, and therefore it is a good indicator of reliability in practice. Our code is publicly available.

Psilocybin Facilitates Fear Extinction: Importance of Dose, Context, and Serotonin Receptors.

A variety of classic psychedelics and MDMA have been shown to enhance fear extinction in rodent models. This has translational significance because a standard treatment for post-traumatic stress disorder (PTSD) is prolonged exposure therapy. However, few studies have investigated psilocybin's potential effect on fear learning paradigms. More specifically, the extents to which dose, timing of administration, and serotonin receptors may influence psilocybin's effect on fear extinction are not understood. In this study, we used a delay fear conditioning paradigm to determine the effects of psilocybin on fear extinction, extinction retention, and fear renewal in male and female mice. Psilocybin robustly enhances fear extinction when given acutely prior to testing for all doses tested. Psilocybin also exerts long-term effects to elevate extinction retention and suppress fear renewal in a novel context, although these changes were sensitive to dose. Analysis of sex differences showed that females may respond to a narrower range of doses than males. Administration of psilocybin prior to fear learning or immediately after extinction yielded no change in behavior, indicating that concurrent extinction experience is necessary for the drug's effects. Cotreatment with a 5-HT2A receptor antagonist blocked psilocybin's effects for extinction, extinction retention, and fear renewal, whereas 5-HT1A receptor antagonism attenuated only the effect on fear renewal. Collectively, these results highlight dose, context, and serotonin receptors as crucial factors in psilocybin's ability to facilitate fear extinction. The study provides preclinical evidence to support investigating psilocybin as a pharmacological adjunct for extinction-based therapy for PTSD.

Cross-Image Pixel Contrasting for Semantic Segmentation.

This work studies the problem of image semantic segmentation. Current approaches focus mainly on mining "local" context, i.e., dependencies between pixels within individual images, by specifically-designed, context aggregation modules (e.g., dilated convolution, neural attention) or structure-aware optimization objectives (e.g., IoU-like loss). However, they ignore "global" context of the training data, i.e., rich semantic relations between pixels across different images. Inspired by recent advance in unsupervised contrastive representation learning, we propose a pixel-wise contrastive algorithm, dubbed as PiCo, for semantic segmentation in the fully supervised learning setting. The core idea is to enforce pixel embeddings belonging to a same semantic class to be more similar than embeddings from different classes. It raises a pixel-wise metric learning paradigm for semantic segmentation, by explicitly exploring the structures of labeled pixels, which were rarely studied before. Our training algorithm is compatible with modern segmentation solutions without extra overhead during testing. We experimentally show that, with famous segmentation models (i.e., DeepLabV3, HRNet, OCRNet, SegFormer, Segmenter, MaskFormer) and backbones (i.e., MobileNet, ResNet, HRNet, MiT, ViT), our algorithm brings consistent performance improvements across diverse datasets (i.e., Cityscapes, ADE20 K, PASCAL-Context, COCO-Stuff, CamVid). We expect that this work will encourage our community to rethink the current de facto training paradigm in semantic segmentation.

Enhanced domain transfer deep fuzzy echo state network for rotating machinery fault diagnosis based on current signal

Although vibration-based fault diagnosis methods have achieved remarkable results in rotating machinery, they are still limited by various factors, such as environment noise and additional sensor installation. Meanwhile, due to inevitable distribution differences in practical engineering, deep transfer learning models are commonly used in fault diagnostic fields. Although deep architectures can extract representative domain-invariant features, they typically result in substantial computational burdens and training time. Therefore, to address the above problems, an enhanced domain transfer deep fuzzy echo state network (EDFESN) is proposed for rotating machinery fault diagnosis based on current signals. Firstly, current signals derived from the driving motor are directly collected without additional sensors. Secondly, an enhanced domain mapping method is raised to solve the problem of data distribution discrepancies between cross domain. Furthermore, a deep fuzzy echo state network (DFESN) integrated with fuzzy clustering is constructed, which utilizes layer-wise fuzzy-tuning learning paradigm instead of backpropagation step to reduce training time. Two datasets from gearbox and bearing both verify the effectiveness of the proposed method. Compared with other popular transfer methods, EDFESN not only achieves highest accuracy, but also has fastest training speed.

Prototype-Guided Memory Replay for Continual Learning.

Continual learning (CL) is a machine learning paradigm that accumulates knowledge while learning sequentially. The main challenge in CL is catastrophic forgetting of previously seen tasks, which occurs due to shifts in the probability distribution. To retain knowledge, existing CL models often save some past examples and revisit them while learning new tasks. As a result, the size of saved samples dramatically increases as more samples are seen. To address this issue, we introduce an efficient CL method by storing only a few samples to achieve good performance. Specifically, we propose a dynamic prototype-guided memory replay (PMR) module, where synthetic prototypes serve as knowledge representations and guide the sample selection for memory replay. This module is integrated into an online meta-learning (OML) model for efficient knowledge transfer. We conduct extensive experiments on the CL benchmark text classification datasets and examine the effect of training set order on the performance of CL models. The experimental results demonstrate the superiority our approach in terms of accuracy and efficiency.

Implementing and evaluating simple resampling techniques in federated learning for imbalanced data

Federated Learning (FL) has emerged as a promising paradigm for privacy-preserving collaborative machine learning. However, the challenge of data imbalance, exacerbated by the non-IID nature of distributed datasets, significantly impacts model performance and fairness in FL systems. This paper investigates the implementation and evaluation of simple resampling techniques to address data imbalance within the FL framework. Using the MIMIC-III healthcare dataset, a simulated FL environment with ten virtual clients was created to test various resampling methods: SMOTE, random undersampling, and a hybrid approach. The study employed logistic regression models and evaluated performance using common and novel FL-specific metrics. Experimental results demonstrate that the hybrid resampling technique significantly outperforms other methods, improving the F1-score by 13.1% and reducing communication rounds by 25.3%. Statistical analyses, including repeated measures ANOVA and hierarchical linear modeling, confirm the robustness of these findings across varied client data distributions. This research provides a replicable framework for addressing data imbalance in FL, contributing to enhanced model fairness and efficiency in privacy-sensitive applications.

Integrating Feature Selection with Machine Learning for Accurate Reservoir Landslide Displacement Prediction

Accurate prediction of reservoir landslide displacements is crucial for early warning and hazard prevention. Current machine learning (ML) paradigms for predicting landslide displacement demonstrate superior performance, while often relying on various feature engineering techniques, such as decomposing into different temporal lags and feature selection. This study investigates the impact of various feature selection techniques on the performance of ML algorithms for landslide displacement prediction. The Shuping and Baishuihe landslides in China’s Three Gorges Reservoir Area are used to comprehensively benchmark four prevalent ML algorithms. Both static ML models, including backpropagation neural network (BPNN), support vector machine (SVM), and dynamic models, such as long short-term memory (LSTM), and gated recurrent unit (GRU), are included. Each ML model is evaluated under three feature engineering techniques: raw multivariate time series, and feature selection under maximal information coefficient-partial autocorrelation function (MIC-PACF), or grey relational analysis-PACF (GRA-PACF). The results demonstrate that appropriate feature selection methods could significantly improve the performance of static ML models. In contrast, dynamic models effectively leverage inherent capabilities in capturing temporal dynamics within raw multivariate time series, seeing marginal gains with extensive feature engineering compared to no feature selection strategy. The optimal feature selection approach varies based on the ML model and specific landslide, highlighting the importance of case-specific assessments. The findings in this study offer guidance on integrating feature selection techniques with different machine learning models to maximize the robustness and generalizability of data-driven landslide displacement prediction frameworks.

A multi-agent deep reinforcement learning paradigm to improve the robustness and resilience of grid connected electric vehicle charging stations against the destructive effects of cyber-attacks

The rising deployment of electric vehicle charging stations (EVCS) in existing power grids and their integration with electric vehicles through communication systems are increasingly vulnerable to cyber-attacks, such as false data injection (FDI) and uncertainties of communication system. This research introduces an EVCS evaluation from a techno-economic standpoint, utilizing the developed and data-oriented TSKFS&MADRL technique to identify and rectify inaccuracies stemming from cyber-attacks like FDI and communication system delays. This method aims to enhance the resilience of EVCSs against sabotage attacks by ensuring fast dynamic responses. Initially, the study models the potential targets of hackers, such as communication systems and transducer sensors, and employs the Euclidean norm theory, weighted least square error method, and residual error technique to identify cyber-attacks resulting from FDI through the comparison of measured data with reference values based on probability distribution functions. Subsequently, the network control and recovery requirements are facilitated by the proposed controller. The proposed approach application has been demonstrated in the IEEE 33 bus, showcasing a 7.33 % lower experimental operation cost compared to the RL method and 12.15 % lower than the CNN method. Additionally, the FDI detection time is observed to be 40 % quicker than alternative methods based on the experimental outcomes.

TLF: Triple learning framework for intracranial aneurysms segmentation from unreliable labeled CTA scans

Intracranial aneurysm (IA) is a prevalent disease that poses a significant threat to human health. The use of computed tomography angiography (CTA) as a diagnostic tool for IAs remains time-consuming and challenging. Deep neural networks (DNNs) have made significant advancements in the field of medical image segmentation. Nevertheless, training large-scale DNNs demands substantial quantities of high-quality labeled data, making the annotation of numerous brain CTA scans a challenging endeavor. To address these challenges and effectively develop a robust IAs segmentation model from a large amount of unlabeled training data, we propose a triple learning framework (TLF). The framework primarily consists of three learning paradigms: pseudo-supervised learning, contrastive learning, and confident learning. This paper introduces an enhanced mean teacher model and voxel-selective strategy to conduct pseudo-supervised learning on unreliable labeled training data. Concurrently, we construct the positive and negative training pairs within the high-level semantic feature space to improve the overall learning efficiency of the TLF through contrastive learning. In addition, a multi-scale confident learning is proposed to correct unreliable labels, which enables the acquisition of broader local structural information instead of relying on individual voxels. To evaluate the effectiveness of our method, we conducted extensive experiments on a self-built database of hundreds of cases of brain CTA scans with IAs. Experimental results demonstrate that our method can effectively learn a robust CTA scan-based IAs segmentation model using unreliable labeled data, outperforming state-of-the-art methods in terms of segmentation accuracy. Codes are released at https://github.com/XueShuangqian/TLF.

Impact of TOC (Teacher Organizational Commitment) and VTLE (Virtual Teaching Learning Environment) on Teachers' Innovative Work Behavior (TIWB) Influencing ICT Skills and Employees Creativity

The use of technology in education has challenged traditional paradigms of teaching and learning, resulting in instructor adaptability and creativity. The rapid advancement of digitalization has resulted in an unparalleled array of tools and platforms that educators can employ to augment the learning process. Consequently, the objective of this study was to examine the relationships among teacher organizational commitment (TOC), teachers' innovative work behavior (TIWB), virtual teaching-learning environments (VTLE), employee creativity and ICT abilities. It also explored how technology readiness and virtual integration play a moderating influence. A quantitative study approach was used, with a broad sample of teachers from various educational institutions participating. Data was collected from 502 teachers using validated self-report surveys that measured the dimensions under research. The variables' relationships were examined using structural equation modeling. The results showed that teacher organizational commitment and creative work behavior are positively correlated. Virtual teaching-learning environments were discovered to have a considerable impact on innovative work behavior. ICT skills and employee creativity strongly were associated with innovative work behavior. Furthermore, virtual integration and technological readiness were recognized as important moderators that shape the interactions between commitment, virtual environments, innovation, and creativity. This study advances our understanding of the interactions between technology, commitment, innovation, and virtual worlds in influencing instructors' behavior. It emphasizes the significance of dedication and technological integration in promoting innovative teaching techniques. The study provides guidance for educators, educational organizations, and legislators who want to improve education in the digital era.

Incomplete label distribution learning via label correlation decomposition

Label distribution learning (LDL) has garnered increased attention in recent studies on label ambiguity. However, collecting complete annotations for LDL tasks is often time-consuming and labor-intensive compared to traditional learning paradigms. Therefore, designing effective incomplete LDL algorithms is crucial to broaden LDL’s application scope. In this paper, we propose a novel LDL algorithm, called Incomplete Label Distribution Learning via Label Correlation Decomposition (IncomLDL-LCD), which simultaneously learns label distributions and recovers missing description degrees of labels through label correlations. Specifically, we decompose the label correlation into sparse local label correlation and low-rank global label correlation using a soft-thresholding operator and a singular value thresholding operator, respectively. The former is utilized to capture the related label subsets necessary for reconstructing each possible label, while the latter focuses on extracting the coarse-grained semantic concepts from all labels and exploring the groupings of labels. Additionally, we develop an alternating solution with the accelerated proximal gradient descent method for optimization. Extensive experiments on 16 real-world data sets with varying degrees of missing annotations validate that our algorithm effectively handles incomplete LDL tasks and outperforms state-of-the-art algorithms.

Transfer learning guided discovery of efficient perovskite oxide for alkaline water oxidation

Perovskite oxides show promise for the oxygen evolution reaction. However, numerical chemical compositions remain unexplored due to inefficient trial-and-error methods for material discovery. Here, we develop a transfer learning paradigm incorporating a pre-trained model, ensemble learning, and active learning, enabling the prediction of undiscovered perovskite oxides with enhanced generalizability for this reaction. Screening 16,050 compositions leads to the identification and synthesis of 36 new perovskite oxides, including 13 pure perovskite structures. Pr0.1Sr0.9Co0.5Fe0.5O3 and Pr0.1Sr0.9Co0.5Fe0.3Mn0.2O3 exhibit low overpotentials of 327 mV and 315 mV at 10 mA cm−2, respectively. Electrochemical measurements reveal coexistence of absorbate evolution and lattice oxygen mechanisms for O-O coupling in both materials. Pr0.1Sr0.9Co0.5Fe0.3Mn0.2O3 demonstrates enhanced OH- affinity compared to Pr0.1Sr0.9Co0.5Fe0.5O3, with the emergence of oxo-bridged Mn-Co conjugate facilitating charge redistribution and dynamic reversibility of Olattice/VO, thereby slowing down Co dissolution. This work paves the way for accelerated discovery and development of high-performance perovskite oxide electrocatalysts for this reaction.

Long-tailed multi-domain generalization for fault diagnosis of rotating machinery under variable operating conditions

As the operating conditions (also known as domains) of rotating machinery become increasingly diverse, fault diagnosis has garnered growing attention. However, fault diagnosis frequently encounters challenges such as long-tailed data distributions, domain shifts in monitoring data, and the unavailability of target-domain data. Existing approaches can only address some of these challenges, limiting their applications. To address these challenges concurrently, we introduce a novel learning paradigm called long-tailed multi-domain generalized fault diagnosis (LMGFD) and propose a two-stage learning framework for LMGFD, comprising domain-invariant feature learning and balanced classifier learning. In the first stage, we leverage a balanced multi-order moment matching (BMMM) module to align subdomains with long-tailed distributions. Additionally, a balanced prototypical supervised contrastive (BPSC) module is developed to effectively alleviate the contrastive imbalance. The combination of BMMM and BPSC enables the effective learning of long-tailed domain-invariant features. In the second stage, we extend the focal loss to a multi-class version and re-weight it using effective sample numbers to strengthen tailed-class loss, thereby mitigating the overfitting problem. Experimental results on both a public dataset and a private dataset support the competitiveness and effectiveness of the proposed method. The findings suggest that we present a promising solution for fault diagnosis of rotating machinery under variable operating conditions.

SINNER: A Reward-Sensitive Algorithm for Imbalanced Malware Classification Using Neural Networks with Experience Replay

Reports produced by popular malware analysis services showed a disparity in samples available for different malware families. The unequal distribution between such classes can be attributed to several factors, such as technological advances and the application domain that seeks to infect a computer virus. Recent studies have demonstrated the effectiveness of deep learning (DL) algorithms when learning multi-class classification tasks using imbalanced datasets. This can be achieved by updating the learning function such that correct and incorrect predictions performed on the minority class are more rewarded or penalized, respectively. This procedure can be logically implemented by leveraging the deep reinforcement learning (DRL) paradigm through a proper formulation of the Markov decision process (MDP). This paper proposes SINNER, i.e., a DRL-based multi-class classifier that approaches the data imbalance problem at the algorithmic level by exploiting a redesigned reward function, which modifies the traditional MDP model used to learn this task. Based on the experimental results, the proposed formula appears to be successful. In addition, SINNER has been compared to several DL-based models that can handle class skew without relying on data-level techniques. Using three out of four datasets sourced from the existing literature, the proposed model achieved state-of-the-art classification performance.

Developing a privacy-preserving deep learning model for glaucoma detection: a multicentre study with federated learning

BackgroundDeep learning (DL) is promising to detect glaucoma. However, patients’ privacy and data security are major concerns when pooling all data for model development. We developed a privacy-preserving DL model...

An ensemble learning paradigm for subsurface stratigraphy from sparse measurements and augmented training images

The performance of computer vision-based techniques for stratigraphic modeling relies heavily on qualified training images to capture the complex stratigraphic connectivity. In geotechnical engineering, only limited training images are available for a specific site. Stochastic simulation modelling based on limited training data may be biased as the collected images that reflect prior geological knowledge may not encompass all potential stratigraphic patterns. Therefore, it is crucial to establish a high-quality, domain-specific training image database for effective stratigraphic modelling. In this study, an ensemble learning paradigm is proposed to tackle this issue and develop subsurface geological cross-sections from sparse data by reconstruction and redistribution of stratigraphic statistics revealed from limited training images. A domain-specific training image database is first established using generative adversarial networks (GAN) that enable the generation of arbitrary sized image samples from a single training image. Subsequently, multiple qualified image samples that are compatible with site-specific data are adaptively selected and utilized for the ensemble learning of geological cross-sections. The performance of the proposed framework is demonstrated using real geological cross-sections collected from a reclamation site and a tunnelling project in Hong Kong. The results indicate that the proposed method can effectively generate diverse image samples that encompass stratigraphic features beyond those reflected in a single training image. More importantly, the ensemble learning framework can capture the complex spatial stratigraphic connectivity of soil layers with enhanced prediction accuracy.

Predicting major adverse cardiac events using radiomics nomogram of pericoronary adipose tissue based on CCTA: A multi-center study.

The evolution of coronary atherosclerotic heart disease (CAD) is intricately linked to alterations in the pericoronary adipose tissue (PCAT). In recent epochs, characteristics of the PCAT have progressively ascended as focal points of research in CAD risk stratification and individualized clinical decision-making. Harnessing radiomic methodologies allows for the meticulous extraction of imaging features from these adipose deposits. Coupled with machine learning paradigms, we endeavor to establish predictive models for the onset of major adverse cardiovascular events (MACE). To appraise the predictive utility of radiomic features of PCAT derived from coronary computed tomography angiography (CCTA) in forecasting MACE. We retrospectively incorporated data from 314 suspected or confirmed CAD patients admitted to our institution from June 2019 to December 2022. An additional cohort of 242 patients from two external institutions was encompassed for external validation. The endpoint under consideration was the occurrence of MACE after a 1-year follow-up. MACE was delineated as cardiovascular mortality, newly diagnosed myocardial infarction, hospitalization (or re-hospitalization) for heart failure, and coronary target vessel revascularization occurring more than 30days post-CCTA examination. All enrolled patients underwent CCTA scanning. Radiomic features were meticulously extracted from the optimal diastolic phase axial slices of CCTA images. Feature reduction was achieved through a composite feature selection algorithm, laying the groundwork for the radiomic signature model. Both univariate and multivariate analyses were employed to assess clinical variables. A multifaceted logistic regression analysis facilitated the crafting of a clinical-radiological-radiomic combined model (or nomogram). Receiver operating characteristic (ROC) curves, calibration, and decision curve analyses (DCA) were delineated, with the area under the ROC curve (AUCs) computed to gauge the predictive prowess of the clinical model, radiomic model, and the synthesized ensemble. A total of 12 radiomic features closely associated with MACE were identified to establish the radiomic model. Multivariate logistic regression results demonstrated that smoking, age, hypertension, and dyslipidemia were significantly correlated with MACE. In the integrated nomogram, which amalgamated clinical, imaging, and radiomic parameters, the diagnostic performance was as follows: 0.970 AUC, 0.949 accuracy (ACC), 0.833 sensitivity (SEN), 0.981 specificity (SPE), 0.926 positive predictive value (PPV), and 0.955 negative predictive value (NPV). The calibration curve indicated a commendable concordance of the nomogram, and the decision curve analysis underscored its superior clinical utility. The integration of radiomic signatures from PCAT based on CCTA, clinical indices, and imaging parameters into a nomogram stands as a promising instrument for prognosticating MACE events.