Simple Learning Model Research Articles

Deep learning hybrid models with multivariate variational mode decomposition for estimating daily solar radiation

Solar energy is one of the renewable and clean energy sources. Accurate solar radiation (SR) estimates are therefore needed in solar energy applications. Firstly, two deep learning models, including gated recurrent unit (GRU) and long short-term memory (LSTM), were developed in this study. Next, a data pre-processing technique named multivariate variational mode decomposition (MVMD) was used to construct the MVMD-GRU and MVMD-LSTM hybrid models. To better test the performance of proposed simple and hybrid models, four stations located in the Illinois State of the USA (i.e., Dixon Springs, Fairfield, Rend Lake, and Carbondale) were considered as the study sites. Whole the simple and hybrid models were established under two different strategies, i.e., local and external. In the local strategy, SR of each location was estimated using the minimum and maximum air temperatures from the same station. While, minimum and maximum air temperatures as well as SR data from the nearby station were utilized in external strategy to estimate SR time series of any target site. Root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2) metrics were used when evaluating the models performances. The overall results revealed that the proposed MVMD-GRU and MVMD-LSTM hybrid models illustrated better SR estimates compared to the simple GRU and LSTM in both the local and external strategies. The values of error metrics obtained for the superior hybrid models (i.e., MVMD-LSTM) during the testing period were as: RMSE = 2.532 MJ/m2.day, MAE = 1.921 MJ/m2.day, R2 = 0.916 at Dixon Springs; RMSE = 2.476 MJ/m2.day, MAE = 1.878 MJ/m2.day, R2 = 0.921 at Fairfield; RMSE = 2.359 MJ/m2.day, MAE = 1.780 MJ/m2.day, R2 = 0.924 at Rend Lake; RMSE = 2.576 MJ/m2.day, MAE = 1.941 MJ/m2.day, R2 = 0.914 at Carbondale. Therefore, the coupled models proposed in this study can be possibly recommended as suitable alternatives to the simple deep learning models with a reliable precision in estimating SR time series.

Perceptions About Monetary Policy

Abstract We estimate perceptions about the Federal Reserve’s monetary policy rule from panel data on professional forecasts of interest rates and macroeconomic conditions. The perceived dependence of the federal funds rate on economic conditions varies substantially over time, in particular over the monetary policy cycle. Forecasters update their perceptions about the Fed’s policy rule in response to monetary policy actions, measured by high-frequency interest rate surprises, suggesting that they have imperfect information about the rule. Monetary policy perceptions matter for monetary transmission, as they affect the sensitivity of interest rates to macroeconomic news, term premia in long-term bonds, and the response of the stock market to monetary policy surprises. A simple learning model with forecaster heterogeneity and incomplete information about the policy rule motivates and explains our empirical findings.

Spatio-Temporal Attention and Gaussian Processes for Personalized Video Gaze Estimation.

Gaze is an essential prompt for analyzing human behavior and attention. Recently, there has been an increasing interest in determining gaze direction from facial videos. However, video gaze estimation faces significant challenges, such as understanding the dynamic evolution of gaze in video sequences, dealing with static backgrounds, and adapting to variations in illumination. To address these challenges, we propose a simple and novel deep learning model designed to estimate gaze from videos, incorporating a specialized attention module. Our method employs a spatial attention mechanism that tracks spatial dynamics within videos. This technique enables accurate gaze direction prediction through a temporal sequence model, adeptly transforming spatial observations into temporal insights, thereby significantly improving gaze estimation accuracy. Additionally, our approach integrates Gaussian processes to include individual-specific traits, facilitating the personalization of our model with just a few labeled samples. Experimental results confirm the efficacy of the proposed approach, demonstrating its success in both within-dataset and cross-dataset settings. Specifically, our proposed approach achieves state-of-the-art performance on the Gaze360 dataset, improving by 2.5° without personalization. Further, by personalizing the model with just three samples, we achieved an additional improvement of 0.8°. The code and pre-trained models are available at https://github.com/jswati31/stage.

A federated learning approach to network intrusion detection using residual networks in industrial IoT networks

This paper introduces a sophisticated approach to network security, with a primary emphasis on utilizing deep learning for intrusion detection. In real-world scenarios, the high dimensionality of training data poses challenges for simple deep learning models and can lead to vanishing gradient issues with complex neural networks. Additionally, uploading network traffic data to a central server for training raises privacy concerns. To tackle these issues, the paper introduces a Residual Network (ResNet)-based deep learning model trained using a federated learning approach. The ResNet effectively tackles the vanishing gradient problem, while federated learning enables multiple Internet Service Providers (ISPs) or clients to engage in joint training without sharing their data with third parties. This approach enhances accuracy through collaborative learning while maintaining privacy. Experimental results on the X-IIoTID dataset indicate that the proposed model outperforms conventional deep learning and machine learning methods in terms of accuracy and other metrics used for evaluation. Specifically, the proposed methodology achieved 99.43% accuracy in a centralized environment and 99.16% accuracy in a federated environment.

Deepfake on Social Media: Harnessing Deep Learning for Identifying Falsified Tweet

Recent advancements in natural language production have enabled the creation of sophisticated deepfake social media messages, posing a significant threat to public discourse. In response, this study focuses on the development of reliable detection methods for identifying automated text on websites such as Twitter. Leveraging Tweepfake, a publicly accessible dataset, a simple deep learning model employing tf-tf-idf, word2vec and tokenizer from keras library word embeddings and a typical architecture for a convolutional neural network (CNN) is proposed. Comparative analysis against baseline methods, including various feature-based approaches and various forms of deep learning, such Long Short-Term Memory (LSTM), demonstrates the suggested method's greater performance. Experimental results showcase an impressive accuracy of 91% in accurately classifying tweet data either bot-generated or human-generated. This research contributes to the ongoing efforts to combat the proliferation of deepfake content on social media platforms. Keywords: Deep Learning, Machine Learning, Deepfake Detection, Text Classification

Exploring simple triplet representation learning

Fully supervised learning methods necessitate a substantial volume of labelled training instances, a process that is typically both labour-intensive and costly. In the realm of medical image analysis, this issue is further amplified, as annotated medical images are considerably more scarce than their unlabelled counterparts. Consequently, leveraging unlabelled images to extract meaningful underlying knowledge presents a formidable challenge in medical image analysis. This paper introduces a simple triple-view unsupervised representation learning model (SimTrip) combined with a triple-view architecture and loss function, aiming to learn meaningful inherent knowledge efficiently from unlabelled data with small batch size. With the meaningful representation extracted from unlabelled data, our model demonstrates exemplary performance across two medical image datasets. It achieves this using only partial labels and outperforms other state-of-the-art methods. The method we present herein offers a novel paradigm for unsupervised representation learning, establishing a baseline that is poised to inspire the development of more intricate SimTrip-based methods across a spectrum of computer vision applications. Code and user guide are released at https://github.com/JerryRollingUp/SimTripSystem, the system also runs at http://43.131.9.159:5000/.

Motor learning changes the axon initial segment of the spinal motoneuron.

We are studying the mechanisms of H-reflex operant conditioning, a simple form of learning. Modelling studies in the literature and our previous data suggested that changes in the axon initial segment (AIS) might contribute. To explore this, we used blinded quantitative histological and immunohistochemical methods to study in adult rats the impact of H-reflex conditioning on the AIS of the spinal motoneuron that produces the reflex. Successful, but not unsuccessful, H-reflex up-conditioning was associated with greater AIS length and distance from soma; greater length correlated with greater H-reflex increase. Modelling studies in the literature suggest that these increases may increase motoneuron excitability, supporting the hypothesis that they may contribute to H-reflex increase. Up-conditioning did not affect AIS ankyrin G (AnkG) immunoreactivity (IR), p-p38 protein kinase IR, or GABAergic terminals. Successful, but not unsuccessful, H-reflex down-conditioning was associated with more GABAergic terminals on the AIS, weaker AnkG-IR, and stronger p-p38-IR. More GABAergic terminals and weaker AnkG-IR correlated with greater H-reflex decrease. These changes might potentially contribute to the positive shift in motoneuron firing threshold underlying H-reflex decrease; they are consistent with modelling suggesting that sodium channel change may be responsible. H-reflex down-conditioning did not affect AIS dimensions. This evidence that AIS plasticity is associated with and might contribute to H-reflex conditioning adds to evidence that motor learning involves both spinal and brain plasticity, and both neuronal and synaptic plasticity. AIS properties of spinal motoneurons are likely to reflect the combined influence of all the motor skills that share these motoneurons. KEY POINTS: Neuronal action potentials normally begin in the axon initial segment (AIS). AIS plasticity affects neuronal excitability in development and disease. Whether it does so in learning is unknown. Operant conditioning of a spinal reflex, a simple learning model, changes the rat spinal motoneuron AIS. Successful, but not unsuccessful, H-reflex up-conditioning is associated with greater AIS length and distance from soma. Successful, but not unsuccessful, down-conditioning is associated with more AIS GABAergic terminals, less ankyrin G, and more p-p38 protein kinase. The associations between AIS plasticity and successful H-reflex conditioning are consistent with those between AIS plasticity and functional changes in development and disease, and with those predicted by modelling studies in the literature. Motor learning changes neurons and synapses in spinal cord and brain. Because spinal motoneurons are the final common pathway for behaviour, their AIS properties probably reflect the combined impact of all the behaviours that use these motoneurons.

Conditioning and pseudoconditioning differently change intrinsic excitability of inhibitory interneurons in the neocortex.

Many studies indicate a broad role of various classes of GABAergic interneurons in the processes related to learning. However, little is known about how the learning process affects intrinsic excitability of specific classes of interneurons in the neocortex. To determine this, we employed a simple model of conditional learning in mice where vibrissae stimulation was used as a conditioned stimulus and a tail shock as an unconditioned one. In vitro whole-cell patch-clamp recordings showed an increase in intrinsic excitability of low-threshold spiking somatostatin-expressing interneurons (SST-INs) in layer 4 (L4) of the somatosensory (barrel) cortex after the conditioning paradigm. In contrast, pseudoconditioning reduced intrinsic excitability of SST-LTS, parvalbumin-expressing interneurons (PV-INs), and vasoactive intestinal polypeptide-expressing interneurons (VIP-INs) with accommodating pattern in L4 of the barrel cortex. In general, increased intrinsic excitability was accompanied by narrowing of action potentials (APs), whereas decreased intrinsic excitability coincided with AP broadening. Altogether, these results show that both conditioning and pseudoconditioning lead to plastic changes in intrinsic excitability of GABAergic interneurons in a cell-specific manner. In this way, changes in intrinsic excitability can be perceived as a common mechanism of learning-induced plasticity in the GABAergic system.

Machine Learning-Based Prediction of Glioma IDH Gene Mutation Status Using Physio-Metabolic MRI of Oxygen Metabolism and Neovascularization (A Bicenter Study).

The mutational status of the isocitrate dehydrogenase (IDH) gene plays a key role in the treatment of glioma patients because it is known to affect energy metabolism pathways relevant to glioma. Physio-metabolic magnetic resonance imaging (MRI) enables the non-invasive analysis of oxygen metabolism and tissue hypoxia as well as associated neovascularization and microvascular architecture. However, evaluating such complex neuroimaging data requires computational support. Traditional machine learning algorithms and simple deep learning models were trained with radiomic features from clinical MRI (cMRI) or physio-metabolic MRI data. A total of 215 patients (first center: 166 participants + 16 participants for independent internal testing of the algorithms versus second site: 33 participants for independent external testing) were enrolled using two different physio-metabolic MRI protocols. The algorithms trained with physio-metabolic data demonstrated the best classification performance in independent internal testing: precision, 91.7%; accuracy, 87.5%; area under the receiver operating curve (AUROC), 0.979. In external testing, traditional machine learning models trained with cMRI data exhibited the best IDH classification results: precision, 84.9%; accuracy, 81.8%; and AUROC, 0.879. The poor performance for the physio-metabolic MRI approach appears to be explainable by site-dependent differences in data acquisition methodologies. The physio-metabolic MRI approach potentially supports reliable classification of IDH gene status in the presurgical stage of glioma patients. However, non-standardized protocols limit the level of evidence and underlie the need for a reproducible framework of data acquisition techniques.

A Refined Wind Power Forecasting Method with High Temporal Resolution Based on Light Convolutional Neural Network Architecture

With a large proportion of wind farms connected to the power grid, it brings more pressure on the stable operation of power systems in shorter time scales. Efficient and accurate scheduling, operation control and decision making require high time resolution power forecasting algorithms with higher accuracy and real-time performance. In this paper, we innovatively propose a high temporal resolution wind power forecasting method based on a light convolutional architecture—DC_LCNN. The method starts from the source data and novelly designs the dual-channel data input mode to provide different combinations of feature data for the model, thus improving the upper limit of the learning ability of the whole model. The dual-channel convolutional neural network (CNN) structure extracts different spatial and temporal constraints of the input features. The light global maximum pooling method replaces the flat operation combined with the fully connected (FC) forecasting method in the traditional CNN, extracts the most significant features of the global method, and directly performs data downscaling at the same time, which significantly improves the forecasting accuracy and efficiency of the model. In this paper, the experiments are carried out on the 1 s resolution data of the actual wind field, and the single-step forecasting task with 1 s ahead of time and the multi-step forecasting task with 1~10 s ahead of time are executed, respectively. Comparing the experimental results with the classical deep learning models in the current field, the proposed model shows absolute accuracy advantages on both forecasting tasks. This also shows that the light architecture design based on simple deep learning models is also a good solution in performing high time resolution wind power forecasting tasks.

Development of a Simple Game-Based PJOK Learning Model to Improve Locomotor Basic Movements and Cooperation of Lower Grade Elementary School Students

This study aims to determine the correlation between the Problem-Based Learning learning model and the Discovery Learning model with students' creativity and learning motivation. This research uses a quantitative approach of correlation type. The sample in this study used purposive sampling, namely grade 6 with a total of 40 students, consisting of 19 students at SD Negeri Awu Awu and 21 students at SD Negeri Wingkoharjo. Data collection using observation sheets and questionnaire statements. Data analysis in this study used Product moment Correlation Test and t-Independent Test. The results of this study indicate that there is a significant correlation between the application of the Problem-Based Learning learning model and the learning creativity of students at SD Negeri Awu Awu with a pearson correlation value = 0.660 and a p-value (sig-2tailed) of 0.002. There is a significant correlation between the application of the Problem-Based Learning learning model and student learning motivation at SD Negeri Awu Awu with a pearson correlation value = 0.834 and a p-value (sig-2tailed) of 0.000. There is a significant correlation between the application of the Discovery Learning learning model and students' learning creativity at SD Negeri Wingkoharjo with a pearson correlation value = 0.660 and a p-value (sig-2tailed) of 0.001. There is a significant correlation between the application of the Discovery Learning learning model and student learning motivation at SD Negeri Wingkoharjo with a pearson correlation value = 0.501 and a p-value (sig-2tailed) of 0.021. There is no significant difference between creativity and motivation of students in the application of both learning models with a significance value of 0.064 for learning creativity and with a significance value of 0.477 for learning motivation.

Network structure shapes the impact of diversity in collective learning

It is widely believed that diversity arising from different skills enhances the performance of teams, and in particular, their ability to learn and innovate. However, diversity has also been associated with negative effects on the communication and coordination within collectives. Yet, despite the importance of diversity as a concept, we still lack a mechanistic understanding of how its impact is shaped by the underlying social network. To fill this gap, we model skill diversity within a simple model of collective learning and show that its effect on collective performance differs depending on the complexity of the task and the network density. In particular, we find that diversity consistently impairs performance in simple tasks. In contrast, in complex tasks, link density modifies the effect of diversity: while homogeneous populations outperform diverse ones in sparse networks, the opposite is true in dense networks, where diversity boosts collective performance. Our findings also provide insight on how to forge teams in an increasingly interconnected world: the more we are connected, the more we can benefit from diversity to solve complex problems.

Problem-Based Learning Model: Its Effectiveness in Improving Creative Thinking Skills of Students with Different Academic Abilities

This study aims to: 1) Test the effectiveness of using simple and complex problem-based learning models in improving students' creative thinking abilities; 2) Test the effectiveness of simple and complex problem-based learning models in improving students' creative thinking skills based on different academic abilities. This research is quasi-experimental with a pretest-posttest nonequivalent control group design involved 96 students grade VII taken by cluster random sampling included a control class with a simple problem-solving model (PSS), experimental class 1 using the Creative Problem Solving (CPS) learning model, and experimental class 2 using the IDEAL-Problem solving (IPS) learning model. The hypothesis test using N-Gain Score and ANCOVA. The results showed: 1) problem-based learning models are generally effective in improving students' creative thinking abilities. PSS learning model (N-Gain 46%, less effective); IPS learning model (N-Gain 60% is quite effective; CPS learning model (N-Gain 71% effective). The order of effectiveness of the three learning models is CPS, IPS, PSS.; 2) Problem-based learning models are generally effective in improving students’ creative thinking abilities with different academic abilities. The percentage of effectiveness of the three models is the PSS is ineffective (20.13%) or less effective (34.38%); IPS is quite effective (62.50%); CPS is effective (43.75%). Based on the study's results, it can be concluded that 1) Problem-based learning models (PSS, CPS, IPS) can improve students' creative thinking skills. 2) The CPS is most effective in improving students' creative thinking skills.

Individual evolutionary learning in repeated beauty contest games

The Individual Evolutionary Learning (IEL) algorithm was proposed as a portable learning model for games with large strategy spaces. In principle, IEL benchmark simulations could substitute or supplement expensive experiments with human subjects. We evaluate the ability of the IEL model to replicate experimental findings observed in repeated Keynesian Beauty Contest (KBC) games, which have a large strategy space. The IEL specification with standard parameter values is able to capture major dynamical features and differences between treatments in both one-dimensional (Nagel, 1995; Duffy and Nagel, 1997) and two-dimensional (Anufriev et al., 2022b) versions of KBC games. We compare IEL with some other simple learning models and find that it performs relatively better across multiple treatments. We also use IEL to predict behavior in repeated KBC games that have not yet been conducted experimentally.

Constraining below-threshold radio source counts with machine learning

We propose a machine-learning-based technique to determine the number density of radio sources as a function of their flux density, for use in next-generation radio surveys. The method uses a convolutional neural network trained on simulations of the radio sky to predict the number of sources in several flux bins. To train the network, we adopt a supervised approach wherein we simulate training data stemming from a large domain of possible number count models going down to fluxes a factor of 100 below the threshold for source detection. We test the model reconstruction capabilities as well as benchmark the expected uncertainties in the model predictions, observing good performance for fluxes down to a factor of ten below the threshold. This work demonstrates that the capabilities of simple deep learning models for radio astronomy can be useful tools for future surveys.

Don't throw the associative baby out with the Bayesian bathwater: Children are more associative when reasoning retrospectively under information processing demands.

Causal reasoning is a fundamental cognitive ability that enables individualsto learn about the complex interactions in the world around them. However, the mechanisms that underpin causal reasoning are not well understood. For example, it remains unresolved whether children's causal inferences are best explained by Bayesian inference or associative learning. The two experiments and computational models reported here were designed to examine whether 5- and 6-year-olds will retrospectively reevaluate objects-that is, adjust their beliefs about the causal status of some objects presented at an earlier point in time based on the observed causal status of other objects presented at a later point in time-when asked to reason about 3 and 4 objects and under varying degrees of information processing demands. Additionally, the experiments and models were designed to determine whether children's retrospective reevaluations were best explained by associative learning, Bayesian inference, or some combination of both. The results indicated that participants retrospectively reevaluated causal inferences under minimal information-processing demands (Experiment 1) but failed to do so under greater information processing demands (Experiment 2) and that their performance was better captured by an associative learning mechanism, with less support for descriptions that rely on Bayesian inference. RESEARCH HIGHLIGHTS: Five- and 6-year-old children engage in retrospective reevaluation under minimal information-processing demands (Experiment 1). Five- and 6-year-old children do not engage in retrospective reevaluation under more extensive information-processing demands (Experiment 2). Across both experiments, children's retrospective reevaluations were better explained by a simple associative learning model, with only minimal support for a simple Bayesian model. These data contribute to our understanding of the cognitive mechanisms by which children make causal judgements.

Perceptions about Monetary Policy

We estimate perceptions about the Federal Reserve’s monetary policy rule from panel data on professional forecasts of interest rates and macroeconomic conditions. The perceived dependence of the federal funds rate on economic conditions varies substantially over time, including over the monetary policy cycle. Forecasters update their perceptions about the Fed’s policy rule in response to monetary policy actions, measured by high-frequency interest rate surprises, suggesting that they have imperfect information about this rule. Monetary policy perceptions matter for monetary transmission, as they affect the sensitivity of interest rates to macroeconomic news, term premia in long-term bonds, and the response of the stock market to monetary policy surprises. A simple learning model with forecaster heterogeneity and incomplete information about the policy rule motivates and explains our empirical findings.

The pigeon as a machine: Complex category structures can be acquired by a simple associative model

Never known for its smarts, the pigeon has proven to be a prodigious classifier of complex visual stimuli. What explains its surprising success? Does it possess elaborate executive functions akin to those deployed by humans? Or does it effectively deploy an unheralded, but powerful associative learning mechanism? In a series of experiments, we first confirm that pigeons can learn a variety of category structures- some devised to foil the use of advanced cognitive processes. We then contrive a simple associative learning model to see how effectively the model learns the same tasks given to pigeons. The close fit of the associative model to pigeons' categorization behavior provides unprecedented support for associative learning as a viable mechanism for mastering complex category structures and for the pigeon's using this mechanism to adapt to a rich visual world. This model will help guide future neuroscientific research into the biological substrates of visual cognition.

Interpretability of simple RNN and GRU deep learning models used to map land susceptibility to gully erosion

Gully erosion possess a serious hazard to critical resources such as soil, water, and vegetation cover within watersheds. Therefore, spatial maps of gully erosion hazards can be instrumental in mitigating its negative consequences. Among the various methods used to explore and map gully erosion, advanced learning techniques, especially deep learning (DL) models, are highly capable of spatial mapping and can provide accurate predictions for generating spatial maps of gully erosion at different scales (e.g., local, regional, continental, and global). In this paper, we applied two DL models, namely a simple recurrent neural network (RNN) and a gated recurrent unit (GRU), to map land susceptibility to gully erosion in the Shamil-Minab plain, Hormozgan province, southern Iran. To address the inherent black box nature of DL models, we applied three novel interpretability methods consisting of SHaply Additive explanation (SHAP), ceteris paribus and partial dependence (CP-PD) profiles and permutation feature importance (PFI). Using the Boruta algorithm, we identified seven important features that control gully erosion: soil bulk density, clay content, elevation, land use type, vegetation cover, sand content, and silt content. These features, along with an inventory map of gully erosion (based on a 70 % training dataset and 30 % test dataset), were used to generate spatial maps of gully erosion using DL models. According to the Kolmogorov–Smirnov (KS) statistic performance assessment measure, the simple RNN model (with KS = 91.6) outperformed the GRU model (with KS = 66.6). Based on the results from the simple RNN model, 7.4 %, 14.5 %, 18.9 %, 31.2 % and 28 % of total area of the plain were classified as very-low, low, moderate, high and very-high hazard classes, respectively. According to SHAP plots, CP-PD profiles, and PFI measures, soil silt content, vegetation cover (NDVI) and land use type had the highest impact on the model's output. Overall, the DL modelling techniques and interpretation methods used in this study proved to be helpful in generating spatial maps of soil erosion hazard, especially gully erosion. Their interpretability can support watershed sustainable management.

A Deep Learning Approach to Intrusion Detection and Segmentation in Pellet Fuels Using Microscopic Images.

Pellet fuels are nowadays commonly used as a heat source for food preparation. Unfortunately, they may contain intrusions which might be harmful for humans and the environment. The intrusions can be identified precisely using immersed microscopy analysis. The aim of this study is to investigate the possibility of autonomous identification of selected classes of intrusions using relatively simple deep learning models. The semantic segmentation was chosen as a method for impurity identification in the microscopic image. Three architectures of deep networks based on UNet architecture were examined. The networks contained the same depth as UNet but with a successively limited number of filters. The input image influence on the segmentation results was also examined. The efficiency of the network was assessed using the intersection over union index. The results showed an easily observable impact of the filter used on segmentation efficiency. The influence of the input image resolution is not so clear, and even the lowest (256 × 256 pixels) resolution used gave satisfactory results. The biggest (but still smaller than originally proposed UNet) network yielded segmentation quality good enough for practical applications. The simpler one was also applicable, although the quality of the segmentation decreased considerably. The simplest network gave poor results and is not suitable in applications. The two proposed networks can be used as a support for domain experts in practical applications.