Fast Learning Research Articles

Redox-based ion-gating reservoir consisting of (104) oriented LiCoO2 film, assisted by physical masking

Reservoir computing (RC) is a machine learning framework suitable for processing time series data, and is a computationally inexpensive and fast learning model. A physical reservoir is a hardware implementation of RC using a physical system, which is expected to become the social infrastructure of a data society that needs to process vast amounts of information. Ion-gating reservoirs (IGR) are compact and suitable for integration with various physical reservoirs, but the prediction accuracy and operating speed of redox-IGRs using WO3 as the channel are not sufficient due to irreversible Li+ trapping in the WO3 matrix during operation. Here, in order to enhance the computation performance of redox-IGRs, we developed a redox-based IGR using a (104) oriented LiCoO2 thin film with high electronic and ionic conductivity as a trap-free channel material. The subject IGR utilizes resistance change that is due to a redox reaction (LiCoO2 ⟺ Li1−xCoO2 + xLi+ + xe−) with the insertion and desertion of Li+. The prediction error in the subject IGR was reduced by 72% and the operation speed was increased by 4 times compared to the previously reported WO3, which changes are due to the nonlinear and reversible electrical response of LiCoO2 and the high dimensionality enhanced by a newly developed physical masking technique. This study has demonstrated the possibility of developing high-performance IGRs by utilizing materials with stronger nonlinearity and by increasing output dimensionality.

X3DFast model for classifying dairy cow behaviors based on a two-pathway architecture

Behavior is one of the important factors reflecting the health status of dairy cows, and when dairy cows encounter health problems, they exhibit different behavioral characteristics. Therefore, identifying dairy cow behavior not only helps in assessing their physiological health and disease treatment but also improves cow welfare, which is very important for the development of animal husbandry. The method of relying on human eyes to observe the behavior of dairy cows has problems such as high labor costs, high labor intensity, and high fatigue rates. Therefore, it is necessary to explore more effective technical means to identify cow behaviors more quickly and accurately and improve the intelligence level of dairy cow farming. Automatic recognition of dairy cow behavior has become a key technology for diagnosing dairy cow diseases, improving farm economic benefits and reducing animal elimination rates. Recently, deep learning for automated dairy cow behavior identification has become a research focus. However, in complex farming environments, dairy cow behaviors are characterized by multiscale features due to large scenes and long data collection distances. Traditional behavior recognition models cannot accurately recognize similar behavior features of dairy cows, such as those with similar visual characteristics, i.e., standing and walking. The behavior recognition method based on 3D convolution solves the problem of small visual feature differences in behavior recognition. However, due to the large number of model parameters, long inference time, and simple data background, it cannot meet the demand for real-time recognition of dairy cow behaviors in complex breeding environments. To address this, we developed an effective yet lightweight model for fast and accurate dairy cow behavior feature learning from video data. We focused on four common behaviors: standing, walking, lying, and mounting. We recorded videos of dairy cow behaviors at a dairy farm containing over one hundred cows using surveillance cameras. A robust model was built using a complex background dataset. We proposed a two-pathway X3DFast model based on spatiotemporal behavior features. The X3D and fast pathways were laterally connected to integrate spatial and temporal features. The X3D pathway extracted spatial features. The fast pathway with R(2 + 1)D convolution decomposed spatiotemporal features and transferred effective spatial features to the X3D pathway. An action model further enhanced X3D spatial modeling. Experiments showed that X3DFast achieved 98.49% top-1 accuracy, outperforming similar methods in identifying the four behaviors. The method we proposed can effectively identify similar dairy cow behaviors while improving inference speed, providing technical support for subsequent dairy cow behavior recognition and daily behavior statistics.

A new real-time maximum power point tracking scheme for PV-BASED microgrid STABILITY using online DEEP ridge extreme learning machine algorithm

A new deep representation-based Maximum Power Point Tracking (MPPT) controller is proposed in this paper for accurate Control references calculation in Photovoltaic (PV) based Micro Grid (MG) operation. The deep representation is obtained by two-step estimation: Data dimension reduction and MPPT Tracker towards optimal computation. The considered deep learning architecture is targeted for N number (large scale) of PV-based DGs, connected locally in the distribution system (DC link extended to AC utility). The collected data of solar irradiation (in W/m2) and PV panel temperature (in oC) profiles of local DGs are subjected to data dimensions using Extreme Learning Machine (ELM) based on Moore-Penrose inverse technique. The compressed represented PV-DG data is further communicated to the Tertiary Control side MPPT Tracker, where Ridge Regression-based ELM is presented for estimating Maximum Power Point Power (PMPP) and Voltage (VMPP) values for kth instant. The initial randomness present in the proposed Deep Representation based Ridge Regression Extreme Learning Machine (DR-RRELM) is further minimized by adopting Huber's characteristic distribution-based likelihood estimator. The proposed MPPT scheme is effectively implemented for accurate control reference in DC-DC and DC-AC converters in the MG. The proposed controller is also suitable for stability improvement at point of common coupling (PCC). Three different case studies such as past data verification, stability analysis under various operating conditions, irradiant change and source power variation. The efficacy of the proposed deep representation-based MPPT scheme is evidenced in MATLAB-based simulation. The proposed technique provides better tracking ability, faster learning and effective reference generation. The case study with irradiation change is validated in TMS320C6713 (32-bit) based Hardware-in-Loop (HIL) validation.

Cognitive and sensory capacity each contribute to the canine spatial bias.

Dogs interpret cues as being about location, which human infants would relate to objects. This spatial bias could shed light on the evolution of object-centered thought, however, research needs to rule out that this is not a by-product of dogs' weaker (compared to humans) visual capacities. In this study, we used a data set in which dogs were tested in two types of learning tasks (discrimination and reversal learning) with two types of rewarded cues (location and object features). In both tasks, dogs displayed spatial bias, that is, faster learning when the rewarded cue was a location. We investigated how sensory and cognitive capacity each contributes to this spatial bias. To this end, an estimate for general cognitive ability (g) was obtained from a battery of tests for some of the dogs. Cephalic index, a feature targeted in breeding and linked to differences in visual capacity, correlated negatively with the expression of spatial bias only in the easier discrimination learning task, while a negative correlation between g factor and spatial bias scores emerged in the more difficult reversal learning task. We conclude that dogs' spatial bias cannot be reduced to a sensory limitation and is easier to overcome with greater cognitive capacity.

Multi-view spatiotemporal learning for traffic forecasting

The spatiotemporal learning has been one of the most burning research topics in traffic data analytics for traffic forecasting, which is instrumental in developing the intelligent transportation systems. Capturing accurately and efficiently complex spatiotemporal dependencies of the road network is a critical prerequisite of the traffic spatiotemporal forecasting. This study proposes a multi-view spatiotemporal learning (MVSTL) framework, which combines the fast parallel learning (FPL) and the serial learning (SL) to juggle the receptive field with the fitting capability of the model. The FPL is proposed to capture the spatiotemporal correlations synchronously and efficiently by reducing the number of parameters significantly. As a supplement to the FPL, the SL, which extracts the spatial and temporal features serially, can expand the spatial/temporal receptive fields and avoid information overwriting. The evaluation of the MVSTL is carried out by employing four sets of traffic data from the LA freeway system, and the experiment results reveal that the MVSTL is able to perform better than the advanced methods in prediction accuracy.

Fast sparse Bayesian learning-based seismic resolution enhancement

Seismic high resolution processing plays a crucial role in seismic interpretation and reservoir characterization. However, the low and high frequency of seismic signal are lost owing to the effects of earth filtering and diffraction. The missing of low-frequency increases the energy of wavelet side lobe, which not only enhances the interference with adjacent events, but also lead to side lobe artifacts. High frequency attenuation is more serious, which significantly reduces the seismic resolution. In addition, the current resolution enhancement methods are difficult to achieve desirable result in noise contaminated environment. A novel resolution enhancement method that extends high and low frequencies simultaneously and improves the signal-to-noise ratio (SNR) under the framework of sparse decomposition theory is proposed. Firstly, an over-complete dictionary is constructed using non-zero phase Ricker wavelet and then the effective seismic signal is extracted by sparse decomposition using the fast sparse Bayesian learning (FSBL) algorithm. Secondly, the effective frequency range of the seismic signal is determined through a series of atoms obtained by sparse decomposition. Then, a desired broadband spectrum can be identified depending on this range. By approximating the broadband spectrum of seismic data, the weak high-frequency information can be extended. Furthermore, a correction term is introduced in frequency domain to suppress the side lobes of atoms. The side lobes are well suppressed without distorting the main lobe, meanwhile, both the low and high frequencies are extended. Finally, the validity of this method is verified by synthetic and field data. The results show that this method has better performance than the ordinary deconvolution and spectral whitening methods in processing high resolution data.

A reinforcement-probability Bayesian approach for strategic bidding and market clearing for renewable energy sources with uncertainty

With the participation of renewable energy sources (RESs) in the electricity market, the inherent uncertainty brings new challenges for strategic bidding and market clearing in the electricity market. In this paper, the market structure is formulated as a non-cooperative game among power suppliers, consumers, and the market operator. A reinforcement-probability Bayesian approach is proposed to obtain the optimal bidding strategy of RESs and achieve maximal social welfare at the market clearing, considering uncertain cost types of RESs and incomplete market information. First, a reinforcement-probability model is presented to obtain the optimal agent-type-strategy combination considering the uncertainty of cost type selection. A agent-type-probability algorithm is proposed for the model to update the mean of probability density function of cost types according to the expected profit. Second, an incremental-cost-based distributed algorithm is proposed to solve the market clearing problem, which accelerates the convergence rate without violating the system constraints via feedback from the scheduled power. Finally, a Bayesian probability model is proposed for RESs to learn the hidden information of the market and transform the incomplete information electricity market into a complete one. A Bayesian Nash equilibrium is reached in the model. To coordinate the information among market participants, the output of the Bayesian probability model is calculated by the Hysteretic Q-learning algorithm and used as the reward to update the agent-type-probability in the reinforcement-probability model. An illustrative example is presented to demonstrate the effectiveness of the proposed approach, which shows that the proposed approach has a better convergence performance and has a 2–6 times faster learning speed compared with the policy gradient and deep Q-network.

Speed Limits: There’s no such thing as a fast learner

Students master concepts through opportunities to practice them. They start at different levels of proficiency but, when provided with high-quality practice opportunities, learn at about the same speed. Yes, they will end in different places—but that’s because they have different starting lines, not because they are quicker or slower to learn.

THE LABELING BENEFIT IN FAST MAPPING AND EXPLICIT ENCODING

The aim of this study was exploring the role of labels in acquisition of information about new objects depending on the learning strategy. In order to reach this aim we investigated efficiency of identification of new objects presented through fast mapping or explicit encoding taking into consideration eye movement patterns. 32 subjects were presented with 8 new fruits 8 times each. The images were accompanied with unique audial labels in 50% of the cases. Visual search of the target image among distractors was used as an assessment task. The results showed that fast mapping speeded learning of information about new objects compared to explicit encoding. We found positive impact of unique labels of new fruits on the results of visual search task. This impact was stronger for fast mapping compared to explicit encoding.

52 Memory Learning Curve and in vivo Brain Pathology in Non-Demented Individuals with Autosomal Dominant Alzheimer’s Disease: Findings from the Colombia-Boston Biomarker Study

Objective:Associative memory is impacted early in Alzheimer’s disease (AD). Poorer performance on associative memory tests has been related to greater amyloid and regional tau burden in preclinical AD. Our group previously examined the association of brain pathology and performance on the Free and Cued Selective Reminding Test (FCSRT) in Autosomal Dominant Alzheimer’s Disease (ADAD), finding that associative memory summary scores distinguished non-demented mutation carriers from non-carriers several years before clinical onset of cognitive impairment. In the current study, we examined whether FCSRT learning slopes were associated with brain pathology in a sample of ADAD carriers and non-carriers.Participants and Methods:There were 119 participants including 57 non-demented carriers of the Colombian kindred with the Presenilin1 E280A mutation and 62 non-carrier family members (mean age= 36.3, 60% female). Participants were administered the Mini-Mental State Examination (MMSE), a measure of global cognitive status, and the FCSRT, which consists of three trials in which participants are asked to freely recall the same list of 16 items. It is a well-established measure known to be sensitive to early changes in AD. A subsample of 69 participants (32 carriers and 37 non-carriers) underwent positron emission tomography (PET) to measure in vivo cortical amyloid-beta (Pittsburgh compound B, PiB), and regional tau (Flortaucipir, FTP) burden in entorhinal and precuneus regions, which are among the earliest sites of tau accumulation in this ADAD population. Mann Whitney U tests, Spearman correlations, and chi-square tests were used to examine group differences and relations among variables of interest. Learning slope was calculated by subtracting the number of items freely recalled in FCSRT Trial 1 from the number of items freely recalled in Trial 3.Results:Compared to non-carriers, carriers had greater cortical amyloid-ß and regional tau burden, lower MMSE scores (mean [SD]: carriers= 27.5 [2.7]; non-carriers= 28.8 [1.0]), and lower scores on total immediate/ delayed free/ cued recall scores on the FCSRT (all p<.01). The groups did not differ on age, sex, or education level (all p> 0.05). In the whole sample and in carriers only, we found that higher MMSE scores were associated with higher learning slope, meaning faster learning (whole group p= 0.25, p= 0.006; carriers p= 0.30, p=0.029). In the whole sample, we found that lower learning slope was associated with higher levels of amyloid (p=-.34, p=.006) and tau in the left, right, and bilateral precuneus region (p=-.43, p<.001; p=-.46, p<.001; p=-.45, p<.001). In carriers only, lower learning slope was associated with higher tau burden in the left, right, and bilateral precuneus specifically (p=-.43, p=.017; p=-.48, p=.008; p=-.46, p=.010, respectively). No significant associations were found in non-carriers.Conclusions:These findings suggest that learning curves on an associative memory test may be sensitive to preclinical pathological changes in AD, specifically within the precuneus, a brain region known to be involved in cue reactivity, episodic memory retrieval, and mental imagery strategies. Future studies with larger samples are warranted to further examine associations between the FCSRT learning curves and regional tau accumulation in individuals with ADAD.

Theoretical limits on the speed of learning inverse models explain the rate of adaptation in arm reaching tasks

An essential aspect of human motor learning is the formation of inverse models, which map desired actions to motor commands. Inverse models can be learned by adjusting parameters in neural circuits to minimize errors in the performance of motor tasks through gradient descent. However, the theory of gradient descent establishes limits on the learning speed. Specifically, the eigenvalues of the Hessian of the error surface around a minimum determine the maximum speed of learning in a task. Here, we use this theoretical framework to analyze the speed of learning in different inverse model learning architectures in a set of isometric arm-reaching tasks. We show theoretically that, in these tasks, the error surface and, thus the speed of learning, are determined by the shapes of the force manipulability ellipsoid of the arm and the distribution of targets in the task. In particular, rounder manipulability ellipsoids generate a rounder error surface, allowing for faster learning of the inverse model. Rounder target distributions have a similar effect. We tested these predictions experimentally in a quasi-isometric reaching task with a visuomotor transformation. The experimental results were consistent with our theoretical predictions. Furthermore, our analysis accounts for the speed of learning in previous experiments with incompatible and compatible virtual surgery tasks, and with visuomotor rotation tasks with different numbers of targets. By identifying aspects of a task that influence the speed of learning, our results provide theoretical principles for the design of motor tasks that allow for faster learning.

Surface Roughness Prediction in Ultra-Precision Milling: An Extreme Learning Machine Method with Data Fusion.

This paper pioneers the use of the extreme learning machine (ELM) approach for surface roughness prediction in ultra-precision milling, leveraging the excellent fitting ability with small datasets and the fast learning speed of the extreme learning machine method. By providing abundant machining information, the machining parameters and force signal data are fused on the feature level to further improve ELM prediction accuracy. An ultra-precision milling experiment was designed and conducted to verify our proposed data-fusion-based ELM method. The results show that the ELM with data fusion outperforms other state-of-art methods in surface roughness prediction. It achieves an impressively low mean absolute percentage error of 1.6% while requiring a mere 18 s for model training.

Joint DR-DME grading classification using optimal feature selection-based deep graph correlation network

Diabetic macular edema (DME) and diabetic retinopathy (DR) are the leading causes of human blindness, and accurate grading of individual DME, individual DR, and the joint DR-DME is very important for the diagnosis of human eye diseases. However, the conventional methods failed to separate the features such as disease-specific, disease-dependent, and joint DR-DME, which resulted in poor grading accuracy. In addition, optimal feature selection is also vital in DR-DME grading classification for improving the performance of joint DR-DME grading classification. Therefore, this work focuses on the implementation of an advanced deep graph correlation learning model based on a joint DR-DME network (JDD-Net) for disease detection and classification from color fundus images. Initially, convolutional block attention module (CBAM) and joint disease attention (JDA) modules are combined to extract the DR-specific, DME-specific, and joint DR-DME disease-dependent features. Here, interdependent DR and DME features are separated by a CBAM-based channel-spatial split attention mechanism. In addition, an iterative random forest network (IRF-Net) is used to select the optimal features by adopting fast machine learning properties. Finally, a deep graph correlation network (DGCN) is used to classify the different diseases using a pre-trained model. The simulations conducted on the Indian Diabetic Retinopathy Image Dataset (IDRiD) disclose that the proposed JDD-Net results in improved individual DR, individual DME, and joint DR-DME performance as compared to state-of-the-art approaches with DR, DME, and joint DR-DME accuracy of 99.53%, 99.1%, and 99.01%, respectively.

Pendampingan Pembelajaran Membaca AlQuran Menggunakan Metode Tartil di Taman Pendidikan Quran Jamiatul Muksinin

The tartil method is considered a practical and fast Alquran learning method to help students read the Alquran well. The aim of this assistance is to provide learning pedagogy for teaching staff at TPQ Jamiatul Muksisnin in accordance with the methods applied, and to provide supporting facilities for teaching staff at TPQ Jamiatul Muksisnin. The service partners are all students at TPQ Jamiatul Muksisnin. The method used in this assistance is through a Participatory Action Research (PAR) approach oriented towards community empowerment where this assistance is carried out to solve problems that exist in the community, precisely at TPQ Jamiatul Muksinin, by applying learning through the tartil method it can provide satisfaction and increasing students' ability to read the Koran. The results of this assistance show that students are very enthusiastic and enjoy this activity because it brings other benefits in training their Alquran reading skills. It can be seen that students are able to understand the tartil method so they can read the Alquran well, correctly and fluently. These results also show that the tartil method can improve students' ability to read the Alquran precisely and accurately.

Cellulosic ethanol production: Assessment of the impacts of learning and plant capacity

Cellulosic ethanol has great potential to displace fossil fuels and reduce greenhouse gas emissions, but supply is low due to high cost. Learning-by-doing is a strategy to reduce cost with increases in capacity. This study attempts to understand how learning-by-doing affects the growth of cellulosic ethanol in synergy with plant capacity. We modeled a three-phase learning effect using a hybrid general equilibrium model to project the U.S cellulosic ethanol production. The reference case projects an annual production of 5.49 million gallons (MG) from 2030 to 2040. With strong policy support of a high mandate, the U.S. is projected to produce 171 MG of cellulosic ethanol in 2040. However, the learning-by-doing effect cannot be initiated because only 4 large-scale centralized plants will be built. When plant size reduces, learning-by-doing is enabled, and 2040 cellulosic ethanol production can reach 3.27 billion gallons. Similar result is predicted for the cases representing an actual biorefinery. Fast and widespread learning is estimated to greatly increase production when plants have lower capital cost. The findings show that policy incentives are critical to technology development when cost is a significant barrier. Additionally, building small-size plants is a strategy that will enable and facilitate the learning-by-doing effect.

Modulating temporal dynamics of performance across retinotopic locations enhances the generalization of perceptual learning

Human visual perception can be improved through perceptual learning. However, such learning is often specific to stimulus and learning conditions. Here, we explored how temporal dynamics of performance across conditions impact learning generalization. Participants performed a visual task, with the target at retinotopic location A. Then, the target was presented at location B either immediately after location A (same-session performance) or following a 48h consolidation period (different-session performance). Long-term generalization was measured the following week. Following initial training, both groups demonstrated generalization, consistent with previous accounts of fast learning. However, long-term generalization was enhanced in the same-session performance group. Consistently, improvements at locations A and B were correlated only following same-session performance, implying an integrated learning process across locations. The results support a new account of perceptual learning and generalization dynamics, suggesting that the temporal proximity of learning and consolidation of different conditions may integrate correlated learning processes, facilitating generalized learning.

Utilizing Mixed Training and Multi-Head Attention to Address Data Shift in AI-Based Electromagnetic Solvers for Nano-Structured Metamaterials.

When designing nano-structured metamaterials with an iterative optimization method, a fast deep learning solver is desirable to replace a time-consuming numerical solver, and the related issue of data shift is a subtle yet easily overlooked challenge. In this work, we explore the data shift challenge in an AI-based electromagnetic solver and present innovative solutions. Using a one-dimensional grating coupler as a case study, we demonstrate the presence of data shift through the probability density method and principal component analysis, and show the degradation of neural network performance through experiments dealing with data affected by data shift. We propose three effective strategies to mitigate the effects of data shift: mixed training, adding multi-head attention, and a comprehensive approach that combines both. The experimental results validate the efficacy of these approaches in addressing data shift. Specifically, the combination of mixed training and multi-head attention significantly reduces the mean absolute error, by approximately 36%, when applied to data affected by data shift. Our work provides crucial insights and guidance for AI-based electromagnetic solvers in the optimal design of nano-structured metamaterials.

Learning in Cooperative Multiagent Systems Using Cognitive and Machine Models

Developing effective multi-agent systems (MASs) is critical for many applications requiring collaboration and coordination with humans. Despite the rapid advance of multi-agent deep reinforcement learning (MADRL) in cooperative MASs, one of the major challenges that remain is the simultaneous learning and interaction of independent agents in dynamic environments in the presence of stochastic rewards. State-of-the-art MADRL models struggle to perform well in Coordinated Multi-agent Object Transportation Problems (CMOTPs) wherein agents must coordinate with each other and learn from stochastic rewards. In contrast, humans often learn rapidly to adapt to non-stationary environments that require coordination among people. In this article, motivated by the demonstrated ability of cognitive models based on Instance-based Learning Theory (IBLT) to capture human decisions in many dynamic decision-making tasks, we propose three variants of multi-agent IBL models (MAIBLs). The idea of these MAIBL algorithms is to combine the cognitive mechanisms of IBLT and the techniques of MADRL models to deal with coordination MASs in stochastic environments from the perspective of independent learners. We demonstrate that the MAIBL models exhibit faster learning and achieve better coordination in a dynamic CMOTP task with various settings of stochastic rewards compared to current MADRL models. We discuss the benefits of integrating cognitive insights into MADRL models.

Nonexact oracle inequalities, r-learnability, and fast rates

As an extension of the standard paradigm in statistical learning theory, we introduce the concept of r-learnability, 0<r≤1, which is a notion very closely related to that of nonexact oracle inequalities (see Lecue and Mendelson (2012) [7]). The r-learnability concept can enable so-called fast learning rates (along with corresponding sample complexity-type bounds) to be established at the cost of multiplying the approximation error term by an extra (1+r)-factor in the learning error estimate. We establish a new, general r-learning bound (nonexact oracle inequality) yielding fast learning rates in probability (up to at most a logarithmic factor) for proper learning in the general setting of an agnostic model, essentially only assuming a uniformly bounded squared loss function and a hypothesis class of finite VC-dimension (that is, finite pseudo-dimension).

Winning the Lottery With Neural Connectivity Constraints: Faster Learning Across Cognitive Tasks With Spatially Constrained Sparse RNNs.

Recurrent neural networks (RNNs) are often used to model circuits in the brain and can solve a variety of difficult computational problems requiring memory, error correction, or selection (Hopfield, 1982; Maass etal., 2002; Maass, 2011). However, fully connected RNNs contrast structurally with their biological counterparts, which are extremely sparse (about 0.1%). Motivated by the neocortex, where neural connectivity is constrained by physical distance along cortical sheets and other synaptic wiring costs, we introduce locality masked RNNs (LM-RNNs) that use task-agnostic predetermined graphs with sparsity as low as 4%. We study LM-RNNs in a multitask learning setting relevant to cognitive systems neuroscience with a commonly used set of tasks, 20-Cog-tasks (Yang etal., 2019). We show through reductio ad absurdum that 20-Cog-tasks can be solved by a small pool of separated autapses that we can mechanistically analyze and understand. Thus, these tasks fall short of the goal of inducing complex recurrent dynamics and modular structure in RNNs. We next contribute a new cognitive multitask battery, Mod-Cog, consisting of up to 132 tasks that expands by about seven-fold the number of tasks and task complexity of 20-Cog-tasks. Importantly, while autapses can solve the simple 20-Cog-tasks, the expanded task set requires richer neural architectures and continuous attractor dynamics. On these tasks, we show that LM-RNNs with an optimal sparsity result in faster training and better data efficiency than fully connected networks.