Dictionary-based Learning Research Articles

Dictionary-enabled efficient training of ConvNets for image classification

Convolutional networks (ConvNets) are computationally expensive but well known for their performance on image data. One way to reduce their complexity is to explore inherited data sparsity. However, since the gradients involved in ConvNets require dynamic updates, applying data sparsity in the training step is not straightforward. Dictionary-based learning methods can be useful since they encode the original data in a sparse form. This paper proposes a new dictionary-based training paradigm for ConvNets by exploiting redundancy in the training data while keeping the distinctive features intact. The ConvNet is then trained on the reduced, sparse dataset. The new approach significantly reduces the training time without compromising accuracy.To the best of our knowledge, this is the first implementation of ConvNet on dictionary-based sparse training data. The proposed method is validated on three publicly available datasets –MNIST, USPS, and MNIST FASHION. The experimental results show a significant reduction of 4.5 times in the overall computational burden of vanilla ConvNet for all the datasets. Whereas the accuracy is intact at 97.21% for MNIST, 96.81% for USPS, and 88.4% for FASHION datasets. These results are comparable to state-of-the-art algorithms, such as ResNet-{18,34,50}, trained on the full training dataset.

A Blended Soft Computing Model for Stock Value Prediction

Stock investments play a crucial role in deciding the global economic growth of the country. Investors can optimize profit and avoid risk through accurate stock value prediction models, which motivates researchers to work on various aspects of correlated features and predictive models for stock value prediction. The existing stock value prediction models used data like Twitter, microblogs, price history, and Google trends. On the other hand, Domain-specific dictionary-based deep learning evolved as a competitive model for alternative models in stock value prediction. But accuracy of these models depends on the quality of the input, the correlation among the features, and the correctness of the sentiment scores generated for the dictionary terms. Financial news sentiment analysis for stock value prediction with dictionary-based learning needs attention in improving the quality of the input and dictionary term’s sentiment score generation. The present research aims to develop a Blended soft computing model for stock value prediction (BSCM) with cooperative fusion and dictionary-based deep learning. In the current work, six Indian stocks that cover uptrend, sideways, and downtrends characteristics are considered with stock price histories and news headlines from 8th August 2016 to 31st March 2023, i.e., 2427 days. The number of records in the price history dataset is 14,562, and the news headlines dataset is 46,213. The performance of the stock value prediction can be improved by taking advantage of multi-source information and context-aware learning. The present research aims to achieve three objectives: 1. Apply cooperative fusion to combine the news headlines and price history of stocks collected from multiple sources to improve the quality of the input with correlated features. 2. Build a dictionary, FNSentiment, with a novel strategy. 3. Predict stock values using FNSentiment and News Sentiment Prediction Model (NSPM) integration. In the experimentation, the proposed model outperformed the state-of-the-art models with an accuracy of 91.11, RMSE of 10.35, MAPE of 0.02, and MAE of 2.74.

Discovery of nonlinear dynamical systems using a Runge-Kutta inspired dictionary-based sparse regression approach.

In this work, we blend machine learning and dictionary-based learning with numerical analysis tools to discover differential equations from noisy and sparsely sampled measurement data of time-dependent processes. We use the fact that given a dictionary containing large candidate nonlinear functions, dynamical models can often be described by a few appropriately chosen basis functions. As a result, we obtain parsimonious models that can be better interpreted by practitioners, and potentially generalize better beyond the sampling regime than black-box modelling. In this work, we integrate a numerical integration framework with dictionary learning that yields differential equations without requiring or approximating derivative information at any stage. Hence, it is utterly effective for corrupted and sparsely sampled data. We discuss its extension to governing equations, containing rational nonlinearities that typically appear in biological networks. Moreover, we generalized the method to governing equations subject to parameter variations and externally controlled inputs. We demonstrate the efficiency of the method to discover a number of diverse differential equations using noisy measurements, including a model describing neural dynamics, chaotic Lorenz model, Michaelis–Menten kinetics and a parameterized Hopf normal form.

Reconstruction of high spectral resolution multispectral image using dictionary-based learning and sparse coding

With the rising development of imaging spectroscopy, there is an increase in demand for images with high spectral, spatial, and temporal resolution. This study proposes a reconstruction from low spectral resolution multispectral imagery to higher spectral resolution hyperspectral imagery to overcome the problem of low spatial resolution and less availability of multi-temporal hyperspectral images. This study proposes a novel approach to dictionary learning based on nonlinear unmixing for dense mixtures and linear unmixing for sparse mixtures. Additionally, least square-based sparse coding has been applied in this framework to reconstruct high-resolution spectra from low-resolution spectra. The proposed method has been implemented on standard as well as real datasets and the performance of reconstruction has been validated through parametric techniques. The proposed method has been compared with states of the art methods like Joint Sparse & Low-Ranking Method and Derivative based Learning Method and it is observed that the proposed method outperforms the other methods. Nonlinear Spectral Unmixing has been performed and classification accuracy has been assessed on regenerated image data and observed to produce satisfactory output. In this study, the regenerated high spectral resolution image has been used for species-level classification and mapping of pure and mixed patches of mangrove species and the output compared with the Hyperion hyperspectral dataset of Henry Island, Sunderbans, India. Similarly, the proposed algorithm has also been applied on the Indian Pines, Salinas, Cuprite and Jasper Ridge datasets and the regenerated high spectral resolution images have been used for subpixel level target detection and classification.

Single-shell NODDI using dictionary-learner-estimated isotropic volume fraction.

Neurite orientation dispersion and density imaging (NODDI) enables the assessment of intracellular, extracellular, and free water signals from multi-shell diffusion MRI data. It is an insightful approach to characterize brain tissue microstructure. Single-shell reconstruction for NODDI parameters has been discouraged in previous studies caused by failure when fitting, especially for the neurite density index (NDI). Here, we investigated the possibility of creating robust NODDI parameter maps with single-shell data, using the isotropic volume fraction (fISO ) as a prior. Prior estimation was made independent of the NODDI model constraint using a dictionary learning approach. First, we used a stochastic sparse dictionary-based network (DictNet), which is trained with data obtained from in vivo and simulated diffusion MRI data, to predict fISO . In single-shell cases, the mean diffusivity and raw T2 signal with no diffusion weighting (S0 ) was incorporated in the dictionary for the fISO estimation. Then, the NODDI framework was used with the known fISO to estimate the NDI and orientation dispersion index (ODI). The fISO estimated using our model was compared with other fISO estimators in the simulation. Further, using both synthetic data simulation and human data collected on a 3 T scanner (both high-quality HCP and clinical dataset), we compared the performance of our dictionary-based learning prior NODDI (DLpN) with the original NODDI for both single-shell and multi-shell data. Our results suggest that DLpN-derived NDI and ODI parameters for single-shell protocols are comparable with original multi-shell NODDI, and the protocol with b= 2000 s/mm2 performs the best (error ~ 5% in white and gray matter). This may allow NODDI evaluation of studies on single-shell data by multi-shell scanning of two subjects for DictNet fISO training.

Hyperparameter-Free Transmit-Nonlinearity Mitigation Using a Kernel-Width Sampling Technique

Nonlinear device characteristics present a severe performance-bottleneck for several upcoming next-generation wireless communication systems and prevent them from delivering high data-rates to the end-users. In this context, reproducing kernel Hilbert space (RKHS) based signal processing methods have gained widespread deployment and have been found to outperform classical polynomial-filtering-based solutions significantly. Furthermore, recent RKHS based techniques that rely on explicit feature-maps called random Fourier features (RFF) have emerged. These techniques alleviate the dependence on learning a dictionary and avoid the computations and errors incurred in dictionary-based learning. However, the performance of existing RKHS based solutions depends on choosing a suitable kernel-width. For the widely-used Gaussian kernel, we propose a methodology of assigning kernel-bandwidths that capitalizes on a stochastic sampling of kernel-widths using an ensemble drawn from a pre-designed probability density function. The technique is found to deliver a comparable convergence/error-rate performance to the scenario when the kernel-width is chosen by brute-force trial and error for tuning it for best performance. The desirable properties of the proposed kernel-sampling technique are supported by analytical proofs and are further highlighted by computer-simulations presented in the form of case studies in the context of next-generation communication systems.

Dynamic Upsampling of Smoke through Dictionary-based Learning

Simulating turbulent smoke flows with fine details is computationally intensive. For iterative editing or simply faster generation, efficiently upsampling a low-resolution numerical simulation is an attractive alternative. We propose a novel learning approach to the dynamic upsampling of smoke flows based on a training set of flows at coarse and fine resolutions. Our multiscale neural network turns an input coarse animation into a sparse linear combination of small velocity patches present in a precomputed over-complete dictionary. These sparse coefficients are then used to generate a high-resolution smoke animation sequence by blending the fine counterparts of the coarse patches. Our network is initially trained from a sequence of example simulations to both construct the dictionary of corresponding coarse and fine patches and allow for the fast evaluation of a sparse patch encoding of any coarse input. The resulting network provides an accurate upsampling when the coarse input simulation is well approximated by patches present in the training set (e.g., for re-simulation), or simply visually plausible upsampling when input and training sets differ significantly. We show a variety of examples to ascertain the strengths and limitations of our approach and offer comparisons to existing approaches to demonstrate its quality and effectiveness.

Dictionary-based Fidelity Measure for Virtual Traffic.

Aiming at objectively measuring the realism of virtual traffic flows and evaluating the effectiveness of different traffic simulation techniques, this paper introduces a general, dictionary-based learning method to evaluate the fidelity of any traffic trajectory data. First, a traffic pattern dictionary that characterizes common patterns of real-world traffic behavior is built offline from pre-collected ground truth traffic data. The corresponding learning error is set as the benchmark of the dictionary-based traffic representation. With the aid of the constructed dictionary, the realism of input simulated traffic flow data can be evaluated by comparing its dictionary-based reconstruction error with the dictionary error benchmark. This evaluation metric can be robustly applied to any simulated traffic flow data; in other words, it is independent of how the traffic data are generated. We demonstrated the effectiveness and robustness of this metric through many experiments on real-world traffic data and various simulated traffic data, comparisons with the state-of-the-art entropy-based similarity metric for aggregate crowd motions, and perceptual evaluation studies.

Analysing concordancing: a simple or multifaceted construct?

ABSTRACTIn pedagogical contexts, concordancing is often researched in relation to a non-concordanced-based learning condition, such as dictionary-based learning, and studies have generally shown concordancing to be a valuable language learning resource, as indicated by Tom Cobb and Alex Boutlon's recent meta-analysis. However, studies framed in terms of concordancing often overlook potentially significant differences in concordancing conditions by treating concordances as a simple, unidimensional construct. This study identifies variables within concordancing that may provide learners with diverging affordances, affordances that are not mutually compatible, and explores concordancing through a survey of concordance users’ preferences. The data is analysed through a series of factor analyses which indicate that concordancing is a complex, multidimensional construct. It finds that there are at least four factors underlying the construct of concordancing: how citations are formatted, the type of corpus used, the order citations are presented in, and the task engaged in. Each of these factors represents a dimension of variance amongst concordance users. If it is safe to assume that concordance users’ preferences are unlikely to be arbitrary, and are likely to indicate the affordances of concordancing in a particular configuration, then each factor represents a potentially significant variable in learner use of concordances. The paper concludes by discussing the implications of the construct identified in relation to learner concordance use.

Ambiguously Labeled Learning Using Dictionaries

We propose a dictionary-based learning method for ambiguously labeled multiclass classification, where each training sample has multiple labels and only one of them is the correct label. The dictionary learning problem is solved using an iterative alternating algorithm. At each iteration of the algorithm, two alternating steps are performed: 1) a confidence update and 2) a dictionary update. The confidence of each sample is defined as the probability distribution on its ambiguous labels. The dictionaries are updated using either soft or hard decision rules. Furthermore, using the kernel methods, we make the dictionary learning framework nonlinear based on the soft decision rule. Extensive evaluations on four unconstrained face recognition datasets demonstrate that the proposed method performs significantly better than state-of-the-art ambiguously labeled learning approaches.