Multiresponse Sparse Regression Research Articles

Visual Navigation Using Inverse Reinforcement Learning and an Extreme Learning Machine

In this paper, we focus on the challenges of training efficiency, the designation of reward functions, and generalization in reinforcement learning for visual navigation and propose a regularized extreme learning machine-based inverse reinforcement learning approach (RELM-IRL) to improve the navigation performance. Our contributions are mainly three-fold: First, a framework combining extreme learning machine with inverse reinforcement learning is presented. This framework can improve the sample efficiency and obtain the reward function directly from the image information observed by the agent and improve the generation for the new target and the new environment. Second, the extreme learning machine is regularized by multi-response sparse regression and the leave-one-out method, which can further improve the generalization ability. Simulation experiments in the AI-THOR environment showed that the proposed approach outperformed previous end-to-end approaches, thus, demonstrating the effectiveness and efficiency of our approach.

Parsimonious Minimal Learning Machine via Multiresponse Sparse Regression.

The training procedure of the minimal learning machine (MLM) requires the selection of two sets of patterns from the training dataset. These sets are called input reference points (IRP) and output reference points (ORP), which are used to build a mapping between the input geometric configurations and their corresponding outputs. In the original MLM, the number of input reference points is the hyper-parameter and the patterns are chosen at random. Therefore, the conventional proposal does not consider which patterns will belong to each reference point group, since the model does not implement an appropriate way of selecting the most suitable patterns as reference points. Such an approach can impact on the decision function in terms of smoothness, resulting in high complexity models. This paper introduces a new approach to select IRP for MLM applied to classification tasks. The optimally selected minimal learning machine (OS-MLM) relies on the multiresponse sparse regression (MRSR) ranking method and the leave-one-out (LOO) criterion to sort the patterns in terms of relevance and select an appropriate number of input reference points, respectively. The experimental assessment conducted on UCI datasets reports the proposal was able to produce sparser models and achieve competitive performance when compared to the regular strategy of selecting MLM input RPs.

Forecasting solar radiation using an optimized hybrid model by Cuckoo Search algorithm

Due to energy crisis and environmental problems, it is very urgent to find alternative energy sources nowadays. Solar energy, as one of the great potential clean energies, has widely attracted the attention of researchers. In this paper, an optimized hybrid method by CS (Cuckoo Search) on the basis of the OP-ELM (Optimally Pruned Extreme Learning Machine), called CS-OP-ELM, is developed to forecast clear sky and real sky global horizontal radiation. First, MRSR (Multiresponse Sparse Regression) and LOO-CV (leave-one-out cross-validation) can be applied to rank neurons and prune the possibly meaningless neurons of the FFNN (Feed Forward Neural Network), respectively. Then, Direct strategy and Direct-Recursive strategy based on OP-ELM are introduced to build a hybrid model. Furthermore, CS (Cuckoo Search) optimized algorithm is employed to determine the proper weight coefficients. In order to verify the effectiveness of the developed method, hourly solar radiation data from six sites of the United States has been collected, and methods like ARMA (Autoregression moving average), BP (Back Propagation) neural network and OP-ELM can be compared with CS-OP-ELM. Experimental results show the optimized hybrid method CS-OP-ELM has the best forecasting performance.

Constructive multi-output extreme learning machine with application to large tanker motion dynamics identification

In this paper, a novel constructive multi-output extreme learning machine (CM-ELM) is proposed to deal with a large tanker motion dynamics identification. The significant contributions are as follows. (1) Driven by generated tanker dynamics data from the reference model, the CM-ELM method is proposed to identify multi-output dynamic models. (2) The candidate pool for CM-ELM is randomly generated by the ELM strategy, and ranked chunk-by-chunk based on a novel improved multi-response sparse regression (I-MRSR) incorporated with λweighting. (3) Consequently, the constructive model selection works with fast speed due to chunk-type training process, which also benefits stable hidden node selection and corresponding generalization. (4) Furthermore, output weight update on the final CM-ELM model randomly selected from the elite subset is conducted to enhance the overall performance of the resulting CM-ELM scheme. Finally, the convincing performance of the complete CM-ELM paradigm is verified by simulation studies on not only tanker motion dynamics identification but also benchmark multi-output regressions. Comprehensive comparisons of the CM-ELM with ELM and OP-ELM indicate the remarkable superiority in terms of generalization capability and stable compact structure. Conclusions are steadily drawn that the CM-ELM method is feasibly effective for tanker motion dynamics identification and multi-output regressions.

OP-KNN: Method and Applications

This paper presents a methodology named Optimally Pruned K-Nearest Neighbors (OP-KNNs) which has the advantage of competing with state-of-the-art methods while remaining fast. It builds a one hidden-layer feedforward neural network using K-Nearest Neighbors as kernels to perform regression. Multiresponse Sparse Regression (MRSR) is used in order to rank each kth nearest neighbor and finally Leave-One-Out estimation is used to select the optimal number of neighbors and to estimate the generalization performances. Since computational time of this method is small, this paper presents a strategy using OP-KNN to perform Variable Selection which is tested successfully on eight real-life data sets from different application fields. In summary, the most significant characteristic of this method is that it provides good performance and a comparatively simple model at extremely high-learning speed.