Greedy Learning Research Articles

The learning performance of the weak rescaled pure greedy algorithms

We investigate the regression problem in supervised learning by means of the weak rescaled pure greedy algorithm (WRPGA). We construct learning estimator by applying the WRPGA and deduce the tight upper bounds of the K-functional error estimate for the corresponding greedy learning algorithms in Hilbert spaces. Satisfactory learning rates are obtained under two prior assumptions on the regression function. The application of the WRPGA in supervised learning considerably reduces the computational cost while maintaining its powerful generalization capability when compared with other greedy learning algorithms.

The role of LightGBM model in management efficiency enhancement of listed agricultural companies

Abstract This paper first explores the principle of the the LightGBM model, which uses a gradient-based one-sided sampling GOSS procedure to sample the sample data to ensure the accuracy of the procedure in information gain evaluation. The mutually exclusive feature bundling Histogram procedure is used to bind the feature variables of the data and the Leaf-wise strategy for differential splitting to enhance the training speed of the model. Then the steps of the LightGBM procedure are discussed, using a a decision tree as the base learner, then iterative training in sequence, and greedy learning using forward distributed procedure and the original feature fetches must be identified from the bundled features to reduce the training complexity. Finally, the LightGBM procedure is associated and investigated with the other four procedures in terms of discriminative accuracy and discriminative time of the factors affecting the efficiency of enterprise management. In terms of level discrimination accuracy, the level discrimination rate of the LightGBM model is basically above 0.7, and the discrimination rate reaches 0.9 at level 6, which is better than other procedures. The discriminatory time of the LightGBM model is only 28 seconds, which is about 16 times faster than the CART model and 20 times faster than the support vector machine model.

Growing axons: greedy learning of neural networks with application to function approximation

Abstract We propose a new method for learning deep neural network models, which is based on a greedy learning approach: we add one basis function at a time, and a new basis function is generated as a non-linear activation function applied to a linear combination of the previous basis functions. Such a method (growing deep neural network by one neuron at a time) allows us to compute much more accurate approximants for several model problems in function approximation.

Learning capability of the rescaled pure greedy algorithm with non-iid sampling

<abstract><p>We consider the rescaled pure greedy learning algorithm (RPGLA) with the dependent samples drawn according to a non-identical sequence of probability distributions. The generalization performance is provided by applying the independent-blocks technique and adding the drift error. We derive the satisfactory learning rate for the algorithm under the assumption that the process satisfies stationary $ \beta $-mixing, and also find that the optimal rate $ O(n^{-1}) $ can be obtained for i.i.d. processes.</p></abstract>

Optimality of the rescaled pure greedy learning algorithms

We propose the Rescaled Pure Greedy Learning Algorithm (RPGLA) for solving the kernel-based regression problem. The computational complexity of the RPGLA is less than the Orthogonal Greedy Learning Algorithm (OGLA) and Relaxed Greedy Learning Algorithm (RGLA). We obtain the convergence rates of the RPGLA for continuous kernels. When the kernel is infinitely smooth, we derive a convergence rate that can be arbitrarily close to the best rate [Formula: see text] under a mild assumption of the regression function.

UAV-Enabled Data Collection Over Clustered Machine-Type Communication Networks: AEM Modeling and Trajectory Planning

Machine-type communications (MTC) is a key technology for Internet-of-Things (IoT) services in 5G mobile communications and beyond. An essential design problem for an MTC network is the efficient and scalable data collection from low-power machine-type communication devices (MTCDs). This paper uses unmanned aerial vehicles (UAVs) to facilitate data collection from a clustered MTC network on the ground. The notion of artificial energy map (AEM) is introduced as a novel modeling technique for energy efficiency analysis, which is critical to the subject of investigation here considering the limited energy of battery-powered MTCDs and UAVs. The proposed design framework first determines the number of MTCD clusters <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L_\mathrm{{opt}}$</tex-math></inline-formula> according to a certain criterion. A greedy learning clustering (GLC) algorithm is then employed to divide the MTCDs into <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L_\mathrm{{opt}}$</tex-math></inline-formula> clusters. For each MTCD cluster, an AEM is constructed, and the optimal UAV hovering strategy within the cluster can be obtained accordingly. Finally, the UAV stations travel across the clusters and collect data from each cluster while hovering above it. This AEM-based modeling technique leads to a solution that can effectively improve the energy efficiency (EE) of UAV-enabled data collection. However, the MTCD clustering strategy, UAV hovering strategy, and UAV flying strategy all have impacts on the overall energy efficiency, which results in a coupled optimization problem that is difficult to solve. The GLC-AEM method is proposed to decouple the original EE optimization problem into sub-problems that can be handled easily by standard optimization techniques. Simulation results show that the GLC-AEM algorithm can be applied to UAV-enabled data collection scenarios with single and multiple UAV stations, and it can improve the overall EE effectively. Besides, the GLC-AEM algorithm shows good scalability and consistent performance in clustered MTC networks. The more MTCDs, the higher the achieved EE.

Optimality of the Approximation and Learning by the Rescaled Pure Super Greedy Algorithms

We propose the Weak Rescaled Pure Super Greedy Algorithm (WRPSGA) for approximation with respect to a dictionary D in Hilbert space. The WRPSGA is simpler than some popular greedy algorithms. We show that the convergence rate of the RPSGA on the closure of the convex hull of the μ-coherent dictionary D is optimal. Then, we design the Rescaled Pure Super Greedy Learning Algorithm (RPSGLA) for kernel-based supervised learning. We prove that the convergence rate of the RPSGLA can be arbitrarily close to the best rate O(m−1) under some mild assumptions.

Deep image retrieval using artificial neural network interpolation and indexing based on similarity measurement

In content-based image retrieval (CBIR), primitive image signatures are critical because they represent the visual characteristics. Image signatures, which are algorithmically descriptive and accurately recognized visual components, are used to appropriately index and retrieve comparable results. To differentiate an image in the category of qualifying contender, feature vectors must have image information's like colour, objects, shape, spatial viewpoints. Previous methods such as sketch-based image retrieval by salient contour (SBIR) and greedy learning of deep Boltzmann machine (GDBM) used spatial information to distinguish between image categories. This requires interest points and also feature analysis emerged image detection problems. Thus, a proposed model to overcome this issue and predict the repeating pattern as well as series of pixels that conclude similarity has been necessary. In this study, a technique called CBIR-similarity measure via artificial neural network interpolation (CBIR-SMANN) has been presented. By collecting datasets, the images are resized then subject to Gaussian filtering in the pre-processing stage, then by permitting them to the Hessian detector, the interesting points are gathered. Based on Skewness, mean, kurtosis and standard deviation features were extracted then given to ANN for interpolation. Interpolated results are stored in a database for retrieval. In the testing stage, the query image was inputted that is subjected to pre-processing, and feature extraction was then fed to the similarity measurement function. Thus, ANN helps to get similar images from the database. CBIR-SMANN have been implemented in the python tool and then evaluated for its performance. Results show that CBIR-SMANN exhibited a high recall value of 78% with a minimum retrieval time of 980 ms. This showed the supremacy of the proposed model was comparatively greater than the previous ones.

Active Learning for Biomedical Text Classification Based on Automatically Generated Regular Expressions

Biomedical text classification algorithms, which currently support clinical decision-making processes, call for expensive training texts due to the low availability of labeled corpus and the cost of manual annotation by specialized professionals. The active learning (AL) approach to classification heavily lessens such cost by reducing the number of labeled documents required to achieve specified performance. This article introduces a query strategy and a stopping criterion that transform CREGEX, a regular-expressions-based text classification algorithm, in an AL biomedical text classifier. The query strategy samples the training dataset, trading off the greedy learning achieved by the regular expressions classification precision and the conservative learning induced by text sequence alignment classification. The sustained reduction in the variance of the query strategy scores is used as a stopping criterion. The AL classifier was compared with Support Vector Machine (SVM), Naïve Bayes (NB), and a classifier based on Bidirectional Encoder Representations from Transformers (BERT), using three datasets with biomedical information in Spanish on smoking habits, obesity, and obesity types. The learning curve results indicate that AL in CREGEX allowed to efficiently reduce the number of training examples for equal performance than the rest of the classifiers, obtaining areas under the learning curve greater than 85% in all cases. The stopping criterion applied to the AL process allowed to use, on average, approximately 32% to 50% of the total training examples with differences in performance concerning the maximum value of the learning curve not exceeding 2%. This performance demonstrates the effectiveness of using AL in a biomedical text classifier based on regular expressions, which is attributable to such expressions' ability to represent intricate sequential patterns in training texts considered most informative.

Greedy Broad Learning System

In order to overcome the extremely time-consuming drawback of deep learning (DL), broad learning system (BLS) was proposed as an alternative method. This model is simple, fast, and easy to update. To ensure the fitting and generalization ability of BLS, the hidden layer neurons are often set too many, in fact, a lot of neurons are not needed. Greedy BLS (GBLS) is proposed in this paper to deal with the redundancy of the hidden layer in BLS from another perspective. Different from BLS, the structure of GBLS can be seen as a combination of unsupervised multi-layer feature representation and supervised classification or regression. It trains with a greedy learning scheme, performs principal component analysis (PCA) on the previous hidden layer to form a set of compressed nodes, which are transformed into enhancement nodes and then activated by nonlinear functions. The new hidden layer is composed of all newly generated compressed nodes and enhancement nodes, and so on. The last hidden layer of the network contains the higher-order and abstract essential features of the original data, which is connected to the output layer. Each time a new layer is added to the model, and there is no need to retrain from the beginning, only the previous layer is trained. Experimental results demonstrate that the proposed GBLS model outperforms BLS both in classification and regression.

Disjunctive Fuzzy Neural Networks: A New Splitting-Based Approach to Designing a T–S Fuzzy Model

This article proposes a new network approach toward the implementation of Takagi–Sugeno (T–S) fuzzy models referred to as disjunctive fuzzy neural networks (DJFNNs). The proposed DJFNN involves a novel network architecture and a greedy learning algorithm. Being different from the existing grid-based and clustering-based network architectures, the proposed architecture adds an OR neural layer positioned between the fuzzification layer and the rule layer. In this way, the implied constraint between the number of rules and the number of fuzzy labels is excluded so that a curse of dimensionality can be overcome and more interpretable models are formed. Furthermore, inspired by the core algorithm for building a decision tree, a top–down, nonbacktracking, and greedy algorithm is proposed to learn the unknown parameters of the networks. The input space splits into smaller and smaller subspace along the predefined fuzzy grids in a supervised manner meanwhile the associated conditions of the T–S fuzzy model are identified. The greedy algorithm is applicable to high-dimensional problems since there is no exponential growth in time or space as the dimensionality increases. The new network architecture and greedy learning algorithm make the proposed DJFNN a regression model of high interpretability and good prediction capability, particularly suitable for solving the high-dimensional problems. The DJFNN was experimented with using a synthetic dataset and 28 real-world datasets and compared with classical and state-of-the-art methods through nonparametric statistical tests. The results confirmed the effectiveness of the DJFNN in terms of accuracy, interpretability, and computational cost.

Research on predicting 2D-HP protein folding using reinforcement learning with full state space

BackgroundProtein structure prediction has always been an important issue in bioinformatics. Prediction of the two-dimensional structure of proteins based on the hydrophobic polarity model is a typical non-deterministic polynomial hard problem. Currently reported hydrophobic polarity model optimization methods, greedy method, brute-force method, and genetic algorithm usually cannot converge robustly to the lowest energy conformations. Reinforcement learning with the advantages of continuous Markov optimal decision-making and maximizing global cumulative return is especially suitable for solving global optimization problems of biological sequences.ResultsIn this study, we proposed a novel hydrophobic polarity model optimization method derived from reinforcement learning which structured the full state space, and designed an energy-based reward function and a rigid overlap detection rule. To validate the performance, sixteen sequences were selected from the classical data set. The results indicated that reinforcement learning with full states successfully converged to the lowest energy conformations against all sequences, while the reinforcement learning with partial states folded 50% sequences to the lowest energy conformations. Reinforcement learning with full states hits the lowest energy on an average 5 times, which is 40 and 100% higher than the three and zero hit by the greedy algorithm and reinforcement learning with partial states respectively in the last 100 episodes.ConclusionsOur results indicate that reinforcement learning with full states is a powerful method for predicting two-dimensional hydrophobic-polarity protein structure. It has obvious competitive advantages compared with greedy algorithm and reinforcement learning with partial states.

Computationally efficient deep neural network for computed tomography image reconstruction.

Deep neural network-based image reconstruction has demonstrated promising performance in medical imaging for undersampled and low-dose scenarios. However, it requires large amount of memory and extensive time for the training. It is especially challenging to train the reconstruction networks for three-dimensional computed tomography (CT) because of the high resolution of CT images. The purpose of this work is to reduce the memory and time consumption of the training of the reconstruction networks for CT to make it practical for current hardware, while maintaining the quality of the reconstructed images. We unrolled the proximal gradient descent algorithm for iterative image reconstruction to finite iterations and replaced the terms related to the penalty function with trainable convolutional neural networks (CNN). The network was trained greedily iteration by iteration in the image domain on patches, which requires reasonable amount of memory and time on mainstream graphics processing unit (GPU). To overcome the local-minimum problem caused by greedy learning, we used deep UNet as the CNN and incorporated separable quadratic surrogate with ordered subsets for data fidelity, so that the solution could escape from easy local minimums and achieve better image quality. The proposed method achieved comparable image quality with state-of-the-art neural network for CT image reconstruction on two-dimensional (2D) sparse-view and limited-angle problems on the low-dose CT challenge dataset. The difference in root-mean-square-error (RMSE) and structural similarity index (SSIM) was within [-0.23,0.47]HU and [0,0.001], respectively, with 95% confidence level. For three-dimensional (3D) image reconstruction with ordinary-size CT volume, the proposed method only needed 2GB graphics processing unit (GPU) memory and 0.45s per training iteration as minimum requirement, whereas existing methods may require 417GB and 31min. The proposed method achieved improved performance compared to total variation- and dictionary learning-based iterative reconstruction for both 2D and 3D problems. We proposed a training-time computationally efficient neural network for CT image reconstruction. The proposed method achieved comparable image quality with state-of-the-art neural network for CT reconstruction, with significantly reduced memory and time requirement during training. The proposed method is applicable to 3D image reconstruction problems such as cone-beam CT and tomosynthesis on mainstream GPUs.

DeFe: indoor localization based on channel state information feature using deep learning

With the development of mobile devices, more and more location-based services (LBS) on the devices are needed and fingerprint indoor localization has become one most important technique because of its low cost and high accuracy. In this paper, we use the fingerprinting method which based on Channel State Information (CSI) for indoor localization. Furthermore, we extract the raw phase information from the multiple antennas and multiple sub-carriers through the IEEE 802.11n network interface card (NIC 5300) on several special models. Then we extract the required phase information and introduce two methods of mathematical statistics to analyse the feature of CSI signals. We replace the processed phase information with the signal features obtained from the analysis. For the offline stage, we employ a deep network with three hidden layers to train the signal features data, and use weights to represent fingerprints. Introducing a greedy learning algorithm to train the weights layer-by-layer to reduce the computational complexity, and the sub-network between two continuous layers forms a restricted Boltzmann machine (RBM). For the online location estimation, we use a probabilistic method based on the radial basis function (RBF). The neural network method we used this time is tested under two different scenarios, and different data are compared. The final conclusion is that the new method we combined is superior to the previous.

Greedy Learning of Deep Boltzmann Machine (GDBM)’s Variance and Search Algorithm for Efficient Image Retrieval

Despite extensive research on content-based image retrieval, challenges such as low accuracy, incapability to handle complex queries and high time consumption persist. Initially, a preprocessing technique is introduced in this study, a technique that uses a median filter to remove noise to achieve improved accuracy and reliability. Then, Fourier and circularity descriptors are extract in an effective manner correspondent to the texture and affine shape adaptation features. In addition, various descriptors, such as color histogram, color moment, color autocorrelogram and color coherency vector, are extracted as the invariant color features. The multiple ant colony optimization (MACOBTC) approach is implemented with whole features to find relevant features. Finally, the relevant features are utilized for the greedy learning of deep Boltzmann machine classifier (GDBM). The proposed approach obtains effective performance and accurate results on four datasets and is analyzed with various parameters such as accuracy, precision, recall, Jaccard, Dice, and Kappa coefficients. The GDBM provides a 25% increase in accuracy compared with existing techniques, such as the a priori classification algorithm.

An Energy-Efficient Online-Learning Stochastic Computational Deep Belief Network

Deep neural networks (DNNs) are effective machine learning models to solve a large class of recognition problems, including the classification of nonlinearly separable patterns. The training of DNNs is, however, particularly difficult due to the large size and high energy consumption of the networks. Recently, stochastic computation (SC) has been considered to implement DNNs to reduce the hardware cost. However, it requires a large number of random number generators (RNGs) and long stochastic sequences that lower the energy efficiency of the network. To overcome these limitations, we propose the design of an energy-efficient deep belief network (DBN) with online learning capacity based on stochastic computation. In the SC-DBN, a reconfigurable structure is utilized to implement the fast greedy learning algorithm and an adaptive moment estimation (ADAM) circuit is designed to improve the speed of the training process. An approximate SC activation unit (A-SCAU) is further designed to implement different types of activation functions in the neurons. The A-SCAU is immune to signal correlations, so the RNGs can be shared among all neurons in the same layer with no accuracy loss. The area and energy of the proposed design are less than 5.5% and 3.7% (or 29.3% and 33.3%) of a pipelined 32-bit floating-point (or an 8-bit fixed-point) implementation. The proposed SC-DBN design achieves a higher classification accuracy compared with the fixed-point implementation. The accuracy is in a range of 0.12% to 0.37% lower than the floating-point design with a significantly lower (or slightly higher) energy consumption than the pipelined (or non-pipelined) circuit for both online learning and inference processes.

Statistical monitoring of a wastewater treatment plant: A case study

The efficient operation of wastewater treatment plants (WWTPs) is key to ensuring a sustainable and friendly green environment. Monitoring wastewater processes is helpful not only for evaluating the process operating conditions but also for inspecting product quality. This paper presents a flexible and efficient fault detection approach based on unsupervised deep learning to monitor the operating conditions of WWTPs. Specifically, this approach integrates a deep belief networks (DBN) model and a one-class support vector machine (OCSVM) to separate normal from abnormal features by simultaneously taking advantage of the feature-extraction capability of DBNs and the superior predicting capacity of OCSVM. Here, the DBN model, which is a powerful tool with greedy learning features, accounts for the nonlinear aspects of WWTPs, while OCSVM is used to reliably detect the faults. The developed DBN-OCSVM approach is tested through a practical application on data from a decentralized WWTP in Golden, CO, USA. The results from the DBN-OCSVM are compared with two other detectors: DBN-based K-nearest neighbor and K-means algorithms. The results show the capability of the developed strategy to monitor the WWTP, suggesting that it can raise an early alert to the abnormal conditions.

Obstacle Detection for Intelligent Transportation Systems Using Deep Stacked Autoencoder and &lt;inline-formula&gt; &lt;tex-math notation="LaTeX"&gt;$k$ &lt;/tex-math&gt; &lt;/inline-formula&gt;-Nearest Neighbor Scheme

Obstacle detection is an essential element for the development of intelligent transportation systems so that accidents can be avoided. In this paper, we propose a stereovision-based method for detecting obstacles in urban environment. The proposed method uses a deep stacked auto-encoders (DSA) model that combines the greedy learning features with the dimensionality reduction capacity and employs an unsupervised $k$ -nearest neighbors (KNN) algorithm to accurately and reliably detect the presence of obstacles. We consider obstacle detection as an anomaly detection problem. We evaluated the proposed method by using practical data from three publicly available data sets, the Malaga stereovision urban data set, the Daimler urban segmentation data set, and the Bahnhof data set. Also, we compared the efficiency of DSA-KNN approach to the deep belief network-based clustering schemes. Results show that the DSA-KNN is suitable to visually monitor urban scenes.

Adaptive Learning Algorithms for Simulation-Based Dynamic Traffic User Equilibrium

We propose two classes of algorithms for achieving the user equilibrium in simulation-based dynamic traffic assignment with special attention given to the interactions between travel information and route choice behavior. A driver is assumed to perform day-to-day route choice repeatedly and experience payoffs with unknown noise. The driver adaptively changes his/her action in accordance with the payoffs that are initially unknown and must be estimated over time due to noisy observations. To solve this problem, we develop a multi-agent version of Q-learning to estimate the payoff functions using novel forms of the e−greedy learning policy. We apply this Q-learning scheme for the simulation-based DTA in which traffic flow and travel times of routes in the traffic network are generated by a microscopic traffic simulator based on cellular automaton. Finally, we provide simulation examples to show convergence of our algorithms to Nash equilibrium and effectiveness of the best-route provision services.

Generic Process Visualization Using Parametric t-SNE

In this work, a generic process visualization method is introduced using parametric t-SNE and used to visualize real-time process information and correlations among variables on a 2D map. A deep network is used to learn the Kullback-Leibler divergence between the original high-dimensional space and the latent space. In practice, it is observed that a model trained with historical data is not robust enough to visualize shifts into unknown states. Due to the effect of greedy learning, the response of the model is biased toward the most-contributing inputs. To relieve this effect, combinatorial variation creation is applied in the training stage to allow the model to respond to each input more evenly. The proposed method is tested on the Tennessee Eastman Process (TEP) data for four types of faults. The result indicates that the proposed method outperforms conventional methods such as PCA and Isomap, and is able to provide clear visual indication of process changes.