Weights-and-structure-determination Research Articles

Credit and Loan Approval Classification Using a Bio-Inspired Neural Network.

Numerous people are applying for bank loans as a result of the banking industry's expansion, but because banks only have a certain amount of assets to lend to, they can only do so to a certain number of applicants. Therefore, the banking industry is very interested in finding ways to reduce the risk factor involved in choosing the safe applicant in order to save lots of bank resources. These days, machine learning greatly reduces the amount of work needed to choose the safe applicant. Taking this into account, a novel weights and structure determination (WASD) neural network has been built to meet the aforementioned two challenges of credit approval and loan approval, as well as to handle the unique characteristics of each. Motivated by the observation that WASD neural networks outperform conventional back-propagation neural networks in terms of sluggish training speed and being stuck in local minima, we created a bio-inspired WASD algorithm for binary classification problems (BWASD) for best adapting to the credit or loan approval model by utilizing the metaheuristic beetle antennae search (BAS) algorithm to improve the learning procedure of the WASD algorithm. Theoretical and experimental study demonstrate superior performance and problem adaptability. Furthermore, we provide a complete MATLAB package to support our experiments together with full implementation and extensive installation instructions.

Customer churn classification through a weights and structure determination neural network

In today's corporate world, acquiring and keeping clients are the most important priorities. Every business’s market is expanding quickly, which is increasing the number of subscribers. Because neglect could result in a drop in profitability from a major standpoint, it has become imperative for service providers to limit churn rates. These days, identifying which customers are most likely to leave a business requires a lot less work thanks to machine learning. Taking this into account, a novel weights and structure determination (WASD) neural network has been built to meet the aforementioned challenge of customer churn classification, as well as to handle its unique characteristics. Motivated by the observation that WASD neural networks outperform conventional back-propagation neural networks in terms of slow training speed and trapping in a local minimum, we enhance the WASD algorithm's learning process with a new activation function for best adapting to the customer churn model. Superior performance and flexibility to problems are demonstrated in an experimental investigation using a dataset from a telecommunications provider.

A Fresnel Cosine Integral WASD Neural Network for the Classification of Employee Attrition

Employee attrition, defined as the voluntary resignation of a subset of a company’s workforce, represents a direct threat to the financial health and overall prosperity of a firm. From lost reputation and sales to the undermining of the company’s long-term strategy and corporate secrets, the effects of employee attrition are multidimensional and, in the absence of thorough planning, may endanger the very existence of the firm. It is thus impeccable in today’s competitive environment that a company acquires tools that enable timely prediction of employee attrition and thus leave room either for retention campaigns or for the formulation of strategical maneuvers that will allow the firm to undergo their replacement process with its economic activity left unscathed. To this end, a weights and structure determination (WASD) neural network utilizing Fresnel cosine integrals in the determination of its activation functions, termed FCI-WASD, is developed through a process of three discrete stages. Those consist of populating the hidden layer with a sufficient number of neurons, fine-tuning the obtained structure through a neuron trimming process, and finally, storing the necessary portions of the network that will allow for its successful future recreation and application. Upon testing the FCI-WASD on two publicly available employee attrition datasets and comparing its performance to that of five popular and well-established classifiers, the vast majority of them coming from MATLAB’s classification learner app, the FCI-WASD demonstrated superior performance with the overall results suggesting that it is a competitive as well as reliable model that may be used with confidence in the task of employee attrition classification.

A Weights Direct Determination Neural Network for International Standard Classification of Occupations

Multiclass classification is one of the most popular machine learning tasks. The main focus of this paper is to classify occupations according to the International Standard Classification of Occupations (ISCO) using a weights and structure determination (WASD)-based neural network. In general, WASD-trained neural networks are known to overcome the drawbacks of conventional back-propagation trained neural networks, such as slow training speed and local minimum. However, WASD-based neural networks have not yet been applied to address the challenges of multiclass classification. As a result, a novel WASD for multiclass classification (WASDMC)-based neural network is introduced in this paper. When applied to two publicly accessible ISCO datasets, the WASDMC-based neural network displayed superior performance across all measures, compared to some of the best-performing classification models that the MATLAB classification learner app has to offer.

A bio-inspired weights and structure determination neural network for multiclass classification: Applications in occupational classification systems

<abstract><p>Undoubtedly, one of the most common machine learning challenges is multiclass classification. In light of this, a novel bio-inspired neural network (NN) has been developed to address multiclass classification-related issues. Given that weights and structure determination (WASD) NNs have been acknowledged to alleviate the disadvantages of conventional back-propagation NNs, such as slow training pace and trapping in a local minimum, we developed a bio-inspired WASD algorithm for multiclass classification problems (BWASDC) by using the metaheuristic beetle antennae search (BAS) algorithm to enhance the WASD algorithm's learning process. The BWASDC's effectiveness is then evaluated through applications in occupational classification systems. It is important to mention that systems of occupational classification serve as a fundamental indicator of occupational exposure. For this reason, they are highly significant in social science research. According to the findings of four occupational classification experiments, the BWASDC model outperformed some of the most modern classification models obtainable through MATLAB's classification learner app on all fronts.</p></abstract>

A multi-input with multi-function activated weights and structure determination neuronet for classification problems and applications in firm fraud and loan approval

Neuronets trained by a weights-and-structure-determination (WASD) algorithm are known to resolve the shortcomings of traditional back-propagation neuronets such as slow training speed and local minimum. A multi-input multi-function activated WASD neuronet (MMA-WASDN) model is introduced in this paper, combined with a novel multi-function activated WASD (MA-WASD) algorithm, for handling binary classification problems. Using multiple power activation functions, the MA-WASD algorithm finds the optimal weights and structure of the MMA-WASDN and uses cross-validation to address bias and prevent being stuck in local optima during the training process. As a result, neuronets trained with the MA-WASD algorithm have higher precision and accuracy than neuronets trained with traditional WASD algorithms. Applications on firm fraud and loan approval classification validate our MMA-WASDN model in order to demonstrate its outstanding learning and predicting performance. Since these applications use real-world datasets that include strings and missing values, an algorithmic method for preparing data is also suggested to make them manageable from the MMA-WASDN. A comparison of the MMA-WASDN model to five other high-performing neuronet models is included, as well as a MATLAB package that is publicly available through GitHub to support and promote the findings of this research.

A weights direct determination neuronet for time‐series with applications in the industrial indices of the Federal Reserve Bank of St. Louis

AbstractThe shortcomings of conventional back‐propagation neuronets, such as slow training speed and local minimum, are known to be addressed by neuronets trained under the weights‐and‐structure‐determination (WASD) algorithm. Derived from power activation feed‐forward neuronets, a multi‐input WASD for time‐series neuronet (MI‐WASDTSN) model is presented in this paper. The MI‐WASDTSN is equipped with a novel WASD for time‐series (WASDTS) algorithm, for handling time‐series modeling and forecasting problems. Employing a power sigmoid activation function, the WASDTS algorithm handles the model fitting and validation by determining the optimal input variables number and the weights of the MI‐WASDTSN. More specifically, the WASDTS algorithm finds and holds only the activation function powers that reduce the model's error during validation. Applications on Federal Reserve Bank of St. Louis (FRED) industrial indices under three different patterns of time‐series validate our MI‐WASDTSN model in order to demonstrate its outstanding learning and forecasting performance. In addition, to support and advance the findings of this work, we created a MATLAB repository for interested users, which is freely available via GitHub.

Multi-input bio-inspired weights and structure determination neuronet with applications in European Central Bank publications

This paper introduces a 3-layer feed-forward neuronet model, trained by novel beetle antennae search weights-and-structure-determination (BASWASD) algorithm. On the one hand, the beetle antennae search (BAS) algorithm is a memetic meta-heuristic optimization algorithm capable of solving combinatorial optimization problems. On the other hand, neuronets trained by a weights-and-structure-determination (WASD) algorithm are known to resolve the shortcomings of traditional back-propagation neuronets, including slow speed of training and local minimum. Combining the BAS and WASD algorithms, a novel BASWASD algorithm is created for training neuronets, and a multi-input BASWASD neuronet (MI-BASWASDN) model is introduced. Using a power sigmoid activation function and while managing the model fitting and validation, the BASWASD algorithm finds the optimal weights and structure of the MI-BASWASDN. Four financial datasets, taken from the European Central Bank publications, validate and demonstrate the MI-BASWASDN model’s outstanding learning and predicting performance. Also included is a comparison of the MI-BASWASDN model to three other well-performing neural network models, as well as a MATLAB kit that is publicly available on GitHub to promote and support this research.

Time-varying Black–Litterman portfolio optimization using a bio-inspired approach and neuronets

The Black–Litterman model is a very important analytical tool for active portfolio management because it allows investment analysts to incorporate investor’s views into market equilibrium returns. In this paper, we define and study the time-varying Black–Litterman portfolio optimization under nonlinear constraints (TV-BLPONC) problem as a nonlinear programming (NLP) problem. More precisely, the nonlinear constraints refer to transaction costs and cardinality constraints. Furthermore, a speedy weights-and-structure-determination (WASD) algorithm for the power-activation feed-forward neuronet (PFN) is presented to solve time-series modeling and forecasting problems. Inhere, the investor’s views in the TV-BLPONC problem are considered as a forecasting problem and, thus, they are produced by the WASD-based PFN. In addition, using the beetle antennae search (BAS) algorithm a computational method is introduced to solve the TV-BLPONC problem. For all we know, this is an innovative approach that integrates modern neural network and meta-heuristic optimization methods to provide a solution to the TV-BLPONC problem in large portfolios. Our approach is tested on portfolios of up to 90 stocks with real-world data, and the results show that it is more than 30 times faster than other methods. Our technique’s speed and precision are verified in this way, showing that it is an outstanding alternative to ordinary methods. In order to support and promote the findings of this work, we have constructed two complete MATLAB packages for the interested user, which are freely available through GitHub.

From WASD to BLS with application to pattern classification

The single-hidden-layer feedforward neural network (SLFN) exhibits an excellent approximation ability, despite its simple structure. This study examines two neural network models developed based on the general structure of the SLFN, i.e., the weights-and-structure-determination (WASD) neural network and broad learning system (BLS). The BLS and WASD neural networks exhibit many similarities in terms of their structure and algorithm, while BLS incorporates certain distinct ideas that render it an improved version of the WASD neural network. The research described in this paper is the first approach to establish a connection between the BLS and WASD neural networks. Moreover, pattern classification experiments on a foot dataset and several benchmark datasets are conducted using these two kinds of neural network models, and the performance of different models is compared and analyzed.Experimental results show that dividing the feature nodes into one group is the best choice for our foot dataset, and that the cross-layer connectivity and sparse coding of BLS can effectively improve the classification accuracy. In addition, it is shown that the dynamic update algorithm of BLS can improve the accuracy and save a lot of time when updating the model structure, but the final accuracy depends on the initial structure and parameters. Comparative experiments show that BLS achieves 89.12% classification accuracy for the foot dataset, which is the highest among all the demonstrated WASD neural networks and two other SLFN models. Finally, extended experiments on five benchmark datasets show that BLS can achieve satisfactory accuracy on multiple types of datasets, while WASD neural networks are possibly inadequate in processing image datasets in contrast.

Forecasting of Chinese Hydropower Generation Using WASD-Neuronet

Hydropower resource is one of the renewable energy sources. With the increasing Chinese economy, people are paying much more attention to sustainable development. The increasing hydropower load is the basis of the development of power industry. Due to the characteristics of electrical energy, predicting the hydropower accurately is a potentially beneficial way to plan hydropower reasonably. This paper presents a neural network method to predict hydropower generation whose data is influenced by several factors such as social economic, population and climate. By using the past 52-year rough data, a 3-layer feedforward neuronet equipped with the weights and structure determination (WASD) method is constructed for the prediction of the Chinese hydropower generation in this paper. By processing mass of data, we could basically predict the hydropower generation using such a WASD neuronet. To a large extent, the trend of developing Chinese hydropower generation in the next years will keep growing.

WASD Algorithm with Pruning-While-Growing and Twice-Pruning Techniques for Multi-Input Euler Polynomial Neural Network

Differing from the conventional back-propagation (BP) neural networks, a novel multi-input Euler polynomial neural network, in short, MIEPNN (specifically, 4-input Euler polynomial neural network, 4IEPNN) is established and investigated in this paper. In order to achieve satisfactory performance of the established MIEPNN, a weights and structure determination (WASD) algorithm with pruning-while-growing (PWG) and twice-pruning (TP) techniques is built up for the established MIEPNN. By employing the weights direct determination (WDD) method, the WASD algorithm not only determines the optimal connecting weights between hidden layer and output layer directly, but also obtains the optimal number of hidden-layer neurons. Specifically, a sub-optimal structure is obtained via the PWG technique, then the redundant hidden-layer neurons are further pruned via the TP technique. Consequently, the optimal structure of the MIEPNN is obtained. To provide a reasonable choice in practice, several different MATLAB computing routines related to the WDD method are studied. Comparative numerical-experiment results of the 4IEPNN using these different MATLAB computing routines and the standard multi-layer perceptron (MLP) neural network further verify the superior performance and efficacy of the proposed MIEPNN equipped with the WASD algorithm including PWG and TP techniques in terms of training, testing and predicting.

CP-activated WASD neuronet approach to Asian population prediction with abundant experimental verification

Data fitting and prediction are important in many domains. As computing power improves, artificial intelligence fitting methods such as the WASD (weights and structure-determination) neuronet become more operable. Although the WASD neuronet has been applied to several other issues, its application on fitting data needs to be explored and discussed more specifically. This paper is committed to introducing the WASD-neuronet model activated by Chebyshev polynomials of class 1 for data fitting and to exploring its capability of data prediction. The learning–checking method and the concept of global minimum point are introduced to improve the prediction performance and extend the application of the WASD-neuronet model. Applying such a model to Asian population prediction substantiates its excellent performance. With numerical experiments validating the predicting performance and a final prediction based on historical data, this paper presents a reasonable tendency of Asian population.

Weights and structure determination of multiple-input feed-forward neural network activated by Chebyshev polynomials of Class 2 via cross-validation

Differing from conventional improvements on backpropagation (BP) neural network, a novel neural network is proposed and investigated in this paper to overcome the BP neural-network weaknesses, which is called the multiple-input feed-forward neural network activated by Chebyshev polynomials of Class 2 (MINN-CP2). In addition, to obtain the optimal number of hidden-layer neurons and the optimal linking weights of the MINN-CP2, the paper develops an algorithm of weights and structure determination (WASD) via cross-validation. Numerical studies show the effectiveness and superior abilities (in terms of approximation and generalization) of the MINN-CP2 equipped with the algorithm of WASD via cross-validation. Moreover, an application to gray image denoising demonstrates the effective implementation and application prospect of the proposed MINN-CP2 equipped with the algorithm of WASD via cross-validation.