Optimal Output Weights Research Articles

Weight-restricted approach on constant returns to scale DEA models: Efficiency of internet banking in Turkey

The performance measurement of decision-making units (DMUs) can be optimised by strategically choosing the optimal input and output weights. This approach circumvents the potential complexities associated with establishing a common basis for ranking homogeneous units and managing the results from multiple efficient frontiers. Efficient frontier ranking is simplified by limiting the weights based on the intrinsic or technical connections between relevant inputs and outputs. It also creates a coherent framework that coordinates various considerations across all under-evaluated DMUs and variables. This method effectively examines the complexities related to the inputs and outputs of homogeneous units using conventional data envelopment analysis (DEA) models to evaluate the efficiency of these units.In the banking sector, the relationship between inputs and outputs is complex. This complexity requires careful attention, particularly in cases where the existence of one input or output is conditional on the existence of another. This dynamic is accentuated by the integration of advanced fintech solutions, which inherently provide significant benefits to banking operations. The use of appropriate analytical models for banking data is important. The synergy between advanced financial technologies and analytical methods has reached its peak in reducing losses and strengthening profits. This, in turn, facilitates the optimal allocation of financial resources to growth-oriented projects.The empirical application of weight-restricted DEA models to Turkey's mobile-internet-banking sector underscores their efficacy. These modified models contrast with the standard DEA versions, which often encounter challenges stemming from the allocation of weights to pivotal variables. This study demonstrates the tangible benefits of our approach and its potential to revolutionise the efficiency assessment landscape in the banking sector.The proposed unconventional model which is applied to Turkish Internet banking for the period extending from 2006 to 2018 and which takes a comprehensive weight restriction approach to DEA and encompasses all relevant input and output variables, resulted in strictly positive weights and a reduced count of inflated efficient units along with their scores. This is practically beneficent to banking sector in so far as it facilitates decision-making and related economic and financial planning, and as it proves to be a reliable benchmark by local or international banks intending to provide or develop Internet banking services.

Reservoir Computing for Solving Ordinary Differential Equations

There is a wave of interest in using physics-informed neural networks for solving differential equations. Most of the existing methods are based on feed-forward networks, while recurrent neural networks solvers have not been extensively explored. We introduce a reservoir computing (RC) architecture, an echo-state recurrent neural network capable of discovering approximate solutions that satisfy ordinary differential equations (ODEs). We suggest an approach to calculate time derivatives of recurrent neural network outputs without using back-propagation. The internal weights of an RC are fixed, while only a linear output layer is trained, yielding efficient training. However, RC performance strongly depends on finding the optimal hyper-parameters, which is a computationally expensive process. We use Bayesian optimization to discover optimal sets in a high-dimensional hyper-parameter space efficiently and numerically show that one set is robust and can be transferred to solve an ODE for different initial conditions and time ranges. A closed-form formula for the optimal output weights is derived to solve first-order linear equations in a one-shot backpropagation-free learning process. We extend the RC approach by solving nonlinear systems of ODEs using a hybrid optimization method consisting of gradient descent and Bayesian optimization. Evaluation of linear and nonlinear systems of equations demonstrates the efficiency of the RC ODE solver.

A finite time discrete distributed learning algorithm using stochastic configuration network

In this paper, we propose a finite-time discrete distributed learning algorithm, denoted as the FTDDL algorithm, using the Stochastic Configuration Network (SCN) and Zero Gradient Sum (ZGS) strategy. In distributed scenarios, the communication networks that store data are modeled as algebraic graphs and the learning problems are formulated as distributed optimization problems. The FTDDL algorithm persists the universal approximation of the SCN learner. The process of the FTDDL algorithm is divided into two phases, namely “distributed hidden parameters configuration” and “distributed output weights determination”. It has been theoretically proved that the FTDDL algorithm works in distributed manner and achieves the same efficiency as the centralized SCN. According to the FTDDL algorithm, each node over the communication network transmits the updated output weights of the SCN with its direct neighbors. Thus, the FTDDL algorithm preserves data privacy and saves network bandwidth. The FTDDL algorithm is theoretically proved to converge to the global optimal output weights of the SCN within finite time. Compared with the other distributed learning algorithms using SCNs, the FTDDL algorithm can significantly reduce the iteration times. At last, some experiments are shown in Simulations part to verify the effectiveness of the FTDDL algorithm.

A Polak-Ribière-Polyak Conjugate Gradient Algorithm Optimized Broad Learning System for Lithium-ion Battery State of Health Estimation

Accurate state of health (SOH) estimation plays a significant role in the battery management system. This paper investigates a Polak-Ribière-Polyak conjugate gradient (PRPCG) algorithm optimized broad learning system (BLS) for lithium-ion battery SOH estimation. Firstly, effective health indicators (HIs) are extracted from the voltage curve in the constant current charge process. Secondly, a hybrid four layers BLS structure with mapped feature nodes and enhancement nodes connecting to the output is established to build both the linear and nonlinear relationships between the HIs and SOH, in which only the output weights require to be trained. Again, the PRPCG algorithm is adopted for searching optimal output weights without matrix inverse calculation during the training process. Furthermore, certain Gaussian noises are added to enhance the training data for solving the locally low accuracy problem. Finally, under the Oxford battery degradation data set, experiments validate the investigated algorithm has high accuracy in SOH estimation with the mean absolute error below 1%. The enhanced data can efficiently improve the model generalization ability.

Bayesian stochastic configuration networks for robust data modeling

AbstractThe SCN networks is incrementally generated by stochastic configuration (SC) algorithms. It randomly assigns the input weights and deviations of hidden nodes through a supervisory mechanism, which can be trained by solving linear modeling problems. The version that uses least squares to estimate the output weight performs well. This article introduces an alternative strategy for performing complete Bayesian inference (BI) of SCN networks. Different from the traditional way, the Bayesian training algorithm we proposed can obtain an entire probability distribution on the optimal output weight of the SCN networks, instead of a single pointwise estimate. The advantage of the Bayesian inference method lies in the possibility of introducing other prior knowledge during the training process, providing a way to measure uncertainty during the testing phase and the ability to automatically infer hyper‐parameters from given data. The BI algorithm proposed in this article for regression problem can be implemented through an iterative process under some practical assumptions. Experimental results show that our proposed Bayesian SCN algorithm performs well in solving data modeling problems with a large number of outliers.

Projectile drag coefficient identification based on extreme learning

Aerodynamic parameters play a decisive role in the ballistic characteristics of the projectile. How to accurately obtain the aerodynamic parameters of the projectile is an important task in the development process of the projectile. In order to further improve the identification accuracy of the projectile drag coefficient, this paper generates huge ballistic data through numerical simulation and uses the extreme learning method to identify the ballistic drag coefficient under three kinds of noise conditions. The method avoids the iterative updating process of weights and thresholds by randomly generating the input weights and threshold values of hidden layer neurons and overcomes the problem of long identification time of the traditional back propagation (BP) neural network algorithm. Based on the least squares principle, the Moore–Penrose generalized inverse matrix of the hidden layer output matrix was solved to determine the optimal output weight of the network, and then, the projectile drag coefficient was accurately identified. Comparing the extreme learning method with the traditional BP neural network method, the results show that the proposed method has higher identification accuracy and faster convergence speed and can effectively identify the projectile drag coefficient, which can meet the practical needs of engineering.

Minimum error entropy criterion‐based randomised autoencoder

The extreme learning machine-based autoencoder (ELM-AE) has attracted a lot of attention due to its fast learning speed and promising representation capability. However, the existing ELM-AE algorithms only reconstruct the original input and generally ignore the probability distribution of the data. The minimum error entropy (MEE), as an optimal criterion considering the distribution statistics of the data, is robust in handling non-linear systems and non-Gaussian noises. The MEE is equivalent to the minimisation of the Kullback–Leibaler divergence. Inspired by these advantages, a novel randomised AE is proposed by adopting the MEE criterion as the loss function in the ELM-AE (in short, the MEE-RAE) in this study. Instead of solving the output weight by the Moore–Penrose generalised inverse, the optimal output weight is obtained by the fixed-point iteration method. Further, a quantised MEE (QMEE) is applied to reduce the computational complexity of. Simulations have shown that the QMEE-RAE not only achieves superior generalisation performance but is also more robust to non-Gaussian noises than the ELM-AE.

Optimal solutions of multiplier DEA models

Conventional models of data envelopment analysis (DEA) are based on the constant and variable returns-to-scale production technologies. Any optimal input and output weights of the multiplier DEA models based on these technologies are interpreted as being the most favorable for the decision making unit (DMU) under the assessment when the latter is benchmarked against the set of all observed DMUs. In this paper we consider a very large class of DEA models based on arbitrary polyhedral technologies, which includes almost all known convex DEA models. We highlight the fact that the conventional interpretation of the optimal input and output weights in such models is generally incorrect, which raises a question about the meaning of multiplier models. We address this question and prove that the optimal solutions of such models show the DMU under the assessment in the best light in comparison to the entire technology, but not necessarily in comparison to the set of observed DMUs. This result allows a clear and meaningful interpretation of the optimal solutions of multiplier models, including known models with a complex constraint structure whose interpretation has been problematic and left unaddressed in the existing literature.

Artificial Immune System in Doing 2-Satisfiability Based Reverse Analysis Method via a Radial Basis Function Neural Network

A radial basis function neural network-based 2-satisfiability reverse analysis (RBFNN-2SATRA) primarily depends on adequately obtaining the linear optimal output weights, alongside the lowest iteration error. This study aims to investigate the effectiveness, as well as the capability of the artificial immune system (AIS) algorithm in RBFNN-2SATRA. Moreover, it aims to improve the output linearity to obtain the optimal output weights. In this paper, the artificial immune system (AIS) algorithm will be introduced and implemented to enhance the effectiveness of the connection weights throughout the RBFNN-2SATRA training. To prove that the introduced method functions efficiently, five well-established datasets were solved. Moreover, the use of AIS for the RBFNN-2SATRA training is compared with the genetic algorithm (GA), differential evolution (DE), particle swarm optimization (PSO), and artificial bee colony (ABC) algorithms. In terms of measurements and accuracy, the simulation results showed that the proposed method outperformed in the terms of Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), Schwarz Bayesian Criterion (SBC), and Central Process Unit time (CPU time). The introduced method outperformed the existing four algorithms in the aspect of robustness, accuracy, and sensitivity throughout the simulation process. Therefore, it has been proven that the proposed AIS algorithm effectively conformed to the RBFNN-2SATRA in relation to (or in terms of) the average value of training of RMSE rose up to 97.5%, SBC rose up to 99.9%, and CPU time by 99.8%. Moreover, the average value of testing in MAE was rose up to 78.5%, MAPE was rose up to 71.4%, and was capable of classifying a higher percentage (81.6%) of the test samples compared with the results for the GA, DE, PSO, and ABC algorithms.

Maximum Correntropy Criterion-Based Hierarchical One-Class Classification.

Due to the effectiveness of anomaly/outlier detection, one-class algorithms have been extensively studied in the past. The representatives include the shallow-structure methods and deep networks, such as the one-class support vector machine (OC-SVM), one-class extreme learning machine (OC-ELM), deep support vector data description (Deep SVDD), and multilayer OC-ELM (ML-OCELM/MK-OCELM). However, existing algorithms are generally built on the minimum mean-square-error (mse) criterion, which is robust to the Gaussian noises but less effective in dealing with large outliers. To alleviate this deficiency, a robust maximum correntropy criterion (MCC)-based OC-ELM (MC-OCELM) is first proposed and then further extended to a hierarchical network to enhance its capability in characterizing complex and large data (named HC-OCELM). The gradient derivation combining with a fixed-point iterative updation scheme is adopted for the output weight optimization. Experiments on many benchmark data sets are conducted for effectiveness validation. Comparisons to many state-of-the-art approaches are provided for the superiority demonstration.

Robust Maximum Mixture Correntropy Criterion-Based Semi-Supervised ELM With Variable Center

The recent correntropy criterion based semi-supervised random neural network extreme learning machine (RC-SSELM) achieved outstanding performance in dealing with datasets with large outliers and non-Gaussian noises. To further improve the effectiveness and flexibility of the algorithm in combating large and complex outliers, we explore a more effective semi-supervised ELM data learning algorithm with the robust maximum mixture correntropy criterion (MMCC) based optimization scheme in this brief. Meanwhile, the generalized correntropy criterion kernel function with the variable kernel center is applied to MMCC, and the resultant novel semi-supervised learning algorithm is abbreviated as MC-SSELM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">vc</sup> . The fixed-point iteration learning algorithm is adopted for the output weight optimization of MC-SSELM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">vc</sup> . Experiment conducted on many benchmark datasets are given to show the effectiveness of MC-SSELM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">vc</sup> and comparisons to several state-of-the-art semi-supervised learning algorithms are provided for the superiority demonstration.

A Novel Orthogonal Extreme Learning Machine for Regression and Classification Problems

An extreme learning machine (ELM) is an innovative algorithm for the single hidden layer feed-forward neural networks and, essentially, only exists to find the optimal output weight so as to minimize output error based on the least squares regression from the hidden layer to the output layer. With a focus on the output weight, we introduce the orthogonal constraint into the output weight matrix, and propose a novel orthogonal extreme learning machine (NOELM) based on the idea of optimization column by column whose main characteristic is that the optimization of complex output weight matrix is decomposed into optimizing the single column vector of the matrix. The complex orthogonal procrustes problem is transformed into simple least squares regression with an orthogonal constraint, which can preserve more information from ELM feature space to output subspace, these make NOELM more regression analysis and discrimination ability. Experiments show that NOELM has better performance in training time, testing time and accuracy than ELM and OELM.

Investigation on the Efficiency of Financial Companies in Malaysia with Data Envelopment Analysis Model

Financial ratio and risk are important financial indicators to evaluate the financial performance or efficiency of the companies. Therefore, financial ratio and risk factor are needed to be taken into consideration to evaluate the efficiency of the companies with Data Envelopment Analysis (DEA) model. In DEA model, the efficiency of the company is measured as the ratio of sum-weighted outputs to sum-weighted inputs. The objective of this paper is to propose a DEA model by incorporating the financial ratio and risk factor in evaluating and comparing the efficiency of the financial companies in Malaysia. In this study, the listed financial companies in Malaysia from year 2004 until 2015 are investigated. The results of this study show that AFFIN, ALLIANZ, APEX, BURSA, HLCAP, HLFG, INSAS, LPI, MNRB, OSK, PBBANK, RCECAP and TA are ranked as efficient companies. This implies that these efficient companies have utilized their resources or inputs optimally to generate the maximum outputs. This study is significant because it helps to identify the efficient financial companies as well as determine the optimal input and output weights in maximizing the efficiency of financial companies in Malaysia.

Bayesian Random Vector Functional-Link Networks for Robust Data Modeling.

Random vector functional-link (RVFL) networks are randomized multilayer perceptrons with a single hidden layer and a linear output layer, which can be trained by solving a linear modeling problem. In particular, they are generally trained using a closed-form solution of the (regularized) least-squares approach. This paper introduces several alternative strategies for performing full Bayesian inference (BI) of RVFL networks. Distinct from standard or classical approaches, our proposed Bayesian training algorithms allow to derive an entire probability distribution over the optimal output weights of the network, instead of a single pointwise estimate according to some given criterion (e.g., least-squares). This provides several known advantages, including the possibility of introducing additional prior knowledge in the training process, the availability of an uncertainty measure during the test phase, and the capability of automatically inferring hyper-parameters from given data. In this paper, two BI algorithms for regression are first proposed that, under some practical assumptions, can be implemented by a simple iterative process with closed-form computations. Simulation results show that one of the proposed algorithms, Bayesian RVFL, is able to outperform standard training algorithms for RVFL networks with a proper regularization factor selected carefully via a line search procedure. A general strategy based on variational inference is also presented, with an application to data modeling problems with noisy outputs or outliers. As we discuss in this paper, using recent advances in automatic differentiation this strategy can be applied to a wide range of additional situations in an immediate fashion.

Second Order Training of a Smoothed Piecewise Linear Network

In this paper, we introduce a smoothed piecewise linear network (SPLN) and develop second order training algorithms for it. An embedded feature selection algorithm is developed which minimizes training error with respect to distance measure weights. Then a method is presented which adjusts center vector locations in the SPLN. We also present a gradient method for optimizing the SPLN output weights. Results with several data sets show that the distance measure optimization, center vector optimization, and output weight optimization, individually and together, reduce testing errors in the final network.

Probabilistic Forecasting of Traffic Flow Using Multikernel Based Extreme Learning Machine

Real-time and accurate prediction of traffic flow is the key to intelligent transportation systems (ITS). However, due to the nonstationarity of traffic flow data, traditional point forecasting can hardly be accurate, so probabilistic forecasting methods are essential for quantification of the potential risks and uncertainties for traffic management. A probabilistic forecasting model of traffic flow based on a multikernel extreme learning machine (MKELM) is proposed. Moreover, the optimal output weights of MKELM are obtained by utilizing Quantum-behaved particle swarm optimization (QPSO) algorithm. To verify its effectiveness, traffic flow probabilistic prediction using QPSO-MKELM was compared with other learning methods. Experimental results show that QPSO-MKELM is more effective for practical applications. And it will help traffic managers to make right decisions.

Properties of a Batch Training Algorithm for Feedforward Networks

We examine properties for a batch training algorithm known as the output weight optimization---hidden weight optimization (OWO---HWO). Using the concept of equivalent networks, we analyze the effect of input transformation on BP. We introduce new theory of affine invariance and partial affine invariance for neural networks and prove this property for OWO---HWO. Finally, we relate HWO to BP and show that every iteration of HWO is equivalent to BP applied to whitening transformed data. Experimental results validate the connection between OWO---HWO and OWO---BP.

Artificial Neural Networks Applied to Flow Prediction: A Use Case for the Tomebamba River

The main aim of this research is to create a model based on Artificial Neural Networks (ANN) that allows predicting the flow in Tomebamba river, at real time and in a specific day of a year. As inputs, this research is using information of rainfall and flow of the stations along of the river. This information is organized in scenarios and each scenario is prepared to a specific area. For this article, we have selected two scenarios. The information is acquired from the hydrological stations placed in the watershed using an electronic system developed at real time and it supports any kind or brands of this type of sensors. The prediction works very good three days in advance. This research includes two ANN models: Backpropagation and a hybrid model between back propagation and OWO-HWO (output weight optimization–hidden weight optimization) to select the initial weights of the connection. These last two models have been tested in a preliminary research. To validate the results we are using some error indicators such as MSE, RMSE, EF, CD and BIAS. The results of this research reached high levels of reliability and the level of error is minimal. These predictions are useful to avoid floods in the city of Cuenca in Ecuador.

Echo State Network with Bayesian Regularization for Forecasting Short-Term Power Production of Small Hydropower Plants

As a novel recurrent neural network (RNN), an echo state network (ESN) that utilizes a reservoir with many randomly connected internal units and only trains the readout, avoids increased complexity of training procedures faced by traditional RNN. The ESN can cope with complex nonlinear systems because of its dynamical properties and has been applied in hydrological forecasting and load forecasting. Due to the linear regression algorithm usually adopted by generic ESN to train the output weights, an ill-conditioned solution might occur, degrading the generalization ability of the ESN. In this study, the ESN with Bayesian regularization (BESN) is proposed for short-term power production forecasting of small hydropower (SHP) plants. According to the Bayesian theory, the weights distribution in space is considered and the optimal output weights are obtained by maximizing the posterior probabilistic distribution. The evidence procedure is employed to gain optimal hyperparameters for the BESN model. The recorded data obtained from the SHP plants in two different counties, located in Yunnan Province, China, are utilized to validate the proposed model. For comparison, the feed-forward neural networks with Levenberg-Marquardt algorithm (LM-FNN) and the generic ESN are also employed. The results indicate that BESN outperforms both LM-FNN and ESN.

Daily rainfall-runoff forecasting using Bayesian echo state network

The echo state network (ESN) is simpler and costs less training time than traditional recurrent neural networks. Due to linear regression algorithm usually adopted by standard ESN to calibrate model parameters, the over-fitting phenomenon easily occurs. To overcome this shortcoming, a Bayesian echo state network (BESN) model is proposed for daily rainfall-runoff forecasting. The BESN model combined Bayesian theory and ESN obtains the optimal output weights via maximizing posterior probabilistic density and improves its generalization ability. Two Case studies on daily inflow forecasting for Ansha Reservoir and Xinfengjiang Reservoir show that the BESN model is effective and feasible and can provide better forecast accuracy than the traditional BP neural network and ESN models.