Nonlinear Regression Algorithm Research Articles

Simultaneous sEMG Classification of Hand/Wrist Gestures and Forces.

Surface electromyography (sEMG) signals represent a promising approach for decoding the motor intention of amputees to control a multifunctional prosthetic hand in a non-invasive way. Several approaches based on proportional amplitude methods or simple thresholds on sEMG signals have been proposed to control a single degree of freedom at time, without the possibility of increasing the number of controllable multiple DoFs in a natural manner. Myoelectric control based on PR techniques have been introduced to add multiple DoFs by keeping low the number of electrodes and allowing the discrimination of different muscular patterns for each class of motion. However, the use of PR algorithms to simultaneously decode both gestures and forces has never been studied deeply. This paper introduces a hierarchical classification approach with the aim to assess the desired hand/wrist gestures, as well as the desired force levels to exert during grasping tasks. A Finite State Machine was introduced to manage and coordinate three classifiers based on the Non-Linear Logistic Regression algorithm. The classification architecture was evaluated across 31 healthy subjects. The “hand/wrist gestures classifier,” introduced for the discrimination of seven hand/wrist gestures, presented a mean classification accuracy of 98.78%, while the “Spherical and Tip force classifier,” created for the identification of three force levels, reached an average accuracy of 98.80 and 96.09%, respectively. These results were confirmed by Linear Discriminant Analysis (LDA) with time domain features extraction, considered as ground truth for the final validation of the performed analysis. A Wilcoxon Signed-Rank test was carried out for the statistical analysis of comparison between NLR and LDA and statistical significance was considered at p < 0.05. The comparative analysis reports not statistically significant differences in terms of F1Score performance between NLR and LDA. Thus, this study reveals that the use of non-linear classification algorithm, as NLR, is as much suitable as the benchmark LDA classifier for implementing an EMG pattern recognition system, able both to decode hand/wrist gestures and to associate different performed force levels to grasping actions.

Особливості моделювання твірної поверхні стовбурів дуба звичайного

Узагальнено досвід дослідження форми стовбурів дерев, який дав змогу зробити висновок про переваги математичного моделювання твірної поверхні для аналізу збігу стовбурів. На основі дослідного матеріалу, зібраного в дубових насадженнях Придніпровського правобережного Лісостепу на 17 тимчасових пробних площах, проаналізовано залежність форми стовбурів дуба звичайного (Quercus robur L.) від висоти та діаметра на висоті 1,3 м. За допомогою графічного аналізу та t-тесту середніх значень коефіцієнтів форми Шиффеля на 5 %-му рівні значущості виявлено відмінність у формі стовбурів дуба різних груп діаметрів. За розрядами висот статистично значущої різниці, за характером збігу не встановлено. Виконаний аналіз дав змогу обґрунтувати доцільність моделювання збігу стовбурів дуба звичайного незалежно від висоти, але в межах 3-х груп діаметрів: 20-28 см, 32-40 см та 44-64 см. Як моделі твірної поверхні стовбурів апробовано показниково-степеневі математичні рівняння A. Kozak (1988) і R. Newnham (1992). Підбір параметрів рівнянь виконано для вибірки з 171 модельного дерева в середовищі R за допомогою функції nls − нелінійна мінімізація суми квадратів відхилень. Систематична помилка обох рівнянь виявилася близькою до нуля (0,9-1,7 %), що дало змогу прийняти їх для моделювання твірної поверхні стовбурів різної форми для встановлених груп діаметрів. Розроблені моделі дають змогу прогнозувати діаметри стовбурів дуба на будь-якій висоті зі стандартною помилкою 1,5-3,2 см. Все ж аналіз розподілу залишків математичних моделей засвідчив переваги моделі A. Kozak (1988), яку рекомендовано використовувати надалі для моделювання розмірно-якісної структури стовбурів дуба звичайного.

Using Text Mining to Predicate Exchange Rates with Sentiment Indicators

Recent innovations in text mining facilitate the use of novel data for sentiment analysis related to financial markets, and promise new approaches to the field of behavioral finance. Traditionally, text mining has allowed a near-real time analysis of available news feeds. The recent dissemination of web 2.0 has seen a drastic increase of user participation, providing comments on websites, social networks and blogs, creating a novel source of rich and personal sentiment data potentially of value to behavioral finance. This study explores the efficacy of using novel sentiment indicators from Market Psych, which analyses social media in addition to newsfeeds to quantify various levels of individual’s emotions, as a predictor for financial time series returns of the Australian Dollar (AUD)-US Dollar (USD) exchange rate. As one of the first studies evaluating both news and social media sentiment indicators as explanatory variables for linear and nonlinear regression algorithms, our study aims to make an original contribution to behavioral finance, combining technical and behavioral aspects of model building. An empirical out-of-sample evaluation with multiple input structures compares Multivariate Linear Regression models (MLR) with multilayer perceptron (MLP) neural networks for descriptive modelling. The results indicate that sentiment indicators are explanatory for market movements of exchange rate returns, with nonlinear MLPs showing superior accuracy over linear regression models with a directional out-of-sample accuracy of 60.26% using cross validation.

Abstract WP162: Automatic Detection of Middle Cerebral Artery Plaque on T2-weighted Vessel Wall Imaging

Background: Intracranial vessel wall imaging (VWI) has been reported to be an efficient approach for evaluating ICAD, yet the reliability of intracranial plaque identification may vary depending on the experience of institutions or readers. We aim to automatically detect middle cerebral artery plaque on T2-weighted VWI using machine learning algorithms and thus reduce inter-reader variability. Methods: Patients were selected from our institutional vessel wall imaging database if they had acute stroke in the MCA territory on diffusion-weighted imaging (DWI) and MCA plaque seen on VWI, or if they had no history of stroke or transient ischemic attack and no plaque seen on VWI. Bilateral MCAs of all patients were analyzed individually. The cross-sectional images of MCA were graded into a scale of 1-3 based on the confidence of the reader regarding the presence of plaque in the cross-sections: Grade 1, the plaque is not present, used as control group; Grade 2, the plaque may or may not present, not included in further analysis; Grade 3, the plaque is present, used as plaque group. Cross-sections with insufficient image quality result from motion artifacts or with non-cross-sectional images of MCA were excluded. The annotated vessel regions on T2-weighted images were used as input to a classification algorithm based on a nonlinear spectral regression algorithm. The model was evaluated under a crossvalidation in order to predict the presence of plaque in the vessel (Grade 1 versus Grade 3). Results: A total of 104 patients with stroke and 27 controls were included, leading to 453 slices with plaque and 246 slices for the control group. After a 3-fold crossvalidation, the model reached an accuracy of 87.2% in detecting correctly the presence of plaque in the vessel regions. Conclusion: Our experiments demonstrate that a nonlinear machine learning approach can achieve high performance in detecting presence of plaque in the vessel, which is the first step in further analysis of plaque-related parameters and more objective diagnoses.

An integrated multi-response optimisation route combining principal component analysis, fuzzy inference system, nonlinear regression and JAYA algorithm: a case experimental study on machining of GFRP (epoxy) composites

Machining (drilling) operations have been performed on glass fibre reinforced polymer (GFRP) (epoxy) composites. The work intended to evaluate the most favourable setting of controllable process parameters which could simultaneously satisfy multi-requirements of process performance yield; in view of product quality as well as productivity. During drilling, three process parameters viz. drill rotational speed, feed rate and drill diameter have been considered to optimise thrust, torque and delamination factor (entry and exit both), simultaneously. Owing to the limitations of traditional Taguchi method-based optimisation approaches, the study proposes an integrated optimisation module combining principal component analysis (PCA), fuzzy inference system (FIS), nonlinear regression and JAYA algorithm towards optimising correlated multi-response features during machining of GFRP (epoxy) composites. JAYA is parameter (algorithm-specific)-less algorithm; which is used to solve constrained and unconstrained optimisation problems. Application potential of the aforesaid integrated optimisation route has been compared to that of teaching-learning-based optimisation (TLBO) algorithm; good agreement has been observed.

An integrated multi-response optimisation route combining principal component analysis, fuzzy inference system, nonlinear regression and JAYA algorithm: a case experimental study on machining of GFRP (epoxy) composites

An integrated multi-response optimisation route combining principal component analysis, fuzzy inference system, nonlinear regression and JAYA algorithm: a case experimental study on machining of GFRP (epoxy) composites

Nonlinear regression via incremental decision trees

We study sequential nonlinear regression and introduce an online algorithm that elegantly mitigates, via an adaptively incremental hierarchical structure, convergence and undertraining issues of conventional nonlinear regression methods. Particularly, we present a piecewise linear (or nonlinear) regression algorithm that partitions the regressor space and learns a linear model at each region to combine. Unlike the conventional approaches, our algorithm effectively learns the optimal regressor space partition with the desired complexity in a completely sequential and data driven manner. Our algorithm sequentially and asymptotically achieves the performance of the optimal twice differentiable regression function for any data sequence without any statistical assumptions. The introduced algorithm can be efficiently implemented with a computational complexity that is only logarithmic in the length of data. In our experiments, we demonstrate significant gains for the well-known benchmark real data sets when compared to the state-of-the-art techniques.

Quantifying Vegetation Biophysical Variables from Imaging Spectroscopy Data: A Review on Retrieval Methods.

An unprecedented spectroscopic data stream will soon become available with forthcoming Earth-observing satellite missions equipped with imaging spectroradiometers. This data stream will open up a vast array of opportunities to quantify a diversity of biochemical and structural vegetation properties. The processing requirements for such large data streams require reliable retrieval techniques enabling the spatiotemporally explicit quantification of biophysical variables. With the aim of preparing for this new era of Earth observation, this review summarizes the state-of-the-art retrieval methods that have been applied in experimental imaging spectroscopy studies inferring all kinds of vegetation biophysical variables. Identified retrieval methods are categorized into: (1) parametric regression, including vegetation indices, shape indices and spectral transformations; (2) nonparametric regression, including linear and nonlinear machine learning regression algorithms; (3) physically based, including inversion of radiative transfer models (RTMs) using numerical optimization and look-up table approaches; and (4) hybrid regression methods, which combine RTM simulations with machine learning regression methods. For each of these categories, an overview of widely applied methods with application to mapping vegetation properties is given. In view of processing imaging spectroscopy data, a critical aspect involves the challenge of dealing with spectral multicollinearity. The ability to provide robust estimates, retrieval uncertainties and acceptable retrieval processing speed are other important aspects in view of operational processing. Recommendations towards new-generation spectroscopy-based processing chains for operational production of biophysical variables are given.

Machining performance optimization during plasma arc cutting of AISI D2 steel: application of FIS, nonlinear regression and JAYA optimization algorithm

The work focuses on assessing the optimal machining conditions which could simultaneously satisfy multiple process performance indices during machining of AISI D2 steel. The main characteristic indices that have been considered here for evaluating plasma arc machining are surface roughness and material removal rate; the corresponding machining parameters are cutting speed, gas pressure and torch height. The study proposes an integrated optimization module combining fuzzy inference system, nonlinear regression and JAYA algorithm towards optimizing correlated multi-response features during machining of AISI D2 steel. Optimum value of machining parameters found as cutting speed of 4000 m/min, gas pressure of 95 psi and torch height of 0.5 mm using aforementioned methodology. Application potential of the aforesaid integrated optimization route has been compared to that of teaching–learning based optimization (TLBO) algorithm and genetic algorithm. It has been concluded that JAYA algorithm possesses less convergence time and hence execution is faster as compared to TLBO and genetic algorithm.

Normative human brain volume growth.

OBJECTIVE While there is a long history of interest in measuring brain growth, as of yet there is no definitive model for normative human brain volume growth. The goal of this study was to analyze a variety of candidate models for such growth and select the model that provides the most statistically applicable fit. The authors sought to optimize clinically applicable growth charts that would facilitate improved treatment and predictive management for conditions such as hydrocephalus. METHODS The Weibull, two-term power law, West ontogenic, and Gompertz models were chosen as potential models. Normative brain volume data were compiled from the NIH MRI repository, and the data were fit using a nonlinear least squares regression algorithm. Appropriate statistical measures were analyzed for each model, and the best model was characterized with prediction bound curves to provide percentile estimates for clinical use. RESULTS Each model curve fit and the corresponding statistics were presented and analyzed. The Weibull fit had the best statistical results for both males and females, while the two-term power law generated the worst scores. The statistical measures and goodness of fit parameters for each model were provided to assure reproducibility. CONCLUSIONS The authors identified the Weibull model as the most effective growth curve fit for both males and females. Clinically usable growth charts were developed and provided to facilitate further clinical study of brain volume growth in conditions such as hydrocephalus. The authors note that the homogenous population from which the normative MRI data were compiled limits the study. Gaining a better understanding of the dynamics that underlie childhood brain growth would yield more predictive growth curves and improved neurosurgical management of hydrocephalus.

Protein A affinity chromatography of Chinese hamster ovary (CHO) cell culture broths containing biopharmaceutical monoclonal antibody (mAb): Experiments and mechanistic transport, binding and equilibrium modeling

Protein A-based affinity chromatography is a highly-efficient separation method to capture, purify and isolate biosimilar monoclonal antibodies (mAb) – an important medical product of biopharmaceutical industrial manufacturing. It is considered the most expensive step in purification downstream operations; therefore, its performance optimization offers a great cost saving in the overall production expenditure. The biochemical mixture-separating specific interaction experiments with Chinese hamster ovary (CHO) cell culture harvest, containing glycosylated extracellular immunoglobulins (Ig), were made using five different state-of-the-art commercial resins. Packing breakthrough curves were recorded at an array of prolonged residence times. A mathematical simulation model was developed, applied and validated in combination with non-linear regression algorithms on bed effluent concentrations to determine the previously-unknown binding properties of stationary phase materials. Apart from the columns' differential partitioning, the whole external system was also integrated. It was confirmed that internal pore diffusion is the global rate-limiting resistance of the compound retention process. Immobilizing substrate characteristics, obtained in this engineering study, are indispensable for the scale-up of the periodic counter-current control with mechanistic load, elution and wash reduction. Furthermore, unit's volumetric flow screening measurements revealed dynamic effect correlation to eluate quality parameters, like the presence of aggregates, the host cell-related impurities at supernatant's extended feeding, and titre. Numerical sensitivity outputs demonstrated the impacts of fluidics (e.g. axial dispersion coefficient), thermodynamics (Langmuir adsorption) and mass transfer fluxes.

Heuristic nonlinear regression strategy for detecting phishing websites

In this paper, we propose a method of phishing website detection that utilizes a meta-heuristic-based nonlinear regression algorithm together with a feature selection approach. In order to validate the proposed method, we used a dataset comprised of 11055 phishing and legitimate webpages, and select 20 features to be extracted from the mentioned websites. This research utilizes two feature selection methods: decision tree and wrapper to select the best feature subset, while the latter incurred the detection accuracy rate as high as 96.32%. After the feature selection process, two meta-heuristic algorithms are successfully implemented to predict and detect the fraudulent websites: harmony search (HS) which was deployed based on nonlinear regression technique and support vector machine (SVM). The nonlinear regression approach was used to classify the websites, where the parameters of the proposed regression model were obtained using HS algorithm. The proposed HS algorithm uses dynamic pitch adjustment rate and generated new harmony. The nonlinear regression based on HS led to accuracy rates of 94.13 and 92.80% for train and test processes, respectively. As a result, the study finds that the nonlinear regression-based HS results in better performance compared to SVM.

Analysis of Flight Fuel Consumption Based on Nonlinear Regression

For to increase fuel utilization and reduce carbon emissions of flights, the relationship between each factor and fuel consumption was established using real flight data and nonlinear regression algorithm. By counting the flight data of different flights (model A330), the fitting model of fuel consumption and fuel consumption was established, and the contribution of minor change of each factor to fuel consumption was also analyzed. The results show that this method can achieve better generalization ability and describe the relationship between the fuel consumption and the various factors. This method serves as a means to evaluate the effectiveness of energy saving and emission reduction, and provides technical guidance for dispatchers to analyze fuel economy.

Runtime Energy Estimation and Route Optimization for Autonomous Underwater Vehicles

This paper is focused on improving the self-awareness of autonomous underwater vehicles (AUVs) operating in unknown environments. A runtime estimation framework is introduced to derive energy usage and navigation performance metrics in the presence of external disturbances, such as slowly varying sea currents. These are calculated by a state-of-the-art nonlinear regression algorithm (LWPR) using measurements commonly available on-board modern AUVs without relying on external sensors or a priori knowledge about the environment. The proposed framework is validated on two vehicles, an IVER3 AUV and a Nessie VII AUV, in the context of real sea trials with no modification required for the AUVs or their missions. Derived metrics are used to estimate the feasibility of underwater missions employing the concept of probability of mission completion (PoMC). If environmental effects modify the vehicle's effectiveness, a mission plan update is performed. This is based on an energy-aware route optimization algorithm that is also introduced in the paper. This algorithm, known as energy-aware orienteering problem (EA-OP), shows a practical usage for the runtime metrics. It allows an AUV to optimize its navigation and to maximize its mission's outcome according to measured performances. Simulation results are also presented for inspection scenarios. These show average improvements of 5%-20% for the mission's outcome when using the proposed strategy in the presence of environmental disturbances.

Implementation of reduced-order physics-based model and multi-parameters identification strategy for lithium-ion battery

Physics-based models for lithium-ion battery have been regarded as a promising alternative to equivalent circuit models due to their ability to describe internal electrochemical states of battery. However, the huge computational burden and numerous parameters of these models impede their application in embedded battery management system. To deal with the above problem, a reduced-order physics-based model for lithium-ion battery with better tradeoff between the model fidelity and computational complexity is developed. A strategy is proposed to extend the operation from a fixed point to full state of charge range. As the model consists of constant, varying, identifiable and unidentifiable parameters, it is impractical to identify the full set of parameters only using the current-voltage data. To sort out the identifiable parameters, a criterion based on calculating the determinant and condition number of Fisher information matrix (FIM) is employed. A subset with maximum nine identifiable parameters is obtained and then identified by nonlinear least square regression algorithm with confidence region calculated by FIM. Compared with the outputs from commercial software, the effectiveness of the battery model and extending strategy are verified. The estimated parameters deviate from the true values slightly, and produce small voltage errors at different current profiles.

Learning-based focusing through scattering media

We present a machine-learning-based method for light focusing through scattering media. In this method, the optical process in a scattering medium is computationally inverted based on a nonlinear regression algorithm with a number of training input-output pairs through the medium, and an input optimized for a target output is calculated. We experimentally demonstrate focusing via a process involving randomness due to a scattering medium and nonlinearity due to double modulation with a spatial light modulator. Our approach realizes model-free control of optical fields, where optical processes or models are unknown.

Research and Application for Transmission Line Sag Measurement Method Based on Aerial Image Sequence

Sag is an important parameter of the transmission lines, which directly affects the safe and stable operation of the transmission lines. This paper proposes a new sag measurement method based on the aerial image. It selects the spacer rod on the transmission lines as the recognized target, and adopts the normalized correlation image matching algorithm and the wavelet pyramid layered search strategy to measure the pixel coordinates of the center of the spacer rod in the sequence image. And then this paper measures the three-dimensional space coordinates of the center of the spacer rod according to the active vision technology, and finally calculates the actual sag via the nonlinear least square regression algorithm. The experimental results prove that this method can improve the efficiency of the sag measurement and it is also suitable for application in complex geographical conditions.

Phase-averaged flow statistics in compressors using a rotated hot-wire technique

A technique based on a rotated hot wire has been developed to characterise the unsteady, three-dimensional flow field between compressor blade rows. Data are acquired from a slanted hot wire rotated through a number of orientations at each measurement point. Phase-averaged velocity statistics are obtained by solving a set of sensor response equations using a weighted, non-linear regression algorithm. The accuracy and robustness of the method were verified a priori by conducting a series of tests using synthetic data. The method is demonstrated by acquiring a full set of phase-averaged flow statistics in the wake of a compressor stator blade row. The technique allows three components of phase-averaged velocity, six components of phase-averaged deterministic stress, and six components of phase-averaged Reynolds stress to be recovered using a single rotated hot-wire probe.

A comparison of Bayesian and non-linear regression methods for robust estimation of pharmacokinetics in DCE-MRI and how it affects cancer diagnosis

The aim of this work is to compare Bayesian Inference for nonlinear models with commonly used traditional non-linear regression (NR) algorithms for estimating tracer kinetics in Dynamic Contrast Enhanced Magnetic Resonance Imaging (DCE-MRI). The algorithms are compared in terms of accuracy, and reproducibility under different initialization settings. Further it is investigated how a more robust estimation of tracer kinetics affects cancer diagnosis. The derived tracer kinetics from the Bayesian algorithm were validated against traditional NR algorithms (i.e. Levenberg-Marquardt, simplex) in terms of accuracy on a digital DCE phantom and in terms of goodness-of-fit (Kolmogorov-Smirnov test) on ROI-based concentration time courses from two different patient cohorts. The first cohort consisted of 76 men, 20 of whom had significant peripheral zone prostate cancer (any cancer-core-length (CCL) with Gleason>3+3 or any-grade with CCL>=4mm) following transperineal template prostate mapping biopsy. The second cohort consisted of 9 healthy volunteers and 24 patients with head and neck squamous cell carcinoma. The diagnostic ability of the derived tracer kinetics was assessed with receiver operating characteristic area under curve (ROC AUC) analysis. The Bayesian algorithm accurately recovered the ground-truth tracer kinetics for the digital DCE phantom consistently improving the Structural Similarity Index (SSIM) across the 50 different initializations compared to NR. For optimized initialization, Bayesian did not improve significantly the fitting accuracy on both patient cohorts, and it only significantly improved the ve ROC AUC on the HN population from ROC AUC=0.56 for the simplex to ROC AUC=0.76. For both cohorts, the values and the diagnostic ability of tracer kinetic parameters estimated with the Bayesian algorithm weren’t affected by their initialization. To conclude, the Bayesian algorithm led to a more accurate and reproducible quantification of tracer kinetic parameters in DCE-MRI, improving their ROC-AUC and decreasing their dependence on initialization settings.

Highly efficient hierarchical online nonlinear regression using second order methods

We introduce highly efficient online nonlinear regression algorithms that are suitable for real life applications. We process the data in a truly online manner such that no storage is needed, i.e., the data is discarded after being used. For nonlinear modeling we use a hierarchical piecewise linear approach based on the notion of decision trees where the space of the regressor vectors is adaptively partitioned based on the performance. As the first time in the literature, we learn both the piecewise linear partitioning of the regressor space as well as the linear models in each region using highly effective second order methods, i.e., Newton–Raphson Methods. Hence, we avoid the well known over fitting issues by using piecewise linear models, however, since both the region boundaries as well as the linear models in each region are trained using the second order methods, we achieve substantial performance compared to the state of the art. We demonstrate our gains over the well known benchmark data sets and provide performance results in an individual sequence manner guaranteed to hold without any statistical assumptions. Hence, the introduced algorithms address computational complexity issues widely encountered in real life applications while providing superior guaranteed performance in a strong deterministic sense.