Nonlinear Input-output Relationship Research Articles

Cognitive Assessment Prediction in Alzheimer's Disease by Multi-Layer Multi-Target Regression.

Accurate and automatic prediction of cognitive assessment from multiple neuroimaging biomarkers is crucial for early detection of Alzheimer's disease. The major challenges arise from the nonlinear relationship between biomarkers and assessment scores and the inter-correlation among them, which have not yet been well addressed. In this paper, we propose multi-layer multi-target regression (MMR) which enables simultaneously modeling intrinsic inter-target correlations and nonlinear input-output relationships in a general compositional framework. Specifically, by kernelized dictionary learning, the MMR can effectively handle highly nonlinear relationship between biomarkers and assessment scores; by robust low-rank linear learning via matrix elastic nets, the MMR can explicitly encode inter-correlations among multiple assessment scores; moreover, the MMR is flexibly and allows to work with non-smooth ℓ2,1-norm loss function, which enables calibration of multiple targets with disparate noise levels for more robust parameter estimation. The MMR can be efficiently solved by an alternating optimization algorithm via gradient descent with guaranteed convergence. The MMR has been evaluated by extensive experiments on the ADNI database with MRI data, and produced high accuracy surpassing previous regression models, which demonstrates its great effectiveness as a new multi-target regression model for clinical multivariate prediction.

Identifying key factors of the seawater intrusion model of Dagu river basin, Jiaozhou Bay

Seawater intrusion is a complex groundwater - seawater interaction process, and it is influenced by many factors from ground surface to underground, from groundwater to seawater. Generally, for seawater intrusion model, some model parameters and boundary conditions are always specified by model users’ personal experiences or literature's reference value. The defective model would damage the groundwater management for controlling and preventing seawater intrusion when making decisions are based on this model. In order to improve the reliability of seawater intrusion model, the influences of model inputs on output should be identified prior at optimizing model inputs.Dagu river basin, Jiaozhou Bay is one of the most serious areas of seawater intrusion in China, and it is chosen as the study area in this study. The seawater intrusion model of Dagu river basin is built based on a general program SEAWAT4. The key influence factors of model output are analyzed by two sensitivity analysis methods, i.e., stepwise regression and mutual entropy. The results demonstrated that the most important influence factors which have largest sensitivities to groundwater Cl- concentration are the precipitation rate and groundwater pumping in agriculture area. In addition, the hydraulic conductivity of zone 1 has a non-negligible influence on seawater intrusion process. Stepwise regression analysis is capable of identifying most important influence factor, and it can’t handle complicated nonlinear input-output relationship. Mutual entropy analysis is reliable for identifying the influence factors for complex seawater intrusion model.

All-optical regenerator of multi-channel signals

One of the main reasons why nonlinear-optical signal processing (regeneration, logic, etc.) has not yet become a practical alternative to electronic processing is that the all-optical elements with nonlinear input–output relationship have remained inherently single-channel devices (just like their electronic counterparts) and, hence, cannot fully utilise the parallel processing potential of optical fibres and amplifiers. The nonlinear input–output transfer function requires strong optical nonlinearity, e.g. self-phase modulation, which, for fundamental reasons, is always accompanied by cross-phase modulation and four-wave mixing. In processing multiple wavelength-division-multiplexing channels, large cross-phase modulation and four-wave mixing crosstalks among the channels destroy signal quality. Here we describe a solution to this problem: an optical signal processor employing a group-delay-managed nonlinear medium where strong self-phase modulation is achieved without such nonlinear crosstalk. We demonstrate, for the first time to our knowledge, simultaneous all-optical regeneration of up to 16 wavelength-division-multiplexing channels by one device. This multi-channel concept can be extended to other nonlinear-optical processing schemes.

Modeling and optimization of furan molding sand system using design of experiments and particle swarm optimization

The present research work is focussed on establishing the complex nonlinear input–output relations of a furan resin-based molding sand system. Further, a set of input parameters, which will result in optimized mold properties, is determined. Grain fineness number, setting time, percentage of resin, and hardener are considered as process variables. Mold properties, such as green compression strength, shear strength, mold hardness, gas evolution, permeability, and collapsibility are treated as the process outputs. Nonlinear input–output relations have been developed and statistical analysis has been carried out by utilizing design of experiments, central composite design. Surface plots are developed to study and analyze the input–output relations. The input parameters that will result in best molding conditions and improve casting quality characteristics are determined by utilizing desirability function approach and multiple particle swarm optimization-based crowding distance (MOPSO-CD) techniques. The optimum value for the process variables namely grain fineness number, furan resin, hardener, and setting time are found to be equal to 55, 1.85, 1.2, and 60, respectively. The quality characteristics of the castings namely yield strength, ultimate tensile strength, hardness, density, and secondary dendrite arm spacing are found to improve by 14.03%, 15.08%, 14.14%, 12%, 2.22%, and 12.24%, respectively for the castings made in optimized molding sand conditions.

An improved variable selection method for support vector regression in NIR spectral modeling

Support vector regression (SVR) has become increasingly popular in analyzing near-infrared spectroscopic data. As an alternative to the conventional partial least squares (PLS) methods, the advantages of SVR include better generalization performance and its robustness against nonlinear input-output relationships. Since most variable selection methods in literature focus on identification of linear input-output relationships, these methods are suboptimal when nonlinear SVR models are constructed. In this paper, we propose a variable selection method using recursive feature elimination with mutual information-based bias correction (RFE-MICBC). The proposed method is able to handle nonlinearity and correlations within the feature space. Through two case studies (one with simulated data and one with diesel NIR spectra), the performance of SVR-RFE-MICBC is compared against variable importance in projection (VIP) based feature selection method for PLS and a probabilistic prediction method for SVR [1]. The results demonstrate that the new method is superior in performance especially when the data set contains a high level of nonlinearities. Lastly, we also show that variable selection is beneficial in improving model efficiency and interpretability. The resulting wavelength bands considered to be important provide valuable guidance for future spectral analysis of diesel fuels.

Side-channel attacks and learning-vector quantization

The security of cryptographic systems is a major concern for cryptosystem designers, even though cryptography algorithms have been improved. Side-channel attacks, by taking advantage of physical vulnerabilities of cryptosystems, aim to gain secret information. Several approaches have been proposed to analyze side-channel information, among which machine learning is known as a promising method. Machine learning in terms of neural networks learns the signature (power consumption and electromagnetic emission) of an instruction, and then recognizes it automatically. In this paper, a novel experimental investigation was conducted on field-programmable gate array (FPGA) implementation of elliptic curve cryptography (ECC), to explore the efficiency of side-channel information characterization based on a learning vector quantization (LVQ) neural network. The main characteristics of LVQ as a multi-class classifier are that it has the ability to learn complex non-linear input-output relationships, use sequential training procedures, and adapt to the data. Experimental results show the performance of multi-class classification based on LVQ as a powerful and promising approach of side-channel data characterization.

Multi-Target Regression via Robust Low-Rank Learning.

Multi-target regression has recently regained great popularity due to its capability of simultaneously learning multiple relevant regression tasks and its wide applications in data mining, computer vision and medical image analysis, while great challenges arise from jointly handling inter-target correlations and input-output relationships. In this paper, we propose Multi-layer Multi-target Regression (MMR) which enables simultaneously modeling intrinsic inter-target correlations and nonlinear input-output relationships in a general framework via robust low-rank learning. Specifically, the MMR can explicitly encode inter-target correlations in a structure matrix by matrix elastic nets (MEN); the MMR can work in conjunction with the kernel trick to effectively disentangle highly complex nonlinear input-output relationships; the MMR can be efficiently solved by a new alternating optimization algorithm with guaranteed convergence. The MMR leverages the strength of kernel methods for nonlinear feature learning and the structural advantage of multi-layer learning architectures for inter-target correlation modeling. More importantly, it offers a new multi-layer learning paradigm for multi-target regression which is endowed with high generality, flexibility and expressive ability. Extensive experimental evaluation on 18 diverse real-world datasets demonstrates that our MMR can achieve consistently high performance and outperforms representative state-of-the-art algorithms, which shows its great effectiveness and generality for multivariate prediction.

Multitarget Sparse Latent Regression.

Multitarget regression has recently generated intensive popularity due to its ability to simultaneously solve multiple regression tasks with improved performance, while great challenges stem from jointly exploring inter-target correlations and input-output relationships. In this paper, we propose multitarget sparse latent regression (MSLR) to simultaneously model intrinsic intertarget correlations and complex nonlinear input-output relationships in one single framework. By deploying a structure matrix, the MSLR accomplishes a latent variable model which is able to explicitly encode intertarget correlations via -norm-based sparse learning; the MSLR naturally admits a representer theorem for kernel extension, which enables it to flexibly handle highly complex nonlinear input-output relationships; the MSLR can be solved efficiently by an alternating optimization algorithm with guaranteed convergence, which ensures efficient multitarget regression. Extensive experimental evaluation on both synthetic data and six greatly diverse real-world data sets shows that the proposed MSLR consistently outperforms the state-of-the-art algorithms, which demonstrates its great effectiveness for multivariate prediction.

Unconventional Nonlinear Input-Output Response in a Luminescent Molecular Switch by Inner Filtering Effects.

Nonlinear input-output relations are at the basis of the regulation of biochemical processes in living organisms and are important for the development of digital logic circuits based on molecules. In this article we show that a linear change of a chemical input can be translated into an exponential change of a luminescence output in a simple fluorescent acid-base switch based on 8-methoxyquinoline. Such unconventional behavior arises from the fact that part of the light emitted by the switch in its basic form is reabsorbed by the acid form, and is made possible by the particular spectroscopic properties of the two forms. Systems of this kind could act as noise filters in analog-to-digital conversion, and as control elements to increase the functional complexity of artificial molecular devices.

Solving a Class of Nonlinear Inverse Problems Using a Feedback Control Approach

Inverse problems have applications in many branches of science and engineering. In this paper we propose a new approach to solving inverse problems which is based on using concepts from feedback control systems to determine the inverse of highly nonlinear, discontinuous, and ill-conditioned input-output relationships. The method uses elements from least squares solutions that are formed within a control loop. The stability and convergence of the inverse solution are established. Several examples demonstrate the applicability of the proposed method.

Model Updating Strategy of the DLR-AIRMOD Test Structure

Considerable progresses have been made in computer-aided engineering for the high fidelity analysis of structures and systems. Traditionally, computer models are calibrated using deterministic procedures. However, different analysts produce different models based on different modelling approximations and assumptions. In addition, identically constructed structures and systems show different characteristic between each other. Hence, model updating needs to take account modelling and test-data variability.Stochastic model updating techniques such as sensitivity approach and Bayesian updating are now recognised as powerful approaches able to deal with unavoidable uncertainty and variability. This paper presents a high fidelity surrogate model that allows to significantly reduce the computational costs associated with the Bayesian model updating technique. A set of Artificial Neural Networks are proposed to replace multi non-linear input-output relationships of finite element (FE) models. An application for updating the model parameters of the FE model of the DRL-AIRMOD structure is presented.

Direct and simultaneous estimation of cardiac four chamber volumes by multioutput sparse regression.

Cardiac four-chamber volume estimation serves as a fundamental and crucial role in clinical quantitative analysis of whole heart functions. It is a challenging task due to the huge complexity of the four chambers including great appearance variations, huge shape deformation and interference between chambers. Direct estimation has recently emerged as an effective and convenient tool for cardiac ventricular volume estimation. However, existing direct estimation methods were specifically developed for one single ventricle, i.e., left ventricle (LV), or bi-ventricles; they can not be directly used for four chamber volume estimation due to the great combinatorial variability and highly complex anatomical interdependency of the four chambers. In this paper, we propose a new, general framework for direct and simultaneous four chamber volume estimation. We have addressed two key issues, i.e., cardiac image representation and simultaneous four chamber volume estimation, which enables accurate and efficient four-chamber volume estimation. We generate compact and discriminative image representations by supervised descriptor learning (SDL) which can remove irrelevant information and extract discriminative features. We propose direct and simultaneous four-chamber volume estimation by the multioutput sparse latent regression (MSLR), which enables jointly modeling nonlinear input-output relationships and capturing four-chamber interdependence. The proposed method is highly generalized, independent of imaging modalities, which provides a general regression framework that can be extensively used for clinical data prediction to achieve automated diagnosis. Experiments on both MR and CT images show that our method achieves high performance with a correlation coefficient of up to 0.921 with ground truth obtained manually by human experts, which is clinically significant and enables more accurate, convenient and comprehensive assessment of cardiac functions.

Efficient neural network modeling of reconfigurable microstrip patch antenna through knowledge‐based three‐step strategy

SummaryArtificial neural network (ANN) provides an efficient modeling technique based on input–output data obtained from an engineering problem. Highly nonlinear and complex relationships can be formed by ANN because of its nonlinear nature and representing knowledge at interconnection weights. If an ANN model is not sufficient in respect to the accuracy and time consumption, the knowledge based on design experience can be embedded into the modeling process. This knowledge reduces the complexity of the nonlinear input–output relationships; therefore, the knowledge embedded ANN can be formed easily compared with the conventional ANN model. Three‐step modeling strategy generates the initial knowledge via the conventional ANN modeling and consists of three sequential and also discrete training processes exploiting the knowledge‐based methods such as prior knowledge input and prior knowledge input with difference. The latter step improves the accuracy of the former step, and three‐step modeling provides more accuracy than conventional ANN modeling. The efficiency of three‐step modeling strategy is demonstrated on two data sets, which are obtained by the reconfigurable microstrip patch antenna design problem. The different number of neurons and ANN structures is handled for the comparison as well. In addition, ANN modeling is formed by MATLAB m‐file through neural network toolbox to reveal the efficiency of knowledge‐based three‐step modeling strategy. Copyright © 2016 John Wiley & Sons, Ltd.

Squeeze casting process modeling by a conventional statistical regression analysis approach

During the casting process, the alloy composition, melt treatment modification, processing method, and process variables change the microstructure, thereby affecting the mechanical properties. The hybrid squeeze casting method has been used to limit casting defects, refine the micro-structure, and enhance the mechanical properties. The process variables influence the mechanical and micro-structure properties during squeeze casting. In the present study, we established nonlinear input–output relationships and explored the physical behavior of this process based on the statistical design of experiments and using the response surface methodology. Experiments were conducted to measure the responses in terms of the density, hardness, and secondary dendrite arm spacing. Two nonlinear regression models, i.e., Box–Behnken design and central composite design, were used to conduct experiments, collect experimental data, identify significant process variables, analyze the collected data, and establish the complex input–output relationships. Surface plots were used to explore the effects of the squeeze pressure, pressure duration, pouring, and die temperature on the measured responses. Analysis of variance tests were conducted to evaluate the statistical suitability of the models developed. Furthermore, the accuracies of the predictions made by the models were investigated based on test cases. We found that both of the nonlinear models were statistically adequate and they provided complete insights into the complex nonlinear input–output relationships. Central composite design performed better for the secondary dendrite arm spacing and hardness responses, whereas its performance was the same as that of Box–Behnken design for the density response. The relationships between the responses (i.e., outputs) were established by generating large volumes of input–output data using the nonlinear regression models. We found that the density, hardness, and secondary dendrite arm spacing responses could be obtained by utilizing the nonlinear regression equations and the same set of process variables. Furthermore, the secondary dendrite arm spacing response could be expressed as third order nonlinear functions of density or hardness (structure to property relationship). The results showed that the secondary dendrite arm spacing had inverse relationships with density and hardness, whereas density and hardness had direct relationships.

On using multivariate polynomial regression model with spectral difference for statistical model-based speech enhancement

In this paper, we propose a statistical model-based speech enhancement technique using a multivariate polynomial regression (MPR) based on spectral difference scheme. In the analyzing step, three principal parameters, the weighting parameter in the decision-directed (DD) method, the long-term smoothing parameter for the noise estimation, and the control parameter of the minimum gain value are estimated as optimal operating points technique by using to the spectral difference under various noise conditions. These optimal operating points, which are specific according to different spectral differences, are estimated based on the composite measure, which is a relevant criterion in terms of speech quality. Thus, we apply the MPR technique by incorporating the spectral differences as independent variables in order to estimate the optimal operating points. The MPR technique offers an effective scheme to represent complex nonlinear input-output relationship between the optimal operating points and spectral differences so that operating points can be determined according to various noise conditions in the off-line step. In the on-line speech enhancement step, different parameters are chosen on a frame-by-frame basis through the regression according to the spectral difference. The performance of the proposed method is evaluated using objective and subjective speech quality measures in various noise environments. Our experimental results show that the proposed algorithm yields better performances than conventional algorithms.

A novel hybrid approach utilizing principal component regression and random forest regression to bridge the period of GPS outages

Global Positioning System (GPS) and Inertial Navigation System (INS) are two salient technologies delivering a vehicle׳s position, velocity and attitude parameters for navigation. The standalone GPS undergoes signal outages (in forest area) and multipath effects (in urban areas), whereas the standalone INS solution accuracy deteriorates with time. To overcome the limitations of standalone GPS and INS, an integrated INS/GPS system is required for continuous, accurate, and reliable navigation solution. This paper proposes a hybrid method of Principal Component Regression (PCR) and Random Forest Regression (RFR) for INS and GPS data fusion.Traditional filter based fusion techniques like Kalman filter (KF), linearized KF, and particle filter (PF) have deficiencies related to sensor error modeling, immunity to noise, and computational load. Artificial Neural Networks (ANN) are introduced to overcome these limitations, but they show poor generalization capability in the presence of noise and suffer from over fitting. Recently, RFR is proposed as an efficient fusion technique due to its flexibility to handle non-linear input–output relationships and overcome the over fitting problems. The limitation with RFR is that it cannot handle both linear and non-linear input–output relationships effectively Hence, the paper introduces PCR, which can model linear input–output relationships. Thus, a hybrid approach of RFR and PCR provides enhanced modeling of input–output relationships leading to increased prediction accuracy and thereby improved integrated INS/GPS system performance. The results are validated through five simulated GPS outages taken on real field test data. The proposed model showed a maximum of 45.06% decrement in positional error when compared to RFR.

Probing the Nonlinearity in Neural Systems Using Cross-frequency Coherence Framework

Neural systems can present various types of nonlinear input-output relationships, such as harmonic, subharmonic, and/or intermodulation coupling. This paper aims to introduce a general framework in frequency domain for detecting and characterizing nonlinear coupling in neural systems, called the cross-frequency coherence framework (CFCF). CFCF is an extension of classic coherence based on higher-order statistics. We demonstrate an application of CFCF for identifying nonlinear interactions in human motion control. Our results indicate that CFCF can effectively characterize nonlinear properties of the afferent sensory pathway. We conclude that CFCF contributes to identifying nonlinear transfer in neural systems.

A new Keynesian triangle Phillips curve

We propose a solution to address the observed negative sign on the marginal cost variable in new Keynesian Phillips curve estimations. Our solution is based on an elaborate specification of the cost function faced by firms and the formulation of a reduced-form production function which is characterised by non-linear input–output relations. The resultant Phillips curve features the standard hybrid expectational term, labour share, output gap, speed-limit effects and supply shock variables. In general, GMM estimations of the model for developed and emerging markets yield a positive and significant coefficient on the labour share and the output gap. We conclude that supply shock variables are essential to the empirical validity of the cost-based Phillips curve.

Efficient design of feedforward network for pattern classification

A feedforward neural network is a computing device whose processing units (the nodes) are distributed in adjacent layers connected through unidirectional links (the weights).Feedforward networks are widely used for pattern recognition. Here two feedforward networks are taken into consideration, Multi Layer Perceptron and Radial Basis Network. while designing these networks problem involves in finding the architecture which is efficient in terms of training time. In this paper different data samples will be presented to RBF and Multi Layer network and the best network selection will be done on the basis of minimum time taken by the network for training. Keywords- Feed forward network, Multi Layer Perceptron Neural Networks, Radial Basis Network, Spread. I. INTRODUCTION Pattern recognition is the study of how machines can observe the environment, learn to explore patterns of interest from their background, and make reasonable decisions about the classes of the patterns. the main properties of neural networks are that they have the ability to learn nonlinear input-output relationships, use sequential training procedures, and adapt themselves to the data. the most commonly used neural network family for pattern classification tasks is the feed-forward network, which includes Multi Layer Perceptron(MLP) and Radial-Basis Function (RBF) networks(9). Neural networks can be viewed as massively parallel computing systems consisting of an extremely large number of simple processors with many interconnections. Neural network models attempt to use some organizational principles (such as learning, generalization, adaptivity, fault tolerance and distributed representation. computation) in a network of weighted directed graphs in which the nodes are artificial neurons and directed edges (with weights) are connections between neuron outputs and neuron inputs. The main characteristics of neural networks are that they have the ability to learn complex nonlinear input-output relationships, use sequential training procedures, and adapt themselves to the data.

Hierarchical multivariate regression-based sensitivity analysis reveals complex parameter interaction patterns in dynamic models

Dynamic models of biological systems often possess complex and multivariate mappings between input parameters and output state variables, posing challenges for comprehensive sensitivity analysis across the biologically relevant parameter space. In particular, more efficient and robust ways to obtain a solid understanding of how the sensitivity to each parameter depends on the values of the other parameters are sorely needed.We report a new methodology for global sensitivity analysis based on Hierarchical Cluster-based Partial Least Squares Regression (HC-PLSR)-based approximations (metamodelling) of the input–output mappings of dynamic models, which we expect to be generic, efficient and robust, even for systems with highly nonlinear input–output relationships. The two-step HC-PLSR metamodelling automatically separates the observations (here corresponding to different combinations of input parameter values) into groups based on the dynamic model behaviour, then analyses each group separately with Partial Least Squares Regression (PLSR). This produces one global regression model comprising all observations, as well as regional regression models within each group, where the regression coefficients can be used as sensitivity measures. Thereby a more accurate description of complex interactions between inputs to the dynamic model can be revealed through analysis of how a certain level of one input parameter affects the model sensitivity to other inputs. We illustrate the usefulness of the HC-PLSR approach on a dynamic model of a mouse heart muscle cell, and demonstrate how it reveals interaction patterns of probable biological significance not easily identifiable by a global regression-based sensitivity analysis alone.Applied for sensitivity analysis of a complex, high-dimensional dynamic model of the mouse heart muscle cell, several interactions between input parameters were identified by the two-step HC-PLSR analysis that could not be detected in the single-step global analysis. Hence, our approach has the potential to reveal new biological insight through the identification of complex parameter interaction patterns. The HC-PLSR metamodel complexity can be adjusted according to the nonlinear complexity of the input–output mapping of the analysed dynamic model through adjustment of the number of regional regression models included. This facilitates sensitivity analysis of dynamic models of varying complexities.