Instance-based visualization and analysis of neural networks

Differences in electroencephalographic (EEG) power spectra obtained under similar, but not identical, conditions may be difficult to discern using standard techniques. Statistical analysis may not be useful because of the large number of comparisons necessary. Visual recognition of differences also may be difficult. A new technique, neural network analysis, has been used successfully in other problems of pattern recognition and classification. We examined a number of methods of classifying similar EEG data: standard statistical analysis (analysis of variance), visual recognition, discriminant analysis, and neural network analysis. Twenty-nine volunteers received either thiopental (n = 9), midazolam (n = 10), or propofol (n = 10) in sedative doses in 3 different studies. These drugs produced very similar changes in the EEG power spectra. Except for beta 2 power during thiopental infusion, differences between drugs could not be detected using analysis of variance. Visual categorization was correct in 72% of the baseline EEGs, 70% of thiopental EEGs, 27% of propofol EEGs, and 46% of midazolam EEGs. A classification neural network (Learning Vector Quantization network) containing a Kohonen hidden layer was able to successfully classify 57 of 58 EEG samples (of 4 minutes' duration). Discriminant analysis had a similar rate of success. This level of performance was achieved by dividing the EEG power spectrum from 1 to 30 Hz into 15 2-Hz bandwidths. When the EEG power spectrum was divided into the "classical" frequency bandwidths (alpha, beta 1, beta 2, theta, delta), both neural network and discriminant analysis performance deteriorated. By training the network using only certain inputs we were able to identify drug-specific bandwidths that seemed to be important in correct classification. We conclude that propofol, thiopental, and midazolam produce different effects on the EEG and that both neural network and discriminant analysis are useful in identifying these differences. We also conclude that EEG spectra should be analyzed without using classical EEG bands (alpha, beta, etc.). Additionally, neural networks can be used to identify frequency bands that are "important" in specific drug effects on the EEG. Once a classification algorithm is obtained using either a neural network or discriminant analysis, it could be used as an on-line monitor to recognize drug-specific EEG patterns.

Connectionist models such as neural networks are alternatives to linear, parametric statistical methods. Neural networks are computer-based pattern recognition methods with loose similarities with the nervous system. Individual variables of the network, usually called 'neurones', can receive inhibitory and excitatory inputs from other neurones. The networks can define relationships among input data that are not apparent when using other approaches, and they can use these relationships to improve accuracy. Thus, neural nets have substantial power to recognize patterns even in complex datasets. Neural network methodology has outperformed classical statistical methods in cases where the input variables are interrelated. Because clinical measurements usually derive from multiple interrelated systems it is evident that neural networks might be more accurate than classical methods in multivariate analysis of clinical data. This paper reviews the use of neural networks in medical decision support. A short introduction to the basics of neural networks is given, and some practical issues in applying the networks are highlighted. The current use of neural networks in image analysis, signal processing and laboratory medicine is reviewed. It is concluded that neural networks have an important role in image analysis and in signal processing. However, further studies are needed to determine the value of neural networks in the analysis of laboratory data.

Effects of bounded and unbounded leakage time-varying delays in memristor-based recurrent neural networks with different memductance functions

Objectives. To build neural network models of time series (LSTM, GRU, RNN) and compare the results of forecasting with their mutual help and the results of standard models (ARIMA, ETS), in order to ascertain in which cases a certain group of models should be used.Methods. The paper provides a review of neural network models and considers the structure of RNN, LSTM, and GRU models. They are used for modeling time series in Russian macroeconomic statistics. The quality of model adjustment to the data and the quality of forecasts are compared experimentally. Neural network and standard models can be used both for the entire series and for its parts (trend and seasonality). When building a forecast for several time intervals in the future, two approaches are considered: building a forecast for the entire interval at once, and step-by-step forecasting. In this way there are several combinations of models that can be used for forecasting. These approaches are analyzed in the computational experiment.Results. Several experiments have been conducted in which standard (ARIMA, ETS, LOESS) and neural network models (LSTM, GRU, RNN) are built and compared in terms of proximity of the forecast to the series data in the test period.Conclusions. In the case of seasonal time series, models based on neural networks surpassed the standard ARIMA and ETS models in terms of forecast accuracy for the test period. The single-step forecast is computationally less efficient than the integral forecast for the entire target period. However, it is not possible to accurately indicate which approach is the best in terms of quality for a given series. Combined models (neural networks for trend, ARIMA for seasonality) almost always give good results. When forecasting a non-seasonal heteroskedastic series of share price, the standard approaches (LOESS method and ETS model) showed the best results.

Determination of soaked california bearing ratio (CBR) and compaction characteristics of soils in the laboratory require considerable time and effort. To make a preliminary assessment of the suitability of soils required for a project, prediction models for these engineering properties on the basis of laboratory tests—which are quick to perform, less time consuming and cheap—such as the tests for index properties of soils, are preferable. Nevertheless researchers hold divergent views regarding the most influential parameters to be taken into account for prediction of soaked CBR and compaction characteristics of fine-grained soils. This could be due to the complex behaviour of soils—which, by their very nature, exhibit extreme variability. However this disagreement is a matter of concern as it affects the dependability of prediction models. This study therefore analyses the ability of artificial neural networks and multiple regression to handle different influential parameters simultaneously so as to make accurate predictions on soaked CBR and compaction characteristics of fine-grained soils. The results of simple regression analyses included in this study indicate that optimum moisture content (OMC) and maximum dry density (MDD) of fine-grained soils bear better correlation with soaked CBR of fine-grained soils than plastic limit and liquid limit. Simple regression analyses also indicate that plastic limit has stronger correlation with compaction characteristics of fine-grained soils than liquid limit. On the basis of these correlations obtained using simple regression analyses, neural network prediction models and multiple regression prediction models—with varying number of input parameters are developed. The results reveal that neural network models have more ability to utilize relatively less influential parameters than multiple regression models. The study establishes that in the case of neural network models, the relatively less powerful parameters—liquid limit and plastic limit can also be used effectively along with MDD and OMC for better prediction of soaked CBR of fine-grained soils. Also with the inclusion of less significant parameter—liquid limit along with plastic limit the predictions on compaction characteristics of fine-grained soils using neural network analysis improves considerably. Thus in the case of neural network analysis, the use of relatively less influential input parameters along with stronger parameters is definitely beneficial, unlike conventional statistical methods—for which, the consequence of this approach is unpredictable—giving sometimes not so favourable results. Very weak input parameters alone need to be avoided for neural network analysis. Consequently, when there is ambiguity regarding the most influential input parameters, neural network analysis is quite useful as all such influential parameters can be taken to consideration simultaneously, which will only improve the performance of neural network models. As soils by their very nature, exhibit extreme complexity, it is necessary to include maximum number of influential parameters—as can be determined easily using simple laboratory tests—in the prediction models for soil properties, so as to improve the reliability of these models—for which, use of neural networks is more desirable.

Over the last 10 years, neural networks have been increasingly applied to various areas of finance. Neural networks are more often applied on the assets side than on the liabilities side of the balance sheet. Some major characteristics of the areas of these applications are their data intensity, unstructured nature, high degree of uncertainty, and hidden relationships. Most of the applications use the backpropagation model with one hidden layer. In most of these applications, neural networks out-performed traditional statistical models, such as discriminant and regression analysis. Furthermore, these applications have shown significant success in financial practice, for example, in forecasting T-bills, in asset management, in portfolio selection, and in fraud detection.

Exponential stability analysis of memristor-based recurrent neural networks with time-varying delays

A rapid and intact method has been developed for predicting polyethylene density by near-infrared spectroscopy combined with neural network analysis. Near-infrared spectra in the region of 1.1-2.2 μm wavelength were measured using pellets or powders of twenty-three kinds of polyethylene (PE) with different densities (0.898-0.962 g cm-3). The spectra were used for training a back-propagation neural network after normalized and second-derivative treatments to predict PE density. Although only a small number of spectral data were used for training, a leave-one-out test of neural network analysis has demonstrated good results. In comparison, principal component regression (PCR) analysis and partial least-squares (PLS) regression analysis were applied. The correlation coefficients (R) were calculated to be 1.000, 0.968 and 0.983 for neural network, PCR and PLS analysis, respectively. The root mean square errors of prediction were found to be 0.00026, 0.0043 and 0.0031 g cm-3, respectively. It is found that near-infrared spectroscopy combined with neural network analysis is useful for the efficient and accurate determination of PE density.

More than a year has passed since the coronavirus 2 (SARS-CoV2) pandemic began, and no one has been able to forecast the infection cases of the disease with high accuracy. Nowadays, a lot of studies are devoted to finding out the pattern of infection spreading and forecasting cases of infection. But models used in those studies have large errors in forecasts, and this makes it more challenging to discover the pattern of infection spreading. The choice of appropriate models may vary from country to country. The increase of the number of infection cases is one of the global challenges nowadays not only for virusologists and medicians, but also for data analytics. In our paper, we analyze errors in the forecast in the list of the top 10 countries affected by disease on January 1, 2021 using the following the neural network models, linear and non-linear classical statistical models, and also classical epidemiological SIR model. The other part of our computational experiment is devoted to forecasting the dates of the peak values of time series. It is shown on time series for the regions most affected by pandemics that neural network models allow to forecast this date with high accuracy. We also discover the possible field of application of such algorithms in other fields.

Neural network (NN) models have been widely used in the literature for short-term load forecasting. Their popularity is mainly due to their excellent learning and approximation capability. However, their forecasting performance significantly depends on several factors including initializing parameters, training algorithm, and NN structure. To minimize negative effects of these factors, this paper proposes a practically simple, yet effective and an efficient method to combine forecasts generated by NN models. The proposed method includes three main phases: (i) training NNs with different structures, (ii) selecting best NN models based on their forecasting performance for a validation set, and (iii) combination of forecasts for selected best NNs. Forecast combination is performed through calculating the mean of forecasts generated by best NN models. The performance of the proposed method is examined using real world data set. Comparative studies demonstrate that the accuracy of combined forecasts is significantly superior to those obtained from individual NN models.

Background. Rheumatoid arthritis (RA) is characterized by synovial membrane inflammation that results in joint damage. Many earlier studies have measured cytokines for a better diagnosis of RA. In the present study, three bone biomarkers [osteopontin, stromelysin-1 (MMP3), and vascular endothelial growth factor-A (VEGF)] are examined for their ability to estimate the severity of disease by using artificial neural network (NN) analysis and binary logistic regression analysis. Methods. The study enrolled 87 RA patients and 44 healthy control subjects. The biomarkers were measured by the enzyme-linked immunosorbent assay technique. Disease Activity Score (28 joints) and C-reactive protein (CRP) (DAS28-CRP) was calculated by using DAS28-CRP calculator. The patients with DAS28-CRP 5.1 are considered as having high disease activity (HDA). While patients group with DAS28-CRP 5.1 are considered as moderate disease activity (MDA). The neural network (NN) analysis was used for the differentiation between groups. Results. Results showed that the most sensitive predictor for high disease activity (HDA) of RA is MMP3, followed by osteopontin and VEGF. These three biomarkers can differentiate significantly between HDA and MDA with a relatively high size effect (Partial 2 = 0.323, p 0.001). The HDA group has a significantly higher MMP3, CRP, RF, and anti-citrullinated protein antibodies (ACPA) than the MDA group. MMP3 is strongly associated with two inflammatory indicators; CRP and ESR. Conclusion. There was a significant elevation in the serum level of MMP3 in RA patients with HDA compared to the MDA and control groups. High DAS28, RF, CRP, and ACPA were found in HDA patients compared with the MDA group. The use of the NN analysis indicated that the measured biomarkers help predict the HDA state in RA patients. MMP3 and osteopontin are diagnostic biomarkers for the severity of RA and are related to many disease-related characteristics with a sensitivity of 88.9% and specificity of 68.4%.

In this report we present our findings and outcomes of the NNRDS (analysis of Neural Networks as Random Dynamical Systems) project. The work is largely motivated by the analogy of a large class of neural networks (NNs) with a discretized ordinary differential equation (ODE) schemes. Namely, residual NNs, or ResNets, can be viewed as a discretization of neural ODEs (NODEs) where the NN depth plays the role of the time evolution. We employ several legacy tools from ODE theory, such as stiffness, nonlocality, autonomicity, to enable regularization of ResNets thus improving their generalization capabilities. Furthermore, armed with NN analysis tools borrowed from the ODE theory, we are able to efficiently augment NN predictions with uncertainty overcoming wellknown dimensionality challenges and adding a degree of trust towards NN predictions. Finally, we have developed a Python library QUiNN (Quantification of Uncertainties in Neural Networks) that incorporates improved-architecture ResNets, besides classical feed-forward NNs, and contains wrappers to PyTorch NN models enabling several major classes of uncertainty quantification methods for NNs. Besides synthetic problems, we demonstrate the methods on datasets from climate modeling and materials science.

This study aims to incorporate Artificial Neural Networks into a Marketing Decision Support System (MDSS), specifically, by discovering important variables that influence sales performance of colour television (CTV) sets in the Singapore market using neural networks. Three kinds of variables, expert knowledge, marketing information and environmental data, are examined. The information about the effects of each of these variables has been studied and made available for decision making. However, their combined effect is unknown. This study attempts to explore the combined effect for the benefit of our collaborator, a multinational corporation (MNC) in the consumer electronics industry in Singapore. Putting these three variables together as input variables results in a neural network model. Neural network training is conducted using historical data on CTV sales in Singapore collected over the past one and a half years. Sensitivity analysis is then performed to reduce input variables of neural networks. This is done by analyzing the weights of the input node connections in the trained neural networks using two different methods. The weaker variables can be excluded, and this results in a simpler model. Further, an R-Square value of almost 1 is obtained through the inclusion of an Unknown variable when the network model consisting only of the most influential variables is trained and tested. Knowing the most influential variables, which in this case include Average Price, Screen Size, Stereo Systems, Flat-Square screen type and Seasonal Factors, marketing managers can improve sales performance by paying more attention to them.

The authors investigate the existence and the global stability of periodic solution for dynamical systems with periodic interconnections, inputs and self-inhibitions. The model is very general, the conditions are quite weak and the results obtained are universal.

The modulus of subgrade reaction test is also known as \({k}_{s}\) value test, and it is essentially a plate bearing test. This test is generally used in the design of rigid pavements and raft foundations. In the present research work, the modulus of subgrade reaction has been predicted by an artificial neural network and principle component analysis. For the prediction of \({k}_{s}\) value, the neural network (NN) models of the different number of hidden layers and nodes have been developed in MATLAB R2016b. The range of the number of hidden layers has been selected from one to five, and for each hidden layer, the range of the number of nodes has been selected from two to eleven. Based on the training and validation performance of NN models, the best architectural neural network model is selected. In the present work, the neural network model of two hidden layers with eleven nodes on each hidden layer has been selected as the best architectural neural network. The best architectural neural network has been compared with principle component analysis. From the comparison, it has been concluded that the principle component analysis has predicted modulus of subgrade (\({k}_{s}\) value) with 86.4% accuracy, which is ≈1.11 (1.105) times less than the accuracy of the neural network model.KeywordsArtificial neural networkModulus of subgradePrinciple component analysisRigid pavement