Neural Network-based Regression Research Articles

Vascular Aging Estimation Based on Artificial Neural Network Using Photoplethysmogram Waveform Decomposition: Retrospective Cohort Study.

BackgroundFor the noninvasive assessment of arterial stiffness, a well-known indicator of arterial aging, various features based on the photoplethysmogram and regression methods have been proposed. However, whether because of the existing characteristics not accurately reflecting the characteristics of the incident and reflected waveforms of the photoplethysmogram or because of the lack of expressive power of the regression model, a reliable arterial stiffness assessment technique based on a single photoplethysmogram has not yet been proposed.ObjectiveThe purpose of this study is to discover highly correlated features from the incident and reflected waves decomposed from a photoplethysmogram waveform and to develop an artificial neural network-based regression model for the assessment of vascular aging using newly derived features.MethodsWe obtained photoplethysmograms from 757 participants. All recorded photoplethysmograms were segmented for each beat, and each waveform was decomposed into incident and reflected waves by the Gaussian mixture model. The 26 basic features and 52 combined features were defined from the morphological characteristics of the incident and reflected waves. The regression model of the artificial neural network was developed using the defined features.ResultsIn correlation analysis, the features from the amplitude of the reflected wave and the skewness of the photoplethysmogram showed a relatively strong correlation with the participant’s real age. In the estimation of real age, the artificial neural network model showed 10.0 years of root mean square error. Its estimated age and real age had a strong correlation of 0.63 (P<.001).ConclusionsThis study proved that the features defined from the reflected wave and skewness of the photoplethysmogram are useful to assess vascular aging. Moreover, the regression model of artificial neural network using these features shows the feasibility for the estimation of vascular aging.

A generalized machine learning workflow to visualize mechanical discontinuity

Accurate detection and mapping of mechanical discontinuity in materials has widespread industrial and research applications. We developed a generalized machine-learning framework for visualizing single mechanical discontinuity embedded in material of any composition, velocity, density, porosity, and size with limited data. The proposed visualization of discontinuity requires accurate estimations of the length, location, and orientation of the embedded discontinuity by processing multipoint wave-transmission measurements. k-Wave simulator is used to create a large dataset of elastic waveforms recorded during multi-point wave-transmission measurements through materials containing single mechanical discontinuity. k-Wave simulator considers the wave attenuation, dispersion, and mode conversion in wave motion. Discrete wavelet transform (DWT) and statistical feature extraction are essential for data preprocessing prior to the data-driven model development. DWT also minimizes the effect of noise. Using hyper-parameter tuning and cross validation, gradient boosting regression can visualize the mechanical discontinuity with an accuracy of 0.85, in terms of coefficient of determination. A double-layered neural network-based regression has better performance with an accuracy of 0.95. Use of convolutional neural network converts the predictive task from a waveform processing to an image processing problem. Convolutional neural network achieved a generalization performance of 0.91. The proposed generalized workflow requires robust simulation of wave propagation, signal processing, feature engineering, and model evaluation. Sensors closest to the source and those located opposite the source are the most significant for the desired visualization. Notably, the sensors closest to the source capture the non-linear associations, whereas the sensor on the border opposite to the source capture the linear associations between the measured waveforms and the properties of the mechanical discontinuity.

Self-attention-based Multi-block regression fusion Neural Network for quality-related process monitoring

BackgroundFor plant-wide process with multiple operation units, local-global modeling is an efficient method to achieve quality-related fault detection. However, most of algorithms based on local-global modeling ignore the correlation between sub-blocks. This will result in poor performance of the extracted global quality-related features. MethodsThis paper focus on the correlation between sub-blocks and proposes Self-attention-based Multi-block regression fusion Neural Network (SMNN) to achieve efficient quality-related fault detection for nonlinear multi-unit process. Firstly, to focus on quality-related information, the key variables are selected. Then, to extract quality-related features in each sub-block, a pre-training approach is used, i.e. a deep neural network-based regression network between process variables and quality variables is constructed in each sub-block. Secondly, considering the correlation between the sub-blocks, self-attention mechanism is used to integrate the quality-related feature from each block. With the help of an additional regression network, the quality-related features of sub-blocks are fine-tuned and the global features are extracted. Finally, quality-related statistic is constructed to detect faults. FindingsThe proposed method shows good performance in Tennessee-Eastman process, which demonstrates the effectiveness of the method. It also shows that considering the potential relationships between sub-blocks during model construction helps in the extraction of global features.

The statistical neural network-based regression approach for prediction of optical band gap of CuO

The statistical neural network-based regression approach for prediction of optical band gap of CuO

Optimising longitudinal and lateral calorimeter granularity for software compensation in hadronic showers using deep neural networks

We investigate the effect of longitudinal and transverse calorimeter segmentation on event-by-event software compensation for hadronic showers. To factorize out sampling and detector effects, events are simulated in which a single charged pion is shot at a homogenous lead glass calorimeter, split into longitudinal and transverse segments of varying size, and the total energy loss within each segment is used as the signal. As an approximation of an optimal reconstruction, a neural network-based energy regression is trained based on these signals. The architecture is based on blocks of convolutional kernels customized for shower energy regression using local energy densities; biases at the edges of the training dataset are mitigated using a histogram technique. With this approximation, we find that a longitudinal and transverse segment size less than or equal to 0.5 and 1.3 nuclear interaction lengths, respectively, is necessary to achieve an optimal energy measurement. In addition, an intrinsic energy resolution of 8%/sqrt{E} for pion showers is observed.

Deviation From Model of Normal Aging in Alzheimer’s Disease: Application of Deep Learning to Structural MRI Data and Cognitive Tests

Background. Psychophysiological and cognitive tests as well as other functional studies can detect pre-symptomatic stages of dementia. When assembled with structural data, cognitive tests diagnose NDs more reliably thus becoming a multimodal diagnostic tool. Objective. Our main goal is to improve screening for dementia by studying an association between the brain structure and its function. Hypothetically, the brain structure-function association has features specific for either diseaserelated cognitive deterioration or normal neurocognitive slowing while aging. Materials and methods. We studied a total number of 287 cognitively normal cases, 646 of mild cognitive impairment, and 369 of Alzheimer’s disease. To work out a new marker of neurodegeneration, we created a convolutional neural network-based regression model and predicted the cognitive status of the cognitively preserved examinee from the brain MRI data. This was a model of normal aging. A big deviation from the model suggests a high risk of accelerated cognitive decline. Results. The deviation from the model of normal aging can accurately distinguish cognitively normal subjects from MCI patients (AUC = 0.9957). We also achieved creditable performance in the MCI-versus-AD classification (AUC = 0.9793). We identified a considerable difference in the MMSE test between A-positive and A-negative demented individuals according to ATN-criteria (6.27±1.82 vs 5.32±1.9; p < 0.05). Conclusion. The deviation from the model of normal aging can be potentially used as a marker of dementia and as a tool for differentiating Alzheimer’s disease from non-Alzheimer’s dementia. To find and justify a reliable threshold levels, further research is required.

A Neural Network-Based Regression Study for a Hybrid Battery Thermal Management System under Fast Charging

A Neural Network-Based Regression Study for a Hybrid Battery Thermal Management System under Fast Charging

A convolutional neural network-based regression model to infer the epigenetic crosstalk responsible for CG methylation patterns

BackgroundEpigenetic modifications, including CG methylation (a major form of DNA methylation) and histone modifications, interact with each other to shape their genomic distribution patterns. However, the entire picture of the epigenetic crosstalk regulating the CG methylation pattern is unknown especially in cells that are available only in a limited number, such as mammalian oocytes. Most machine learning approaches developed so far aim at finding DNA sequences responsible for the CG methylation patterns and were not tailored for studying the epigenetic crosstalk.ResultsWe built a machine learning model named epiNet to predict CG methylation patterns based on other epigenetic features, such as histone modifications, but not DNA sequence. Using epiNet, we identified biologically relevant epigenetic crosstalk between histone H3K36me3, H3K4me3, and CG methylation in mouse oocytes. This model also predicted the altered CG methylation pattern of mutant oocytes having perturbed histone modification, was applicable to cross-species prediction of the CG methylation pattern of human oocytes, and identified the epigenetic crosstalk potentially important in other cell types.ConclusionsOur findings provide insight into the epigenetic crosstalk regulating the CG methylation pattern in mammalian oocytes and other cells. The use of epiNet should help to design or complement biological experiments in epigenetics studies.

A robust low data solution: Dimension prediction of semiconductor nanorods

Precise control over dimension of nanocrystals is critical to tune the properties for various applications. However, the traditional control through experimental optimization is slow, tedious and time consuming. Herein a robust deep neural network-based regression algorithm has been developed for precise prediction of length, width, and aspect ratios of semiconductor nanorods (NRs). Given there is limited experimental data available (28 samples), a Synthetic Minority Oversampling Technique for regression (SMOTE-REG) is employed first for data generation. Deep neural network is further applied to develop regression model which demonstrated the well performed prediction on both the original and generated data with a similar distribution. The prediction model is further validated with additional experimental data, showing accurate prediction results. Additionally, Local Interpretable Model-Agnostic Explanations (LIME) is used to interpret the weight for each sample, corresponding to its importance towards the target dimension, which is well validated by experimental observations.

A Fully Convolutional Neural Network-based Regression Approach for Effective Chemical Composition Analysis Using Near-infrared Spectroscopy in Cloud

As one chemical composition, nicotine content has an important influence on the quality of tobacco leaves. Rapid and non-destructive quantitative analysis of nicotine is an important task in the tobacco industry. Near-infrared (NIR) spectroscopy as an effective chemical-composition analysis technique has been widely used. In this paper, we propose a one-dimensional Fully Convolutional Network (1D-FCN) model to quantitatively analyze the nicotine composition of tobacco leaves using NIRspectroscopy data in a cloud environment. This 1D-FCN model uses one-dimension convolution layers to directly extract the complex features from sequential spectroscopy data. It consists of five convolutional layers and two full connection layers with the max-pooling layer replaced by a convolutional layer to avoid information loss.Cloud computing techniques are used to solve the increasing requests of large-size data analysis and implement data sharing and accessing.Experimental results show that the proposed 1D-FCN model can effectively extract the complex characteristics inside the spectrum and more accurately predict the nicotine volumes in tobacco leaves than other approaches. This research provides a deep learning foundation for quantitative analysis of NIR spectra data in the tobacco industry.

Fast automated detection of COVID-19 from medical images using convolutional neural networks

Coronavirus disease 2019 (COVID-19) is a global pandemic posing significant health risks. The diagnostic test sensitivity of COVID-19 is limited due to irregularities in specimen handling. We propose a deep learning framework that identifies COVID-19 from medical images as an auxiliary testing method to improve diagnostic sensitivity. We use pseudo-coloring methods and a platform for annotating X-ray and computed tomography images to train the convolutional neural network, which achieves a performance similar to that of experts and provides high scores for multiple statistical indices (F1 scores > 96.72% (0.9307, 0.9890) and specificity >99.33% (0.9792, 1.0000)). Heatmaps are used to visualize the salient features extracted by the neural network. The neural network-based regression provides strong correlations between the lesion areas in the images and five clinical indicators, resulting in high accuracy of the classification framework. The proposed method represents a potential computer-aided diagnosis method for COVID-19 in clinical practice.

A Fast Charging–Cooling Coupled Scheduling Method for a Liquid Cooling-Based Thermal Management System for Lithium-Ion Batteries

Efficient fast-charging technology is necessary for the extension of the driving range of electric vehicles. However, lithium-ion cells generate immense heat at high-current charging rates. In order to address this problem, an efficient fast charging–cooling scheduling method is urgently needed. In this study, a liquid cooling-based thermal management system equipped with mini-channels was designed for the fast-charging process of a lithium-ion battery module. A neural network-based regression model was proposed based on 81 sets of experimental data, which consisted of three sub-models and considered three outputs: maximum temperature, temperature standard deviation, and energy consumption. Each sub-model had a desirable testing accuracy (99.353%, 97.332%, and 98.381%) after training. The regression model was employed to predict all three outputs among a full dataset, which combined different charging current rates (0.5C, 1C, 1.5C, 2C, and 2.5C (1C = 5 A)) at three different charging stages, and a range of coolant rates (0.0006, 0.0012, and 0.0018 kg·s −1 ). An optimal charging–cooling schedule was selected from the predicted dataset and was validated by the experiments. The results indicated that the battery module’s state of charge value increased by 0.5 after 15 min, with an energy consumption lower than 0.02 J. The maximum temperature and temperature standard deviation could be controlled within 33.35 and 0.8 °C, respectively. The approach described herein can be used by the electric vehicles industry in real fast-charging conditions. Moreover, optimal fast charging–cooling schedule can be predicted based on the experimental data obtained, that in turn, can significantly improve the efficiency of the charging process design as well as control energy consumption during cooling.

Machine learning surrogates for molecular dynamics simulations of soft materials

Molecular dynamics (MD) simulations accelerated by high-performance computing (HPC) methods are powerful tools to investigate and extract the microscopic mechanisms characterizing the properties of soft materials such as self-assembled nanoparticles, virus capsids, confined electrolytes, and polymeric fluids. In this paper, we extend the idea developed in our earlier work of integrating machine learning (ML) methods with HPC-accelerated MD simulations of soft materials in order to enhance their predictive power and advance their applications for research and educational activities. Parallelized MD simulations of self-assembling ions in nanoconfinement are employed to demonstrate our approach. We find that an artificial neural network-based regression model successfully learns nearly all the interesting features associated with the output ionic density profiles over a broad range of ionic system parameters. The ML model generates predictions that are in excellent agreement with the results from MD simulations. The inference time associated with the ML model is over a factor of 10,000 smaller than the corresponding parallel MD simulation time. Through this demonstration, we introduce a “machine learning surrogate” for MD simulations of soft-matter systems. We develop and deploy a web application on nanoHUB to realize the advantages associated with the ML surrogate. The results demonstrate that the performance of MD simulations can be further enhanced by using ML, enabling rapid and accurate simulation-driven exploration of the soft material design space.

Convolutional neural network-based regression for biomarker estimation in corneal endothelium microscopy images.

The morphometric parameters of the corneal endothelium - cell density (ECD), cell size variation (CV), and hexagonality (HEX) - provide clinically relevant information about the cornea. To estimate these parameters, the endothelium is commonly imaged with a non-contact specular microscope and cell segmentation is performed to these images. In previous work, we have developed several methods that, combined, can perform an automated estimation of the parameters: the inference of the cell edges, the detection of the region of interest (ROI), a post-processing method that combines both images (edges and ROI), and a refinement method that removes false edges. In this work, we first explore the possibility of using a CNN-based regressor to directly infer the parameters from the edge images, simplifying the framework. We use a dataset of 738 images coming from a study related to the implantation of a Baerveldt glaucoma device and a standard clinical care regarding DSAEK corneal transplantation, both from the Rotterdam Eye Hospital and both containing images of unhealthy endotheliums. This large dataset allows us to build a large training set that makes this approach feasible. We achieved a mean absolute percentage error (MAPE) of 4.32% for ECD, 7.07% for CV, and 11.74% for HEX. These results, while promising, do not outperform our previous work. In a second experiment, we explore the use of the CNN-based regressor to improve the post-processing method of our previous approach in order to adapt it to the specifics of each image. Our results showed no clear benefit and proved that our previous post-processing is already highly reliable and robust.

A Deep Learning-Based Automatic Mosquito Sensing and Control System for Urban Mosquito Habitats.

Mosquito control is important as mosquitoes are extremely harmful pests that spread various infectious diseases. In this research, we present the preliminary results of an automated system that detects the presence of mosquitoes via image processing using multiple deep learning networks. The Fully Convolutional Network (FCN) and neural network-based regression demonstrated an accuracy of 84%. Meanwhile, the single image classifier demonstrated an accuracy of only 52%. The overall processing time also decreased from 4.64 to 2.47 s compared to the conventional classifying network. After detection, a larvicide made from toxic protein crystals of the Bacillus thuringiensis serotype israelensis bacteria was injected into static water to stop the proliferation of mosquitoes. This system demonstrates a higher efficiency than hunting adult mosquitos while avoiding damage to other insects.

Propagation Index on Airport Delays

This paper explores the propagation effect of flight delays among airports in the aviation system and proposes a new measure, the propagation index, to effectively analyze the interrelationship among airports in relation to flight delays. This index quantifies the effect of delay propagation by measuring the causality among delay time series. To assess the effectiveness of the proposed index on airport delays, three neural network-based regression models are built. The comparative experiments demonstrate that the propagation index proposed is highly correlated with observed airport delays.

Convolutional Neural Networks for Subjective Face Attributes

Describable visual facial attributes are now commonplace in human biometrics and affective computing, with existing algorithms even reaching a sufficient point of maturity for placement into commercial products. These algorithms model objective facets of facial appearance, such as hair and eye color, expression, and aspects of the geometry of the face. A natural extension, which has not been studied to any great extent thus far, is the ability to model subjective attributes that are assigned to a face based purely on visual judgments. For instance, with just a glance, our first impression of a face may lead us to believe that a person is smart, worthy of our trust, and perhaps even our admiration — regardless of the underlying truth behind such attributes. Psychologists believe that these judgments are based on a variety of factors such as emotional states, personality traits, and other physiognomic cues. But work in this direction leads to an interesting question: how do we create models for problems where there is only measurable behavior? In this paper, we introduce a convolutional neural network-based regression framework that allows us to train predictive models of crowd behavior for social attribute assignment. Over images from the AFLW face database, these models demonstrate strong correlations with human crowd ratings.

Applying Information Measure for Predicting Release Time of Open Source Software

Open Source Software (OSS) is updated regularly to meet the requirements posed by the customers. The source code of OSS undergoes frequent change to diffuse new features and update existing features in the system, providing a user friendly interface. The source code changes for fixing bugs and meeting user end requirements again affects the complexity of the code change and creates bugs in the software which are accountable to the next release of software. In this paper, the complexity of code changes in various Bugzilla open source software releases, from version 2.0 on 19th Sep, 1998, to 5.0.1 on 10th Sep, 2015, bugs in each software version release, and the time of release of each software version are considered, and the data used to predict the next release time. The Shannon entropy measure is used to quantify the code change process in terms of entropy for each software release. Observed code changes are utilized to quantify them into entropy units and are further used to predict the next release time. A neural network-based regression model is used to predict the next release time. The performance is compared with the R measure calculated using the multi linear regression model, and a goodness of fit curve is produced.

Felled trap trees as the traditional method for bark beetle control: Can the trapping performance be increased?

Felled trap trees have been traditionally used to control bark beetles in central Europe. There is, however, little research on the method and on factors affecting trap tree performance. We therefore evaluated the effect of felling date, tree diameter, site shading, and weather conditions on the pattern of trap tree infestation by bark beetles. An experiment was conducted at two localities with contrasting forest damage rates and bark beetle outbreak phases in the Czech Republic during three periods: December to April of 2004–2005, 2005–2006, and 2006–2007. For each of the three periods, eight dates of trap tree felling were equally spaced. On each date and at each locality, 5–7 trees located at the open forest edge were felled (295 trap trees in total). The number of Ips typographus (Linnaeus, 1758) entry holes dm−2 counted in May each year was used as the response variable. Neural network-based regression models were used to identify the variables that most affected trapping performance.The performance of the regression models in terms of Pearson’s correlation between observed and predicted entry hole densities ranged from 0.15 to 0.93. The effect of predictor variables on the recorded entry hole density was much lower in the forest with a culminating bark beetle outbreak than in the forest with outbreak in retrogradation phase. For trap trees felled before winter, trapping performance was positively associated with the length of the period that the trap trees were covered by snow. The trap trees exposed to higher heat sums tended to show lower entry hole density than trap trees exposed to lower heat sums.The results indicate that, in forests with high beetle densities, there are few options for increasing trap tree performance based on selection of specific sites, tree characteristics, or felling dates. The identified sensitivity of trap trees to weather indicates that trees should be felled just before beetles emerge if conditions have been relatively warm and snowless but can be felled far in advance under cooler conditions with regular snow. Finally, the results indicate that foresters should continue to use large-diameter trees located on non-shaded sites as felled trap trees.

MODIS-based vegetation index has sufficient sensitivity to indicate stand-level intra-seasonal climatic stress in oak and beech forests

Variation in photosynthetic activity of trees induced by climatic stress can be effectively evaluated using remote sensing data. Although adverse effects of climate on temperate forests have been subjected to increased scrutiny, the suitability of remote sensing imagery for identification of drought stress in such forests has not been explored fully. The study seeks to evaluate the sensitivity of MODIS-based vegetation index to heat and drought stress in temperate forests, and to explore the differences in stress response of oaks and beech. We identified 8 oak and 13 beech pure and mature stands, each covering between 4 and 13 MODIS pixels. For each pixel, we extracted a time series of MODIS NDVI from 2000 to 2010. We identified all sequences of continuous unseasonal NDVI decline to be used as the response variable indicative of environmental stress. Neural network-based regression modelling was then applied to identify the climatic variables that best explain observed NDVI declines. Tested variables explained 84–97 % of the variation in NDVI, whilst air temperature-related climate extremes were found to be the most influential. Beech showed a linear response to the most influential climatic predictors, while oak responded in a unimodal pattern suggesting a better coping mechanism. MODIS NDVI has proved sufficiently sensitive as a stand-level indicator of climatic stress acting upon temperate broadleaf forests, leading to its potential use in predicting drought stress from meteorological observations and improving parameterisation of forest stress indices.