Deep Learning Regression Research Articles

Game Data Mining Competition on Churn Prediction and Survival Analysis Using Commercial Game Log Data

Game companies avoid sharing their game data with external researchers. Only a few research groups have been granted limited access to game data so far. The reluctance of these companies to make data publicly available limits the wide use and development of data mining techniques and artificial intelligence research specific to the game industry. In this work, we developed and implemented an international competition on game data mining using commercial game log data from one of the major game companies in South Korea: NCSOFT. Our approach enabled researchers to develop and apply state-of-the-art data mining techniques to game log data by making the data open. For the competition, data were collected from Blade & Soul, an action role-playing game, from NCSOFT. The data comprised approximately 100 GB of game logs from 10,000 players. The main aim of the competition was to predict whether a player would churn and when the player would churn during two periods between which the business model was changed to a free-to-play model from a monthly subscription. The results of the competition revealed that highly ranked competitors used deep learning, tree boosting, and linear regression.

Prediction of Core Shear Strength in Sandwich Composites using Deep Learning and Support Vector Regression

In the present study, machine learning approaches have been developed to predict the 180 days aged core shear strength of sandwich composites. The characteristics of the sandwich composites depends on the number of factors namely fibre type i.e., Chopped strand Mat, Stitched, Chopped strand Mat and Woven Roving, core density, bond between the core and the face sheets and the ability to bear the load in flexural mode. In the current approach deep learning and SVR models were worked out by taking on six different parameters namely foam density, aging temperature and variety of fiber types as input variables. For each set of these input variables, the 180 days aged shear strength of sandwich composites with a test frequency of 30 days was determined. The paper aims at predicting the core shear strength value of stitch bond sandwich composites using other three aforementioned fibers. To create the model and confirm the accuracy of the algorithm training and test data are considered. The results obtained revealed that the deep learning model develo ped provides better predictive ability than the model of SVR.

Statistical Approaches Based on Deep Learning Regression for Verification of Normality of Blood Pressure Estimates.

Oscillometric blood pressure (BP) monitors currently estimate a single point but do not identify variations in response to physiological characteristics. In this paper, to analyze BP’s normality based on oscillometric measurements, we use statistical approaches including kurtosis, skewness, Kolmogorov-Smirnov, and correlation tests. Then, to mitigate uncertainties, we use a deep learning method to determine the confidence limits (CLs) of BP measurements based on their normality. The proposed deep learning regression model decreases the standard deviation of error (SDE) of the mean error and the mean absolute error and reduces the uncertainties of the CLs and SDEs of the proposed technique. We validate the normality of the distribution of the BP estimation which fits the standard normal distribution very well. We use a rank test in the deep learning technique to demonstrate the independence of the artificial systolic BP and diastolic BP estimations. We perform statistical tests to verify the normality of the BP measurements for individual subjects. The proposed methodology provides accurate BP estimations and reduces the uncertainties associated with the CLs and SDEs using the deep learning algorithm.

Intelligent sales volume forecasting using Google search engine data

Business forecasting is a critical organizational capability for both strategic and tactical business planning. Improving the quality of forecasts is thus an important organization goal. In this paper, the intelligent sales volume forecasting models are constructed using grey analysis, deep learning (DNN), and least-square support vector regression (LSSVR) optimized through particle swarm optimization or genetic algorithm. First, features (predictors) from economic variables are extracted through grey analysis. The selected features together with Google Index, an exogenous variable used widely by researchers, are then used as the inputs to the DNN and LSSVR to build the models. The experimental results indicate that the grey DNN model, an emerging and pioneering artificial intelligence technology, can accurately predict sales volumes based on non-parametric statistical tests. DNN also outperformed the competing models when using Google Index.

Research on soil moisture prediction model based on deep learning.

Soil moisture is one of the main factors in agricultural production and hydrological cycles, and its precise prediction is important for the rational use and management of water resources. However, soil moisture involves complex structural characteristics and meteorological factors, and it is difficult to establish an ideal mathematical model for soil moisture prediction. Existing prediction models have problems such as prediction accuracy, generalization, and multi-feature processing capability, and prediction performance must improve. Based on this, taking the Beijing area as the research object, the deep learning regression network (DNNR) with big data fitting capability was proposed to construct a soil moisture prediction model. By integrating the dataset, analyzing the time series of the predictive variables, and clarifying the relationship between features and predictive variables through the Taylor diagram, selected meteorological parameters can provide effective weights for moisture prediction. Test results prove that the deep learning model is feasible and effective for soil moisture prediction. Its’ good data fitting and generalization capability can enrich the input characteristics while ensuring high accuracy in predicting the trends and values of soil moisture data and provides an effective theoretical basis for water-saving irrigation and drought control.

Prediction of premature all-cause mortality: A prospective general population cohort study comparing machine-learning and standard epidemiological approaches.

BackgroundPrognostic modelling using standard methods is well-established, particularly for predicting risk of single diseases. Machine-learning may offer potential to explore outcomes of even greater complexity, such as premature death. This study aimed to develop novel prediction algorithms using machine-learning, in addition to standard survival modelling, to predict premature all-cause mortality.MethodsA prospective population cohort of 502,628 participants aged 40–69 years were recruited to the UK Biobank from 2006–2010 and followed-up until 2016. Participants were assessed on a range of demographic, biometric, clinical and lifestyle factors. Mortality data by ICD-10 were obtained from linkage to Office of National Statistics. Models were developed using deep learning, random forest and Cox regression. Calibration was assessed by comparing observed to predicted risks; and discrimination by area under the ‘receiver operating curve’ (AUC).Findings14,418 deaths (2.9%) occurred over a total follow-up time of 3,508,454 person-years. A simple age and gender Cox model was the least predictive (AUC 0.689, 95% CI 0.681–0.699). A multivariate Cox regression model significantly improved discrimination by 6.2% (AUC 0.751, 95% CI 0.748–0.767). The application of machine-learning algorithms further improved discrimination by 3.2% using random forest (AUC 0.783, 95% CI 0.776–0.791) and 3.9% using deep learning (AUC 0.790, 95% CI 0.783–0.797). These ML algorithms improved discrimination by 9.4% and 10.1% respectively from a simple age and gender Cox regression model. Random forest and deep learning achieved similar levels of discrimination with no significant difference. Machine-learning algorithms were well-calibrated, while Cox regression models consistently over-predicted risk.ConclusionsMachine-learning significantly improved accuracy of prediction of premature all-cause mortality in this middle-aged population, compared to standard methods. This study illustrates the value of machine-learning for risk prediction within a traditional epidemiological study design, and how this approach might be reported to assist scientific verification.

Early Detection of Parkinson’s Disease by Using SPECT Imaging and Biomarkers

Abstract Precise and timely diagnosis of Parkinson’s disease is important to control its progression among subjects. Currently, a neuroimaging technique called dopaminergic imaging that uses single photon emission computed tomography (SPECT) with 123I-Ioflupane is popular among clinicians for detecting Parkinson’s disease in early stages. Unlike other studies, which consider only low-level features like gray matter, white matter, or cerebrospinal fluid, this study explores the non-linear relation between different biomarkers (SPECT + biological) using deep learning and multivariate logistic regression. Striatal binding ratios are obtained using 123I-Ioflupane SPECT scans from four brain regions which are further integrated with five biological biomarkers to increase the diagnostic accuracy. Experimental results indicate that this investigated approach can differentiate subjects with 100% accuracy. The obtained results outperform the ones reported in the literature. Furthermore, logistic regression model has been developed for estimating the Parkinson’s disease onset probability. Such models may aid clinicians in diagnosing this disease.

Cross selling of investment products and services: a case study of leading financial services organisation

We have utilised 360-degree view of the customer information from multiple aspects of financial transaction, product purchase history and account details and provide insights to marketing teams in identifying the potential customers who can be targeted for cross sell of different financial investment products. Advanced analytics algorithms and statistical techniques like deep learning, neural network and logistic regression were applied on the transformed data to predict likelihood of cross sell response different financial product. The analysis of customer data helps to identify the potential customers who can be targeted for cross sell of different investment products and services. The result indicates deep leaning approach provide outstanding performance compare to neural network and logistic regression.

Cross selling of investment products and services: a case study of leading financial services organisation

We have utilised 360-degree view of the customer information from multiple aspects of financial transaction, product purchase history and account details and provide insights to marketing teams in identifying the potential customers who can be targeted for cross sell of different financial investment products. Advanced analytics algorithms and statistical techniques like deep learning, neural network and logistic regression were applied on the transformed data to predict likelihood of cross sell response different financial product. The analysis of customer data helps to identify the potential customers who can be targeted for cross sell of different investment products and services. The result indicates deep leaning approach provide outstanding performance compare to neural network and logistic regression.

Environmental factors as indicators of dissolved oxygen concentration and zooplankton abundance: Deep learning versus traditional regression approach

Presence of optimal levels of dissolved oxygen (above critical level of 4.5 mg L−1) and presence of zooplankton community are indicators of good water quality of an aquatic ecosystem and also of the health of the same. Reservoirs being artificially created water bodies present hybrid systems containing features of both lotic and lentic systems and thus have unique organization that are representative of both rivers and lakes. Since any reservoir is primarily a fresh water system, presence of a large array of zooplankton (diverse community structure) implies its good health and also presence of optimal dissolved oxygen levels supports sustenance of life. In this study, artificial neural network modelling approach has been utilized to predict the level of dissolved oxygen and zooplankton abundance in the Bakreswar reservoir and their variation in relation to the environmental factors. Use of neural network modelling is exceedingly capable of determining correlation among apparently non correlated environmental data and in the current study these are capable of accurately predicting the variations in the levels of dissolved oxygen as well as the abundance of zooplankton. From this study, it has been observed that chemical factors like productivity, nitrates, salinity, pH, phosphates, total dissolved solids, etc. are mostly responsible for control of dissolved oxygen and zooplankton variation at certain points of the study site (stations 1 and 3) whereas at other points (station 2) physical factors like solar radiation, humidity, etc. are more effective. These models are capable of finding the important environmental controllers of such variations and prove to be a powerful alternative to traditional approaches like multiple regression analysis.

Use of Deep Learning for Detailed Severity Characterization and Estimation of 5-Year Risk Among Patients With Age-Related Macular Degeneration

Although deep learning (DL) can identify the intermediate or advanced stages of age-related macular degeneration (AMD) as a binary yes or no, stratified gradings using the more granular Age-Related Eye Disease Study (AREDS) 9-step detailed severity scale for AMD provide more precise estimation of 5-year progression to advanced stages. The AREDS 9-step detailed scale's complexity and implementation solely with highly trained fundus photograph graders potentially hampered its clinical use, warranting development and use of an alternate AREDS simple scale, which although valuable, has less predictive ability. To describe DL techniques for the AREDS 9-step detailed severity scale for AMD to estimate 5-year risk probability with reasonable accuracy. This study used data collected from November 13, 1992, to November 30, 2005, from 4613 study participants of the AREDS data set to develop deep convolutional neural networks that were trained to provide detailed automated AMD grading on several AMD severity classification scales, using a multiclass classification setting. Two AMD severity classification problems using criteria based on 4-step (AMD-1, AMD-2, AMD-3, and AMD-4 from classifications developed for AREDS eligibility criteria) and 9-step (from AREDS detailed severity scale) AMD severity scales were investigated. The performance of these algorithms was compared with a contemporary human grader and against a criterion standard (fundus photograph reading center graders) used at the time of AREDS enrollment and follow-up. Three methods for estimating 5-year risk were developed, including one based on DL regression. Data were analyzed from December 1, 2017, through April 15, 2018. Weighted κ scores and mean unsigned errors for estimating 5-year risk probability of progression to advanced AMD. This study used 67 401 color fundus images from the 4613 study participants. The weighted κ scores were 0.77 for the 4-step and 0.74 for the 9-step AMD severity scales. The overall mean estimation error for the 5-year risk ranged from 3.5% to 5.3%. These findings suggest that DL AMD grading has, for the 4-step classification evaluation, performance comparable with that of humans and achieves promising results for providing AMD detailed severity grading (9-step classification), which normally requires highly trained graders, and for estimating 5-year risk of progression to advanced AMD. Use of DL has the potential to assist physicians in longitudinal care for individualized, detailed risk assessment as well as clinical studies of disease progression during treatment or as public screening or monitoring worldwide.

Automated Assessment of Bone Age Using Deep Learning and Gaussian Process Regression.

Bone age is an essential measure of skeletal maturity in children with growth disorders. It is typically assessed by a trained physician using radiographs of the hand and a reference model. However, it has been described that the reference models leave room for interpretation leading to a large inter-observer and intra-observer variation. In this work, we explore a novel method for automated bone age assessment to assist physicians with their estimation. It consists of a powerful combination of deep learning and Gaussian process regression. Using this combination, sensitivity of the deep learning model to rotations and flips of the input images can be exploited to increase overall predictive performance compared to only using the deep learning network. We validate our approach retrospectively on a set of 12611 radiographs of patients between 0 and 19 years of age.

Deep learning for biomarker regression: application to osteoporosis and emphysema on chest CT scans.

Biomarker computation using deep-learning often relies on a two-step process, where the deep learning algorithm segments the region of interest and then the biomarker is measured. We propose an alternative paradigm, where the biomarker is estimated directly using a regression network. We showcase this image-to-biomarker paradigm using two biomarkers: the estimation of bone mineral density (BMD) and the estimation of lung percentage of emphysema from CT scans. We use a large database of 9,925 CT scans to train, validate and test the network for which reference standard BMD and percentage emphysema have been already computed. First, the 3D dataset is reduced to a set of canonical 2D slices where the organ of interest is visible (either spine for BMD or lungs for emphysema). This data reduction is performed using an automatic object detector. Second, The regression neural network is composed of three convolutional layers, followed by a fully connected and an output layer. The network is optimized using a momentum optimizer with an exponential decay rate, using the root mean squared error as cost function. The Pearson correlation coefficients obtained against the reference standards are r = 0.940 (p < 0.00001) and r = 0.976 (p < 0.00001) for BMD and percentage emphysema respectively. The deep-learning regression architecture can learn biomarkers from images directly, without indicating the structures of interest. This approach simplifies the development of biomarker extraction algorithms. The proposed data reduction based on object detectors conveys enough information to compute the biomarkers of interest.

Learning Latent Representations of 3D Human Pose with Deep Neural Networks

Most recent approaches to monocular 3D pose estimation rely on Deep Learning. They either train a Convolutional Neural Network to directly regress from an image to a 3D pose, which ignores the dependencies between human joints, or model these dependencies via a max-margin structured learning framework, which involves a high computational cost at inference time. In this paper, we introduce a Deep Learning regression architecture for structured prediction of 3D human pose from monocular images or 2D joint location heatmaps that relies on an overcomplete autoencoder to learn a high-dimensional latent pose representation and accounts for joint dependencies. We further propose an efficient Long Short-Term Memory network to enforce temporal consistency on 3D pose predictions. We demonstrate that our approach achieves state-of-the-art performance both in terms of structure preservation and prediction accuracy on standard 3D human pose estimation benchmarks.

On the Relevance of the Loss Function in the Agatston Score Regression from Non-ECG Gated CT Scans.

In this work, we evaluate the relevance of the choice of loss function in the regression of the Agatston score from 3D heart volumes obtained from non-contrast non-ECG gated chest computed tomography scans. The Agatston score is a well-established metric of cardiovascular disease, where an index of coronary artery disease (CAD) is computed by segmenting the calcifications of the arteries and multiplying each calcification by a factor related to their intensity and their volume, creating a final aggregated index. Recent work has automated such task with deep learning techniques, even skipping the segmentation step and performing a direct regression of the Agatston score. We study the effect of the choice of the loss function in such methodologies. We use a large database of 6983 CT scans to which the Agatston score has been manually computed. The dataset is split into a training set and a validation set of n = 1000. We train a deep learning regression network using such data with different loss functions while keeping the structure of the network and training parameters constant. Pearson correlation coefficient ranges from 0.902 to 0.938 depending on the loss function. Correct risk group assignment measurements range between 59.5% and 81.7%. There is a trade-off between the accuracy of the Pearson correlation coefficient and the risk group measurement, which leads to optimize for one or the other.

Bearing Fault Diagnosis Using Convolution Neural Network and Support Vector Regression

Rolling bearing is one of the most widely used parts in machinery. The vibration signals measured from a rolling bearing can reflect the conditions of the bearing, so the vibration signal is often used in the field of bearing fault diagnosis. A number of diagnostic techniques have been studied based on the vibration signals. Deep learning theory is usually used in image recognition or speech processing, and it has attracted more and more attentions in fault diagnosis. In this paper, a method based on deep learning and support vector regression is proposed. The convolutional neural network which is seldom used for one dimension signal is used for promoting feature extraction capability. Besides, the support vector regression (SVR) is an upgrading of the support vector machine which has great generalization ability and can be improved for classification by regression theory. The proposed method is achieved through a hybrid model which combines these two techniques. The presented method is used to classify the bearing fault patterns and obtains better results than that of the convolutional neural network and support vector regression machine separately.

Hypertension

The field of hypertension has enjoyed some remarkable successes in recent years. Better blood pressure management has been an important contributor to the recent rapid decline in the age-adjusted death rates for coronary heart disease (CHD) and stroke.1 Yet, there is yet so much more that could be accomplished through better blood pressure management. CHD and stroke remain the first and third leading causes of death in the United States.2 And only slightly more than a third of those with hypertension in the United States achieve recommended treatment goal blood pressure levels.3 Control rates are even worse in most other countries.4 Better blood pressure management can save many lives. Here, some pathways to success are considered. Few measurements in medicine are done as poorly and inconsistently as blood pressure measurement. Methods used today in clinical practice and in clinical trials are little changed from the earliest days of measurement.5 Though there is a clear recognition of biological variability, we continue to make decisions largely on measurements taken at random times under poorly controlled conditions. The measurement process is less regulated than measurement of most other major risk factors. Just compare the quality control process for glucose or cholesterol measurement with that for blood pressure measurement in your own local setting. There are not readily available solutions for this issue. Substantial investments in research in this area should be made by government and commercial entities. We must have reliable, reproducible, convenient, and precise measurement tools for blood pressure. These tools should allow for measurement of the important components of blood pressure and mechanisms for integrating and interpreting influence of time of day, activity, …