Multi-task Regression Research Articles

Residual neural networks for the prediction of planetary collision outcomes

ABSTRACT Fast and accurate treatment of collisions in the context of modern N-body planet formation simulations remains a challenging task due to inherently complex collision processes. We aim to tackle this problem with machine learning (ML), in particular via residual neural networks. Our model is motivated by the underlying physical processes of the data-generating process and allows for flexible prediction of post-collision states. We demonstrate that our model outperforms commonly used collision handling methods such as perfect inelastic merging and feed-forward neural networks in both prediction accuracy and out-of-distribution generalization. Our model outperforms the current state of the art in 20/24 experiments. We provide a data set that consists of 10164 Smooth Particle Hydrodynamics (SPH) simulations of pairwise planetary collisions. The data set is specifically suited for ML research to improve computational aspects for collision treatment and for studying planetary collisions in general. We formulate the ML task as a multi-task regression problem, allowing simple, yet efficient training of ML models for collision treatment in an end-to-end manner. Our models can be easily integrated into existing N-body frameworks and can be used within our chosen parameter space of initial conditions, i.e. where similar-sized collisions during late-stage terrestrial planet formation typically occur.

A 3D Multi-task Regression and Ordinal Regression Deep Neural Network for Collateral Imaging from Dynamic Susceptibility Contrast-Enhanced MR perfusion in Acute Ischemic Stroke

Background and ObjectiveCerebral collaterals have been identified as one of the primary determinants for treatment options in acute ischemic stroke. Several works have been proposed, but these have not been adopted for a routine clinical usage due to their manual and heuristic nature as well as inconsistency and instability of the assessment. Herein, we present an advanced deep learning-based method that can automatically generate a multiphase collateral imaging (collateral map) derived from dynamic susceptibility contrast-enhanced MR perfusion (DSC-MRP) in an accurate and robust manner. MethodsWe develop a 3D multi-task regression and ordinal regression deep neural network for generating collateral maps from DSC-MRP, which formulates the prediction of collateral maps as both a regression task and an ordinal regression task. For an ordinal regression task, we introduce a spacing-decreasing discretization (SDD) strategy to represent the intensity of the collateral status on a discrete, ordinal scale. We also devise loss functions to achieve effective and efficient multi-task learning. ResultsWe systematically evaluated the performance of the proposed network using DSC-MRP from 802 patients. On average, the proposed network achieved ≥0.900 squared correlation coefficient (R-Squared), ≥0.916 Tanimoto measure (TM), ≥0.0913 structural similarity index measure (SSIM), and ≤0.564 × 10−1 mean absolute error (MAE), outperforming eight competing models that have been recently developed in medical imaging and computer vision. We also found that the proposed network could provide an improved contrast between the low and high intensity regions in the collateral maps, which is a key to an accurate evaluation of the collateral status. ConclusionsThe experimental results demonstrate that the proposed network is able to generate collateral maps with high accuracy, facilitating a timely and prompt assessment of the collateral status in clinlcs. The future study will entail the optimization of the proposed network and its clinical evalution in a prospective manner.

Multi-Task Weakly-Supervised Attention Network for Dementia Status Estimation With Structural MRI

Accurate prediction of clinical scores (of neuropsychological tests) based on noninvasive structural magnetic resonance imaging (MRI) helps understand the pathological stage of dementia (e.g., Alzheimer's disease (AD)) and forecast its progression. Existing machine/deep learning approaches typically preselect dementia-sensitive brain locations for MRI feature extraction and model construction, potentially leading to undesired heterogeneity between different stages and degraded prediction performance. Besides, these methods usually rely on prior anatomical knowledge (e.g., brain atlas) and time-consuming nonlinear registration for the preselection of brain locations, thereby ignoring individual-specific structural changes during dementia progression because all subjects share the same preselected brain regions. In this article, we propose a multi-task weakly-supervised attention network (MWAN) for the joint regression of multiple clinical scores from baseline MRI scans. Three sequential components are included in MWAN: 1) a backbone fully convolutional network for extracting MRI features; 2) a weakly supervised dementia attention block for automatically identifying subject-specific discriminative brain locations; and 3) an attention-aware multitask regression block for jointly predicting multiple clinical scores. The proposed MWAN is an end-to-end and fully trainable deep learning model in which dementia-aware holistic feature learning and multitask regression model construction are integrated into a unified framework. Our MWAN method was evaluated on two public AD data sets for estimating clinical scores of mini-mental state examination (MMSE), clinical dementia rating sum of boxes (CDRSB), and AD assessment scale cognitive subscale (ADAS-Cog). Quantitative experimental results demonstrate that our method produces superior regression performance compared with state-of-the-art methods. Importantly, qualitative results indicate that the dementia-sensitive brain locations automatically identified by our MWAN method well retain individual specificities and are biologically meaningful.

Spatio-temporal Tensor Multi-Task Learning for Predicting Alzheimer's Disease in a Longitudinal study.

The utilisation of machine learning techniques to predict Alzheimer's Disease (AD) progression will substantially assist researchers and clinicians in establishing effective AD prevention and treatment strategies. In this research, we present a novel Multi-Task Learning (MTL) model for modelling AD progression based on tensor formation from spatio-temporal similarity measures of brain biomarkers. In this model, each patient sample's prediction in the tensor is assigned to a task, with each task sharing a set of latent factors acquired via tensor decomposition. To further improve the performance of the model, we present a novel regularisation term which utilises the convex combination of disease progression to modify longitudinal stability and ensure that two regression models have a minimal variation at successive time points. The model can be utilised to effectively predict AD progression with magnetic resonance imaging (MRI) data and cognitive scores of AD patients at various stages. We conducted extensive experiments to evaluate the performance for the proposed model and algorithm utilising data from the Alzheimer's Disease Neuroimaging Initiative (ADNI). Compared to single-task and state-of-the-art multi-task regression techniques, our proposed method has greater accuracy and stability for predicting AD progress in terms of root mean square error, with an average reduction of 2.60 compared to single-task regression methods and 1.17 compared to multi-task regression methods in the Mini-Mental State Examination (MMSE) questionnaire; with an average reduction of 5.08 compared to single-task regression methods and 2.71 compared to multi-task regression methods in the Alzheimer's Disease Assessment Scale-Cognitive subscale (ADAS-Cog).

Scalable multi-task Gaussian processes with neural embedding of coregionalization

Multi-task regression attempts to exploit the task similarity in order to achieve knowledge transfer across related tasks for improving the quality of prediction and alleviating the demand of big data. The application of Gaussian process (GP) in this scenario yields the non-parametric yet informative Bayesian multi-task regression paradigm. Multi-task GP (MTGP) provides not only the prediction mean but also the associated prediction variance to quantify uncertainty, thus gaining popularity in various scenarios. The linear model of coregionalization (LMC) is a well-known MTGP paradigm which exploits the dependency of tasks through linear combination of several independent and diverse GPs. The LMC however suffers from high model complexity and limited model capability when handling complicated multi-task cases. To this end, we develop the neural embedding of coregionalization that transforms the latent GPs into a high-dimensional latent space to induce rich yet diverse behaviors. Furthermore, we use advanced variational inference as well as sparse approximation to devise a tight and compact evidence lower bound (ELBO) for higher quality of scalable model inference. Extensive numerical experiments have been conducted to verify the higher prediction quality and better generalization of our model, named NSVLMC, on various real-world multi-task datasets and the cross-fluid modeling of unsteady fluidized bed.

A multitask GNN-based interpretable model for discovery of selective JAK inhibitors

The Janus kinase (JAK) family plays a pivotal role in most cytokine-mediated inflammatory and autoimmune responses via JAK/STAT signaling, and administration of JAK inhibitors is a promising therapeutic strategy for several diseases including COVID-19. However, to screen and design selective JAK inhibitors is a daunting task due to the extremely high homology among four JAK isoforms. In this study, we aimed to simultaneously predict pIC50 values of compounds for all JAK subtypes by constructing an interpretable GNN multitask regression model. The final model performance was positive, with R2 values of 0.96, 0.79 and 0.78 on the training, validation and test sets, respectively. Meanwhile, we calculated and visualized atom weights, followed by the rank sum tests and local mean comparisons to obtain key atoms and substructures that could be fine-tuned to design selective JAK inhibitors. Several successful case studies have demonstrated that our approach is feasible and our model could learn the interactions between proteins and small molecules well, which could provide practitioners with a novel way to discover and design JAK inhibitors with selectivity.Graphical

A Novel Tropical Cyclone Size Estimation Model Based on a Convolutional Neural Network Using Geostationary Satellite Imagery

A novel tropical cyclone (TC) size estimation model (TC-SEM) in the western North Pacific was developed based on a convolutional neural network (CNN) using geostationary satellite infrared (IR) images. The proposed TC-SEM was tested using three CNN schemes: a single-task regression model that separately estimated the radius of maximum wind (RMW) and the radius of 34 kt wind (R34) of the TC, a multi-task regression model that estimated the RMW and R34 simultaneously, and a multi-task regression model using best-track TC intensity information. For model training, validation, and testing, 29,730, 2505, and 11,624 geostationary satellite images of the region around the center of the TC, respectively, were used, each containing four IR bands: short-wavelength IR (3.7 µm), water vapor (6.7 µm), IR1 (10.8 µm), and IR2 (12.0 µm). The results showed that the multi-task model performed better than the single-task model due to knowledge sharing and its ability to solve multiple interrelated tasks simultaneously. The inclusion of TC intensity information in the multi-task model further improved the performance of the RMW and R34 estimations, with correlations (mean absolute errors) of 0.95 (2.05 nmi) and 0.93 (9.77 nmi), respectively, which represent significant improvements over the performance of existing linear regression statistical methods. The results suggested that this CNN model using geostationary satellite images may be a powerful tool for estimating TC sizes in operational TC forecasts.

Leveraging pleiotropic association using sparse group variable selection in genomics data

BackgroundGenome-wide association studies (GWAS) have identified genetic variants associated with multiple complex diseases. We can leverage this phenomenon, known as pleiotropy, to integrate multiple data sources in a joint analysis. Often integrating additional information such as gene pathway knowledge can improve statistical efficiency and biological interpretation. In this article, we propose statistical methods which incorporate both gene pathway and pleiotropy knowledge to increase statistical power and identify important risk variants affecting multiple traits.MethodsWe propose novel feature selection methods for the group variable selection in multi-task regression problem. We develop penalised likelihood methods exploiting different penalties to induce structured sparsity at a gene (or pathway) and SNP level across all studies. We implement an alternating direction method of multipliers (ADMM) algorithm for our penalised regression methods. The performance of our approaches are compared to a subset based meta analysis approach on simulated data sets. A bootstrap sampling strategy is provided to explore the stability of the penalised methods.ResultsOur methods are applied to identify potential pleiotropy in an application considering the joint analysis of thyroid and breast cancers. The methods were able to detect eleven potential pleiotropic SNPs and six pathways. A simulation study found that our method was able to detect more true signals than a popular competing method while retaining a similar false discovery rate.ConclusionWe developed feature selection methods for jointly analysing multiple logistic regression tasks where prior grouping knowledge is available. Our method performed well on both simulation studies and when applied to a real data analysis of multiple cancers.

Multi-target regressor chains with repetitive permutation scheme for characterization of built environments with remote sensing

Multi-task learning techniques allow the beneficial joint estimation of multiple target variables. Here, we propose a novel multi-task regression (MTR) method called ensemble of regressor chains with repetitive permutation scheme. It belongs to the family of problem transformation-based MTR methods which foresee the creation of an individual model per target variable. Subsequently, the combination of the separate models allows obtaining an overall prediction. Our method builds upon the concept of so-called ensemble of regressor chains which align single-target models along a flexible permutation, i.e., chain. However, in order to particularly address situations with a small number of target variables, we equip ensemble of regressor chains with a repetitive permutation scheme. Thereby, estimates of the target variables are cascaded to subsequent models as additional features when learning along a chain, whereby one target variable can occupy multiple elements of the chain. We provide experimental evaluation of the method by jointly estimating built-up height and built-up density based on features derived from Sentinel-2 data for the four largest cities in Germany in a comparative setup. We also consider single-target stacking, multi-target stacking, and ensemble of regressor chains without repetitive permutation. Empirical results underline the beneficial performance properties of MTR methods. Our ensemble of regressor chain with repetitive permutation scheme approach achieved most frequently the highest accuracies compared to the other MTR methods, whereby mean improvements across the experiments of 14.5% compared to initial single-target models could be achieved.

Regulation-based probabilistic substance quality index and automated geo-spatial modeling for water quality assessment

Natural environments are recognized as complex heterogeneous structures thus requiring numerous multi-scale observations to yield a comprehensive description. To monitor the current state and identify negative impacts of human activity, fast and precise instruments are in urgent need. This work provides an automated approach to the assessment of spatial variability of water quality using guideline values on the example of 1526 water samples comprising 21 parameters at 448 unique locations across the New Moscow region (Russia). We apply multi-task Gaussian process regression (GPR) to model the measured water properties across the territory, considering not only the spatial but inter-parameter correlations. GPR is enhanced with a Spectral Mixture Kernel to facilitate a hyper-parameter selection and optimization. We use a 5-fold cross-validation scheme along with R^2-score to validate the results and select the best model for simultaneous prediction of water properties across the area. Finally, we develop a novel Probabilistic Substance Quality Index (PSQI) that combines probabilistic model predictions with the regulatory standards on the example of the epidemiological rules and hygienic regulations established in Russia. Moreover, we provide an interactive map of experimental results at 100 m2 resolution. The proposed approach contributes significantly to the development of flexible tools in environment quality monitoring, being scalable to different standard systems, number of observation points, and region of interest. It has a strong potential for adaption to environmental and policy changes and non-unified assessment conditions, and may be integrated into support-decision systems for the rapid estimation of water quality spatial distribution.

An L0 Regularization Method for Imaging Genetics and Whole Genome Association Analysis on Alzheimer's Disease.

Although the neuroimaging measures build a bridge between genetic variants and disease phenotypes, an assessment of single nucleotide variants changes in brain structure and their clinically influence on the progression of Alzheimer's disease remain largely preliminary. Note that each variant has very weak correlation signal to neuroimaging measures or Alzheimer's disease phenotypes. Therefore, traditional sparse regression-based image genetics approaches confront with unresolvable features, relative high regression error or inapplicability of high-dimensional data. Adopting an [Formula: see text] regularization method, we significantly elevate the regression accuracy of imaging genetics compared with group-sparse multitask regression method. With further analysis on the simulation results, we conclude that multiple regression tasks model may be unsuitable for image genetics. In addition, we carried out a whole genome association analysis between genetic variants (about 388 million loci) and phenotypes (cognition normal, mild cognitive impairment and Alzheimer's disease) with using the [Formula: see text] regularization method. After annotating the effect of all variants by Ensembl Variant Effect Predictor (VEP), our method locates 33 missense variants which can explain 40% phenotype variance. Then, we mapped each missense variant to the nearest gene and carried out pathway enrichment analysis. The Notch signaling pathway and Apoptosis pathway have been reported to be related to the formation of Alzheimer's disease.

Multi-task short-term reactive and active load forecasting method based on attention-LSTM model

With the rapid development of power markets, smart grids and large-scale renewable energy generation, it is crucial to be able to accurately predict both reactive and active power loads. In this paper, we propose a novel multi-task load forecasting method for predicting both active and reactive power simultaneously. The long short-term memory (LSTM) architecture is employed in the backbone prediction model, supported by an attention mechanism to prevent performance deterioration. Considering the latent dynamic correlations between the reactive and active loads of a substation, multi-task regression based on hard parameter sharing is adopted to treat the forecasting of both types of loads as parallel subtasks. Meanwhile, we design an adaptive combined-task-wise loss function to optimize the proposed multi-task load forecasting model to avoid biasing of the final model for any subtask. We compare our multi-task attention-LSTM (MTAL) model with other popular single-task load forecasting models and achieve superior accuracy on both subtasks. The results indicate that the proposed method is robust and reliable for practical applications in power systems.

Identification of Potential Biomarkers and Biological Pathways for Poor Clinical Outcome in Mucinous Colorectal Adenocarcinoma.

Simple SummaryPatients with mucinous adenocarcinoma (MAC) have been considered to have a faster disease progression than patients with traditional adenocarcinoma (TAC) in colorectal cancer (CRC). However, to date, the roles of MAC in long-term survival remain controversial due to a small sample size and the nature of its relatively rare occurrence, although it potentially represents entities with different aggressiveness and prognoses. Here, using large-scale population data, we found that the patients with the MAC subtype had a significantly worse overall survival rate and a tendency of worse disease-specific survival rate in stage II compared with the patients with the TAC subtype. Furthermore, key gene signatures were identified using the established predictive models for the disease-specific survival of stage II mucinous CRC.Colorectal cancer (CRC) comprises several histological subtypes, but the influences of the histological subtypes on prognosis remains unclear. We sought to evaluate the prognosis of mucinous adenocarcinoma (MAC), compared to that of traditional adenocarcinoma (TAC). This study used the data of patients diagnosed with CRC between 2004 and 2016, as obtained from the Surveillance, Epidemiology, and End Results database. We established a predictive model for disease-specific survival using conditional survival forest, model, non-linear Cox proportional hazards, and neural multi-task logistic regression model and identified the gene signatures for predicting poor prognosis based on the arrayexpress datasets. In total, 9096 (42.1%) patients with MAC and 12,490 (58.9%) patients with TAC were included. Those with the MAC subtype were more likely to have a poorer overall survival rate compared to those with the TAC subtype in stage II CRC (p = 0.002). The eight major genes including RPS18, RPL30, NME2, USP33, GAB2, RPS3A, RPS25, and CEP57 were found in the interacting network pathway. MAC was found to have a poorer prognosis compared to TAC, especially in Stage II CRC. In addition, our findings suggest that identifying potential biomarkers and biological pathways can be useful in CRC prognosis.

A SHRINKAGE PRINCIPLE FOR HEAVY-TAILED DATA: HIGH-DIMENSIONAL ROBUST LOW-RANK MATRIX RECOVERY.

This paper introduces a simple principle for robust statistical inference via appropriate shrinkage on the data. This widens the scope of high-dimensional techniques, reducing the distributional conditions from sub-exponential or sub-Gaussian to more relaxed bounded second or fourth moment. As an illustration of this principle, we focus on robust estimation of the low-rank matrix Θ* from the trace regression model Y = Tr(Θ*⊤ X) + ϵ. It encompasses four popular problems: sparse linear model, compressed sensing, matrix completion and multi-task learning. We propose to apply the penalized least-squares approach to the appropriately truncated or shrunk data. Under only bounded 2+δ moment condition on the response, the proposed robust methodology yields an estimator that possesses the same statistical error rates as previous literature with sub-Gaussian errors. For sparse linear model and multi-task regression, we further allow the design to have only bounded fourth moment and obtain the same statistical rates. As a byproduct, we give a robust covariance estimator with concentration inequality and optimal rate of convergence in terms of the spectral norm, when the samples only bear bounded fourth moment. This result is of its own interest and importance. We reveal that under high dimensions, the sample covariance matrix is not optimal whereas our proposed robust covariance can achieve optimality. Extensive simulations are carried out to support the theories.

OF-UMRN: Uncertainty-guided multitask regression network aided by optical flow for fully automated comprehensive analysis of carotid artery.

Fully automated comprehensive analysis of carotid artery (localization of range of interest (ROI), direct quantitative measurement and segmentation of lumen diameter (CALD) and intima-media thickness (CIMT), and motion estimation of the carotid wall) is a reliable auxiliary diagnosis of cardiovascular diseases, which relieves physicians from laborious workloads. No work has achieved fully automated comprehensive analysis of carotid artery due to five intractable challenges: (1) The heavy reliance on experienced carotid physicians for the selection of ROI limits fully automated studies. (2) The weak structural information of intima-media thickness increases the difficulty of feature encoding. (3) The radial motion of the carotid wall results in the lack of discriminant features of boundaries. (4) Diseased carotid arteries lose many expression features. (5) Optimal weights of multitask regression are hard to tune manually. In this paper, we propose a novel uncertainty-guided multitask regression network aided by optical flow named OF-UMRN to solve the intractable challenges. The four modules and three innovations of the OF-UMRN take their responsibility. OF-UMRN takes localization and mapping of ROI as a pre-processing. It achieves direct quantitative measurement and segmentation by a multitask regression network. And we creatively model homoscedastic uncertainty to automated tune the weights of the two tasks optimally. The OF-UMRN adopts a bidirectional mechanism to encode the optical flow used to predict the carotid wall's motion fields. More importantly, we creatively propose a dual optimization module based on the co-promotion between segmentation and motion estimation to improve the performance of radially moving and diseased carotid arteries. Therefore, the OF-UMRN makes the most of the pathological relationship between multiple objects and co-promotion between various tasks. Extensive experiments on US sequences of 101 patients have demonstrated the superior performance of OF-UMRN on the fully automated comprehensive analysis of the carotid artery. Therefore the OF-UMRN has excellent potential in clinical disease diagnoses and assessments of the carotid artery.

A feature reconstruction-based multi-task regression model for cyanobacterial distribution forecasting along the water column

Cyanobacterial water pollution has been threatening the cleaner ecosystem and urban sustainability due to the harmfulness to aquatic ecosystems and human health, which triggers the development of an effective forecasting tool for cyanobacterial blooms. Along the water column, the variations in cyanobacteria cell densities show various distribution patterns and are influenced by multiple environmental factors. Most data-driven models treat cyanobacteria forecasting at a specific water depth as a single task, which fails to share knowledge amongst water depths, resulting in unfavourable forecasting accuracy. This is why an increasing number of nonlinear black-box models have been built for cyanobacteria forecasting but at the expense of model interpretability. This study aims to investigate whether forecasting accuracy and model interpretability can be enhanced by (i) using easily accessible predictors and (ii) developing a feature reconstruction-based multi-task regression model with knowledge sharing amongst water depths. Real-world data from a tropical lake are used to evaluate the effectiveness of the model. For the studied lake, the highest average cyanobacteria cell density occurs at 1.0 m, after which it decreases by over 30% at 5.5 m. The correlation coefficients of time-serial cyanobacteria cell densities between adjacent water depths are greater than 0.95 (P < 0.001). The forecasting results indicate that, compared to single-task nonlinear models, 20.59%, 16.25%, and 22.70% error reductions, measured by the mean square error, are achieved for one-day-ahead, two-day-ahead, and three-day-ahead cyanobacterial bloom forecasts. The accurate bloom and non-bloom signals under the proposed model are up to 94.81% and 98.28%. Based on the proposed model, the relative importance of predictors, the sparsity of regression coefficients, and the covariance relationship of regression coefficients can interpret the model adequately and elucidate the mechanism of knowledge sharing and forecasting accuracy improvement.

Direct full quantification of the left ventricle via multitask regression and classification

Left ventricle (LV) quantitative indices, such as the areas of the cavity and myocardium, dimensions of the cavity, regional wall thickness, and phase classification, play a vital role in assessing the overall and local cardiac function in clinical practice. However, due to variations in the cardiac structures of different subjects and the inconvenience of manual feature extraction, it is difficult to accurately estimate left ventricle indices. To solve the above problem, this paper presents a multitask regression and classification network (MTRC-net) based on image representation to improve the accuracy of the quantified results. The network employs an autoencoder network to perform image representation, and then the representative images are fed into the multitask regression and classification network to extract the temporal and spatial features of the cardiac images, which can better help the MTRC-net to learn the potential image features, reduce the experimental errors and speed up the convergence of the model. In the multitask regression and classification network, a pseudo-3D network is used to compress the network model, and the circular hypothesis is applied to fully obtain the temporal and spatial information of the cardiac images. The proposed algorithm was verified on a dataset that includes images of 145 patients from the left ventricle quantification competition of MICCAI2018, and the results are as follows. The average absolute error of MAE in the Areas (cavity area and myocardium area) is 106 ± 88 mm2, the average absolute error of MAE in the Dims (directional dimensions of the cavity) region is 1.54 ± 1.40 mm, the average absolute error of MAE in the RWTs(regional wall thicknesses region) is 0.96 ± 0.70 mm, and the error rate of phase classification is 1.2%. The errors of each index are almost always smaller than those of the indices of the existing methods, and the experimental results illustrate the feasibility and effectiveness of the proposed method.

Deep regression for LiDAR-based localization in dense urban areas

LiDAR-based localization in a city-scale map is a fundamental question in autonomous driving research. As a reasonable localization scheme, the localization can be performed by global retrieval (that suggests potential candidates from the database) followed by geometric registration (that obtains an accurate relative pose). In this work, we develop a novel end-to-end, deep multi-task network that simultaneously performs global retrieval and geometric registration for LiDAR-based localization. Both retrieval and registration are formulated and solved as regression problems, and they can be deployed independently during inference time. We also design two mechanisms to enhance our multi-task regression network’s performance: residual connections for point clouds and a new loss function with learnable parameters. To alleviate the common phenomenon of vanishing gradients in neural networks, we employ residual connections to support constructing a deeper network effectively. At the same time, to solve the problem of huge differences in scale and units between different tasks, we propose a loss function that can automatically balance multi-tasks. Experiments on two public benchmarks validate the state-of-the-art performance of our algorithm in large-scale LiDAR-based localization.

Optimal selection of sample-size dependent common subsets of covariates for multi-task regression prediction

An analyst is given a training set consisting of regression datasets $D_j$ of different sizes, which are distributed according to some $G_j$, $j=1,\ldots,\cal J$, where the distributions $G_j$ are assumed to form a random sample generated by some common source. In particular, the $D_j$'s have a common set of covariates and they are all labeled. The training set is used by the analyst for selection of subsets of covariates denoted by ${P}^*(n)$, whose role is described next. The multi-task problem we consider is as follows: given a number of random labeled datasets (which may be in the training set or not) $D_{J_k}$ of size $n_k$, $k=1,\ldots,K$, estimate separately for each dataset the regression coefficients on the subset of covariates ${P}^*(n_k)$ and then predict future dependent variables given their covariates. Naturally, a large sample size $n_k$ of $D_{J_k}$ allows a larger subset of covariates, and the dependence of the size of the selected covariate subsets on $n_k$ is needed in order to achieve good prediction and avoid overfitting. Subset selection is notoriously difficult and computationally demanding, and requires large samples; using all the regression datasets in the training set together amounts to borrowing strength toward better selection under suitable assumptions. Furthermore, using common subsets for all regressions having a given sample size standardizes and simplifies the data collection and avoids having to select and use a different subset for each prediction task. Our approach is efficient when the relevant covariates for prediction are common to the different regressions, while the models' coefficients may vary between different regressions.

Robust hybrid deep learning models for Alzheimer’s progression detection

The prevalence of Alzheimer’s disease (AD) in the growing elderly population makes accurately predicting AD progression crucial. Due to AD’s complex etiology and pathogenesis, an effective and medically practical solution is a challenging task. In this paper, we developed and evaluated two novel hybrid deep learning architectures for AD progression detection. These models are based on the fusion of multiple deep bidirectional long short-term memory (BiLSTM) models. The first architecture is an interpretable multitask regression model that predicts seven crucial cognitive scores for the patient 2.5 years after their last observations. The predicted scores are used to build an interpretable clinical decision support system based on a glass-box model. This architecture aims to explore the role of multitasking models in producing more stable, robust, and accurate results. The second architecture is a hybrid model where the deep features extracted from the BiLSTM model are used to train multiple machine learning classifiers. The two architectures were comprehensively evaluated using different time series modalities of 1371 subjects participated in the study of the Alzheimer’s disease neuroimaging initiative (ADNI). The extensive, real-world experimental results over ADNI data help establish the effectiveness and practicality of the proposed deep learning models.