Multi-task Convolutional Neural Network Research Articles

Independent evaluation of a multi-view multi-task convolutional neural network breast cancer classification model using Finnish mammography screening data

Background:Development of deep convolutional neural networks for breast cancer classification has taken significant steps towards clinical adoption. It is though unclear how the models perform for unseen data, and what is required to adapt them to different demographic populations. In this retrospective study, we adopt an openly available pre-trained mammography breast cancer multi-view classification model and evaluate it by utilizing an independent Finnish dataset. Methods:Transfer learning was used, and the pre-trained model was finetuned with 8,829 examinations from the Finnish dataset (4,321 normal, 362 malignant and 4,146 benign examinations). Holdout dataset with 2,208 examinations from the Finnish dataset (1,082 normal, 70 malignant and 1,056 benign examinations) was used in the evaluation. The performance was also evaluated on a manually annotated malignant suspect subset. Receiver Operating Characteristic (ROC) and Precision–Recall curves were used to performance measures. Results:The Area Under ROC [95%CI] values for malignancy classification obtained with the finetuned model for the entire holdout set were 0.82 [0.76, 0.87], 0.84 [0.77, 0.89], 0.85 [0.79, 0.90], and 0.83 [0.76, 0.89] for R-MLO, L-MLO, R-CC and L-CC views respectively. Performance on the malignant suspect subset was slightly better. On the auxiliary benign classification task performance remained low. Conclusions:The results indicate that the model performs well also in an out-of-distribution setting. Finetuning allowed the model to adapt to some of the underlying local demographics. Future research should concentrate to identify breast cancer subgroups adversely affecting performance, as it is a requirement for increasing the model’s readiness level for a clinical setting.

Combined image Hough transform based simultaneous multi-parameter optical performance monitoring for intelligent optical networks

The fast-emerging flexible optical networks require intelligent monitoring techniques for supervising the signal quality performance to achieve the desired efficiency. In this paper, we propose a multi-task convolutional neural network (MT-CNN) for multi-parameter monitoring using a unique combined image Hough Transform (CIHT) technique. The proposed system supports simultaneous modulation format identification (MFI), transmission distance detection (TDD), roll-off factor identification (ROFI), Laser linewidth detection (LLWD), centre wavelength detection (CWLD) along with the optical signal-to-noise ratio (OSNR) estimation. The proposed scheme realizes the multi-parameter optical performance monitoring using the advantage of Hough transform (HT) for efficient feature extraction. Rigorous simulations to prove the advantage of using Hough transform, and data augmentation has been performed to conclude the results achieved. The simulation results demonstrate 100% accuracy after 112 epochs for MFI, TDD, ROFI, LLWD, and CWLD, and the mean absolute error (MAE) for OSNR estimation has reduced to 0.354 dB after 180 epochs. Also, rigorous simulations with various conditions are performed to demonstrate the merits of the proposed method.

Explainable multi-task convolutional neural network framework for electronic petition tag recommendation

Electronic petition (e-petition) is an electronic government (e-government) service that allows citizens to file petitions to governments via the internet. The complexity of the e-petition filing process and the unexplainable e-government tools would reduce the perceived ease of use and trust of users, which causes citizens to make errors when tagging their e-petition. These errors may lead to failure and delay in resolving the e-petition, which further restrains citizen engagement and electronic participation (e-participation) adoption on e-petition platforms (EPP). The purpose of this study is to develop an explainable tag recommender system to assist citizens in tagging their e-petitions. Specifically, we design an explainable multi-task learning framework for e-petition tag recommendation based on convolutional neural networks and layer-wise relevance propagation. We also conduct both quantitative and qualitative experiments to demonstrate the recommendation effectiveness as well as interpretability of our model. This is among the first attempt to design an e-petition tag recommender system and an explainable e-government recommender system. The practical implications of our research are two-fold. For citizens, our proposed model recommends more accurate tags with human-understandable explanations, which could assist citizens’ tagging decisions and increase the possibility for an e-petition to be resolved. For governments, the e-petition service quality of governments would be enhanced, which further promotes e-participation adoption, citizen engagement, and e-government success on EPPs.

A Multi-Task Learning for Submarine Cable Magnetic Anomaly Recognition

The recognition of submarine cable magnetic anomaly (SCMA) signals is a challenging task in magnetic signal data processing. In this study, a multi-task convolutional neural network (MTCNN) model is proposed to simultaneously recognize abnormal signals and locate abnormal regions. The residual block is added to the shared feature backbone to improve the ability of the network to extract high-level features and maintain the gradient stability of the model in the training process. The long short-term memory (LSTM) block is added to the classification branch task to learn the internal relationship of the magnetic anomaly time series, so as to improve the network’s ability to recognize magnetic anomalies. Our proposed model can accurately recognize the SCMA signals collected in the East China Sea and the South China Sea. The classification accuracy and the ability to locate the abnormal regions are close to the manual labeling of human analysts. The newly developed model can help analysts reduce the probability of missing and misjudging submarine cable magnetic anomalies, improve the efficiency and accuracy of interpretation, and could even be deployed to an unmanned platform to realize the automatic detection of SCMAs.

Multi-task neural network blind deconvolution and its application to bearing fault feature extraction

Blind deconvolution (BD) is an effective method to extract fault-related characteristics from vibration signals. Previous researches focused on two primary approaches to improve the robustness and effectiveness of BD methods: developing new optimization functions or devising new methods for estimating filter coefficients. However, these methods often suffer from the difficulty of finding the global optimum due to the complex non-convex functions. To address this issue, we propose a novel multi-objective criterion, combining two well-established sparsity criteria: kurtosis and norm, that evaluates signal characteristics in both the time and frequency domains. We observe that this criterion, consisting of two sparsity criteria with opposite monotonicity, can mutually constrain and avoid overfitting that occurs with single-domain optimization. Inspired by multi-task convolutional neural networks, we introduce a multi-task 1DCNN with two branches to optimize the criterion in both domains simultaneously. To our best knowledge, it is the first time a multi-task convolutional neural network is used for BD problems. Experiments show that our method outperforms other state-of-the-art BD methods. We have share our code in https://github.com/asdvfghg/MNNBD

Multi-task convolutional neural network for simultaneous monitoring of lipid and protein oxidative damage in frozen-thawed pork using hyperspectral imaging

Lipid and protein oxidation are the main causes of meat deterioration during freezing. Traditional methods using hyperspectral imaging (HSI) need to train multiple independent models to predict multiple attributes, which is complex and time-consuming. In this study, a multi-task convolutional neural network (CNN) model was developed for visible near-infrared HSI data (400–1002 nm) of 240 pork samples treated with different freeze-thaw cycles (0–9 cycles) to evaluate the feasibility of simultaneously monitoring lipid oxidation (thiobarbituric acid reactive substance content) and protein oxidation (carbonyl content) in pork. The performance of the commonly used partial least squares regression (PLSR) model based on the spectra after pre-processing (Standard normal variate, Savitzky-Golay derivative, and Savitzky-Golay smoothing) and feature selection (Regression coefficients) and single-output CNN model was compared. The results showed that the multi-task CNN model achieved the optimal prediction accuracies for lipid oxidation (R2p = 0.9724, RMSEP = 0.0227, and RPD = 5.2579) and protein oxidation (R2p = 0.9602, RMSEP = 0.0702, and RPD = 4.6668). In final, the changes of lipid and protein oxidation of pork in different freeze-thaw cycles were successfully visualized. In conclusion, the combination of HSI and multi-task CNN method shows the potential of end-to-end prediction of pork oxidative damage. This study provides a new, convenient and automated technique for meat quality detection in the food industry.

Abstract 5356: Collaborating to ensure data-driven drug discovery on the Cancer Genomics Cloud: Realizing the possibilities for MoDaC and ATOM

Abstract Introduction. The NCI-funded Cancer Genomics Cloud (CGC) and the NCI Predicative Oncology Model and Data Clearinghouse (MoDaC) advance NCI computing infrastructure and tools that aim to reduce the burden of cancer on patients. The CGC provides a collaborative cloud base computation infrastructure that collocates computation, bioinformatics workflows, and 3+ PB data to researchers. MoDaC provides a publicly available resource generated from the Joint Design of Advanced Computing Solutions for Cancer (JDACS4C) Program and the Accelerating Therapeutics for Opportunities in Medicine (ATOM Consortium). The NCI-sponsored MoDaC program aims to add machine learning toolsets to identifying novel treatments for Cancer Patients. We present our progress collaborating with MoDaC to make machine learning models available on the Cancer Genomics Cloud. Methods. Our Bioinformatics teams migrated MoDaC tools on the CGC, defined standards for releasing models, and collected recommendations for decreasing the time required to make MoDaC tools available on the Cancer Genomics Cloud. We mirrored the ATOM Modeling Pipeline (AMPL), a drug discovery platform, on the CGC by making the GIT repository cloud accessible through Jupyter Notebook access, supporting interactive analysis. We translated AMPL and JDACS4C models into CWL. Converting these ML models into CWL supports reproducible execution, scalable deployment, and computational portability. Results. The AMPL drug discovery platform on the CGC includes data ingestion & curation, featurization, model training & tuning, prediction generation, visualization & analysis functionality as Jupyter notebooks and CWL workflows. The release consists of chemo-informatics tools for integrating cancer treatment features in deep-learning graph models. JDACS4C ML Models ported to the CGC include classifiers (tumor and normal-tumor pairs), autoencoders (Gene Expression), drug response predictors (single and combination), and Multitask Convolutional Neural Networks (extract information from cancer pathology reports). Conclusion. We optimized the MoDaC Drug Discovery and Machine Learning tools into cloud-native resources on the CGC, supporting interactive and GUI-driven analysis. The release supports technical and newer users to machine learning, allowing access to a broader user base than those who traditionally have access to ML toolsets. These MoDaC toolsets will support pre-clinical study evaluation, treatment identification, and experimental design. Moreover, existing MoDaC-AMPL tutorials on the CGC support distributed ML-Drug Discovery training. Lastly, collaborating with MoDaC teams identified standardization approaches that can reduce the time and effort to make these tools widely available across the NIH-NCI computational infrastructure. Citation Format: Soner Koc, Vojislav Varjacic, Miona Rankovic, Marijeta Slavkovic-Ilic, Aleksandar Danicic, Sean Black, Naomi Ohashi, Titli Sarkar, Zelia Worman, Jack DiGiovanna, Brandi Davis-Dusenbery, Dennis A. Dean. Collaborating to ensure data-driven drug discovery on the Cancer Genomics Cloud: Realizing the possibilities for MoDaC and ATOM. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5356.

Gearbox fault diagnosis method based on deep learning multi-task framework

PurposeIn gearbox fault diagnosis, identifying the fault type and severity simultaneously, as well as the compound fault containing multiple faults, is necessary.Design/methodology/approachTo diagnose multiple faults simultaneously, this paper proposes a multichannel and multi-task convolutional neural network (MCMT-CNN) model.FindingsExperiments were conducted on a bearing dataset containing different fault types and severities and a gearbox compound fault dataset. The experimental results show that MCMT-CNN can effectively extract features of different tasks from vibration signals, with a diagnosis accuracy of more than 97%.Originality/valueVibration signals at different positions and in different directions are taken as the MC inputs to ensure the integrity of the fault features. Fault labels are established to retain and distinguish the unique features of different tasks. In MCMT-CNN, multiple task branches can connect and share all neurons in the hidden layer, thus enabling multiple tasks to share information.

Hybrid Multitask Learning Reveals Sequence Features Driving Specificity in the CRISPR/Cas9 System

CRISPR/Cas9 technology is capable of precisely editing genomes and is at the heart of various scientific and medical advances in recent times. The advances in biomedical research are hindered because of the inadvertent burden on the genome when genome editors are employed-the off-target effects. Although experimental screens to detect off-targets have allowed understanding the activity of Cas9, that knowledge remains incomplete as the rules do not extrapolate well to new target sequences. Off-target prediction tools developed recently have increasingly relied on machine learning and deep learning techniques to reliably understand the complete threat of likely off-targets because the rules that drive Cas9 activity are not fully understood. In this study, we present a count-based as well as deep-learning-based approach to derive sequence features that are important in deciding on Cas9 activity at a sequence. There are two major challenges in off-target determination-the identification of a likely site of Cas9 activity and the prediction of the extent of Cas9 activity at that site. The hybrid multitask CNN-biLSTM model developed, named CRISP-RCNN, simultaneously predicts off-targets and the extent of activity on off-targets. Employing methods of integrated gradients and weighting kernels for feature importance approximation, analysis of nucleotide and position preference, and mismatch tolerance have been performed.

EMI Threat Assessment of UAV Data Link Based on Multi-Task CNN

In this work, a multi-task convolutional neural network with multi-input (MIMT-CNN) is proposed for electromagnetic interference (EMI) signals recognition and electromagnetic environment risk evaluation of the data link of unmanned aerial vehicle (UAV). The visualized performance parameters, short-time Fourier transform (STFT) spectrograms, and constellation diagrams are obtained by experiment on the electromagnetic susceptibility of UAV’s datalink. In particular, the constellation diagram is further enhanced by calculating the density distribution of sampling points to obtain the normalized density constellation. Taking the above different categories of images as the input of the expected model, the multi-element and high correlation EMI features are extracted and fused in the MIMT-CNN. Besides, the structure of series-parallel connection is adopted in the trained model and the Bayesian optimization is also used to select hyperparameters. In this case, the perception model with higher reliability can be obtained. On this basis, the performance and complexity of the obtained model with different input channels are compared. The results show that with the input of constellation diagram, especially the normalized density constellation, can significantly improve the accuracy of the model. Besides the normalized density constellation, the model with visualized performance parameters and STFT spectrogram as inputs has a much better performance.

Robust multi-task learning and online refinement for spacecraft pose estimation across domain gap

This work presents Spacecraft Pose Network v2 (SPNv2), a Convolutional Neural Network (CNN) for pose estimation of noncooperative spacecraft across domain gap. SPNv2 is a multi-scale, multi-task CNN which consists of a shared multi-scale feature encoder and multiple prediction heads that perform different tasks on a shared feature output. These tasks are all related to detection and pose estimation of a target spacecraft from an image, such as prediction of pre-defined satellite keypoints, direct pose regression, and binary segmentation of the satellite foreground. It is shown that by jointly training on different yet related tasks with extensive data augmentations on synthetic images only, the shared encoder learns features that are common across image domains that have fundamentally different visual characteristics compared to synthetic images. This work also introduces Online Domain Refinement (ODR) which refines the parameters of the normalization layers of SPNv2 on the target domain images online at deployment. Specifically, ODR performs self-supervised entropy minimization of the predicted satellite foreground, thereby improving the CNN’s performance on the target domain images without their pose labels and with minimal computational efforts. The GitHub repository for SPNv2 is available at https://github.com/tpark94/spnv2.

A multi-task and multi-channel convolutional neural network for semi-supervised neonatal artefact detection

Objective. Automated artefact detection in the neonatal electroencephalogram (EEG) is crucial for reliable automated EEG analysis, but limited availability of expert artefact annotations challenges the development of deep learning models for artefact detection. This paper proposes a semi-supervised deep learning approach for artefact detection in neonatal EEG that requires few labelled data by training a multi-task convolutional neural network (CNN). Approach. An unsupervised and a supervised objective were jointly optimised by combining an autoencoder and an artefact classifier in one multi-output model that processes multi-channel EEG inputs. The proposed semi-supervised multi-task training strategy was compared to a classical supervised strategy and other existing state-of-the-art models. The models were trained and tested separately on two different datasets, which contained partially annotated multi-channel neonatal EEG. Models were evaluated using the F1-statistic and the relevance of the method was investigated in the context of a functional brain age (FBA) prediction model. Main results. The proposed multi-task and multi-channel CNN methods outperformed state-of-the-art methods, reaching F1 scores of 86.2% and 95.7% on two separate datasets. The proposed semi-supervised multi-task training strategy was shown to be superior to a classical supervised training strategy when the amount of labels in the dataset was artificially reduced. Finally, we found that the error of a brain age prediction model correlated with the amount of automatically detected artefacts in the EEG segment. Significance. Our results show that the proposed semi-supervised multi-task training strategy can train CNNs successfully even when the amount of labels in the dataset is limited. Therefore, this method is a promising semi-supervised technique for developing deep learning models with scarcely labelled data. Moreover, a correlation between the error of FBA estimates and the amount of detected artefacts in the corresponding EEG segments indicates the relevance of artefact detection for robust automated EEG analysis.

Online visual monitoring method for liquid rocket engine nozzle welding based on a multi-task deep learning model

Online quality monitoring is important in liquid rocket engine nozzle (LREN) welding procedures. Convolutional neural networks (CNNs) are widely applied in welding quality monitoring. However, most CNNs perform single visual tasks and provide limited information. They are not competent in LREN welding quality monitoring. This paper presents an online visual sensing method for LREN welding quality monitoring based on a multi-task deep learning model. Weld-detection-segmentation network (WDS-net), a multi-task CNN for simultaneous detection and segmentation is proposed for weld defect detection and weld width measurement. WDS-net obtains more information from welding images and depicts the welding process more adequately compared with single-task models. A machine vision system is constructed to monitor the LREN welding process. Then, experiments are conducted to obtain LREN welding images and construct a dataset. WDS-net is trained and tested on the dataset and obtains a 95.85% mean average precision (mAP) for weld defect detection and a 92.63% intersection over union (IoU) for weld zone segmentation. Finally, WDS-net is tested on LREN welding image sequences, and the results are compared with the offline measurement results of corresponding welds. WDS-net detects, locates the defects without omission, and measures weld width with a mean error within 0.1 mm compared with the actual weld. WDS-net’s monitoring performance and its average inference rate of 147 frames per second meet the requirements of online monitoring of LREN welding.

Learning a Novel LiDAR Submap-Based Observation Model for Global Positioning in Long-Term Changing Environments

In complex environments with long-term changes such as light, seasonal, and viewpoint changes, robust, accurate, and high-frequency global positioning based on light detection and ranging (LiDAR) map is still a challenge, which is crucial for autonomous vehicles or robots. To this end, a novel observation model that relies on the siamese multitask convolutional neural networks (CNNs) with multimodule cascaded is creatively presented in this article. In particular, a new pseudoimage representing LiDAR submap is designed to enrich scene texture and enhance rotation invariance. Besides, novel siamese CNNs that are coupled by NeXtVLAD and long short-term memory is designed for the first time, which can reliably predict similarity and quaternion at the same time. Finally, the predicted quaternion observation is integrated into the extended Kalman filter framework for multisensor fusion to achieve robust high-frequency global pose estimation. Extensive evaluations on KITTI, NCLT, and real-world datasets suggest that the proposed method not only obtains the remarkable precision-recall performance, but also effectively enhances and improves the robustness and accuracy of long-term positioning.

Electromechanical Impedance Temperature Compensation and Bolt Loosening Monitoring Based on Modified Unet and Multitask Learning

Bolts are frequently subjected to loosening due to time varying external loads during service. The electromechanical impedance (EMI) technique based on piezoelectric ceramic wafers (PZT) is sensitive to the initial bolt preload looseness. However, the change in environmental temperature has a great effect on EMI monitoring. Deep convolutional neural network (CNN) is a promising technique for EMI monitoring. Nevertheless, it is difficult to train a deep CNN with limited training data to accurately identify damages under a wide range of temperature variations. To this end, this study proposes a multitask CNN for identifying bolts loosening. The network consists of a temperature compensation subnetwork to compensate for the temperature effect, and a lightweight damage identification subnetwork to identify bolt loosening states. The temperature compensation subnetwork is a modified Unet, and both the impedance and temperature are used as its input. The damage identification subnetwork is connected in series behind the temperature compensation subnetwork. A multiloss function is proposed in which a TV regularizer is used. Experimental results show that the validation accuracy of the multitask network is 97.71% when the network is trained by only about 30 samples from each loosening state. Moreover, the generalization abilities of the proposed multitask model to unexpected temperatures and bolt torques are investigated. The model is interpreted by the integrated gradients method, and is also compared with single-task damage identification CNNs. It is proved that the multitask network trained by limited samples can achieve accurate damage identification in temperature varying environments.

A semantic robotic grasping framework based on multi-task learning in stacking scenes

Autonomous robotic grasping is an essential skill for service robots to perform specified tasks in unstructured scenarios. Previous work focus on simple pick-and-place tasks, and it is not satisfactory for real-world scenes that have requirements for manipulation. In this paper, we present a modular intelligent robot architecture via multi-task convolutional neural network which can be used for specific object grasping and manipulation in a stacked and cluttered environment. Firstly, an end-to-end, multi-task semantic grasping convolutional neural network (MSG-ConvNet) that simultaneously outputs the results of grasp detection and semantic segmentation is proposed to recognize the affiliations between objects and grasps in cluttered scenarios. Secondly, we propose a post-processing method which allows the robot to select an optimal grasping area in an active perception way through simply reasoning on the multi-modal information output by the proposed model. The proposed multi-task network has a great improvement in both recognition accuracy and detection speed on the public multi-object dataset GraspNet-1Billion compared with the benchmark. The proposed grasp detection method also yields state-of-the-art performance with accuracies of 95.06% and 98.6% on the public single-object Jacquard Dataset and Cornell Dataset, respectively. In addition, the experiments in a real-world scene demonstrate that our proposed method has stronger robustness and adaptability than the simple direct grasping strategy in the environment with higher mutual occlusion.

A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery.

The current study presents a multi-task end-to-end deep learning model for real-time blood accumulation detection and tools semantic segmentation from a laparoscopic surgery video. Intraoperative bleeding is one of the most problematic aspects of laparoscopic surgery. It is challenging to control and limits the visibility of the surgical site. Consequently, prompt treatment is required to avoid undesirable outcomes. This system exploits a shared backbone based on the encoder of the U-Net architecture and two separate branches to classify the blood accumulation event and output the segmentation map, respectively. Our main contribution is an efficient multi-task approach that achieved satisfactory results during the test on surgical videos, although trained with only RGB images and no other additional information. The proposed multi-tasking convolutional neural network did not employ any pre- or postprocessing step. It achieved a Dice Score equal to 81.89% for the semantic segmentation task and an accuracy of 90.63% for the event detection task. The results demonstrated that the concurrent tasks were properly combined since the common backbone extracted features proved beneficial for tool segmentation and event detection. Indeed, active bleeding usually happens when one of the instruments closes or interacts with anatomical tissues, and it decreases when the aspirator begins to remove the accumulated blood. Even if different aspects of the presented methodology could be improved, this work represents a preliminary attempt toward an end-to-end multi-task deep learning model for real-time video understanding.

Coarse-to-Fine: Progressive Knowledge Transfer-Based Multitask Convolutional Neural Network for Intelligent Large-Scale Fault Diagnosis.

In modern industry, large-scale fault diagnosis of complex systems is emerging and becoming increasingly important. Most deep learning-based methods perform well on small number of fault diagnosis, but cannot converge to satisfactory results when handling large-scale fault diagnosis because the huge number of fault types will lead to the problems of intra/inter-class distance unbalance and poor local minima in neural networks. To address the above problems, a progressive knowledge transfer-based multitask convolutional neural network (PKT-MCNN) is proposed. First, to construct the coarse-to-fine knowledge structure intelligently, a structure learning algorithm is proposed via clustering fault types in different coarse-grained nodes. Thus, the intra/inter-class distance unbalance problem can be mitigated by spreading similar tasks into different nodes. Then, an MCNN architecture is designed to learn the coarse and fine-grained task simultaneously and extract more general fault information, thereby pushing the algorithm away from poor local minima. Last but not least, a PKT algorithm is proposed, which can not only transfer the coarse-grained knowledge to the fine-grained task and further alleviate the intra/inter-class distance unbalance in feature space, but also regulate different learning stages by adjusting the attention weight to each task progressively. To verify the effectiveness of the proposed method, a dataset of a nuclear power system with 66 fault types was collected and analyzed. The results demonstrate that the proposed method can be a promising tool for large-scale fault diagnosis.

Direct reconstruction for simultaneous dual-tracer PET imaging based on multi-task learning

BackgroundSimultaneous dual-tracer positron emission tomography (PET) imaging can observe two molecular targets in a single scan, which is conducive to disease diagnosis and tracking. Since the signals emitted by different tracers are the same, it is crucial to separate each single tracer from the mixed signals. The current study proposed a novel deep learning-based method to reconstruct single-tracer activity distributions from the dual-tracer sinogram.MethodsWe proposed the Multi-task CNN, a three-dimensional convolutional neural network (CNN) based on a framework of multi-task learning. One common encoder extracted features from the dual-tracer dynamic sinogram, followed by two distinct and parallel decoders which reconstructed the single-tracer dynamic images of two tracers separately. The model was evaluated by mean squared error (MSE), multiscale structural similarity (MS-SSIM) index and peak signal-to-noise ratio (PSNR) on simulated data and real animal data, and compared to the filtered back-projection method based on deep learning (FBP-CNN).ResultsIn the simulation experiments, the Multi-task CNN reconstructed single-tracer images with lower MSE, higher MS-SSIM and PSNR than FBP-CNN, and was more robust to the changes in individual difference, tracer combination and scanning protocol. In the experiment of rats with an orthotopic xenograft glioma model, the Multi-task CNN reconstructions also showed higher qualities than FBP-CNN reconstructions.ConclusionsThe proposed Multi-task CNN could effectively reconstruct the dynamic activity images of two single tracers from the dual-tracer dynamic sinogram, which was potential in the direct reconstruction for real simultaneous dual-tracer PET imaging data in future.

A multi-task convolutional neural network for classification and segmentation of chronic venous disorders

Chronic Venous Disorders (CVD) of the lower limbs are one of the most prevalent medical conditions, affecting 35% of adults in Europe and North America. Due to the exponential growth of the aging population and the worsening of CVD with age, it is expected that the healthcare costs and the resources needed for the treatment of CVD will increase in the coming years. The early diagnosis of CVD is fundamental in treatment planning, while the monitoring of its treatment is fundamental to assess a patient’s condition and quantify the evolution of CVD. However, correct diagnosis relies on a qualitative approach through visual recognition of the various venous disorders, being time-consuming and highly dependent on the physician’s expertise. In this paper, we propose a novel automatic strategy for the joint segmentation and classification of CVDs. The strategy relies on a multi-task deep learning network, denominated VENet, that simultaneously solves segmentation and classification tasks, exploiting the information of both tasks to increase learning efficiency, ultimately improving their performance. The proposed method was compared against state-of-the-art strategies in a dataset of 1376 CVD images. Experiments showed that the VENet achieved a classification performance of 96.4%, 96.4%, and 97.2% for accuracy, precision, and recall, respectively, and a segmentation performance of 75.4%, 76.7.0%, 76.7% for the Dice coefficient, precision, and recall, respectively. The joint formulation increased the robustness of both tasks when compared to the conventional classification or segmentation strategies, proving its added value, mainly for the segmentation of small lesions.