Location Annotation Research Articles

IID 2021: towards context-specific protein interaction analyses by increased coverage, enhanced annotation and enrichment analysis.

Improved bioassays have significantly increased the rate of identifying new protein-protein interactions (PPIs), and the number of detected human PPIs has greatly exceeded early estimates of human interactome size. These new PPIs provide a more complete view of disease mechanisms but precise understanding of how PPIs affect phenotype remains a challenge. It requires knowledge of PPI context (e.g. tissues, subcellular localizations), and functional roles, especially within pathways and protein complexes. The previous IID release focused on PPI context, providing networks with comprehensive tissue, disease, cellular localization, and druggability annotations. The current update adds developmental stages to the available contexts, and provides a way of assigning context to PPIs that could not be previously annotated due to insufficient data or incompatibility with available context categories (e.g. interactions between membrane and cytoplasmic proteins). This update also annotates PPIs with conservation across species, directionality in pathways, membership in large complexes, interaction stability (i.e. stable or transient), and mutation effects. Enrichment analysis is now available for all annotations, and includes multiple options; for example, context annotations can be analyzed with respect to PPIs or network proteins. In addition to tabular view or download, IID provides online network visualization. This update is available at http://ophid.utoronto.ca/iid.

Genetic Spectrum of Inherited/Congenital Hemolytic Anemias in Indian Patients

IntroductionHemolytic anemias are a group of disorders caused by the premature destruction of red blood cells with reticulocytosis. Common causes of inherited/congenital hemolysis are hemoglobinopathies and thalassemia syndromes, red blood cell membrane, and enzyme disorders. Most of the common causes (thalassemia, glucose-6-phosphate dehydrogenase (G6PD) deficiency, hereditary spherocytosis, etc.) are diagnosed based on laboratory testing; however, for remaining causes laboratory tests are either inaccessible or cumbersome. We follow a stepwise diagnostic pipeline and red cell morphology is helpful with membrane disorders. Phenotypes vary from severe hemolysis (transfusion-dependent) to mild/asymptomatic patients. Undiagnosed haemolytic anemias are taken up for multi-gene panel-based targeted resequencing which is rapid, accurate, and cost-effective. The use of these panels expedites the diagnoses of inherited hemolytic anemias and is eventually helpful for evidence-based genetic counseling.ObjectivesThis study aimed to determine the genetic defects in inherited/congenital hemolytic anemias which remained unexplained after routine laboratory tests.MethodsSeventy-five families were enrolled based on the clinical and laboratory features of inherited/congenital hemolytic anemias. Common causes of inherited hemolysis are G6PD deficiency, hemoglobinopathies and thalassemia syndromes, autoimmune hemolytic anemias, hereditary spherocytosis, and pyruvate kinase (PK) deficiency were excluded on the basis of biochemical and molecular tests. DNA extraction was done QIAamp DNA Blood Mini Kit. Quantity and quality of DNA were verified using NanoDrop and Qubit Fluorometer respectively. DNA libraries were prepared using Amplicon custom panels for genes implicated in hemolytic anemias and sequenced on Illumina MiSeq Sequencer. Alignment and variant calling were done in Illumina Local run Manager and Variant annotation was done in Basespace VariantInterpretor. Sanger sequencing was done as orthogonal validation in the index case. Predictive testing was performed for the family members.ResultsAfter targeted resequencing of the total 75 index cases, 19 patients were found to have red blood cell enzymopathies, 15 patients had stomatocytosis, 13 had membranopathies and three patients had unstable hemoglobins. In 8 patients cause was not established either only heterozygous variant was found for autosomal recessive or due to the lack of samples of family members for screening. Seventeen cases remained unexplained even after next-generation sequencing.Out of 19 patients, unexpected PK deficiency was found in 12 patients and G6PD deficiency was found in 3 patients; despite the enzyme assay being normal in these cases. We also found 2 patients with glucose-6-phosphate isomerase deficiency. One case each with hexokinase deficiency and glutathione synthetase deficiency was found.Among 15 patients with stomatocytosis, 8 had Mediterranean stomatocytosis/macrothrombocytopenia (ABCG5/ABCG8). These 8 patients showed the presence of stomatocytosis along with giant platelets on peripheral smear evaluation. Of the remaining 7 cases , 2 were found to have overhydrated hereditary stomatocytosis (RHAG) and dehydrated Stomatocytosis/xerocytosis was found in 5 (PIEZO1/KCNN4).We also found 13 cases of hemolytic anemia to have a genetic defect in red blood cell membrane protein-coding genes. Of these 5 had probably pathogenic variants in the ANK1 gene, 5 had a pathogenic variant in SPTA1, 2 had SPTB 2, and 1 patient SLC4A1.We also encountered 3 cases of unstable hemoglobins where no abnormality was noted in Hb-HPLC patterns.A total of seven patients underwent splenectomy and are transfusion free.ConclusionsOur cohort of 75 families of hemolytic anemia of unexplained etiology showed a highly heterogeneous genetic spectrum. Of the total cases, the confirmed diagnosis was achieved in 67% of the patients. This approach of using a multi-gene panel is cost-effective and can provide a rapid and accurate diagnosis. Unexpected PK deficiency, G6PD deficiency, and unstable hemoglobins suggest that such cases can be missed. Providing accurate diagnosis in such cases provides evidence-based counseling and saves the families from inappropriate treatments. DisclosuresNo relevant conflicts of interest to declare.

Weakly supervised pneumonia localization in chest X-rays using generative adversarial networks.

Automatic localization of pneumonia on chest X-rays (CXRs) is highly desirable both as an interpretive aid to the radiologist and for timely diagnosis of the disease. However, pneumonia's amorphous appearance on CXRs and complexity of normal anatomy in the chest present key challenges that hinder accurate localization. Existing studies in this area are either not optimized to preserve spatial information of abnormality or depend on expensive expert-annotated bounding boxes. We present a novel generative adversarial network (GAN)-based machine learning approach for this problem, which is weakly supervised (does not require any location annotations), was trained to retain spatial information, and can produce pixel-wise abnormality maps highlighting regions of abnormality (as opposed to bounding boxes around abnormality). Our method is based on the Wasserstein GAN framework and, to the best of our knowledge, the first application of GANs to this problem. Specifically, from an abnormal CXR as input, we generated the corresponding pseudo normal CXR image as output. The pseudo normal CXR is the "hypothetical" normal, if the same abnormal CXR were not to have any abnormalities. We surmise that the difference between the pseudo normal and the abnormal CXR highlights the pixels suspected to have pneumonia and hence is our output abnormality map. We trained our algorithm on an "unpaired" data set of abnormal and normal CXRs and did not require any location annotations such as bounding boxes/segmentations of abnormal regions. Furthermore, we incorporated additional prior knowledge/constraints into the model and showed that they help improve localization performance. We validated the model on a data set consisting of 14184 CXRs from the Radiological Society of North America pneumonia detectionchallenge. We evaluated our methods by comparing the generated abnormality maps with radiologist annotated bounding boxes using receiver operating characteristic (ROC) analysis, image similarity metrics such as normalized cross-correlation/mutual information, and abnormality detection rate.We also present visual examples of the abnormality maps, covering various scenarios of abnormality occurrence. Results demonstrate the ability to highlight regions of abnormality with the best method achieving an ROC area under the curve (AUC) of 0.77 and a detection rate of 85%.The GAN tended to perform better as prior knowledge/constraints were incorporated into themodel. We presented a novel GAN based approach for localizing pneumonia on CXRs that (1) does not require expensive hand annotated location ground truth; and (2) was trained to produce abnormality maps at the pixel level as opposed to bounding boxes. We demonstrated the efficacy of our methods via quantitative and qualitative results.

DULoc: quantitatively unmixing protein subcellular location patterns in immunofluorescence images based on deep learning features.

Knowledge of subcellular locations of proteins is of great significance for understanding their functions. The multi-label proteins that simultaneously reside in or move between more than one subcellular structure usually involve with complex cellular processes. Currently, the subcellular location annotations of proteins in most studies and databases are descriptive terms, which fail to capture the protein amount or fractions across different locations. This highly limits the understanding of complex spatial distribution and functional mechanism of multi-label proteins. Thus, quantitatively analyzing the multiplex location patterns of proteins is an urgent and challenging task. In this study, we developed a deep-learning-based pattern unmixing pipeline for protein subcellular localization (DULoc) to quantitatively estimate the fractions of proteins localizing in different subcellular compartments from immunofluorescence images. This model used a deep convolutional neural network to construct feature representations, and combined multiple nonlinear decomposing algorithms as the pattern unmixing method. Our experimental results showed that the DULoc can achieve over 0.93 correlation between estimated and true fractions on both real and synthetic datasets. In addition, we applied the DULoc method on the images in the human protein atlas database on a large scale, and showed that 70.52% of proteins can achieve consistent location orders with the database annotations. The datasets and code are available at: https://github.com/PRBioimages/DULoc. Supplementary data are available at Bioinformatics online.

AI-Based Quantification of Planned Radiation Therapy Dose to Cardiac Structures and Coronary Arteries in Patients With Breast Cancer

The purpose of this work is to develop and evaluate an automatic deep learning method for segmentation of cardiac chambers and large arteries, and localization of the 3 main coronary arteries in radiation therapy planning on computed tomography (CT). In addition, a second purpose is to determine the planned radiation therapy dose to cardiac structures for breast cancer therapy. Eighteen contrast-enhanced cardiac scans acquired with a dual-layer-detector CT scanner were included for method development. Manual reference annotations of cardiac chambers, large arteries, and coronary artery locations were made in the contrast scans and transferred to virtual noncontrast images, mimicking noncontrast-enhanced CT. In addition, 31 noncontrast-enhanced radiation therapy treatment planning CTs with corresponding dose-distribution maps of breast cancer cases were included for evaluation. For reference, cardiac chambers and large vessels were manually annotated in two 2-dimensional (2D) slices per scan (26 scans, totaling 52 slices) and in 3-dimensional (3D) scan volumes in 5 scans. Coronary artery locations were annotated on 3D imaging. The method uses an ensemble of convolutional neural networks with 2 output branches that perform 2 distinct tasks: (1) segmentation of the cardiac chambers and large arteries and (2) localization of coronary arteries. Training was performed using reference annotations and virtual noncontrast cardiac scans. Automatic segmentation of the cardiac chambers and large vessels and the coronary artery locations was evaluated in radiation therapy planning CT with Dice score (DSC) and average symmetrical surface distance (ASSD). The correlation between dosimetric parameters derived from the automatic and reference segmentations was evaluated with R2. For cardiac chambers and large arteries, median DSC was 0.76 to 0.88, and the median ASSD was 0.17 to 0.27 cm in 2D slice evaluation. 3D evaluation found a DSC of 0.87 to 0.93 and an ASSD of 0.07 to 0.10 cm. Median DSC of the coronary artery locations ranged from 0.80 to 0.91. R2 values of dosimetric parameters were 0.77 to 1.00 for the cardiac chambers and large vessels, and 0.76 to 0.95 for the coronary arteries. The developed and evaluated method can automatically obtain accurate estimates of planned radiation dose and dosimetric parameters for the cardiac chambers, large arteries, and coronary arteries.

Lung nodule malignancy classification with weakly supervised explanation generation.

Purpose: Explainable AI aims to build systems that not only give high performance but also are able to provide insights that drive the decision making. However, deriving this explanation is often dependent on fully annotated (class label and local annotation) data, which are not readily available in the medical domain. Approach: This paper addresses the above-mentioned aspects and presents an innovative approach to classifying a lung nodule in a CT volume as malignant or benign, and generating a morphologically meaningful explanation for the decision in the form of attributes such as nodule margin, sphericity, and spiculation. A deep learning architecture that is trained using a multi-phase training regime is proposed. The nodule class label (benign/malignant) is learned with full supervision and is guided by semantic attributes that are learned in a weakly supervised manner. Results: Results of an extensive evaluation of the proposed system on the LIDC-IDRI dataset show good performance compared with state-of-the-art, fully supervised methods. The proposed model is able to label nodules (after full supervision) with an accuracy of 89.1% and an area under curve of 0.91 and to provide eight attributes scores as an explanation, which is learned from a much smaller training set. The proposed system's potential to be integrated with a sub-optimal nodule detection system was also tested, and our system handled 95% of false positive or random regions in the input well by labeling them as benign, which underscores its robustness. Conclusions: The proposed approach offers a way to address computer-aided diagnosis system design under the constraint of sparse availability of fully annotated images.

Putting human behavior predictability in context

Various studies have investigated the predictability of different aspects of human behavior such as mobility patterns, social interactions, and shopping and online behaviors. However, the existing researches have been often limited to a single or to the combination of few behavioral dimensions, and they have adopted the perspective of an outside observer who is unaware of the motivations behind the specific behaviors or activities of a given individual. The key assumption of this work is that human behavior is deliberated based on an individual’s own perception of the situation that s/he is in, and that therefore it should also be studied under the same perspective. Taking inspiration from works in ubiquitous and context-aware computing, we investigate the role played by four contextual dimensions (or modalities), namely time, location, activity being carried out, and social ties, on the predictability of individuals’ behaviors, using a month of collected mobile phone sensor readings and self-reported annotations about these contextual modalities from more than two hundred study participants. Our analysis shows that any target modality (e.g. location) becomes substantially more predictable when information about the other modalities (time, activity, social ties) is made available. Multi-modality turns out to be in some sense fundamental, as some values (e.g. specific activities like “shopping”) are nearly impossible to guess correctly unless the other modalities are known. Subjectivity also has a substantial impact on predictability. A location recognition experiment suggests that subjective location annotations convey more information about activity and social ties than objective information derived from GPS measurements. We conclude the paper by analyzing how the identified contextual modalities allow to compute the diversity of personal behavior, where we show that individuals are more easily identified by rarer, rather than frequent, context annotations. These results offer support in favor of developing innovative computational models of human behaviors enriched by a characterization of the context of a given behavior.

AlfaNET: A Database of Alfalfa-Bacterial Stem Blight Protein-Protein Interactions Revealing the Molecular Features of the Disease-causing Bacteria.

Alfalfa has emerged as one of the most important forage crops, owing to its wide adaptation and high biomass production worldwide. In the last decade, the emergence of bacterial stem blight (caused by Pseudomonas syringae pv. syringae ALF3) in alfalfa has caused around 50% yield losses in the United States. Studies are being conducted to decipher the roles of the key genes and pathways regulating the disease, but due to the sparse knowledge about the infection mechanisms of Pseudomonas, the development of resistant cultivars is hampered. The database alfaNET is an attempt to assist researchers by providing comprehensive Pseudomonas proteome annotations, as well as a host–pathogen interactome tool, which predicts the interactions between host and pathogen based on orthology. alfaNET is a user-friendly and efficient tool and includes other features such as subcellular localization annotations of pathogen proteins, gene ontology (GO) annotations, network visualization, and effector protein prediction. Users can also browse and search the database using particular keywords or proteins with a specific length. Additionally, the BLAST search tool enables the user to perform a homology sequence search against the alfalfa and Pseudomonas proteomes. With the successful implementation of these attributes, alfaNET will be a beneficial resource to the research community engaged in implementing molecular strategies to mitigate the disease. alfaNET is freely available for public use at http://bioinfo.usu.edu/alfanet/.

Deep learning for guidewire detection in intravascular ultrasound images

Abstract Algorithms for automated analysis of intravascular ultrasound (IVUS) images can be disturbed by guidewires, which are often encountered when treating bifurcations in percutaneous coronary interventions. Detecting guidewires in advance can therefore help avoiding potential errors. This task is not trivial, since guidewires appear rather small compared to other relevant objects in IVUS images. We employed CNNs with additional multi-task learning as well as different guidewire-specific regularizations to enable and improve guidewire detection. In this context, we developed a network block which generates heatmaps that highlight guidewires without the need of localization annotations. The guidewire detection results reach values of 0.931 in terms of the F1-score and 0.996 in terms of area under curve (AUC). Comparing thresholded guidewire heatmaps with ground truth segmentation masks leads to a Dice score of 23.1 % and an average Hausdorff distance of 1.45 mm. Guidewire detection has proven to be a task that CNNs can handle quite well. Employing multi-task learning and guidewire-specific regularizations further improve detection results and enable generation of heatmaps that indicate the position of guidewires without actual labels.

Spectrum of Protein Location in Proteomes Captures Evolutionary Relationship Between Species

The native subcellular location (also referred to as localization or cellular compartment) of a protein is the one in which it acts most frequently; it is one aspect of protein function. Do ten eukaryotic model organisms differ in their location spectrum, i.e., the fraction of its proteome in each of seven major cellular compartments? As experimental annotations of locations remain biased and incomplete, we need prediction methods to answer this question. After systematic bias corrections, the complete but faulty prediction methods appeared to be more appropriate to compare location spectra between species than the incomplete more accurate experimental data. This work compared the location spectra for ten eukaryotes: Homo sapiens (human), Gorilla gorilla (gorilla), Pan troglodytes (chimpanzee), Mus musculus (mouse), Rattus norvegicus (rat), Drosophila melanogaster (fruit/vinegar fly), Anopheles gambiae (African malaria mosquito), Caenorhabitis elegans (nematode), Saccharomyces cerevisiae (baker’s yeast), and Schizosaccharomyces pombe (fission yeast). The two largest classes were predicted to be the nucleus and the cytoplasm together accounting for 47–62% of all proteins, while 7–21% of the proteins were predicted in the plasma membrane and 4–15% to be secreted. Overall, the predicted location spectra were largely similar. However, in detail, the differences sufficed to plot trees (UPGMA) and 2D (PCA) maps relating the ten organisms using a simple Euclidean distance in seven states (location classes). The relations based on the simple predicted location spectra captured aspects of cross-species comparisons usually revealed only by much more detailed evolutionary comparisons. Most interestingly, known phylogenetic relations were reproduced better by paralog-only than by ortholog-only trees.

Semi-supervised training of deep convolutional neural networks with heterogeneous data and few local annotations: An experiment on prostate histopathology image classification

Convolutional neural networks (CNNs) are state-of-the-art computer vision techniques for various tasks, particularly for image classification. However, there are domains where the training of classification models that generalize on several datasets is still an open challenge because of the highly heterogeneous data and the lack of large datasets with local annotations of the regions of interest, such as histopathology image analysis. Histopathology concerns the microscopic analysis of tissue specimens processed in glass slides to identify diseases such as cancer. Digital pathology concerns the acquisition, management and automatic analysis of digitized histopathology images that are large, having in the order of 100′0002 pixels per image. Digital histopathology images are highly heterogeneous due to the variability of the image acquisition procedures. Creating locally labeled regions (required for the training) is time-consuming and often expensive in the medical field, as physicians usually have to annotate the data. Despite the advances in deep learning, leveraging strongly and weakly annotated datasets to train classification models is still an unsolved problem, mainly when data are very heterogeneous. Large amounts of data are needed to create models that generalize well. This paper presents a novel approach to train CNNs that generalize to heterogeneous datasets originating from various sources and without local annotations. The data analysis pipeline targets Gleason grading on prostate images and includes two models in sequence, following a teacher/student training paradigm. The teacher model (a high-capacity neural network) automatically annotates a set of pseudo-labeled patches used to train the student model (a smaller network). The two models are trained with two different teacher/student approaches: semi-supervised learning and semi-weekly supervised learning. For each of the two approaches, three student training variants are presented. The baseline is provided by training the student model only with the strongly annotated data. Classification performance is evaluated on the student model at the patch level (using the local annotations of the Tissue Micro-Arrays Zurich dataset) and at the global level (using the TCGA-PRAD, The Cancer Genome Atlas-PRostate ADenocarcinoma, whole slide image Gleason score). The teacher/student paradigm allows the models to better generalize on both datasets, despite the inter-dataset heterogeneity and the small number of local annotations used. The classification performance is improved both at the patch-level (up to κ=0.6127±0.0133 from κ=0.5667±0.0285), at the TMA core-level (Gleason score) (up to κ=0.7645±0.0231 from κ=0.7186±0.0306) and at the WSI-level (Gleason score) (up to κ=0.4529±0.0512 from κ=0.2293±0.1350). The results show that with the teacher/student paradigm, it is possible to train models that generalize on datasets from entirely different sources, despite the inter-dataset heterogeneity and the lack of large datasets with local annotations.

Discourse Relations and Connectives in Higher Text Structure

The present article investigates possibilities and limits of local (shallow) analysis of discourse coherence with respect to the phenomena of global coherence and higher composition of texts. We study corpora annotated with local discourse relations in Czech and partly in English to try and find clues in the local annotation indicating a higher discourse structure. First, we classify patterns of subsequent or overlapping pairs of local relations, and hierarchies formed by nested local relations. Special attention is then given to relations crossing paragraph boundaries and their semantic types, and to paragraph-initial discourse connectives. In the third part, we examine situations in which annotators incline to marking a large argument (larger than one sentence) of a discourse relation even with a minimality principle annotation rule in place. Our analyses bring (i) new linguistic insights regarding coherence signals in local and higher contexts, e.g. detection and description of hierarchies of local discourse relations up to 5 levels in Czech and English, description of distribution differences in semantic types in cross-paragraph and other settings, identification of Czech connectives only typical for higher structures, or the detection of prevalence of large left-sided arguments in locally annotated data; (ii) as another type of contribution, some new reflections on methodologies of the approaches under scrutiny.

Combining weakly and strongly supervised learning improves strong supervision in Gleason pattern classification

BackgroundOne challenge to train deep convolutional neural network (CNNs) models with whole slide images (WSIs) is providing the required large number of costly, manually annotated image regions. Strategies to alleviate the scarcity of annotated data include: using transfer learning, data augmentation and training the models with less expensive image-level annotations (weakly-supervised learning). However, it is not clear how to combine the use of transfer learning in a CNN model when different data sources are available for training or how to leverage from the combination of large amounts of weakly annotated images with a set of local region annotations. This paper aims to evaluate CNN training strategies based on transfer learning to leverage the combination of weak and strong annotations in heterogeneous data sources. The trade-off between classification performance and annotation effort is explored by evaluating a CNN that learns from strong labels (region annotations) and is later fine-tuned on a dataset with less expensive weak (image-level) labels.ResultsAs expected, the model performance on strongly annotated data steadily increases as the percentage of strong annotations that are used increases, reaching a performance comparable to pathologists (kappa = 0.691 pm 0.02). Nevertheless, the performance sharply decreases when applied for the WSI classification scenario with kappa = 0.307 pm 0.133. Moreover, it only provides a lower performance regardless of the number of annotations used. The model performance increases when fine-tuning the model for the task of Gleason scoring with the weak WSI labels kappa = 0.528 pm 0.05.ConclusionCombining weak and strong supervision improves strong supervision in classification of Gleason patterns using tissue microarrays (TMA) and WSI regions. Our results contribute very good strategies for training CNN models combining few annotated data and heterogeneous data sources. The performance increases in the controlled TMA scenario with the number of annotations used to train the model. Nevertheless, the performance is hindered when the trained TMA model is applied directly to the more challenging WSI classification problem. This demonstrates that a good pre-trained model for prostate cancer TMA image classification may lead to the best downstream model if fine-tuned on the WSI target dataset. We have made available the source code repository for reproducing the experiments in the paper: https://github.com/ilmaro8/Digital_Pathology_Transfer_Learning

Schubert Winterreise Dataset

This article presents a multimodal dataset comprising various representations and annotations of Franz Schubert’s song cycle Winterreise . Schubert’s seminal work constitutes an outstanding example of the Romantic song cycle—a central genre within Western classical music. Our dataset unifies several public sources and annotations carefully created by music experts, compiled in a comprehensive and consistent way. The multimodal representations comprise the singer’s lyrics, sheet music in different machine-readable formats, and audio recordings of nine performances, two of which are freely accessible for research purposes. By means of explicit musical measure positions, we establish a temporal alignment between the different representations, thus enabling a detailed comparison across different performances and modalities. Using these alignments, we provide for the different versions various musicological annotations describing tonal and structural characteristics. This metadata comprises chord annotations in different granularities, local and global annotations of musical keys, and segmentations into structural parts. From a technical perspective, the dataset allows for evaluating algorithmic approaches to tasks such as automated music transcription, cross-modal music alignment, or tonal analysis, and for testing these algorithms’ robustness across songs, performances, and modalities. From a musicological perspective, the dataset enables the systematic study of Schubert’s musical language and style in Winterreise and the comparison of annotations regarding different annotators and granularities. Beyond the research domain, the data may serve further purposes such as the didactic preparation of Schubert’s work and its presentation to a wider public by means of an interactive multimedia experience. With this article, we provide a detailed description of the dataset, indicate its potential for computational music analysis by means of several studies, and point out possibilities for future research.

Residual multi-task learning for facial landmark localization and expression recognition

Facial landmark localization and expression recognition are two important and highly relevant topics in facial analysis. However, few works focus on using the complementary information between the two tasks to improve the performance. In this paper, we propose a residual multi-task learning framework to predict the two tasks simultaneously. Different from previous multi-task learning methods which directly train a deep multi-task network with additional branches and losses, we propose a novel residual learning module to further strengthen the linkages between the two tasks. Benefit from the proposed residual learning module, one task can learn complementary information from the other task, leading to the performance promotion. Another problem for the multi-task learning is the lack of training data with multi-task labels. For example, there is no landmark localization annotation for the two widely-used FER dataset (AffectNet and RAF), vice versa. To solve this problem, we propose an association learning method to further enhance the connection between the two tasks. Based on this connection, the dataset with single-task labels can be used in the multi-task learning. Extensive experiments are conducted on four popular datasets (i.e. 300-W, AFLW for landmark localization and AffectNet, RAF for expression recognition), demonstrating the effectiveness of the proposed algorithm.

Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China, Italy, Japan.

The recent outbreak of Coronavirus Disease 2019 (COVID-19) has led to urgent needs for reliable diagnosis and management of SARS-CoV-2 infection. The current guideline is using RT-PCR for testing. As a complimentary tool with diagnostic imaging, chest Computed Tomography (CT) has been shown to be able to reveal visual patterns characteristic for COVID-19, which has definite value at several stages during the disease course. To facilitate CT analysis, recent efforts have focused on computer-aided characterization and diagnosis with chest CT scan, which has shown promising results. However, domain shift of data across clinical data centers poses a serious challenge when deploying learning-based models. A common way to alleviate this issue is to fine-tune the model locally with the target domains local data and annotations. Unfortunately, the availability and quality of local annotations usually varies due to heterogeneity in equipment and distribution of medical resources across the globe. This impact may be pronounced in the detection of COVID-19, since the relevant patterns vary in size, shape, and texture. In this work, we attempt to find a solution for this challenge via federated and semi-supervised learning. A multi-national database consisting of 1704 scans from three countries is adopted to study the performance gap, when training a model with one dataset and applying it to another. Expert radiologists manually delineated 945 scans for COVID-19 findings. In handling the variability in both the data and annotations, a novel federated semi-supervised learning technique is proposed to fully utilize all available data (with or without annotations). Federated learning avoids the need for sensitive data-sharing, which makes it favorable for institutions and nations with strict regulatory policy on data privacy. Moreover, semi-supervision potentially reduces the annotation burden under a distributed setting. The proposed framework is shown to be effective compared to fully supervised scenarios with conventional data sharing instead of model weight sharing.

DM3Loc: multi-label mRNA subcellular localization prediction and analysis based on multi-head self-attention mechanism.

Subcellular localization of messenger RNAs (mRNAs), as a prevalent mechanism, gives precise and efficient control for the translation process. There is mounting evidence for the important roles of this process in a variety of cellular events. Computational methods for mRNA subcellular localization prediction provide a useful approach for studying mRNA functions. However, few computational methods were designed for mRNA subcellular localization prediction and their performance have room for improvement. Especially, there is still no available tool to predict for mRNAs that have multiple localization annotations. In this paper, we propose a multi-head self-attention method, DM3Loc, for multi-label mRNA subcellular localization prediction. Evaluation results show that DM3Loc outperforms existing methods and tools in general. Furthermore, DM3Loc has the interpretation ability to analyze RNA-binding protein motifs and key signals on mRNAs for subcellular localization. Our analyses found hundreds of instances of mRNA isoform-specific subcellular localizations and many significantly enriched gene functions for mRNAs in different subcellular localizations.

CTC-Based Learning of Chroma Features for Score–Audio Music Retrieval

This paper deals with a scoreaudio music retrieval task where the aim is to find relevant audio recordings of Western classical music, given a short monophonic musical theme in symbolic notation as a query. Strategies for comparing score and audio data are often based on a common mid-level representation, such as chroma features, which capture melodic and harmonic properties. Recent studies demonstrated the effectiveness of deep neural networks that learn task-specific mid-level representations. Usually, such supervised learning approaches require scoreaudio pairs where individual note events of the score are aligned to the corresponding time positions of the audio excerpt. However, in practice, it is tedious to generate such strongly aligned training pairs. As one contribution of this paper, we show how to apply the Connectionist Temporal Classification (CTC) loss in the training procedure, which only uses weakly aligned training pairs. In such a pair, only the time positions of the beginning and end of a theme occurrence are annotated in an audio recording, rather than requiring local alignment annotations. We evaluate the resulting features in our theme retrieval scenario and show that they improve the state of the art for this task. As a main result, we demonstrate that with the CTC-based training procedure using weakly annotated data, we can achieve results almost as good as with strongly annotated data. Furthermore, we assess our chroma features in depth by inspecting their temporal smoothness or granularity as an important property and by analyzing the impact of different degrees of musical complexity on the features.

FPCB Surface Defect Detection: A Decoupled Two-Stage Object Detection Framework

In the integrated circuit (IC) packaging, the surface defect detection of flexible printed circuit boards (FPCBs) is important to control the quality of IC. Although various computer vision (CV)-based object detection frameworks have been widely used in industrial surface defect detection scenarios, FPCB surface defect detection is still challenging due to non-salient defects and the similarities between diverse defects on FPCBs. To solve this problem, a decoupled two-stage object detection framework based on convolutional neural networks (CNNs) is proposed, wherein the localization task and the classification task are decoupled through two specific modules. Specifically, to effectively locate non-salient defects, a multi-hierarchical aggregation (MHA) block is proposed as a location feature (LF) enhancement module in the defect localization task. Meanwhile, to accurately classify similar defects, a locally non-local (LNL) block is presented as a SEF enhancement module in the defect classification task. What is more, an FPCB surface defect detection dataset (FPCB-DET) is built with corresponding defect category and defect location annotations. Evaluated on the FPCB-DET, the proposed framework achieves state-of-the-art (SOTA) accuracy to 94.15% mean average precision (mAP) compared with the existing surface defect detection networks. Soon, source code and dataset will be available at https://github.com/SCUTyzy/decoupled-two-stage-framework.

MULocDeep: A deep-learning framework for protein subcellular and suborganellar localization prediction with residue-level interpretation

Prediction of protein localization plays an important role in understanding protein function and mechanisms. In this paper, we propose a general deep learning-based localization prediction framework, MULocDeep, which can predict multiple localizations of a protein at both subcellular and suborganellar levels. We collected a dataset with 44 suborganellar localization annotations in 10 major subcellular compartments—the most comprehensive suborganelle localization dataset to date. We also experimentally generated an independent dataset of mitochondrial proteins in Arabidopsis thaliana cell cultures, Solanum tuberosum tubers, and Vicia faba roots and made this dataset publicly available. Evaluations using the above datasets show that overall, MULocDeep outperforms other major methods at both subcellular and suborganellar levels. Furthermore, MULocDeep assesses each amino acid’s contribution to localization, which provides insights into the mechanism of protein sorting and localization motifs. A web server can be accessed at http://mu-loc.org.