Sparse Labeling Research Articles

SpinalTRAQ: A novel volumetric cervical spinal cord atlas identifies the corticospinal tract synaptic projectome in healthy and post-stroke mice.

Descending corticospinal tract (CST) connections to the neurons of the cervical spinal cord are vital for performance of forelimb-specific fine motor skills. In rodents, CST axons are almost entirely crossed at the level of the medullary decussation. While specific contralateral axon projections have been well-characterized using anatomic and molecular approaches, the field currently lacks a cohesive imaging modality allowing rapid quantitative assessment of the entire, bilateral cervical cord projectome at the level of individual laminae and cervical levels. This is potentially important as the CST is known to undergo marked structural remodeling in development, injury, and disease. We developed SpinalTRAQ (Spinal cord Tomographic Registration and Automated Quantification), a novel volumetric cervical spinal cord atlas and machine learning-driven microscopy acquisition and analysis pipeline that uses serial two-photon tomography- images to generate unbiased, region-specific quantification of the fluorescent pixels of anterograde AAV-labeled CST pre-synaptic terminals. In adult mice, the CST synaptic projectome densely innervates the contralateral hemicord, particularly in laminae 5 and 7, with sparse, monosynaptic input to motoneurons in lamina 9. Motor pools supplying axial musculature in the upper cervical cord are bilaterally innervated. The remainder of the ipsilateral cord has sparse labeling in a distinct distribution compared to the contralateral side. Following a focal stroke of the motor cortex, there is a complete loss of descending corticospinal axons from the injured side. Consistent with prior reports of axon collateralization, the CST spinal projectome increases at four weeks post-stroke and continues to elevate by six weeks post stroke. At six weeks post-stroke, we observed striking synapse formation in the denervated hemicord from the uninjured CST in a homotopic distribution. Additionally, CST synaptic reinnervation increases in the denervated lamina 9 in nearly all motoneuron pools, exhibiting novel patterns of connectivity. Detailed level- and lamina-specific quantification of the bilateral cervical spinal cord synaptic projectome reveals previously undescribed patterns of CST connectivity in health and injury-related plasticity.

Ultrack: pushing the limits of cell tracking across biological scales.

Tracking live cells across 2D, 3D, and multi-channel time-lapse recordings is crucial for understanding tissue-scale biological processes. Despite advancements in imaging technology, achieving accurate cell tracking remains challenging, particularly in complex and crowded tissues where cell segmentation is often ambiguous. We present Ultrack, a versatile and scalable cell-tracking method that tackles this challenge by considering candidate segmentations derived from multiple algorithms and parameter sets. Ultrack employs temporal consistency to select optimal segments, ensuring robust performance even under segmentation uncertainty. We validate our method on diverse datasets, including terabyte-scale developmental time-lapses of zebrafish, fruit fly, and nematode embryos, as well as multi-color and label-free cellular imaging. We show that Ultrack achieves state-of-the-art performance on the Cell Tracking Challenge and demonstrates superior accuracy in tracking densely packed embryonic cells over extended periods. Moreover, we propose an approach to tracking validation via dual-channel sparse labeling that enables high-fidelity ground truth generation, pushing the boundaries of long-term cell tracking assessment. Our method is freely available as a Python package with Fiji and napari plugins and can be deployed in a high-performance computing environment, facilitating widespread adoption by the research community.

Haploinsufficiency of intraflagellar transport protein 172 causes autism-like behavioral phenotypes in mice through BDNF

IntroductionPrimary cilia are hair-like solitary organelles growing on most mammalian cells that play fundamental roles in embryonic patterning and organogenesis. Defective cilia often cause a suite of inherited diseases called ciliopathies with multifaceted manifestations. Intraflagellar transport (IFT), a bidirectional protein trafficking along the cilium, actively facilitates the formation and absorption of primary cilia. IFT172 is the largest component of the IFT-B complex, and its roles in Bardet-Biedl Syndrome (BBS) have been appreciated with unclear mechanisms. ObjectivesWe performed a battery of behavioral tests with Ift172 haploinsufficiency (Ift172+/-) and WT littermates. We use RNA sequencing to identify the genes and signaling pathways that are differentially expressed and enriched in the hippocampus of Ift172+/- mice. Using AAV-mediated sparse labeling, electron microscopic examination, patch clamp and local field potential recording, western blot, luciferase reporter assay, chromatin immunoprecipitation, and neuropharmacological approach, we investigated the underlying mechanisms for the aberrant phenotypes presented by Ift172+/- mice. ResultsIft172+/- mice displayed excessive self-grooming, elevated anxiety, and impaired cognition. RNA sequencing revealed enrichment of differentially expressed genes in pathways relevant to axonogenesis and synaptic plasticity, which were further confirmed by less spine density and synaptic number. Ift172+/- mice demonstrated fewer parvalbumin-expressing neurons, decreased inhibitory synaptic transmission, augmented theta oscillation, and sharp-wave ripples in the CA1 region. Moreover, Ift172 haploinsufficiency caused less BDNF production and less activated BDNF-TrkB signaling pathway through transcription factor Gli3. Application of 7,8-Dihydroxyflavone, a potent small molecular TrkB agonist, fully restored BDNF-TrkB signaling activity and abnormal behavioral phenotypes presented by Ift172+/- mice. With luciferase and chip assays, we provided further evidence that Gli3 may physically interact with BDNF promoter I and regulate BDNF expression. ConclusionsOur data suggest that Ift172 per se drives neurotrophic effects and, when defective, could cause neurodevelopmental disorders reminiscent of autism-like disorders.

Effective descriptor extraction strategies for correspondence matching in coronary angiography images

The importance of 3D reconstruction of coronary arteries using multiple coronary angiography (CAG) images has been increasingly recognized in the field of cardiovascular disease management. This process relies on the camera matrix’s optimization, needing correspondence info for identical point positions across two images. Therefore, an automatic method for determining correspondence between two CAG images is highly desirable. Despite this need, there is a paucity of research focusing on image matching in the CAG images. Additionally, standard deep learning image matching techniques often degrade due to unique features and noise in CAG images. This study aims to fill this gap by applying a deep learning-based image matching method specifically tailored for the CAG images. We have improved the structure of our point detector and redesigned loss function to better handle sparse labeling and indistinct local features specific to CAG images. Our method include changes to training loss and introduction of a multi-head descriptor structure leading to an approximate 6% improvement. We anticipate that our work will provide valuable insights into adapting techniques from general domains to more specialized ones like medical imaging and serve as an improved benchmark for future endeavors in X-ray image-based correspondence matching.

Incomplete label distribution learning via label correlation decomposition

Label distribution learning (LDL) has garnered increased attention in recent studies on label ambiguity. However, collecting complete annotations for LDL tasks is often time-consuming and labor-intensive compared to traditional learning paradigms. Therefore, designing effective incomplete LDL algorithms is crucial to broaden LDL’s application scope. In this paper, we propose a novel LDL algorithm, called Incomplete Label Distribution Learning via Label Correlation Decomposition (IncomLDL-LCD), which simultaneously learns label distributions and recovers missing description degrees of labels through label correlations. Specifically, we decompose the label correlation into sparse local label correlation and low-rank global label correlation using a soft-thresholding operator and a singular value thresholding operator, respectively. The former is utilized to capture the related label subsets necessary for reconstructing each possible label, while the latter focuses on extracting the coarse-grained semantic concepts from all labels and exploring the groupings of labels. Additionally, we develop an alternating solution with the accelerated proximal gradient descent method for optimization. Extensive experiments on 16 real-world data sets with varying degrees of missing annotations validate that our algorithm effectively handles incomplete LDL tasks and outperforms state-of-the-art algorithms.

MDGCL: Graph Contrastive Learning Framework with Multiple Graph Diffusion Methods

In recent years, some classical graph contrastive learning(GCL) frameworks have been proposed to address the problem of sparse labeling of graph data in the real world. However, in node classification tasks, there are two obvious problems with existing GCL frameworks: first, the stochastic augmentation methods they adopt lose a lot of semantic information; second, the local–local contrasting mode selected by most frameworks ignores the global semantic information of the original graph, which limits the node classification performance of these frameworks. To address the above problems, this paper proposes a novel graph contrastive learning framework, MDGCL, which introduces two graph diffusion methods, Markov and PPR, and a deterministic–stochastic data augmentation strategy while retaining the local–local contrasting mode. Specifically, before using the two stochastic augmentation methods (FeatureDrop and EdgeDrop), MDGCL first uses two deterministic augmentation methods (Markov diffusion and PPR diffusion) to perform data augmentation on the original graph to increase the semantic information, this step ensures subsequent stochastic augmentation methods do not lose too much semantic information. Meanwhile, the diffusion matrices carried by the augmented views contain global semantic information of the original graph, allowing the framework to utilize the global semantic information while retaining the local-local contrasting mode, which further enhances the node classification performance of the framework. We conduct extensive comparative experiments on multiple benchmark datasets, and the results show that MDGCL outperforms the representative baseline frameworks on node classification tasks. Among them, compared with COSTA, MDGCL’s node classification accuracy has been improved by 1.07% and 0.41% respectively on two representative datasets, Amazon-Photo and Coauthor-CS. In addition, we also conduct ablation experiments on two datasets, Cora and CiteSeer, to verify the effectiveness of each improvement work of our framework.

Pancreatic Schwann cell reprogramming supports cancer-associated neuronal remodeling.

The peripheral nervous system is a key regulator of cancer progression. In pancreatic ductal adenocarcinoma (PDAC), the sympathetic branch of the autonomic nervous system inhibits cancer development. This inhibition is associated with extensive sympathetic nerve sprouting in early pancreatic cancer precursor lesions. However, the underlying mechanisms behind this process remain unclear. This study aimed to investigate the roles of pancreatic Schwann cells in the structural plasticity of sympathetic neurons. We examined the changes in the number and distribution of Schwann cells in a transgenic mouse model of PDAC and in a model of metaplastic pancreatic lesions induced by chronic inflammation. Schwann cells proliferated and expanded simultaneously with new sympathetic nerve sprouts in metaplastic/neoplastic pancreatic lesions. Sparse genetic labeling showed that individual Schwann cells in these lesions had a more elongated and branched structure than those under physiological conditions. Schwann cells overexpressed neurotrophic factors, including glial cell-derived neurotrophic factor (GDNF). Sympathetic neurons upregulated the GDNF receptors and exhibited enhanced neurite growth in response to GDNF in vitro. Selective genetic deletion of Gdnf in Schwann cells completely blocked sympathetic nerve sprouting in metaplastic pancreatic lesions in vivo. This study demonstrated that pancreatic Schwann cells underwent adaptive reprogramming during early cancer development, supporting a protective antitumor neuronal response. These finding could help to develop new strategies to modulate cancer associated neural plasticity.

Coronary artery calcification and cardiovascular outcome as assessed by intravascular OCT and artificial intelligence.

Coronary artery calcification (CAC) is a marker of atherosclerosis and is thought to be associated with worse clinical outcomes. However, evidence from large-scale high-resolution imaging data is lacking. We proposed a novel deep learning method that can automatically identify and quantify CAC in massive intravascular OCT data trained using efficiently generated sparse labels. 1,106,291 OCT images from 1,048 patients were collected and utilized to train and evaluate the method. The Dice similarity coefficient for CAC segmentation and the accuracy for CAC classification are 0.693 and 0.932, respectively, close to human-level performance. Applying the method to 1259 ST-segment elevated myocardial infarction patients imaged with OCT, we found that patients with a greater extent and more severe calcification in the culprit vessels were significantly more likely to have major adverse cardiovascular and cerebrovascular events (MACCE) (p < 0.05), while the CAC in non-culprit vessels did not differ significantly between MACCE and non-MACCE groups.

A Semi-Automatic Magnetic Resonance Imaging Annotation Algorithm Based on Semi-Weakly Supervised Learning.

The annotation of magnetic resonance imaging (MRI) images plays an important role in deep learning-based MRI segmentation tasks. Semi-automatic annotation algorithms are helpful for improving the efficiency and reducing the difficulty of MRI image annotation. However, the existing semi-automatic annotation algorithms based on deep learning have poor pre-annotation performance in the case of insufficient segmentation labels. In this paper, we propose a semi-automatic MRI annotation algorithm based on semi-weakly supervised learning. In order to achieve a better pre-annotation performance in the case of insufficient segmentation labels, semi-supervised and weakly supervised learning were introduced, and a semi-weakly supervised learning segmentation algorithm based on sparse labels was proposed. In addition, in order to improve the contribution rate of a single segmentation label to the performance of the pre-annotation model, an iterative annotation strategy based on active learning was designed. The experimental results on public MRI datasets show that the proposed algorithm achieved an equivalent pre-annotation performance when the number of segmentation labels was much less than that of the fully supervised learning algorithm, which proves the effectiveness of the proposed algorithm.

TP-GCL: graph contrastive learning from the tensor perspective.

Graph Neural Networks (GNNs) have demonstrated significant potential as powerful tools for handling graph data in various fields. However, traditional GNNs often encounter limitations in information capture and generalization when dealing with complex and high-order graph structures. Concurrently, the sparse labeling phenomenon in graph data poses challenges in practical applications. To address these issues, we propose a novel graph contrastive learning method, TP-GCL, based on a tensor perspective. The objective is to overcome the limitations of traditional GNNs in modeling complex structures and addressing the issue of sparse labels. Firstly, we transform ordinary graphs into hypergraphs through clique expansion and employ high-order adjacency tensors to represent hypergraphs, aiming to comprehensively capture their complex structural information. Secondly, we introduce a contrastive learning framework, using the original graph as the anchor, to further explore the differences and similarities between the anchor graph and the tensorized hypergraph. This process effectively extracts crucial structural features from graph data. Experimental results demonstrate that TP-GCL achieves significant performance improvements compared to baseline methods across multiple public datasets, particularly showcasing enhanced generalization capabilities and effectiveness in handling complex graph structures and sparse labeled data.

Noise source localization using deep learning

Summary Ambient noise source localization is of great significance for estimating seismic noise source distribution, understanding source mechanisms and imaging subsurface structures. The commonly used methods for source localization, such as the matched field processing and the full-waveform inversion, are time-consuming and not applicable for time-lapse monitoring of the noise source distribution. We propose an efficient alternative of using deep learning for noise source localization. In the neural network, the input data are noise cross-correlation functions and the output are matrices containing the information of noise source distribution. It is assumed that the subsurface structure is a horizontally layered earth model and the model parameters are known. A wavefield superposition method is employed to efficiently simulate ambient noise data with quantities of local noise sources labelled as training datasets. We use a weighted binary cross-entropy loss function to address the prediction inaccuracy caused by a sparse label matrix during training. The proposed deep learning framework is validated by synthetic tests and two field data examples. The successful applications to locate an anthropogenic noise source and a carbon dioxide (CO2) degassing area demonstrate the accuracy and efficiency of the proposed deep learning method for noise source localization, which has great potential for monitoring the changes of the noise source distribution in a survey area.

WBNet: Weakly-supervised salient object detection via scribble and pseudo-background priors

Weakly supervised salient object detection (WSOD) methods endeavor to boost sparse labels to get more salient cues in various ways. Among them, an effective approach is using pseudo labels from multiple unsupervised self-learning methods, but inaccurate and inconsistent pseudo labels could ultimately lead to detection performance degradation. To tackle this problem, we develop a new multi-source WSOD framework, WBNet, that can effectively utilize pseudo-background (non-salient region) labels combined with scribble labels to obtain more accurate salient features. We first design a comprehensive salient pseudo-mask generator from multiple self-learning features. Then, we pioneer the exploration of generating salient pseudo-labels via point-prompted and box-prompted Segment Anything Models (SAM). Then, WBNet leverages a pixel-level Feature Aggregation Module (FAM), a mask-level Transformer-decoder (TFD), and an auxiliary Boundary Prediction Module (EPM) with a hybrid loss function to handle complex saliency detection tasks. Comprehensively evaluated with state-of-the-art methods on five widely used datasets, the proposed method significantly improves saliency detection performance. The code and results are publicly available at https://github.com/yiwangtz/WBNet.

Mapping inhibitory and excitatory projections to the preBӧtzinger complex

Breathing relies on rhythmic activity generated by the preBӧtzinger complex (preBӧtC), a small brainstem region. Because breathing is influenced by many different physiological, behavioral, and emotional contexts, the preBӧtC is not only highly connected with nearby respiratory related regions in the hindbrain, but also has widespread connections with other mid-and forebrain regions not generally implicated in the regulation of breathing. The specific functions for many of these preBӧtC inputs remain unclear. Although afferent pathways indiscriminately target excitatory and inhibitory neurons in the preBӧtC, we hypothesized that inputs to the preBӧtC would map to distinct brain regions based on the inhibitory or excitatory phenotype of the projecting neurons. To test this, mice that express tdTomato only when both Cre and FlpO recombinases are present (Ai65) were bred with mice that express Cre in inhibitory neurons ( VgatCre). Adult offspring ( VgatCre; Ai65) received a unilateral preBӧtC injection of a retrograde AAV that expresses FlpO recombinase (AAVrg-Flpo), thereby specifically activating tdTomato expression in any inhibitory neurons that project directly to the preBӧtC. Whole-brain imaging revealed many labeled neurons along the ipsilateral ventral respiratory column (VRC), far rostral and caudal of the injection site, extending along the intermediate reticular zone (IRt) from the VII nucleus through the caudal end of the ventral respiratory group (VRG). Interestingly, this well-defined and continuous column of labeled neurons was confined to the ventral IRt at its caudal end but moved dorsally at its rostral end. Only a small number of neurons were labeled along the contralateral VRC. Sparse labeling was present in the medial parabrachial, Kolliker-Fuse, and reticular regions of the pons. Many projecting neurons were observed across subregions of the amygdala, which were almost exclusively ipsilateral. The hypothalamus contained diffuse projecting neurons along its rostral-caudal extent that were also mostly ipsilateral. However, denser labeling was present in a distinctly curved cluster of the lateral hypothalamus, which was observed on both the ipsilateral and contralateral side. Consistent with the preBӧtC having distinct inhibitory and excitatory input pathways, we did not observe significant labelling in other regions with known projections to the preBotC identified previously using non-cell-type specific tracing strategies. These included cortex, lateral parabrachial, NTS, and RVM. This will be tested further by replicating this experiment with targeting of glutamatergic ( Vglut2) neurons. Defining these cell-type-specific inputs to the preBӧtC will provide an essential foundation for future functional studies to determine how these circuits may regulate breathing. R01 HL1660317 (Baertsch) R00 HL145004 (Baertsch). This is the full abstract presented at the American Physiology Summit 2024 meeting and is only available in HTML format. There are no additional versions or additional content available for this abstract. Physiology was not involved in the peer review process.

Multi-task self-supervised time-series representation learning

Time-series representation learning is crucial for extracting meaningful representations from time-series data with temporal dynamics and sparse labels. Contrastive learning, a powerful technique for exploiting the inherent data patterns, has been applied to explore the diverse consistencies in time-series data, achieved through careful selection of contrastive pairs and design of appropriate loss. Encouraging such consistency is essential for acquiring comprehensive representations of time-series data. In this paper, we propose a new framework for time-series representation learning that combines the advantages of contextual, temporal, and transformation consistencies. This framework enables the network to learn general representations suitable for different tasks and domains. First, positive and negative pairs are generated to establish a multi-task learning setup. Then, contrastive losses are formulated to explore contextual, temporal, and transformation consistencies, which are jointly optimized to learn general time-series representations. In addition, we investigate an uncertainty weighting approach to enhance the effectiveness of multi-task learning. To evaluate the performance of our framework, we conduct experiments on three downstream tasks: time-series classification, forecasting, and anomaly detection. The experimental results demonstrate the superior performance of our framework compared to benchmark models across different tasks. Furthermore, our framework shows efficiency in cross-domain transfer learning scenarios.

Generative Adversarial Network and Mutual-Point Learning Algorithm for Few-Shot Open-Set Classification of Hyperspectral Images

Existing approaches addressing the few-shot open-set recognition (FSOSR) challenge in hyperspectral images (HSIs) often encounter limitations stemming from sparse labels, restricted category numbers, and low openness. These limitations compromise stability and adaptability. In response, an open-set HSI classification algorithm based on data wandering (DW) is introduced in this research. Firstly, a K-class classifier suitable for a closed set is trained, and its internal encoder is leveraged to extract features and estimate the distribution of known categories. Subsequently, the classifier is fine-tuned based on feature distribution. To address the scarcity of samples, a sample density constraint based on the generative adversarial network (GAN) is employed to generate synthetic samples near the decision boundary. Simultaneously, a mutual-point learning method is incorporated to widen the class distance between known and unknown categories. In addition, a dynamic threshold method based on DW is devised to enhance the open-set performance. By categorizing drifting synthetic samples into known and unknown classes and retraining them together with the known samples, the closed-set classifier is optimized, and a (K + 1)-class open-set classifier is trained. The experimental results in this research demonstrate the superior FSOSR performance of the proposed method across three benchmark HSI datasets.

Advancing urban traffic accident forecasting through sparse spatio-temporal dynamic learning

Traffic accidents have emerged as one of the most public health safety matters, raising concerns from both the public and urban administrators. The ability to accurately predict traffic accident not only supports the governmental decision-making in advance but also enhances public confidence in safety measures. However, the efficacy of traditional spatio-temporal prediction models are compromised by the skewed distributions and sparse labeling of accident data. To this end, we propose a Sparse Spatio-Temporal Dynamic Hypergraph Learning (SST-DHL) framework that captures higher-order dependencies in sparse traffic accidents by combining hypergraph learning and self-supervised learning. The SST-DHL model incorporates a multi-view spatiotemporal convolution block to capture local correlations and semantics of traffic accidents, a cross-regional dynamic hypergraph learning model to identify global spatiotemporal dependencies, and a two-supervised self-learning paradigm to capture both local and global spatiotemporal patterns. Through experimentation on New York City and London accident datasets, we demonstrate that our proposed SST-DHL exhibits significant improvements compared to optimal baseline models at different sparsity levels. Additionally, it offers enhanced interpretability of results by elucidating complex spatio-temporal dependencies among various traffic accident instances. Our study demonstrates the effectiveness of the SST-DHL framework in accurately predicting traffic accidents, thereby enhancing public safety and trust.

353 Eph/Ephrin Signaling Influences Innervation of Outer Hair Cells in Cochlea

OBJECTIVES/GOALS: 48,000,000 people in the U.S. have hearing loss, negatively impacting quality of life and work. Unveiling axon guidance for auditory type II spiral ganglia neurons (SGNs) will aid development of new therapies. I study the role of Eph/Ephrin and planar cell polarity (PCP) signaling during type II SGN turning and outer hair cell (OHC) innervation. METHODS/STUDY POPULATION: This quantitative study was conducted on Efna3 and Vangl2 null mice possessing Neurog1CreERT2 and R26RtdTomato mutations. Spontaneous Cre activity within the Neurogenin1CreERT2 line causes recombination and expression of fluorescent Rosa26 Reporter (R26R)tdTomatoin a restricted number of SGNs, including type IIs. Together, these lines permit SGN sparse labeling. Immunostaining and confocal imaging were used to analyze dsRed in Efna3 and Vangl2 mice and quantify type II SGN turning. In combination, Imaris 3D renderings were used to quantify type II SGN turning, branching, navigation features and temporal effects of EPHRIN-A3-Fc on type IIs via cochlear cultures (a gain-of-function manipulation). For both sexes, 5-6 cochleae per genotype were analyzed and compared by t-test to wildtype (WT) controls. RESULTS/ANTICIPATED RESULTS: Efna3 nulls showed a small rise in type II SGNs incorrectly turning toward the apex at an error rate of 16.0% compared to WTs (n=6; p=0.05). P0 Efna3 nulls had reduced branch number compared to WTs, 4.1 and 7.2, respectively (n=129; p=&lt;0.0001), suggesting EPHRIN-A3 acts as a positive growth cue. In cochlear cultures, EPHRIN-A3-Fc led to type II SGN collapse at E15.5, indicating a repulsive function. However, at P0, EPHRIN-A3-Fc treatment led to type II SGNs with elevated branch numbers compared to Control-Fc treatment: 18.1 and 11.4, respectively (n=116; p=&lt;0.0001). This indicates a positive growth function. At E16.5, EPHRIN-A3 protein immunoreactivity on Deiters’ and pillar cells appears reduced in Vangl2 nulls compared to WT cochleae, suggesting that EPHRIN-A3 acts downstream of PCP signaling. DISCUSSION/SIGNIFICANCE: Results suggest that Eph/Ephrin signaling acts downstream of PCP signaling to mediate type II SGN guidance and EPHRIN-A3 switches its mode of activation. The clinical implications of these findings are that therapeutics targeting EPHRIN-A3 and/or VANGL2 in their given pathways could stimulate new OHC innervation following auditory damage.

Fully Data-Driven Pseudo Label Estimation for Pointly-Supervised Panoptic Segmentation

The core of pointly-supervised panoptic segmentation is estimating accurate dense pseudo labels from sparse point labels to train the panoptic head. Previous works generate pseudo labels mainly based on hand-crafted rules, such as connecting multiple points into polygon masks, or assigning the label information of labeled pixels to unlabeled pixels based on the artificially defined traversing distance. The accuracy of pseudo labels is limited by the quality of the hand-crafted rules (polygon masks are rough at object contour regions, and the traversing distance error will result in wrong pseudo labels). To overcome the limitation of hand-crafted rules, we estimate pseudo labels with a fully data-driven pseudo label branch, which is optimized by point labels end-to-end and predicts more accurate pseudo labels than previous methods. We also train an auxiliary semantic branch with point labels, it assists the training of the pseudo label branch by transferring semantic segmentation knowledge through shared parameters. Experiments on Pascal VOC and MS COCO demonstrate that our approach is effective and shows state-of-the-art performance compared with related works. Codes are available at https://github.com/BraveGroup/FDD.

Taming the Sigmoid Bottleneck: Provably Argmaxable Sparse Multi-Label Classification

Sigmoid output layers are widely used in multi-label classification (MLC) tasks, in which multiple labels can be assigned to any input. In many practical MLC tasks, the number of possible labels is in the thousands, often exceeding the number of input features and resulting in a low-rank output layer. In multi-class classification, it is known that such a low-rank output layer is a bottleneck that can result in unargmaxable classes: classes which cannot be predicted for any input. In this paper, we show that for MLC tasks, the analogous sigmoid bottleneck results in exponentially many unargmaxable label combinations. We explain how to detect these unargmaxable outputs and demonstrate their presence in three widely used MLC datasets. We then show that they can be prevented in practice by introducing a Discrete Fourier Transform (DFT) output layer, which guarantees that all sparse label combinations with up to k active labels are argmaxable. Our DFT layer trains faster and is more parameter efficient, matching the F1@k score of a sigmoid layer while using up to 50% fewer trainable parameters. Our code is publicly available at https://github.com/andreasgrv/sigmoid-bottleneck.

The superior colliculus projection upon the macaque inferior olive.

Saccade accommodation is a productive model for exploring the role of the cerebellum in behavioral plasticity. In this model, the target is moved during the saccade, gradually inducing a change in the saccade vector as the animal adapts. The climbing fiber pathway from the inferior olive provides a visual error signal generated by the superior colliculus that is believed to be crucial for cerebellar adaptation. However, the primate tecto-olivary pathway has only been explored using large injections of the central portion of the superior colliculus. To provide a more detailed picture, we have made injections of anterograde tracers into various regions of the macaque superior colliculus. As shown previously, large central injections primarily label a dense terminal field within the C subdivision at caudal end of the contralateral medial inferior olive. Several, previously unobserved, sites of sparse terminal labeling were noted: bilaterally in the dorsal cap of Kooy and ipsilaterally in theC subdivision of the medial inferior olive. Small, physiologically directed, injections into the rostral, small saccade portion of the superior colliculus produced terminal fields in the same regions of the medial inferior olive, but with decreased density. Small injections of the caudal superior colliculus, where large amplitude gaze changes are encoded, again labeled a terminal field located in the same areas. The lack of a topographic pattern within the main tecto-olivary projection suggests that either the precise vector of the visual error is not transmitted to the vermis, or that encoding of this error is via non-topographic means.