Labeling Density Research Articles

Combining dense and sparse labeling in optical DNA mapping.

Optical DNA mapping (ODM) is based on fluorescent labeling, stretching and imaging of single DNA molecules to obtain sequence-specific fluorescence profiles, DNA barcodes. These barcodes can be mapped to theoretical counterparts obtained from DNA reference sequences, which in turn allow for DNA identification in complex samples and for detecting structural changes in individual DNA molecules. There are several types of DNA labeling schemes for ODM and for each labeling type one or several types of match scoring methods are used. By combining the information from multiple labeling schemes one can potentially improve mapping confidence; however, combining match scores from different labeling assays has not been implemented yet. In this study, we introduce two theoretical methods for dealing with analysis of DNA molecules with multiple label types. In our first method, we convert the alignment scores, given as output from the different assays, into p-values using carefully crafted null models. We then combine the p-values for different label types using standard methods to obtain a combined match score and an associated combined p-value. In the second method, we use a block bootstrap approach to check for the uniqueness of a match to a database for all barcodes matching with a combined p-value below a predefined threshold. For obtaining experimental dual-labeled DNA barcodes, we introduce a novel assay where we cut plasmid DNA molecules from bacteria with restriction enzymes and the cut sites serve as sequence-specific markers, which together with barcodes obtained using the established competitive binding labeling method, form a dual-labeled barcode. All experimental data in this study originates from this assay, but we point out that our theoretical framework can be used to combine data from all kinds of available optical DNA mapping assays. We test our multiple labeling frameworks on barcodes from two different plasmids and synthetically generated barcodes (combined competitive-binding- and nick-labeling). It is demonstrated that by simultaneously using the information from all label types, we can substantially increase the significance when we match experimental barcodes to a database consisting of theoretical barcodes for all sequenced plasmids.

A Hierarchical Feature Extraction Network for Fast Scene Segmentation.

Semantic segmentation is one of the most active research topics in computer vision with the goal to assign dense semantic labels for all pixels in a given image. In this paper, we introduce HFEN (Hierarchical Feature Extraction Network), a lightweight network to reach a balance between inference speed and segmentation accuracy. Our architecture is based on an encoder-decoder framework. The input images are down-sampled through an efficient encoder to extract multi-layer features. Then the extracted features are fused via a decoder, where the global contextual information and spatial information are aggregated for final segmentations with real-time performance. Extensive experiments have been conducted on two standard benchmarks, Cityscapes and Camvid, where our network achieved superior performance on NVIDIA 2080Ti.

Automatic vertebrae localization and segmentation in CT with a two-stage Dense-U-Net

Automatic vertebrae localization and segmentation in computed tomography (CT) are fundamental for spinal image analysis and spine surgery with computer-assisted surgery systems. But they remain challenging due to high variation in spinal anatomy among patients. In this paper, we proposed a deep-learning approach for automatic CT vertebrae localization and segmentation with a two-stage Dense-U-Net. The first stage used a 2D-Dense-U-Net to localize vertebrae by detecting the vertebrae centroids with dense labels and 2D slices. The second stage segmented the specific vertebra within a region-of-interest identified based on the centroid using 3D-Dense-U-Net. Finally, each segmented vertebra was merged into a complete spine and resampled to original resolution. We evaluated our method on the dataset from the CSI 2014 Workshop with 6 metrics: location error (1.69 ± 0.78 mm), detection rate (100%) for vertebrae localization; the dice coefficient (0.953 ± 0.014), intersection over union (0.911 ± 0.025), Hausdorff distance (4.013 ± 2.128 mm), pixel accuracy (0.998 ± 0.001) for vertebrae segmentation. The experimental results demonstrated the efficiency of the proposed method. Furthermore, evaluation on the dataset from the xVertSeg challenge with location error (4.12 ± 2.31), detection rate (100%), dice coefficient (0.877 ± 0.035) shows the generalizability of our method. In summary, our solution localized the vertebrae successfully by detecting the centroids of vertebrae and implemented instance segmentation of vertebrae in the whole spine.

Classifying 3-D Models of Coral Reefs Using Structure-From-Motion and Multi-View Semantic Segmentation

Benthic quadrat surveys using 2-D images are one of the most common methods of quantifying the composition of coral reef communities, but they and other methods fail to assess changes in species composition as a 3-dimensional system, arguably one of the most important attributes in foundational systems. Structure-from-motion (SfM) algorithms that utilize images collected from various viewpoints to form an accurate 3-D model have become more common among ecologists in recent years. However, there exist few efficient methods that can classify portions of the 3-D model to specific ecological functional groups. This lack of granularity makes it more difficult to identify the class category responsible for changes in the structure of coral reef communities. We present a novel method that can efficiently provide semantic labels of functional groups to 3-D reconstructed models created from commonly used SfM software (i.e., Agisoft Metashape) using fully convolutional networks (FCNs). Unlike other methods, ours involves creating dense labels for each of the images used in the 3-D reconstruction and then reusing the projection matrices created during the SfM process to project semantic labels onto either the point cloud or mesh to create fully classified versions. When quantitatively validating the classification results we found that this method is capable of accurately projecting semantic labels from image-space to model-space with scores as high as 91% pixel accuracy. Furthermore, because each image only needs to be provided with a single set of dense labels this method scales linearly making it useful for large areas or high resolution-models. Although SfM has become widely adopted by ecologists, deep learning presents a steep learning curve for many. To ensure repeatability and ease-of-use, we provide a comprehensive workflow with detailed instructions and open-sourced the programming code to assist others in replicating our methodology. Our method will allow researchers to assess precise changes in 3-D community composition of reef habitats in an entirely novel way, providing more insight into changes in ecological paradigms, such as those that occur during coral-algae shifts.

Phosphine-Catalyzed Annulations Based on [3+3] and [3+2] Trapping of Ketene Intermediates with Thioamides.

With the aim of developing novel annulations via ketene intermediates, allenyl imide and alkynoates bearing good leaving groups are used for their function in a tandem conjugate addition-elimination reaction (SN2' type) promoted by nucleophilic phosphine catalysts. By utilizing thioamides as 1S,3N-bis-nucleophiles, [3+3] and [3+2] annulations have been established to allow rapid access to 1,3-thiazin-4-ones and 5-alkenyl thiazolones in high yields, respectively. Furthermore, the possible reaction mechanisms are proposed on the basis of deuterium labeling experiments and density functional theory calculations.

Dense prediction of label noise for learning building extraction from aerial drone imagery

ABSTRACT Label noise is a commonly encountered problem in learning building extraction tasks; its presence can reduce performance and increase learning complexity. This is especially true for cases where high resolution aerial drone imagery is used, as the labels may not perfectly correspond/align with the actual objects in the imagery. In general machine learning and computer vision context, labels refer to the associated class of data, and in remote sensing-based building extraction refer to pixel-level classes. Dense label noise in building extraction tasks has rarely been formalized and assessed. We formulate a taxonomy of label noise models for building extraction tasks, which incorporates both pixel-wise and dense models. While learning dense prediction under label noise, the differences between the ground truth clean label and observed noisy label can be encoded by error matrices indicating locations and type of noisy pixel-level labels. In this work, we explicitly learn to approximate error matrices for improving building extraction performance; essentially, learning dense prediction of label noise as a subtask of a larger building extraction task. We propose two new model frameworks for learning building extraction under dense real-world label noise, and consequently two new network architectures, which approximate the error matrices as intermediate predictions. The first model learns the general error matrix as an intermediate step and the second model learns the false positive and false-negative error matrices independently, as intermediate steps. Approximating intermediate error matrices can generate label noise saliency maps, for identifying labels having higher chances of being mis-labelled. We have used ultra-high-resolution aerial images, noisy observed labels from OpenStreetMap, and clean labels obtained after careful annotation by the authors. When compared to the baseline model trained and tested using clean labels, our intermediate false positive-false negative error matrix model provides Intersection-Over-Union gain of 2.74% and F1-score gain of 1.75% on the independent test set. Furthermore, our proposed models provide much higher recall than currently used deep learning models for building extraction, while providing comparable precision. We show that intermediate false positive-false negative error matrix approximation can improve performance under label noise.

Robotic picking in dense clutter via domain invariant learning from synthetic dense cluttered rendering

Robotic picking of diverse range of novel objects is a great challenge in dense clutter, in which objects are stacked together tightly. However, collecting large-scale dataset with dense grasp labels is extremely time-consuming, and there is a huge gap between synthetic and real images. In this paper, we explore suction based grasping from synthetic multi object rendering. To avoid tedious human labeling, we present a pipeline to model stacked objects in simulation and generate photorealistic rendering RGB-D images with dense suction point labels. To reduce simulation-to-reality gap from synthetic images to low-quality RGB-D camera, we propose a novel domain-invariant Suction Quality Neural Network (diSQNN) by training on labeled synthetic dataset with unlabeled real dataset. Specifically, diSQNN fuses photorealistic color feature and adversarial depth feature, and uses a domain discriminator on depth extractor to align depth feature distribution from synthetic and real images. We evaluate our proposed method by comparing with other baseline and suction detection method. The results demonstrate the effectiveness of our synthetic dense cluttered rendering. And through feature alignment, our domain invariant learning method can learn grasp related features, while ignoring domain related disturbing features, which maintains a high transfer performance on real RGB-D images. On a physical robot with vacuum-based gripper, the proposed method achieves average picking success rate of 91% and 88% for known objects and novel objects in a tote without using any manual labels.

Exploiting Complex Fluorophore Interactions to Monitor Virus Capsid Disassembly.

Supramolecular protein complexes are the corner stone of biological processes; they are essential for many biological functions. Unraveling the interactions responsible for the (dis)assembly of these complexes is required to understand nature and to exploit such systems in future applications. Virus capsids are well-defined assemblies of hundreds of proteins and form the outer shell of non-enveloped viruses. Due to their potential as a drug carriers or nano-reactors and the need for virus inactivation strategies, assessing the intactness of virus capsids is of great interest. Current methods to evaluate the (dis)assembly of these protein assemblies are experimentally demanding in terms of instrumentation, expertise and time. Here we investigate a new strategy to monitor the disassembly of fluorescently labeled virus capsids. To monitor surfactant-induced capsid disassembly, we exploit the complex photophysical interplay between multiple fluorophores conjugated to capsid proteins. The disassembly of the capsid changes the photophysical interactions between the fluorophores, and this can be spectrally monitored. The presented data show that this low complexity method can be used to study and monitor the disassembly of supramolecular protein complexes like virus capsids. However, the range of labeling densities that is suitable for this assay is surprisingly narrow.

Deep learning enables fast and dense single-molecule localization with high accuracy.

Single-molecule localization microscopy (SMLM) has had remarkable success in imaging cellular structures with nanometer resolution, but standard analysis algorithms necessitate the activation of only single isolated emitters which limits imaging speed and labeling density. Here, we overcome this major limitation using deep learning. We developed DECODE, a computational tool that can localize single emitters at high density in 3D with highest accuracy for a large range of imaging modalities and conditions. In a public software benchmark competition, it outperformed all other fitters on 12 out of 12 data-sets when comparing both detection accuracy and localization error, often by a substantial margin. DECODE allowed us to take fast dynamic live-cell SMLM data with reduced light exposure and to image microtubules at ultra-high labeling density. Packaged for simple installation and use, DECODE will enable many labs to reduce imaging times and increase localization density in SMLM.

Multi-task fully convolutional network for tree species mapping in dense forests using small training hyperspectral data

This work proposes a multi-task fully convolutional architecture for tree species mapping in dense forests from sparse and scarce polygon-level annotations using hyperspectral UAV-borne data. Our model implements a partial loss function that enables dense tree semantic labeling outcomes from non-dense training samples, and a distance regression complementary task that enforces tree crown boundary constraints and substantially improves the model performance. Our multi-task architecture uses a shared backbone network that learns common representations for both tasks and two task-specific decoders, one for the semantic segmentation output and one for the distance map regression. We report that introducing the complementary task boosts the semantic segmentation performance compared to the single-task counterpart in up to 11% reaching an average user’s accuracy of 88.63% and an average producer’s accuracy of 88.59%, achieving state-of-art performance for tree species classification in tropical forests.

Weakly-supervised contrastive learning in path manifold for Monte Carlo image reconstruction

Image-space auxiliary features such as surface normal have significantly contributed to the recent success of Monte Carlo (MC) reconstruction networks. However, path-space features, another essential piece of light propagation, have not yet been sufficiently explored. Due to the curse of dimensionality, information flow between a regression loss and high-dimensional path-space features is sparse, leading to difficult training and inefficient usage of path-space features in a typical reconstruction framework. This paper introduces a contrastive manifold learning framework to utilize path-space features effectively. The proposed framework employs weakly-supervised learning that converts reference pixel colors to dense pseudo labels for light paths. A convolutional path-embedding network then induces a low-dimensional manifold of paths by iteratively clustering intra-class embeddings, while discriminating inter-class embeddings using gradient descent. The proposed framework facilitates path-space exploration of reconstruction networks by extracting low-dimensional yet meaningful embeddings within the features. We apply our framework to the recent image- and sample-space models and demonstrate considerable improvements, especially on the sample space. The source code is available at https://github.com/Mephisto405/WCMC.

Live Fluorescence Imaging of F-Actin Organization in Chick Whole Embryo Cultures Using SiR-Actin.

Morphogenesis is a continuous process of pattern formation so complex that it requires in vivo monitoring for better understanding. Changes in tissue shape are initiated at the cellular level, where dynamic intracellular F-actin networks determine the shape and motility of cells, influence differentiation and cytokinesis and mediate mechanical signaling. Here, we stain F-actin with the fluorogenic probe SiR-actin for live fluorescence imaging of whole chick embryos. We found that 50 nM SiR-actin in the culture medium is a safe and effective concentration for this purpose, as it provides high labeling density without inducing morphological malformations.

Antibody Self-Assembly Maximizes Cytoplasmic Immunostaining Accuracy of Compact Quantum Dots

Antibody conjugates of quantum dots (QDs) are expected to transform immunofluorescence staining by expanding multiplexed analysis and improving target quantification. Recently, a new generation of small QDs coated with multidentate polymers has improved QD labeling density in diverse biospecimens, but new challenges prevent their routine use. In particular, these QDs exhibit nonspecific binding to fixed cell nuclei and their antibody conjugates have random attachment orientations. This report describes four high-efficiency chemical approaches to conjugate antibodies to compact QDs. Methods include click chemistry and self-assembly through polyhistidine coordination, both with and without adaptor proteins that directionally orient antibodies. Specific and nonspecific labeling are independently analyzed after application of diverse blocking agent classes, and a new assay is developed to quantitatively measure intracellular labeling density based on microtubule stain connectivity. Results show that protein conjugation to the QD surface is required to simultaneously eliminate nonspecific binding and maintain antigen specificity. Of the four conjugation schemes, polyhistidine-based coordination of adaptor proteins with antibody self-assembly yields the highest intracellular staining density and the simplest conjugation procedure. Therefore, antibody and adaptor protein orientation, in addition to blocking optimization, are important determinants of labeling outcomes, insights that can inform translational development of these more compact nanomaterials.

Classifier aided training for semantic segmentation

Semantic segmentation is a prominent problem in scene understanding expressed as a dense labeling task with deep learning models being one of the main methods to solve it. Traditional training algorithms for semantic segmentation models produce less than satisfactory results when not combined with post-processing techniques such as CRFs. In this paper, we propose a method to train segmentation models using an approach which utilizes classification information in the training process of the segmentation network. Our method employs the use of classification network that detects the presence of classes in the segmented output. These class scores are then used to train the segmentation model. This method is motivated by the fact that by conditioning the training of the segmentation model with these scores, higher order features can be captured. Our experiments show significantly improved performance of the segmentation model on the CamVid and CityScapes datasets with no additional post processing.

The Effect of Vitamin D Metabolic Status and Endometrial Immune Patterns on the Outcomes of ART Programs

The objective of this study was to assess the possibility of predicting the outcomes of in vitro fertilization (IVF) and IVF/ICSI (intracytoplasmic injection [transfer] of sperm into eggs), protocols based on the serum vitamin D content and endometrial expression of vitamin D receptor (VDR), CD4+T cells, and CD8+T cells. Methods and Results: The results of IVF and IVF/ICSI programs were analyzed in 147 patients of older reproductive age (36-44 years of age) with tubal-peritoneal factor of infertility. Depending on the outcomes of ART programs, 2 groups were formed. Group 1 included 37 women with the onset of pregnancy; Group 2 included 110 women with negative outcomes of ART programs. The level of 25(OH)D in the blood serum and follicular fluid (FF) was assessed. The sampling of the material (blood and FF) was carried out on the day of ovarian puncture, the 25(OH)D content – at least three days after the last administration of the drug. Anti-Müllerian hormone (AMH) was determined using AMH reagent (AMH Gen II ELISA) and the Beckman Coulter Access® 2 Immunoassay System (Beckman Coulter, Inc. USA). Morphological examination of the endometrium was performed after biopsy sampling by pipelle biopsy on Day 7 after confirmed ovulation in the cycle preceding ART. The endometrial expression of VDR, CD4+Tcells, and CD8+Tcells was evaluated in 68 women of the sample: 18 – with pregnancy, 50 – with negative outcomes in ART programs. Vitamin D deficiency (<20ng/ml) in peripheral blood was detected only in Group 2. Low levels of vitamin D (20-<30ng/ml) in women of this group were four times more frequent (p=0.008) than in Group 1. Women with the onset of clinical pregnancy were more often distinguished by the optimal content of vitamin D in biological fluids – almost twice as much as those with ineffective attempts to treat infertility (P=0.009 for peripheral blood). The mean value of the 25(OH)D content in the blood serum in both groups did not differ; in FF it was significantly higher in Group 1 than in Group 2 (p = 0.003). A strong correlation was revealed between the content of 25(OH)D in the blood serum and FF (r=0.67, P=0.003 in both groups), a weak correlation – in cases where it was impossible to sample oocytes and/or embryos (r=0.48, P=0.06). The VDR expression in stromal epithelium was higher in Group 2 than in Group 1 (P=0,016). It is calculated that a 1% decrease in VDR expression in the stroma increases the chance of a favorable outcome by 1.35 times.The density of labels of CD4+T cells and CD8+T cells in the endometrial stroma did not differ significantly between groups. The expression of CD4+T cells and CD8+T cells in endometrial samples taken from the same infertile women in different phases of the menstrual cycle showed no statistically significant differences. Conclusion: Morphofunctional peculiarities of the endometrium of infertile women are the basis for predicting the outcomes of ART programs and preparing for pregnancy.

The distribution of phosphatidylinositol 4,5-bisphosphate in the budding yeast plasma membrane.

Phosphatidylinositol 4,5-bisphosphate (PtdIns(4,5)P2) is generated through phosphorylation of phosphatidylinositol 4-phosphate (PtdIns(4)P) by Mss4p, the only PtdIns phosphate 5-kinase in yeast cells. PtdIns(4,5)P2 is involved in various kinds of yeast functions. PtdIns(4)P is not only the immediate precursor of PtdIns(4,5)P2, but also an essential signaling molecule in the plasma membrane, Golgi, and endosomal system. To analyze the distribution of PtdIns(4,5)P2 and PtdIns(4)P in the yeast plasma membrane at a nanoscale level, we employed a freeze-fracture electron microscopy (EM) method that physically immobilizes lipid molecules in situ. It has been reported that the plasma membrane of budding yeast can be divided into three distinct areas: furrowed, hexagonal, and undifferentiated flat. Previously, using the freeze-fracture EM method, we determined that PtdIns(4)P is localized in the undifferentiated flat area, avoiding the furrowed and hexagonal areas of the plasma membrane. In the present study, we found that PtdIns(4,5)P2 was localized in the cytoplasmic leaflet of the plasma membrane, and concentrated in the furrowed area. There are three types of PtdIns 4-kinases which are encoded by stt4, pik1, and lsb6. The labeling density of PtdIns(4)P in the plasma membrane significantly decreased in both pik1ts and stt4ts mutants. However, the labeling densities of PtdIns(4,5)P2 in the plasma membrane of both the pik1ts and stt4ts mutants were comparable to that of the wild type yeast. These results suggest that PtdIns(4)P produced by either Pik1p or Stt4p is immediately phosphorylated by Mss4p and converted to PtdIns(4,5)P2 at the plasma membrane.

Exploiting Learnable Joint Groups for Hand Pose Estimation

In this paper, we propose to estimate 3D hand pose by recovering the 3D coordinates of joints in a group-wise manner, where less-related joints are automatically categorized into different groups and exhibit different features. This is different from the previous methods where all the joints are considered holistically and share the same feature. The benefits of our method are illustrated by the principle of multi-task learning (MTL), i.e., by separating less-related joints into different groups (as different tasks), our method learns different features for each of them, therefore efficiently avoids the negative transfer (among less related tasks/groups of joints). The key of our method is a novel binary selector that automatically selects related joints into the same group. We implement such a selector with binary values stochastically sampled from a Concretedistribution, which is constructed using Gumbel softmax on trainable parameters. This enables us to preserve the differentiable property of the whole network. We further exploit features from those less-related groups by carrying out an additional feature fusing scheme among them, to learn more discriminative features. This is realized by implementing multiple 1x1 convolutions on the concatenated features, where each joint group contains a unique 1x1convolution for feature fusion. The detailed ablation analysis and the extensive experiments on several benchmark datasets demonstrate the promising performance of the proposed method over the state-of-the-art (SOTA) methods. Besides, our method achieves top-1 among all the methods that do not exploit the dense 3D shape labels on the most recently released FreiHAND competition at the submission date. The source code and models are available at https://github.com/moranli-aca/LearnableGroups-Hand.

A Comparison of Feature Density for Large Scale Online Maps

Large scale maps as provided by Google, Microsoft Bing, and Mapbox among others provide users an important source of information for local environments. Comparing maps from these services helps to evaluate both the quality of the underlying spatial data and the rendering process. A comparison procedure is used that generates large scale map pairs at random locations using the Application Programmer Interface (API) for three different mapping services. The quality of each representation is then evaluated for feature and label density. The comparison is done for three different continents. For North America, it was found that maps from Google had consistently higher feature and label density than those from Microsoft Bing and Mapbox. Google Maps also held an advantage in Europe. Maps from Microsoft Bing were more detailed in Sub-Saharan Africa in comparison to both Google Maps and Mapbox. Maps from Mapbox, that relies exclusively on data from OpenStreetMap, had the lowest feature and label density for all three areas.

A New Pooling Approach Based on Zeckendorf's Theorem for Texture Transfer Information.

The pooling layer is at the heart of every convolutional neural network (CNN) contributing to the invariance of data variation. This paper proposes a pooling method based on Zeckendorf’s number series. The maximum pooling layers are replaced with Z pooling layer, which capture texels from input images, convolution layers, etc. It is shown that Z pooling properties are better adapted to segmentation tasks than other pooling functions. The method was evaluated on a traditional image segmentation task and on a dense labeling task carried out with a series of deep learning architectures in which the usual maximum pooling layers were altered to use the proposed pooling mechanism. Not only does it arbitrarily increase the receptive field in a parameterless fashion but it can better tolerate rotations since the pooling layers are independent of the geometric arrangement or sizes of the image regions. Different combinations of pooling operations produce images capable of emphasizing low/high frequencies, extract ultrametric contours, etc.

Missing multi-label learning with non-equilibrium based on two-level autoencoder

For a multi-label learning framework, each instance may belong to multiple labels simultaneously. The classification accuracy can be improved significantly by exploiting various correlations, such as label correlations, feature correlations, or the correlations between features and labels. There are few studies on how to combine the feature and label correlations, and they deal more with complete data sets. However, missing labels or other phenomena often occur because of the cost or technical limitations in the data acquisition process. A few label completion algorithms currently suitable for missing multi-label learning, ignore the noise interference of the feature space. At the same time, the threshold of the discriminant function often affects the classification results, especially those of the labels near the threshold. All these factors pose considerable difficulties in dealing with missing labels using label correlations. Therefore, we propose a missing multi-label learning algorithm with non-equilibrium based on a two-level autoencoder. First, label density is introduced to enlarge the classification margin of the label space. Then, a new supplementary label matrix is augmented from the missing label matrix with the non-equilibrium label completion method. Finally, considering feature space noise, a two-level kernel extreme learning machine autoencoder is constructed to implement the information feature and label correlation. The effectiveness of the proposed algorithm is verified by many experiments on both missing and complete label data sets. A statistical analysis of hypothesis validates our approach.