Representation Of Segments Research Articles

Unsupervised Speech Segmentation and Variable Rate Representation Learning Using Segmental Contrastive Predictive Coding

Typically, unsupervised segmentation of speech into the phone- and word-like units are treated as separate tasks and are often done via different methods which do not fully leverage the inter-dependence of the two tasks. Here, we unify them and propose a technique that can jointly perform both, showing that these two tasks indeed benefit from each other. Recent attempts employ self-supervised learning, such as contrastive predictive coding (CPC), where the next frame is predicted given past context. However, CPC only looks at the audio signal’s frame-level structure. We overcome this limitation with a segmental contrastive predictive coding (SCPC) framework to model the signal structure at a higher level, e.g., phone level. A convolutional neural network learns frame-level representation from the raw waveform via noise-contrastive estimation (NCE). A differentiable boundary detector finds variable-length segments, which are then used to optimize a segment encoder via NCE to learn segment representations. The differentiable boundary detector allows us to train frame-level and segment-level encoders jointly. Experiments show that our single model outperforms existing phone and word segmentation methods on TIMIT and Buckeye datasets. We analyze the impact of the threshold on boundary detector performance, and our results suggest that automatically learning the boundary threshold can be as effective as manually tuning that threshold. We discover that phone class impacts the boundary detection performance, and the boundaries between successive vowels or semivowels are the most difficult. Finally, we use SCPC to extract speech features at the segment level rather than at the uniformly spaced frame level (e.g., 10 ms) and produce variable rate representations that change according to the contents of the utterance. We can lower the feature extraction rate from the typical 100 Hz to as low as 14.5 Hz on average while still outperforming the hand-crafted features such as MFCC on the linear phone classification task.

A Crossmodal Multiscale Fusion Network for Semantic Segmentation of Remote Sensing Data

Driven by the rapid development of Earth observation sensors, semantic segmentation using multimodal fusion of remote sensing data has drawn substantial research attention in recent years. However, existing multimodal fusion methods based on convolutional neural networks cannot capture long-range dependencies across multiscale feature maps of remote sensing data in different modalities. To circumvent this problem, this work proposes a crossmodal multiscale fusion network (CMFNet) by exploiting the transformer architecture. In contrast to the conventional early, late, or hybrid fusion networks, the proposed CMFNet fuses information of different modalities at multiple scales using the cross-attention mechanism. More specifically, the CMFNet utilizes a novel cross-modal attention architecture to fuse multiscale convolutional feature maps of optical remote sensing images and digital surface model data through a crossmodal multiscale transformer (CMTrans) and a multiscale context augmented transformer (MCATrans). The CMTrans can effectively model long-range dependencies across multiscale feature maps derived from multimodal data, while the MCATrans can learn discriminative integrated representations for semantic segmentation. Extensive experiments on two large-scale fine-resolution remote sensing datasets, namely ISPRS Vaihingen and Potsdam, confirm the excellent performance of the proposed CMFNet as compared to other multimodal fusion methods.

Machine learning based feedback on textual student answers in large courses

Many engineering disciplines require problem-solving skills, which cannot be learned by memorization alone. Open-ended textual exercises allow students to acquire these skills. Students can learn from their mistakes when instructors provide individual feedback. However, grading these exercises is often a manual, repetitive, and time-consuming activity. The number of computer science students graduating per year has steadily increased over the last decade. This rise has led to large courses that cause a heavy workload for instructors, especially if they provide individual feedback to students. This article presents CoFee, a framework to generate and suggest co mputer-aided feedback for textual exercises based on machine learning. CoFee utilizes a segment-based grading concept, which links feedback to text segments. CoFee automates grading based on topic modeling and an assessment knowledge repository acquired during previous assessments. A language model builds an intermediate representation of the text segments. Hierarchical clustering identifies groups of similar text segments to reduce the grading overhead. We first demonstrated the CoFee framework in a small laboratory experiment in 2019, which showed that the grading overhead could be reduced by 85%. This experiment confirmed the feasibility of automating the grading process for problem-solving exercises. We then evaluated CoFee in a large course at the Technical University of Munich from 2019 to 2021, with up to 2, 200 enrolled students per course. We collected data from 34 exercises offered in each of these courses. On average, CoFee suggested feedback for 45% of the submissions. 92% (Positive Predictive Value) of these suggestions were precise and, therefore, accepted by the instructors.

Disentangled Representation for Cross-Domain Medical Image Segmentation

Image segmentation is a long-standing problem in medical image analysis to facilitate the clinical diagnosis and intervention. Progress has been made owing to deep learning via supervised training with elaborate human labelling, however, the segmentation models trained by the labeled source domain cannot perform well in the target domain, making existing approaches lack robustness and generalization ability. Considering the acquisition of medical image labels is quite expensive and time-consuming, we propose a novel feature disentanglement-based unsupervised domain adaptation (UDA) method to improve the robustness of the trained model in the target domain. A segmentation network is designed to learn disentangled features with two parts: I. content-related features, which are responsible for the segmentation task and invariant across domains; II. style-related features, which elucidate the discrepancy between different domains. Feature disentanglement (FD) is achieved by multi-task learning and image translation. Meanwhile, knowledge distillation is introduced to improve the performance on fine-grained segmentations. And for objects with regular shape, we incorporate the adversarial training to predict shape-invariant segmentation masks across domains. Comprehensive experiments are conducted on retina vessel segmentation and sinus surgical instrument segmentation to validate the effectiveness of the proposed method. The average Dice of twenty regular transfer directions achieves 79.26% on five public benchmarks of retina vessel segmentation, the average Dice of two transfer directions from regular to UWF attains 72.63%, and the Dice from cadaveric images to live images reaches 68.1% on sinus surgical instrument segmentation. The results demonstrate that the proposed method achieves the state-of-the-art segmentation performance in the UDA setting.

A Pretrained ELECTRA Model for Kinase-Specific Phosphorylation Site Prediction.

Phosphorylation plays a vital role in signal transduction and cell cycle. Identifying and understanding phosphorylation through machine-learning methods has a long history. However, existing methods only learn representations of a protein sequence segment from a labeled dataset itself, which could result in biased or incomplete features, especially for kinase-specific phosphorylation site prediction in which training data are typically sparse. To learn a comprehensive contextual representation of a protein sequence segment for kinase-specific phosphorylation site prediction, we pretrained our model from over 24 million unlabeled sequence fragments using ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately). The pretrained model was applied to kinase-specific site prediction of kinases CDK, PKA, CK2, MAPK, and PKC. The pretrained ELECTRA model achieves 9.02% improvement over BERT and 11.10% improvement over MusiteDeep in the area under the precision-recall curve on the benchmark data.

Metodika získavania vonkajšej geometrie segmentov novej generácie skúšobnej figuríny

The article presents the fulfilment of a partial goal of the research project focused on developing and constructing a biofidelic surrogate of the human body to improve the forensic reconstruction of road traffic accidents. It aims to describe the methods and results of acquiring external geometry of segments of crash test dummy Hybrid III. To create a precise spatial representation of dummy segments, we applied 3D laser scanning using the handheld 3D scanner VIUscan Handyscan 3D. The workflow, used material and application of software support are closely described in the methodical part of the article. The results present 3D models of significant material for the continuous fulfilment of project goals. In the follow-up activities, we will use dummy external geometry data, particularly to develop molds for casting segments of new-generation synthetic human body surrogates.

SODAR: Segmenting Objects by Dynamically Aggregating Neighboring Mask Representations.

Recent state-of-the-art one-stage instance segmentation model SOLO divides the input image into a grid and directly predicts per grid cell object masks with fully-convolutional networks, yielding comparably good performance as traditional two-stage Mask R-CNN yet enjoying much simpler architecture and higher efficiency. We observe SOLO generates similar masks for an object at nearby grid cells, and these neighboring predictions can complement each other as some may better segment certain object part, most of which are however directly discarded by non-maximum-suppression. Motivated by the observed gap, we develop a novel learning-based aggregation method that improves upon SOLO by leveraging the rich neighboring information while maintaining the architectural efficiency. The resulting model is named SODAR. Unlike the original per grid cell object masks, SODAR is implicitly supervised to learn mask representations that encode geometric structure of nearby objects and complement adjacent representations with context. The aggregation method further includes two novel designs: 1) a mask interpolation mechanism that enables the model to generate much fewer mask representations by sharing neighboring representations among nearby grid cells, and thus saves computation and memory; 2) a deformable neighbour sampling mechanism that allows the model to adaptively adjust neighbor sampling locations thus gathering mask representations with more relevant context and achieving higher performance. SODAR significantly improves the instance segmentation performance, e.g., it outperforms a SOLO model with ResNet-101 backbone by 2.2 AP on COCO test set, with only about 3% additional computation. We further show consistent performance gain with the SOLOv2 model.

Signal-Transformer: A Robust and Interpretable Method for Rotating Machinery Intelligent Fault Diagnosis Under Variable Operating Conditions

As well-known, deep learning models have achieved great success in the field of intelligent fault diagnosis. However, once the working condition changed, the diagnostic accuracy of the trained models would be greatly affected, which seriously limits the application of deep learning models in real industry. Therefore, signal-transformer (S-Transformer), an intelligent fault diagnosis model focusing on the problem of variable operating conditions, is proposed in this study. First, signal embedding is employed to complete the segmentation and up-dimensional representation of the 1-D signal, thus enriching the information in a high dimensional space. Then, those embedded subsignals are further processed via the multihead self-attention mechanism to explore the state features of the signal in different spaces for a deep representation. Finally, an attention visualization method is proposed for vibrating signals to increase the interpretability of the proposed model and overcome the drawbacks of black boxes. With an experimental validation, the proposed model outperforms than other six deep learning methods in terms of diagnostic accuracy under unknown operating conditions. According to the principle of the model, the crucial weights for signal are further explained in visualization. Furthermore, it can be foreseen that the S-Transformer can achieve a robust recognition effect with the ability of interpretable feature extraction in signal denoising.

Cluster Representation of the Structural Description of Images for Effective Classification

The problem of image recognition in the computer vision systems is being studied. The results of the development of efficient classification methods, given the figure of processing speed, based on the analysis of the segment representation of the structural description in the form of a set of descriptors are provided. We propose three versions of the classifier according to the following principles: “object–etalon”, “object descriptor–etalon” and “vector description of the object–etalon”, which are not similar in level of integration of researched data analysis. The options for constructing clusters over the whole set of descriptions of the etalon database, separately for each of the etalons, as well as the optimal method to compare sets of segment centers for the etalons and object, are implemented. An experimental rating of the efficiency of the created classifiers in terms of productivity, processing time, and classification quality has been realized of the applied. The proposed methods classify the set of etalons without error. We have formed the inference about the efficiency of classification approaches based on segment centers. The time of image processing according to the developed methods is hundreds of times less than according to the traditional one, without reducing the accuracy.

Construction of auditory bombardment therapy program: a pilot study

Abstract Background Speech sound disorder is a communication disorder in which children have persistent difficulty saying words or sounds correctly. It refers to any difficulty or combination of difficulties with perception, motor production, or phonological representation of speech sounds and speech segments. Aim of the Work to construct an Arabic auditory bombardment therapy program and measure its effectiveness in treatment of functional speech sound disorder. Subjects and Methods This study was applied on 60 participants divided into 2 groups (30 for each group) with age ranging from 3-5 years diagnosed with functional speech sound disorder with or without language disorders, attending at the Phoniatrics outpatient clinic in Ain Shams University Hospitals. The test for identification of phonological processes was applied on 60 patients with speech sound disorder selected to participate in this study. These were divided in to 2 groups (Group (1) received only the conventional therapy while group (2) received auditory bombardment in addition to the conventional therapy for 3 months) and the test was repeated again after therapy. Results Group (2) showed high significant difference (improvement) in consonant assimilation, voicing change, final consonant deletion, palatal fronting, gliding, lateralization and glottal replacement while group (1) showed high significant difference (improvement) in syllable deletion and partial cluster reduction. Conclusion The present study showed that application of auditory bombardment therapy program in addition to conventional therapy has high significant improvement than conventional therapy alone.

Towards 5G: Joint Optimization of Video Segment Caching, Transcoding and Resource Allocation for Adaptive Video Streaming in a Multi-Access Edge Computing Network

Caching and transcoding at multi-access edge computing (MEC) server and wireless resource allocation in eNodeB interact with each other and together determine the quality of experience (QoE) of dynamic adaptive streaming over HTTP (DASH) clients. However, the relationship among the three factors has not been explored, which has led to limited improvement in clients’ QoE. Therefore, we propose a joint optimization framework of video segment caching and transcoding in MEC servers and resource allocation to improve the QoE of DASH clients. Based on the established framework, we develop an MEC caching management mechanism that consists of the MEC caching partition, video segment deletion, and MEC caching space transfer. Then, a joint optimization algorithm that combines the video segment caching and transcoding in the MEC server and resource allocation is proposed. In the algorithm, the clients’ channel state and the playback status and cooperation among MEC servers are employed to estimate the client's priority, video segment representation switch and continuous playback time. Considering the above four factors, we develop a utility function model of clients’ QoE. Then, we formulate a mixed-integer nonlinear programming mathematical model to maximize the total utility of DASH clients, where the video segment caching and transcoding strategy and resource allocation strategy are jointly optimized. To solve this problem, we propose a low-complexity heuristic algorithm that decomposes the original problem into multiple subproblems. The simulation results show that our proposed algorithms efficiently improve client's throughput, received video quality and hit ratio of video segments while decreasing the playback rebuffering time, video segment representation switch and system backhaul traffic.

A river runs through it: Brain representations of segmented environments

Spatial environments are often segmented into multiple regions or compartments. How is this spatial segmentation represented in the brain? Previous studies have suggested three possible mechanisms: grouping (boundaries warp the global map, making locations in different segments appear more distant than they actually are); schematization (locations are coded with respect to environmental boundaries, in a way that generalizes across segments); and remapping (each segment is represented independently, with no integration into a global map). To test these possibilities, we taught participants the locations of 16 objects within a segmented virtual environment and then used fMRI to assess location codes for these objects. The environment consisted of a virtual courtyard transected by a river that divided it into two geometrically identical segments. Visibility and spatial relations between objects were balanced to be identical within and between segments. After training, participants’ distance estimations and free recall order were affected by the spatial segmentation, suggesting that their mental representations were affected by the presence of the river. Analysis of multivoxel fMRI activity patterns revealed that spatial relations between objects were coded in the hippocampus, occipital place area (OPA) and retrosplenial complex (RSC). Notably, OPA and hippocampus coded schematic representation of the individual segments, such that objects in geometrically equivalent locations within the two segments were represented as being spatially similar, while RSC coded a global map of the environment. We did not find evidence for grouping or remapping. Our findings suggest that spatial segmentation can be induced by topographic feature of the environment even when all parts of the environment are co-visible, and that segmented environments are encoded using a combination of schematic representations of the segments and a global map.

A joint inversion-segmentation approach to assisted seismic interpretation

SUMMARY Structural seismic interpretation and quantitative characterization are historically intertwined processes. The latter provides estimates of the properties of the subsurface, which can be used to aid structural interpretation alongside the original seismic data and a number of other seismic attributes. In this work, we redefine this process as an inverse problem which tries to jointly estimate subsurface properties (i.e. acoustic impedance) and a piece-wise segmented representation of the subsurface based on user-defined macroclasses. By inverting for the quantities simultaneously, the inversion is primed with prior knowledge about the regions of interest, whilst at the same time it constrains this belief with the actual seismic measurements. As the proposed functional is separable in the two quantities, these are optimized in an alternating fashion, where each subproblem is solved using the Primal-Dual algorithm. Subsequently, the final segmented model is used as input to an ad hoc workflow that extracts the perimeter of the detected shapes and produces a structural framework (i.e. seismic horizons) consistent with the estimated subsurface properties. The effectiveness of the proposed method is illustrated through numerical examples on synthetic and field data sets.

If pictures are stative, what does this mean for discourse interpretation?

The goal of this paper is to explore the consequences of adopting Abusch (2014)’s hypothesis about pictures—that they are stative depictions of the world—for the interpreta-tion of discourses. We focus on the phenomenon of narrative progression and reject Abusch’s proposal that aspectual differences between linguistic and pictorial narrative do not factor in providing a uniform analysis across media. Based on eventive-stative sequences in linguistic narrative, we develop a new answer within Segmented Discourse Representation Theory. We state a single pragmatic algorithm that exploits the aspectual differences between linguistic and pictorial narratives to derive the correct predictions.

DPN: detail-preserving network with high resolution representation for efficient segmentation of retinal vessels

Retinal vessels are important biomarkers for many ophthalmological and cardiovascular diseases. Hence, it is of great significance to develop automatic models for computer-aided diagnosis. Existing methods, such as U-Net follows the encoder-decoder pipeline, where detailed information is lost in the encoder in order to achieve a large field of view. Although spatial detailed information could be recovered partly in the decoder, while there is noise in the high-resolution feature maps of the encoder. And, we argue this encoder-decoder architecture is inefficient for vessel segmentation. In this paper, we present the detail-preserving network (DPN), which avoids the encoder-decoder pipeline. To preserve detailed information and learn structural information simultaneously, we designed the detail-preserving block (DP-Block). Further, we stacked eight DP-Blocks together to form the DPN. More importantly, there are no down-sampling operations among these blocks. Therefore, the DPN could maintain a high/full resolution during processing, avoiding the loss of detailed information. To illustrate the effectiveness of DPN, we conducted experiments over three public datasets. Experimental results show, compared to state-of-the-art methods, DPN shows competitive/better performance in terms of segmentation accuracy, segmentation speed, and model size. Specifically, (1) Our method achieves comparable segmentation performance on the DRIVE, CHASE_DB1, and HRF datasets. (2) The segmentation speed of DPN is over 20-160 $$\times$$ faster than other methods on the DRIVE dataset. (3) The number of parameters of DPN is around 120k, far less than all comparison methods.

High-resolution representations and multistage region-based network for ship detection and segmentation from optical remote sensing images

Ship detection and segmentation in optical remote sensing (ORS) images is a current concern actively addressed by the academic community owing to its vast applications. Most present methods only detect the location of the ship but do not segment it at the pixel level. Considering the complex background of ORS images, identifying ships from interferences, such as clouds, waves, and some land architectures similar to ships, proves to be difficult. To address this issue, we propose a high-resolution representation and multistage region-based network (HR-MSRN) for ship detection and segmentation from ORS images. HR-MSRN mainly consists of three parts: the high-resolution feature pyramid network (HRFPN), region proposal network (RPN), and multistage detection and segmentation network (MSDSN). First, HRFPN is built as a backbone network to extract and fuse multilevel image feature maps. Second, ship candidate boxes are generated by defining numerous anchors through RPN. Third, using the idea of a cascade mask R-CNN as the reference method, the MSDSN is proposed to obtain the ship localization and mask shape. We utilize the proposed framework to evaluate an Airbus-ship dataset, and the experiments indicate that (1) HRFPN provides better feature representation ability than the ResNet-FPN when maintaining the same detection framework, especially for small ships; (2) the direct flow between mask branches refines the mask information, and the semantic segmentation branch enhances context information, which indicates that MSDSN is effective and promotes further improvements in ship detection and segmentation from ORS images; (3) in comparison to other region-based methods, HR-MSRN obtains superior performance of ship detection and segmentation in the ORS imagery.

Content Dependent Representation Selection Model for Systems Based on MPEG DASH

A standard called MPEG Dynamic Adaptive Streaming over HTTP (MPEG DASH) ensures the interoperability between different streaming services and the highest possible video quality in changing network conditions. The solutions described in the available literature that focus on video segmentation are mostly proprietary, use a high amount of computational power, lack the methodology, model notation, information needed for reproduction, or do not consider the spatial and temporal activity of video sequences. This paper presents a new model for selecting optimal parameters and number of representations for video encoding and segmentation, based on a measure of the spatial and temporal activity of the video content. The model was developed for the H.264 encoder, using Structural Similarity Index Measure (SSIM) objective metrics as well as Spatial Information (SI) and Temporal Information (TI) as measures of video spatial and temporal activity. The methodology that we used to develop the mathematical model is also presented in detail so that it can be applied to adapt the mathematical model to another type of an encoder or a set of encoding parameters. The efficiency of the segmentation made by the proposed model was tested using the Basic Adaptation algorithm (BAA) and Segment Aware Rate Adaptation (SARA) algorithm as well as two different network scenarios. In comparison to the segmentation available in the relevant literature, the segmentation based on the proposed model obtains better SSIM values in 92% of cases and subjective testing showed that it achieves better results in 83.3% of cases.

Instance search via instance level segmentation and feature representation

Instance search is an interesting task as well as a challenging issue due to the lack of effective feature representation. In this paper, an instance level feature representation built upon fully convolutional instance-aware segmentation is proposed. The feature is ROI-pooled from the segmented instance region. So that instances in various sizes and layouts are represented by deep features in uniform length. This representation is further enhanced by the use of deformable ResNeXt blocks. Superior performance is observed in terms of its distinctiveness and scalability on a challenging evaluation dataset built by ourselves. In addition, the proposed enhancement on the network structure also shows superior performance on the instance segmentation task.

Computational Model for Global Contour Precedence Based on Primary Visual Cortex Mechanisms

The edges of an image contains rich visual cognitive cues. However, the edge information of a natural scene usually is only a set of disorganized unorganized pixels for a computer. In psychology, the phenomenon of quickly perceiving global information from a complex pattern is called the global precedence effect (GPE). For example, when one observes the edge map of an image, some contours seem to automatically “pop out” from the complex background. This is a manifestation of GPE on edge information and is called global contour precedence (GCP). The primary visual cortex (V1) is closely related to the processing of edges. In this article, a neural computational model to simulate GCP based on the mechanisms of V1 is presented. There are three layers in the proposed model: the representation of line segments, organization of edges, and perception of global contours. In experiments, the ability to group edges is tested on the public dataset BSDS500. The results show that the grouping performance, robustness, and time cost of the proposed model are superior to those of other methods. In addition, the outputs of the proposed model can also be applied to the generation of object proposals, which indicates that the proposed model can contribute significantly to high-level visual tasks.

On segmental representations in second language phonology: A perceptual account

The extent to which exposure to new phonemic contrasts (i.e. contrasts that are present in the L2 but not in the L1) will lead to the creation of a new phonemic category in L2 speakers, as well as the phonological nature of these categories, remains an open question insofar as there is no consensus on whether acquiring a new contrast would result in abstract, phoneme-like categories, or if they belong to a less abstract level of representation. This work explores the perception of the /ɑ/–/ʌ/ contrast ( cop – cup) in American English by Spanish speakers of L2 English through a discrimination task. The results show that while the interlanguage state of less experienced learners is best described as a case of single-category assimilation, the interlanguage state achieved by advanced learners is not a full phonemic split, despite the increased sensitivity to otherwise within-category perceptual cues; rather, it seems that while the ability to perceive differences is not affected, the ability to create a new phonemic representation is impaired.