Temporal Locality Research Articles

Multinodular Vacuolating Neuronal Tumors: Symptomatic Presentation Versus Incidental Finding: Case Series and Literature Review

Background: Multinodular Vacuolating Neuronal Tumors (MVNTs) are mixed glial–neuronal brain lesions classified as World Health Organization (WHO) CNS grade 1 tumors, often associated with long-term epilepsy. First described by Huse et al. in 2013 and included in the WHO CNS classification in 2016, MVNTs present a range of clinical manifestations, from symptomatic to asymptomatic. They typically affect young to middle-aged adults and exhibit diverse presentations. Radiologically, MVNTs are usually supratentorial, frequently located in the temporal lobe but also observed in the frontal and parietal lobes. MRI is essential for diagnosis, revealing multiple coalescing subcortical or cortical nodules with hyperintense signals on T2-weighted/FLAIR sequences, often without peripheral edema or mass effects. Case Reports: This paper presents two cases: one symptomatic MVNT with significant clinical manifestations, and the other documenting an incidental finding of MVNT in an asymptomatic patient. One case shows typical temporal lobe localization, while the other highlights a rare frontal lobe localization, with clear radiological findings on T2/FLAIR sequences. Conclusions: These cases illustrate the varied clinical presentations of MVNTs and emphasize MRI’s critical role in diagnosis and management. Asymptomatic cases often require conservative management, stressing the avoidance of unnecessary invasive procedures and the importance of regular monitoring.

Histochemical and molecular analyses reveal an insight into the scent volatiles synthesis and emission in ephemeral flowers of Murraya paniculata (L.) Jack.

Temporal histolocalization of floral volatiles in the petal epidermis of Murraya paniculata was found to be linked with the coordinated expression of candidate genes and successive accumulation of an internal pool of volatiles. Murraya paniculata (Rutaceae) is known for its highly fragrant ephemeral flowers that emit volatiles to attract nocturnal pollinators. To unfold the patterns of volatile emission in relation to floral life-span, we studied time-course accumulation and emission rate of scent volatiles at six timepoints of floral maturation, at an interval of 4h starting from the bud stage to the senescence stage on the next day. This study revealed the maximum emission rate of scent volatiles at the anthesis stage at 18:00h. This finding correlates well with the maximum accumulation of volatiles in the internal pool of the flowers at this stage. The key volatiles detected in both emitted and internal pools were benzaldehyde, benzeneacetaldehyde, linalool, caryophyllene, germacrene-D and α-farnesene. In addition, the internal pool also contained substantial amounts of indole, scopoletin, caffeine and osthole. To histochemically localize the temporal accumulation of major volatile groups in the epidermal cells, petal cross sections were stained with NaDi and ferric chloride to visualize terpenes and phenolics, respectively, under light microscope. Histolocalization studies showed a higher accumulation of terpenes at 14:00h and 18:00h, which subsequently was reduced as senescence approached. Significant phenolics in the abaxial and adaxial layers of the petal epidermis accumulated at 18:00h and at the early senescence (06:00h) stages. Furthermore, temporal localization of active shikimate dehydrogenase (SKDH) protein through in-gel activity assay demonstrated higher enzymatic activities at anthesis (18:00h) and fully bloomed (02:00h) stages, supporting the findings of higher accumulation of phenolic volatiles at 18:00h and 06:00h stages. Expression analysis of major candidate genes of floral scent volatiles pathway supported the hypothesis that the emission rate of floral fragrance reached its maximum at the anthesis (18:00h) stage. In contrast, biosynthesis of scent compounds started at the bud (14:00h) stage itself as indicated by the RT-PCR semi-quantitative estimation. As flowers of M. paniculata attract multiple pollinator species, this study could also serve as a springboard for pollination biology in Rutaceae, which includes important fruit crops.

Caching Historical Embeddings in Conversational Search

Rapid response, namely, low latency, is fundamental in search applications; it is particularly so in interactive search sessions, such as those encountered in conversational settings. An observation with a potential to reduce latency asserts that conversational queries exhibit a temporal locality in the lists of documents retrieved. Motivated by this observation, we propose and evaluate a client-side document embedding cache, improving the responsiveness of conversational search systems. By leveraging state-of-the-art dense retrieval models to abstract document and query semantics, we cache the embeddings of documents retrieved for a topic introduced in the conversation, as they are likely relevant to successive queries. Our document embedding cache implements an efficient metric index, answering nearest-neighbor similarity queries by estimating the approximate result sets returned. We demonstrate the efficiency achieved using our cache via reproducible experiments based on Text Retrieval Conference Conversational Assistant Track datasets, achieving a hit rate of up to 75% without degrading answer quality. Our achieved high cache hit rates significantly improve the responsiveness of conversational systems while likewise reducing the number of queries managed on the search back-end.

Semi-supervised sound event detection with dynamic convolution and confidence-aware mean teacher

Recently, sound event detection (SED) has made significant advancements through the application of deep learning, but there are still many difficulties and challenges to be addressed. One of the major challenges is the diversity of sound events, leading to substantial variations in time-frequency domain features. Additionally, most existing SED models can not effectively handle sound events of different scales, particularly those of short duration. Another challenge is the lack of well labeled dataset. The commonly used solution is mean teacher method, but inaccurate pseudo-labels could lead to confirmation bias and performance imbalance. In this paper, we introduce the multi-dimensional frequency dynamic convolution, which endows convolutional kernels with frequency-adaptive dynamic properties to enhance the feature representation capability. Moreover, we propose dual self attention pooling function to achieve more precise temporal localization. Finally, to solve the incorrect pseudo-labels problems, we propose the confidence-aware mean teacher to increase pseudo-labels accuracy and train the student model with high confidence labels. Experimental results on DCASE2017, DCASE2018 and DCASE2023 Task4 dataset validate the superior performance of proposed methods.

TeST: Temporal-spatial separated transformer for temporal action localization

Temporal action localization is a fundamental task in video understanding. Existing methods fall into three categories: anchor-based, actionness-guided, and anchor-free. Anchor-based and actionness-guided models need huge computation resources to process redundant proposals or enumerate every possible proposal. Anchor-free models with lighter parameters become a more attractive option as temporal actions become more complex. However, they typically struggle to achieve high performance due to the need to aggregate global temporal-spatial features at every time step. To overcome this limitation, we design three efficient transformer-based architectures, bringing two advantages: (i) the global receptive field of transformers enables models to aggregate spatial and temporal at each time step, and (ii) the transformers could capture the moment-level feature, enhancing localization performance. Our designed architectures are adapted to any framework, thus we propose a simple but effective anchor-free framework named TeST. Compared to strong baselines, TeST achieves 0.96% to 3.20% improvement on two real-world datasets. Meanwhile, it improves time efficiency by 1.36 times and space efficiency by 1.08 times. Further experiments prove the effectiveness of TeST’s modules. Implementation of our work is available at https://github.com/whr000001/TeST.

A machine learning framework to measure Water Drop Penetration Time (WDPT) for soil water repellency analysis

The heat from wildfires volatilizes soil’s organic compounds which form a waxy layer when condensed on cooler soil particles causing soil to repel water. Timely assessment of soil water repellency (SWR) is critical for prediction and prevention of detrimental impacts of hydrophobic soils such as soil erosion, reduced availability of water to plants, and water runoff after rainfalls leading to floods. The Water Drop Penetration Time (WDPT), i.e., the time elapsed from a drop landing on the soil surface to its complete absorption is commonly used to assess the SWR level. Its manual measurements have variability based on the used instruments and subjective observations. The goal of this work is to design an automated system to perform standardized WDPT tests and assess the SWR levels. It consists of an electronically controlled mechanism to release a water drop, and a video camera to record the water penetration process. The latter is modeled as an “action” in video and Temporal Action Localization (TAL) analytics is used for predicting the WDPT and assessing the SWR level.

Toward Long Form Audio-Visual Video Understanding

We live in a world filled with never-ending streams of multimodal information. As a more natural recording of the real scenario, long form audio-visual videos (LFAVs) are expected as an important bridge for better exploring and understanding the world. In this article, we propose the multisensory temporal event localization task in long form videos and strive to tackle the associated challenges. To facilitate this study, we first collect a large-scale LFAV dataset with 5,175 videos and an average video length of 210 seconds. Each collected video is elaborately annotated with diversified modality-aware events, in a long-range temporal sequence. We then propose an event-centric framework for localizing multisensory events as well as understanding their relations in long form videos. It includes three phases in different levels: snippet prediction phase to learn snippet features, event extraction phase to extract event-level features, and event interaction phase to study event relations. Experiments demonstrate that the proposed method, utilizing the new LFAV dataset, exhibits considerable effectiveness in localizing multiple modality-aware events within long form videos. We hope that our newly collected dataset and novel approach serve as a cornerstone for furthering research in the realm of LFAV understanding. Project page: https://gewu-lab.github.io/LFAV/ .

Temporal localization of upper extremity bilateral synergistic coordination using wearable accelerometers.

The human upper extremity is characterized by inherent motor abundance, allowing a diverse array of tasks with agility and adaptability. Upper extremity functional limitations are a common sequela to Stroke, resulting in pronounced motor and sensory impairments in the contralesional arm. While many therapeutic interventions focus on rehabilitating the weaker arm, it is increasingly evident that it is necessary to consider bimanual coordination and motor control. Participants were recruited to two groups differing in age (Group 1 (n=10): 23.4 ±2.9 years, Group 2 (n=10): 55.9 ±10.6 years) for an exploratory study on the use of accelerometry to quantify bilateral coordination. Three tasks featuring coordinated reaching were selected to investigate the acceleration of the upper arm, forearm, and hand during activities of daily living (ADLs). Subjects were equipped with acceleration and inclination sensors on each upper arm, each forearm, and each hand. Data was segmented in MATLAB to assess inter-limb and intra-limb coordination. Inter-limb coordination was indicated through dissimilarity indices and temporal locations of congruous movement between upper arm, forearm, or hand segments of the right and left limbs. Intra-limb coordination was likewise assessed between upper arm-forearm, upper arm-hand, and forearm-hand segment pairs of the dominant limb. Acceleration data revealed task-specific movement features during the three distinct tasks. Groups demonstrated diminished similarity as task complexity increased. Groups differed significantly in the hand segments during the buttoning task, with Group 1 showing no coordination in the hand segments during buttoning, and strong coordination in reaching each button with the upper arm and forearm guiding extension. Group 2's dissimilarity scores and percentages of similarity indicated longer periods of inter-limb coordination, particularly towards movement completion. Group 1's dissimilarity scores and percentages of similarity indicated longer periods of intra-limb coordination, particularly in the coordination of the upper arm and forearm segments. The Expanding Procrustes methodology can be applied to compute objective coordination scores using accessible and highly accurate wearable acceleration sensors. The findings of task duration, angular velocity, and peak roll angle are supported by previous studies finding older individuals to present with slower movements, reduced movement stability, and a reduction of laterality between the limbs. The theory of a shift towards ambidexterity with age is supported by the finding of greater inter-limb coordination in the group of subjects above the age of thirty-five. The group below the age of thirty was found to demonstrate longer periods of intra-limb coordination, with upper arm and forearm coordination emerging as a possible explanation for the demonstrated greater stability.

Weakly Supervised Temporal Action Localization With Bidirectional Semantic Consistency Constraint.

Weakly supervised temporal action localization (WTAL) aims to classify and localize temporal boundaries of actions for the video, given only video-level category labels in the training datasets. Due to the lack of boundary information during training, existing approaches formulate WTAL as a classification problem, i.e., generating the temporal class activation map (T-CAM) for localization. However, with only classification loss, the model would be suboptimized, i.e., the action-related scenes are enough to distinguish different class labels. Regarding other actions in the action-related scene (i.e., the scene same as positive actions) as co-scene actions, this suboptimized model would misclassify the co-scene actions as positive actions. To address this misclassification, we propose a simple yet efficient method, named bidirectional semantic consistency constraint (Bi-SCC), to discriminate the positive actions from co-scene actions. The proposed Bi-SCC first adopts a temporal context augmentation to generate an augmented video that breaks the correlation between positive actions and their co-scene actions in the inter-video. Then, a semantic consistency constraint (SCC) is used to enforce the predictions of the original video and augmented video to be consistent, hence suppressing the co-scene actions. However, we find that this augmented video would destroy the original temporal context. Simply applying the consistency constraint would affect the completeness of localized positive actions. Hence, we boost the SCC in a bidirectional way to suppress co-scene actions while ensuring the integrity of positive actions, by cross-supervising the original and augmented videos. Finally, our proposed Bi-SCC can be applied to current WTAL approaches and improve their performance. Experimental results show that our approach outperforms the state-of-the-art methods on THUMOS14 and ActivityNet. The code is available at https://github.com/lgzlIlIlI/BiSCC.

TC-ULM: Temporal Correlation-based Ultrasound Localization Microscopy by Deep Learning

Abstract Ultrasound Localization Microscopy (ULM) transcends the limitations of acoustic diffraction by pinpointing minuscule microbubbles (MBs), enabling detailed visualization of microvasculature at resolutions finer than the wavelength. Current approaches in ULM, inclusive of both conventional and deep learning techniques, predominantly analyze data from the ultrasound frame plane (x-y). This leaves the valuable information in the time dimension (t), especially between consecutive frames, largely untapped. This paper introduces a novel approach, Temporal Correlation ULM (TC-ULM), which harnesses temporal correlations within the y-t plane for enhanced ULM imaging. By extracting the variations of MBs over time through a sophisticated deep learning model, TC-ULM effectively reconstructs ULM images with superior quality. Empirical evidence from numerical simulations and in vivo experiments reveals that MBs exhibit a discernible pattern of continuity and local linkage across adjacent frames in the y-t plane. This key finding allows for a significant reduction in the number of frames needed for ULM, without compromising the spatial resolution. Thus, TC-ULM emerges as an innovative technique that not only retains image quality but also potentially expedites the ULM imaging process, marking a notable shift from traditional x-y plane processing methods.

Weakly-supervised temporal action localization using multi-branch attention weighting

Weakly-supervised temporal action localization using multi-branch attention weighting

MKP-Net: Memory knowledge propagation network for point-supervised temporal action localization in livestreaming

Standardized regulation of livestreaming is an important element of cyberspace governance. Temporal action localization (TAL) can localize the occurrence of specific actions to better understand human activities. Due to the short duration and inconspicuous boundaries of human-specific actions, it is very cumbersome to obtain sufficient labeled data for training in untrimmed livestreaming. The point-supervised approach requires only a single-frame annotation for each action instance and can effectively balance cost and performance. Therefore, we propose a memory knowledge propagation network (MKP-Net) for point-supervised temporal action localization in livestreaming, including (1) a plug-and-play memory module is introduced to model prototype features of foreground actions and background knowledge using point-level annotations, (2) the memory knowledge propagation mechanism is used to generate discriminative feature representation in a multi-instance learning pipeline, and (3) localization completeness learning is performed by designing a dual optimization loss for refining and localizing temporal actions. Experimental results show that our method achieves 61.4% and 49.1% SOTAs on THUMOS14 and self-built BJUT-PTAL datasets, respectively, with an inference speed of 711 FPS.

Proximity Ligand Assay to Localize Proteins in DNA Damage Sites.

The DNA damage response is a genetic information safeguard that protects cells from perpetuating damaged DNA. The characterization of the proteins that cooperate in this process allows the identification of alternative targets for therapeutic intervention in several diseases, such as cancer, aging-related diseases, and chronic inflammation. The Proximity Ligand Assay (PLA) emerged as a tool for estimating interaction between proteins as well as spatial proximity among organelles or cellular structures and allows the temporal localization and co-localization analysis under stress conditions, for instance. The method is simple because it is similar to conventional immunofluorescence and allows the staining of an organelle, cellular structure, or a specific marker such as mitochondria, endoplasmic reticulum, PML bodies, or DNA double-strand marker, yH2AX simultaneously. The phosphorylation of the S139 at Histone 2A variant, H2AX, then referred to as yH2AX, is widely used as a very sensitive and specific marker of DNA double-strand breaks. Each focus of yH2AX staining corresponds to one break in DNA that occurs a few minutes after the damage. The analysis of changes in yH2AX foci is the most common assay for studying if the protein of interest is implicated in DNA damage response (DDR). Whether a direct role in the DNA damage site is expected, fluorescence microscopy is used to verify the colocalization of the protein of interest with yH2AX foci. However, except for the new super-resolution fluorescence methods, to conclude, the local interaction with DNA damage sites can be a little subjective. Here, we show an assay to evaluate the localization of proteins in the DDR pathway using yH2AX as a marker of the damage site. This assay can be used to characterize the temporal localization under different insults that cause DNA damage.

Denoising of Dense Spatial Array Data Using the Curvelet Transform

ABSTRACT To remove background noise from seismic data recorded by spatially dense arrays, we have developed a space-based denoising procedure using the discrete curvelet transform. Based on a detailed statistical characterization of noise coefficients through the empirical cumulative distribution function method within a pre-event time window, signal and noise can be separated effectively by nonlinear thresholding. After synthetic test, we applied this method on data from an industry 3D seismic experiment recorded at an array deployed near Utica, Ohio. The denoising results show good waveform consistency with a significantly enhanced signal-to-noise ratio. Our curvelet approach allows a more computationally efficient spatial–temporal localization analysis of seismic data than conventional curvelet techniques by avoiding the assumption of stationary Gaussian-distributed noise and can be implemented as a complement of time-domain wavelet methods with fewer frequency losses after denoising. This new method provides a fast and convenient way to recover signals from noisy recordings with dense 2D arrays, leading to a considerable improvement in data quality compared with conventional Fourier, wavelet, and curvelet methods. The partitioned seismic signals and noise would yield advanced earth structure imaging, small-event detection, ambient noise tomography, and others.

Stacked collaborative transformer network with contrastive learning for video moment localization

Video grounding intends to perform temporal localization in multimedia information retrieval. The temporal bounds of the target video span are determined for the given input query. A novel interactive multi-head self-attention (IMSA) transformer is proposed to localize an unseen moment in the untrimmed video for the given image. A new semantic-trained self-supervised approach is considered in this paper to perform cross-domain learning to match the image query – video segment. It normalizes the convolution function enabling efficient correlation and collecting of semantically related video segments across time based on the image query. A double hostile Contrastive learning with Gaussian distribution parameters method is advanced to learn the representations of video. The proposed approach performs dynamically on various video components to achieve exact semantic synchronization and localization among queries and video. In the proposed approach, the IMSA model localizes frames greatly compared to other approaches. Experiments on benchmark datasets show that the proposed model can significantly increase temporal grounding accuracy. The moment occurrence is identified in the video with a start and end boundary ascertains an average recall of 86.45% and a mAP of 59.3%.

HDR brachytherapy afterloader quality assurance optimization using monolithic silicon strip detectors.

There currently exists no widespread high dose-rate (HDR) brachytherapy afterloader quality assurance (QA) tool for simultaneously assessing the afterloader's positional, temporal, transit velocity and air kerma strength accuracy. The purpose of this study was to develop a precise and rigorous technique for performing daily QA of HDR brachytherapy afterloaders, incorporating QA of: dwell position accuracy, dwell time accuracy, transit velocity consistency and relative air kerma strength (AKS) of an Ir-192 source. A Sharp ProGuide 240mm catheter (Elekta Brachytherapy, Veenendaal, The Netherlands) was fixed 5mm above a 256 channel epitaxial diode array 'dose magnifying glass' (DMG256) (Centre for Medical and Radiation Physics, University of Wollongong). Three dwell positions, each of 5.0s dwell times, were spaced 13.0mm apart along the array with the Flexitron HDR afterloader (Elekta Brachytherapy, Veenendaal, The Netherlands). The DMG256 was connected to a data acquisition system (DAQ) and a computer via USB2.0 link for live readout and post-processing. The outputted data files were analyzed using a Python script to provide positional and temporal localization of the Ir-192 source by tracking the centroid of the detected response. Measurements were repeated on a weekly basis, for a period of 5weeks to determine the consistency of the measured parameters over an extended period. Using the DMG256 for relative AKS measurements resulted in measured values within 0.6%-3.0% of the expected activity over a 7-week period. The sub-millisecond temporal accuracy of the device allowed for measurements of the transit velocity with an average of (10.88±1.01) cm/s for 13mm steps. The dwell position localization for 1, 2, 3, 5, and 10mm steps had an accuracy between 0.1 and 0.3mm (3σ), with a fixed temporal accuracy of 10ms. The DMG256 silicon strip detector allows for clinics to perform rigorous daily QA of HDR afterloader dwell position and dwell time accuracy with greater precision than the current standard methodology using closed circuit television and a stopwatch. Additionally, DMG256 unlocks the ability to perform measurements of transit velocity/time and relative AKS, which are not possible using current standard techniques.

Temporal Attention for Few-Shot Concept Drift Detection in Streaming Data

Concept drift describes unforeseeable changes in the underlying distribution of streaming data over time. Concept drift is a phenomenon in which the statistical properties of a target domain change over time in an arbitrary way. These changes might be caused by changes in hidden variables that cannot be measured directly. With the onset of the big data era, domains such as social networks, meteorology, and finance are generating copious amounts of streaming data. Embedded within these data, the issue of concept drift can affect the attributes of streaming data in various ways, leading to a decline in the accuracy and performance of models. There is a pressing need for new models to re-adapt to the changes in streaming data. Traditional concept drift detection algorithms struggle to effectively capture and utilize the key feature points of concept drift within complex time series, thereby failing to maintain the accuracy and efficiency of the models. In light of these challenges, this study introduces a novel concept drift detection method that incorporates a temporal attention mechanism within a prototypical network. By integrating a temporal attention mechanism during the feature extraction process, our approach enhances the capability to process complex time series data, preserves temporal locality, strengthens the learning of key features, and reduces the amount of labeled data required. This method significantly improves the detection accuracy and efficiency of small sample streaming data by better capturing the local features of the data. Experiments conducted across multiple datasets demonstrate that this method exhibits comprehensive leading performance in terms of accuracy and F1-score, with excellent recall and precision, thereby validating its effectiveness in enhancing concept drift detection in streaming data.

Cross-Video Contextual Knowledge Exploration and Exploitation for Ambiguity Reduction in Weakly Supervised Temporal Action Localization

Cross-Video Contextual Knowledge Exploration and Exploitation for Ambiguity Reduction in Weakly Supervised Temporal Action Localization

Enhancing Single-Frame Supervision for Better Temporal Action Localization.

Temporal action localization aims to identify the boundaries and categories of actions in videos, such as scoring a goal in a football match. Single-frame supervision has emerged as a labor-efficient way to train action localizers as it requires only one annotated frame per action. However, it often suffers from poor performance due to the lack of precise boundary annotations. To address this issue, we propose a visual analysis method that aligns similar actions and then propagates a few user-provided annotations (e.g., boundaries, category labels) to similar actions via the generated alignments. Our method models the alignment between actions as a heaviest path problem and the annotation propagation as a quadratic optimization problem. As the automatically generated alignments may not accurately match the associated actions and could produce inaccurate localization results, we develop a storyline visualization to explain the localization results of actions and their alignments. This visualization facilitates users in correcting wrong localization results and misalignments. The corrections are then used to improve the localization results of other actions. The effectiveness of our method in improving localization performance is demonstrated through quantitative evaluation and a case study.

Co-Occurrence Matters: Learning Action Relation for Temporal Action Localization

Co-Occurrence Matters: Learning Action Relation for Temporal Action Localization