Temporal Locality Research Articles

A Multitemporal Scale and Spatial–Temporal Transformer Network for Temporal Action Localization

Temporal action localization plays an important role in video analysis, which aims to localize and classify actions in untrimmed videos. Previous methods often predict actions on a feature space of a single temporal scale. However, the temporal features of a low-level scale lack sufficient semantics for action classification, while a high-level scale cannot provide the rich details of the action boundaries. In addition, the long-range dependencies of video frames are often ignored. To address these issues, a novel multitemporal-scale spatial–temporal transformer (MSST) network is proposed for temporal action localization, which predicts actions on a feature space of multiple temporal scales. Specifically, we first use refined feature pyramids of different scales to pass semantics from high-level scales to low-level scales. Second, to establish the long temporal scale of the entire video, we use a spatial–temporal transformer encoder to capture the long-range dependencies of video frames. Then, the refined features with long-range dependencies are fed into a classifier for coarse action prediction. Finally, to further improve the prediction accuracy, we propose a frame-level self-attention module to refine the classification and boundaries of each action instance. Most importantly, these three modules are jointly explored in a unified framework, and MSST has an anchor-free and end-to-end architecture. Extensive experiments show that the proposed method can outperform state-of-the-art approaches on the THUMOS14 dataset and achieve comparable performance on the ActivityNet1.3 dataset. Compared with A2Net (TIP20, Avg{0.3:0.7}), Sub-Action (CSVT2022, Avg{0.1:0.5}), and AFSD (CVPR21, Avg{0.3:0.7}) on the THUMOS14 dataset, the proposed method can achieve improvements of 12.6%, 17.4%, and 2.2%, respectively.

Time dependent risk factors for epileptic seizures in glioblastoma patients: A retrospective analysis of 520 cases.

Epilepsy is a common comorbidity of glioblastoma. Seizures may occur in various phases of the disease. We aimed to assess potential risk factors for seizures in accordance with the point of time at which they occurred. We retrospectively analysed medical files of adult patients with de novo glioblastoma treated at our institution between 01/2006 and 01/2020. We categorized seizures as preoperative seizures (POS), early postoperative seizures (EPS; before initiation of radio(chemo)therapy, RCT), seizures during radiotherapy (SDR; during or < 30 days after RCT) and post-therapeutic seizures (PTS; ≥ 30 days after completion of RCT). We addressed associations between patients' characteristics and their seizures. In the final cohort (N = 520), 292 patients experienced seizures. POS, EPS, SDR and/or PTS occurred in 29.6 % (154/520), 6.0 % (31/520), 13.8 % (70/509) and 36.1 % (152/421) of patients, respectively. POS occurred more frequently in patients with higher Karnofsky performance scores (odds ratio (OR) 3.27, p = 0.001) and tumour location in the temporal lobe (OR 1.51, P = 0.034). None of the parameters we analysed was related to the occurrence of EPS. SDR were independently associated with tumour location (parietal lobe, OR 1.86, P = 0.027) and POS, but not EPS, and were independent of RCT. PTS were independently associated with tumour progression (OR 2.32, P < 0.001) and with occurrence of SDR (OR 3.36, P < 0.001), and negatively correlated with temporal lobe location (OR 0.58, p < 0.014). In patients with tumours exclusively located in the temporal lobe, complete tumour resection was associated with a decreased risk of postoperative seizures. Seizures in glioblastoma patients have various, time-dependent risk factors. Temporal lobe localisation was a risk factor for preoperative seizures; surgery may have had a protective effect in these patients. RCT did not have dose-dependent pro- or anticonvulsive effects. PTS were associated with tumour progression.

Re-Cache: Mitigating cache contention by exploiting locality characteristics with reconfigurable memory hierarchy for GPGPUs

Modern GPGPUs have employed multi-threading to hide the long off-chip memory access latency caused by frequent cache misses. However, the limited cache capacity shared by thousands of concurrently running warps will introduce serious cache contention problem, which results in high cache miss rates to hurt the overall GPGPU performance. To address this problem, this paper proposes a novel cache technology (Re-Cache) by exploiting locality characteristics and reconfiguring the memory hierarchy including the unused shared memory space and the idle registers of pending warps without extra storage overhead for GPGPUs. To further improve the utilization efficiency of the Re-Cache space, this paper proposes a dynamic and reconfigurable cache organization for Re-Cache that explores the spatial and temporal locality differences and changes among the running applications. Re-Cache dynamically reconfigures the cache organization as the adaptive prefetch region and victim region according to the differences and changes of the spatial and temporal locality by catching the locality characteristics for each load instruction. For the high-locality load instructions, the prefetch region stores the subsequent data blocks to exploit the spatial locality of the running threads among all streaming multiprocessors and warps, while the victim region collects the evicted data blocks from the upper memory hierarchy to exploit the temporal locality. Experimental results demonstrate that the proposed Re-Cache reduces up to 34.82% and 31.62% cache misses, while improving about 47.17% and 14.96% IPC performance compared with the state-of-the-art designs for cache-sensitive kernels and benchmarks, respectively.

METHOD OF SPATIOTEMPORAL LOCALIZATION AND CLASSIFICATION OF EVENTS ON A VIDEO OF A SPORTS GAME

Abstract. The paper considers the problem of identifying game events in a video game of beach volleyball. As a method for solving the problem, a combined approach is proposed based on recognizing the trajectory of the ball and constructing skeletal representations of the players. Temporal localization of game events is carried out by searching for the changes in the ball trajectory using the construction of a composite Bezier curve. Spatial localization is performed by the positioning of the ball and the players in the frame at the moment of the trajectory change. The classification is based on the skeletal representation of the player touching the ball. The experiments performed show that the algorithm is able to perform spatiotemporal localization with high precision and recall.

Acoustic emission wavelet time-frequency characteristics of fiber reinforced mortar failure process under axial compression

Abstract In order to investigate the wavelet time-frequency characteristics of fiber reinforced mortar under axial compression, acoustic emission signals associated with steel fiber reinforced mortar and polypropylene fiber reinforced mortar with five different fiber contents (0 to 2.0 vol%) under axial compression were recorded. A new method of multivariate statistical analysis of wavelet time-frequency maps based on Morlet wavelet transform is proposed to make signal processing more accurate. Results show in the 15% to 85% loading stage, Type 3 and Type 4 signals are abundant in fiber reinforced mortar than plain mortar, while Type 7 and Type 9 signals are abundant in the 85% to 100% loading stage. According to the temporal localization characteristics of signal frequency quantified by each type of signals, it can be inferred that the fibers improve the toughness of concrete materials. It is also found the strengthening and toughening effect of steel fiber is more sensitive to fiber contents than that of polypropylene fiber. The ratio of Type 1 signals can be used as a warning against the final failure of materials while that of Type 5 signals can be used to evaluate the anti-cracking effect of fibers.

Event-Related Potentials of the Human Brain During the Comparison of Visual Stimuli

The aim of the investigation was to study the features of the brain activity when comparing visual stimuli under conditions of delayed motor response. Event-related potentials (ERP) were studied in 84 healthy subjects in a three-stimulus test, the first two stimuli were a comparison pair, and the third stimulus triggered a motor response. After presentation of the second stimulus ERP were recorded: a complex of two waves with occipital (Oz, most pronounced in the interval 100–150 ms) and posterior temporal localization (P7, P8, 190–270 ms); negative oscillation in the frontal regions (Fz, 240–300 ms) and positive oscillation in the parietal regions (Pz, 270–450 ms). Brain responses differ in amplitude in case of match and discrepancy of visual stimuli. The article discusses the physiological meaning of these waves and their differences under two conditions.

Heartbeat Cardiac Sounds Analysis

Phonocardiography provides the clinician with a complementary tool to record the heart sounds heard during auscultation. This paper presents an algorithm for the detection of heart sounds (the first and second sounds, S1 and S2) and heart murmurs. Secondly the heartbeat cardiac sounds represented by the phonocardiogram (PCG) signal once segmented is one of the useful approaches to explore cardiac activity and extract many features to help researchers develop techniques that can be used for the medical treatment of several heart diseases. For people affected by a heart activity problem, it is a serious health problem that requires special care. In this paper, importance is given to heart murmurs and their severity to show their impact on heart rate. Heart murmurs are very widespread pathologies in the world and depending on their severity they could constitute. This paper is concerned to the segmentation of heart sounds by using state of art Hidden Markov Models technology which used to extract a smooth envelogram which enable us to apply the tests necessary for temporal localization of heart sounds and heart murmurs.

Investigating the magnitude and temporal localization of inertial particle mixing in turbulent channel flows

Mixing of inertial point particles in a turbulent channel flow at Reτ=950 is investigated by means of direct numerical simulations. We consider inertial particles, at varying Stokes number, released from pairs of sources located at different positions inside the channel and analyze the rate at which particles come into close proximity to each other. To do so, we employ a Lagrangian framework, which is suitable for the analysis of trajectories and in general for the study of mixing and dispersion problems. By varying the release position of particles along the wall-normal direction we obtain a thorough description of mixing in an anisotropic turbulent flow. Moreover, we analyze the effects of particle inertia and show that these are not univocal but also depend on the position and alignment of the sources, owing in particular to the dependence of the flow timescales on the distance from the wall.

Point-supervised temporal action localisation based on multi-branch attention

ABSTRACT Temporal action localisation is a key research direction for video understanding in the field of computer vision. Current methods of using an attention mechanism only divides the video frame into an action instance frame and a background frame. As a result, the action context, which should belong to the background is misclassified into an action instance In addition, during the training phase of using point-supervised frame-level labels, action samples and background samples are unbalanced. The lack of background samples leads to the reduction of the activation score of the background so that the imbalance of samples will affect the separation of action examples from the background. All these reduce the accuracy of action classification and temporal localisation. Therefore, this paper proposesa multi-branch attention network and a pseudo-background label generation method. Experimental results show that the proposed method can improve the separation effect of action instances, background, and action context. Moreover, the proposed model achieves excellent performance on the THUMOS-14 dataset.

Self-adjusting grid networks

Emerging networked systems become increasingly flexible, reconfigurable, and “self-⁎”. This introduces an opportunity to adjust networked systems in a demand-aware manner, leveraging spatial and temporal locality in the workload for online optimizations. However, it also introduces a tradeoff: while more frequent adjustments can improve performance, they also entail higher reconfiguration costs. This paper studies self-adjusting grid networks in which frequently communicating nodes (e.g., virtual machines) are moved topologically closer in an online and demand-aware manner, striking a balance between the benefits and costs of reconfigurations. The paper presents a general Ω(log⁡n) lower bound for this problem, even in scenarios where the demand graph is constant once learned. To demonstrate the challenge of adapting a network to pair-wise communication requests, we also design an O(log⁡n)-competitive algorithm for 1-dimensional grids.

Role of the blood-spinal cord barrier: An adheren junction regulation mechanism that promotes chronic postsurgical pain

Chronic postsurgical pain (CPSP) is a serious postoperative complication with high incidence, and its pathogenesis involves neuroimmune interactions and the breakdown of the blood-spinal cord barrier (BSCB), the decreased level of adheren junction (AJ)-related proteins is an important cause of BSCB injury. Vascular endothelial-cadherin (VE-cadherin) and p120 catenin (p120) constitute the endothelial barrier adheren junction. The Src/p120/VE-cadherin pathway is involved in the regulation of the endothelial barrier function. However, the role of the BSCB-AJ regulatory mechanism in CPSP has not been reported. In this study, we established a skin/muscle incision and retraction (SMIR) model and evaluated the paw withdrawal threshold (PWT), the effects of an Src inhibitor and p120 knockdown on p-Src, p120 and VE-cadherin expression, as well as BSCB-AJ function in rat spinal cord were observed to explore the regulation of BSCB-AJ function by the p-Src/p120/VE-cadherin pathway in promoting SMIR-induced CPSP. The levels of p-Src, p120 and VE-cadherin in the spinal cord were detected by Western blot. Meanwhile, BSCB permeability test was used to detect the changes in BCSB function. Finally, the spatial and temporal localization of p120 in spinal cord was detected by immunofluorescence. Our findings indicated that p-Src/p120/VE-cadherin could induce BSCB-AJ dysfunction and promote the development of CPSP.

Efficient training of spiking neural networks with temporally-truncated local backpropagation through time.

Directly training spiking neural networks (SNNs) has remained challenging due to complex neural dynamics and intrinsic non-differentiability in firing functions. The well-known backpropagation through time (BPTT) algorithm proposed to train SNNs suffers from large memory footprint and prohibits backward and update unlocking, making it impossible to exploit the potential of locally-supervised training methods. This work proposes an efficient and direct training algorithm for SNNs that integrates a locally-supervised training method with a temporally-truncated BPTT algorithm. The proposed algorithm explores both temporal and spatial locality in BPTT and contributes to significant reduction in computational cost including GPU memory utilization, main memory access and arithmetic operations. We thoroughly explore the design space concerning temporal truncation length and local training block size and benchmark their impact on classification accuracy of different networks running different types of tasks. The results reveal that temporal truncation has a negative effect on the accuracy of classifying frame-based datasets, but leads to improvement in accuracy on event-based datasets. In spite of resulting information loss, local training is capable of alleviating overfitting. The combined effect of temporal truncation and local training can lead to the slowdown of accuracy drop and even improvement in accuracy. In addition, training deep SNNs' models such as AlexNet classifying CIFAR10-DVS dataset leads to 7.26% increase in accuracy, 89.94% reduction in GPU memory, 10.79% reduction in memory access, and 99.64% reduction in MAC operations compared to the standard end-to-end BPTT. Thus, the proposed method has shown high potential to enable fast and energy-efficient on-chip training for real-time learning at the edge.

Active learning with effective scoring functions for semi-supervised temporal action localization

Temporal Action Localization (TAL) aims to predict both action category and temporal boundary of action instances in untrimmed videos, i.e., start and end time. Existing works usually adopt fully-supervised solutions, however, one of the practical bottlenecks in these solutions is the large amount of labeled training data required. To reduce expensive human label cost, this paper focuses on a rarely investigated yet practical task named semi-supervised TAL and proposes an effective active learning method, named AL-STAL. We leverage four steps for actively selecting video samples with high informativeness and training the localization model, named Train, Query, Annotate, Append. Two scoring functions that consider the uncertainty of localization model are equipped in AL-STAL, thus facilitating the video sample ranking and selection. One takes entropy of predicted label distribution as measure of uncertainty, named Temporal Proposal Entropy (TPE). And the other introduces a new metric based on mutual information between adjacent action proposals, named Temporal Context Inconsistency (TCI). To validate the effectiveness of proposed method, we conduct extensive experiments on three benchmark datasets THUMOS’14, ActivityNet 1.3 and ActivityNet 1.2. Experiment results show that AL-STAL outperforms the existing competitors and achieves satisfying performance compared with fully-supervised learning.

Adaptive Two-Stream Consensus Network for Weakly-Supervised Temporal Action Localization.

Weakly-supervised temporal action localization (W-TAL) aims to classify and localize all action instances in untrimmed videos under only video-level supervision. Without frame-level annotations, it is challenging for W-TAL methods to clearly distinguish actions and background, which severely degrades the action boundary localization and action proposal scoring. In this paper, we present an adaptive two-stream consensus network (A-TSCN) to address this problem. Our A-TSCN features an iterative refinement training scheme: a frame-level pseudo ground truth is generated and iteratively updated from a late-fusion activation sequence, and used to provide frame-level supervision for improved model training. Besides, we introduce an adaptive attention normalization loss, which adaptively selects action and background snippets according to video attention distribution. By differentiating the attention values of the selected action snippets and background snippets, it forces the predicted attention to act as a binary selection and promotes the precise localization of action boundaries. Furthermore, we propose a video-level and a snippet-level uncertainty estimator, and they can mitigate the adverse effect caused by learning from noisy pseudo ground truth. Experiments conducted on the THUMOS14, ActivityNet v1.2, ActivityNet v1.3, and HACS datasets show that our A-TSCN outperforms current state-of-the-art methods, and even achieves comparable performance with several fully-supervised methods.

341 Post-Operative Outcomes and Rare Complications in Pediatric Stereotactic Electroencephalography: A Retrospective Review from a High Volume Pediatric Epilepsy Center

INTRODUCTION: Stereotactic electro-encephalography (sEEG) allows for spatial and temporal localization of epileptic foci for definitive treatment guidance. High volume center studies are more likely to detect rare, although potentially significant complications. METHODS: A retrospective review of 58 sEEG cases performed by a single surgeon at a single pediatric academic institution was performed. Patients &lt;18 years old were included. Patients without 5 years of follow-up were excluded. RESULTS: 25 (49%) of the patients undergoing SEEG were non-lesional on pre-operative imaging. 50 patients (98%) received surgical treatment of their epilepsy following sEEG. 19 underwent open surgical resection of EZ, 16 laser interstitial thermal ablation, 13 responsive neurostimulation, and 1 was treated with DBS. We achieved Engle 1, 2, 3, and 4 level outcomes in 22 (55%), 8 (20%), 5(13%), 5 (13%) of patients respectively. Engle outcome scores were comparable for lesional and non-lesional EZ in patients who underwent resection (Engle 1 outcome in 10/12 and 1/2 respectively) and ablation (6/9 and 6/7, respectively). IPH occurred in 3 cases (5.2%), with only one requiring surgical evacuation. 2 of the 3 hemorrhages caused transient neurologic deficits and the third caused a longstanding hemiparesis. Two (3.4%) cases developed asymptomatic pseudo-aneurysms identified on routine delayed vascular imaging. One was treated successfully with endovascular coiling, however the second was too distal and required surgical clipping. There were no complications from either pseudo-aneurysm treatment. CONCLUSIONS: sEEG offers favorable morbidity, source localization, and seizure control in surgical lesions. We recommend surveillance imaging with CTA or MRA 3-6 months post operatively to rule out pseudoaneurysm formation given.

A multi‐feature‐based intelligent redundancy elimination scheme for cloud‐assisted health systems

AbstractRedundancy elimination techniques are extensively investigated to reduce storage overheads for cloud‐assisted health systems. Deduplication eliminates the redundancy of duplicate blocks by storing one physical instance referenced by multiple duplicates. Delta compression is usually regarded as a complementary technique to deduplication to further remove the redundancy of similar blocks, but our observations indicate that this is disobedient when data have sparse duplicate blocks. In addition, there are many overlapped deltas in the resemblance detection process of post‐deduplication delta compression, which hinders the efficiency of delta compression and the index phase of resemblance detection inquires abundant non‐similar blocks, resulting in inefficient system throughput. Therefore, a multi‐feature‐based redundancy elimination scheme, called MFRE, is proposed to solve these problems. The similarity feature and temporal locality feature are excavated to assist redundancy elimination where the similarity feature well expresses the duplicate attribute. Then, similarity‐based dynamic post‐deduplication delta compression and temporal locality‐based dynamic delta compression discover more similar base blocks to minimise overlapped deltas and improve compression ratios. Moreover, the clustering method based on block‐relationship and the feature index strategy based on bloom filters reduce IO overheads and improve system throughput. Experiments demonstrate that the proposed method, compared to the state‐of‐the‐art method, improves the compression ratio and system throughput by 9.68% and 50%, respectively.

Intracerebral Correlates of Scalp EEG Ictal Discharges Based on Simultaneous Stereo-EEG Recordings.

It remains unknown to what extent ictal scalp EEG can accurately predict the localization of the intracerebral seizure onset in presurgical evaluation of drug-resistant epilepsies. In this study, we aimed to define homogeneous ictal scalp EEG profiles (based on their first ictal abnormality) and assess their localizing value using simultaneously recorded scalp EEG and stereo-EEG. We retrospectively included consecutive patients with drug-resistant focal epilepsy who had simultaneous stereo-EEG and scalp EEG recordings of at least 1 seizure in the epileptology unit in Nancy, France. We analyzed 1 seizure per patient and used hierarchical cluster analysis to group similar seizure profiles on scalp EEG and then performed a descriptive analysis of their intracerebral correlates. We enrolled 129 patients in this study. The hierarchical cluster analysis showed 6 profiles on scalp EEG first modification. None were specific to a single intracerebral localization. The "normal EEG" and "blurred EEG" clusters (early muscle artifacts) comprised only 5 patients each and corresponded to no preferential intracerebral localization. The "temporal discharge" cluster (n = 46) was characterized by theta or delta discharges on ipsilateral anterior temporal scalp electrodes and corresponded to a preferential mesial temporal intracerebral localization. The "posterior discharge" cluster (n = 42) was characterized by posterior ipsilateral or contralateral rhythmic alpha discharges or slow waves on scalp and corresponded to a preferential temporal localization. However, this profile was the statistically most frequent scalp EEG correlate of occipital and parietal seizures. The "diffuse suppression" cluster (n = 9) was characterized by a bilateral and diffuse background activity suppression on scalp and corresponded to mesial, and particularly insulo-opercular, localization. Finally, the "frontal discharge" cluster (n = 22) was characterized by bilateral frontal rhythmic fast activity or preictal spike on scalp and corresponded to preferential ventrodorsal frontal intracerebral localizations. The hierarchical cluster analysis identified 6 seizure profiles regarding the first abnormality on scalp EEG. None of them were specific of a single intracerebral localization. Nevertheless, the strong relationships between the "temporal," "frontal," "diffuse suppression," and "posterior" profiles and intracerebral discharge localizations may contribute to hierarchize hypotheses derived from ictal scalp EEG analysis regarding intracerebral seizure onset.

TriCache: A User-Transparent Block Cache Enabling High-Performance Out-of-Core Processing with In-Memory Programs

Out-of-core systems rely on high-performance cache sub-systems to reduce the number of I/O operations. Although the page cache in modern operating systems enables transparent access to memory and storage devices, it suffers from efficiency and scalability issues on cache misses, forcing out-of-core systems to design and implement their own cache components, which is a non-trivial task. This study proposes TriCache, a cache mechanism that enables in-memory programs to efficiently process out-of-core datasets without requiring any code rewrite. It provides a virtual memory interface on top of the conventional block interface to simultaneously achieve user transparency and sufficient out-of-core performance. A multi-level block cache design is proposed to address the challenge of per-access address translations required by a memory interface. It can exploit spatial and temporal localities in memory or storage accesses to render storage-to-memory address translation and page-level concurrency control adequately efficient for the virtual memory interface. Our evaluation shows that in-memory systems operating on top of TriCache can outperform Linux OS page cache by more than one order of magnitude, and can deliver performance comparable to or even better than that of corresponding counterparts designed specifically for out-of-core scenarios.

Deep cascaded action attention network for weakly-supervised temporal action localization

Deep cascaded action attention network for weakly-supervised temporal action localization

Spatial and temporal localization of ultrasonic heating effect using cylindrical standing waves

The aim of this work was to develop a new method alternative to high intensity focused ultrasound for spatial and temporal localization of ultrasonic heating effects using cylindrical standing waves. Computer simulations and experimental validation of the standing cylindrical waves generation and the viscoelastic media heating inside a cylindrical piezoelectric transducer were carried out.