Temporal View Research Articles

LACOSTE: Exploiting stereo and temporal contexts for surgical instrument segmentation

Surgical instrument segmentation is instrumental to minimally invasive surgeries and related applications. Most previous methods formulate this task as single-frame-based instance segmentation while ignoring the natural temporal and stereo attributes of a surgical video. As a result, these methods are less robust against the appearance variation through temporal motion and view change. In this work, we propose a novel LACOSTE model that exploits Location-Agnostic COntexts in Stereo and TEmporal images for improved surgical instrument segmentation. Leveraging a query-based segmentation model as core, we design three performance-enhancing modules. Firstly, we design a disparity-guided feature propagation module to enhance depth-aware features explicitly. To generalize well for even only a monocular video, we apply a pseudo stereo scheme to generate complementary right images. Secondly, we propose a stereo-temporal set classifier, which aggregates stereo-temporal contexts in a universal way for making a consolidated prediction and mitigates transient failures. Finally, we propose a location-agnostic classifier to decouple the location bias from mask prediction and enhance the feature semantics. We extensively validate our approach on three public surgical video datasets, including two benchmarks from EndoVis Challenges and one real radical prostatectomy surgery dataset GraSP. Experimental results demonstrate the promising performances of our method, which consistently achieves comparable or favorable results with previous state-of-the-art approaches.

The Efficacy of the Temporal View in Detection of Shallow Anterior Chamber

The Efficacy of the Temporal View in Detection of Shallow Anterior Chamber

The Policy Potential of Monitoring Market Proximal Maritime Connectivity Over Time: The Case of Aotearoa New Zealand

AbstractA country’s maritime connectivity is integral to its international trade and regional economic growth. Advancements in technology and complex network theory have elevated maritime connectivity in regional transport policy agendas. This study introduces a novel conceptualisation of the maritime network called the market proximal network which better highlights connectivity concerns and opportunities that leverage a country’s current market position. By implementing a temporal view (using Time-Granularity Graphs) of the market proximal network, connectivity trends can be appreciated. The policy value of this approach is illustrated by the case of Aotearoa New Zealand, a maritime connectivity taker, between January 2018 and June 2021. Five maritime connectivity priorities are identified regarding the country’s gateways onto the major maritime highways. The potential policy value of the temporal view of the market proximal network is illustrated by identifying priorities that are actionable because they are based on the existing business cases that draw shipping companies to call in Aotearoa New Zealand.

Imaging the large-scale and cellular response to focal traumatic brain injury in mouse neocortex.

Traumatic brain injury (TBI) affects neural function at the local injury site and also at distant, connected brain areas. However, the real-time neural dynamics in response to injury and subsequent effects on sensory processing and behavior are not fully resolved, especially across a range of spatial scales. We used in vivo calcium imaging in awake, head-restrained male and female mice to measure large-scale and cellular resolution neuronal activation, respectively, in response to a mild/moderate TBI induced by focal controlled cortical impact (CCI) injury of the motor cortex (M1). Widefield imaging revealed an immediate CCI-induced activation at the injury site, followed by a massive slow wave of calcium signal activation that traveled across the majority of the dorsal cortex within approximately 30 s. Correspondingly, two-photon calcium imaging in primary somatosensory cortex (S1) found strong activation of neuropil and neuronal populations during the CCI-induced traveling wave. A depression of calcium signals followed the wave, during which we observed atypical activity of a sparse population of S1 neurons. Longitudinal imaging in the hours and days after CCI revealed increases in the area of whisker-evoked sensory maps at early time points, in parallel to decreases in cortical functional connectivity and behavioral measures. Neural and behavioral changes mostly recovered over hours to days in our M1-TBI model, with a more lasting decrease in the number of active S1 neurons. Our results in unanesthetized mice describe novel spatial and temporal neural adaptations that occur at cortical sites remote to a focal brain injury.

How to stay competitive: An innovative concept to assess the business competitiveness using online restaurant reviews

Nowadays customers would make decisions by reading online reviews and comparing differences in restaurants before visiting. Therefore how restaurants take advantage from such information is important to attract customers and stay competitive. Many researchers believe that increasing business performance could improve competitiveness. However, with changing customer requirements and business environment, it is challenging to understand which attributes matter the most to customers and how to improve considering competitors. Therefore this study proposed assessing restaurant competitiveness using online reviews. After crawling 38,479 online reviews employing Python, deep learning-based BERT is developed to measure attribute performance and understand the competitiveness through McKinsey Matrix. Then, the competitiveness was analyzed from a temporal dynamic view to present how attributes are changing importance. Notably, the asymmetric effects between attribute performance and satisfaction were considered. Results demonstrated encouraging accuracy in measuring restaurant competitiveness and explained how asymmetric McKinsey Matrix could help formulate efficient competitiveness enhancement strategies.

Transforming the Niche: The Emerging Role of Extracellular Vesicles in Acute Myeloid Leukaemia Progression.

Acute myeloid leukaemia (AML) management remains a significant challenge in oncology due to its low survival rates and high post-treatment relapse rates, mainly attributed to treatment-resistant leukaemic stem cells (LSCs) residing in bone marrow (BM) niches. This review offers an in-depth analysis of AML progression, highlighting the pivotal role of extracellular vesicles (EVs) in the dynamic remodelling of BM niche intercellular communication. We explore recent advancements elucidating the mechanisms through which EVs facilitate complex crosstalk, effectively promoting AML hallmarks and drug resistance. Adopting a temporal view, we chart the evolving landscape of EV-mediated interactions within the AML niche, underscoring the transformative potential of these insights for therapeutic intervention. Furthermore, the review discusses the emerging understanding of endothelial cell subsets' impact across BM niches in shaping AML disease progression, adding another layer of complexity to the disease progression and treatment resistance. We highlight the potential of cutting-edge methodologies, such as organ-on-chip (OoC) and single-EV analysis technologies, to provide unprecedented insights into AML-niche interactions in a human setting. Leveraging accumulated insights into AML EV signalling to reconfigure BM niches and pioneer novel approaches to decipher the EV signalling networks that fuel AML within the human context could revolutionise the development of niche-targeted therapy for leukaemia eradication.

Multi-geometry embedded transformer for facial expression recognition in videos

Dynamic facial expressions in videos express more realistic emotional states, and recognizing emotions from in-the-wild facial expression videos is a challenging task due to the changeable posture, partial occlusion and various light conditions. Although current methods have designed transformer-based models to learn spatial–temporal features, they cannot explore useful local geometry structures from both spatial and temporal views to capture subtle emotional features for the videos with varied poses and facial occlusion. To this end, we propose a novel multi-geometry embedded transformer (MGET), which adapts multi-geometry knowledge into transformers and excavates spatial–temporal geometry information as complementary to learn effective emotional features. Specifically, from a new perspective, we first design a multi-geometry distance learning (MGDL) to capture emotion-related geometry structure knowledge under Euclidean and Hyperbolic spaces. Especially based on the advantages of hyperbolic geometry, it finds the more subtle emotional changes among local spatial and temporal features. Secondly, we combine MGDL with transformer to design spatial–temporal MGETs, which capture important spatial and temporal multi-geometry features to embed them into their corresponding original features, and then perform cross-regions and cross-frame interaction on these multi-level features. Finally, MGET gains superior performance on DFEW, FERV39k and AFEW datasets, where the unweighted average recall (UAR) and weighted average recall (WAR) are 58.65%/69.91%, 41.91%/50.76% and 53.23%/55.40%, respectively, and the gained improvements are 2.55%/0.66%, 3.69%/2.63% and 3.66%/1.14% compared to M3DFEL, Logo-Forme and EST methods.

MVSTT: A Multiview Spatial-Temporal Transformer Network for Traffic-Flow Forecasting.

Accurate traffic-flow prediction remains a critical challenge due to complicated spatial dependencies, temporal factors, and unpredictable events. Most existing approaches focus on single-or dual-view learning and thus face limitations in systematically learning complex spatial-temporal features. In this work, we propose a novel multiview spatial-temporal transformer (MVSTT) network that can effectively learn complex spatial-temporal domain correlations and potential patterns from multiple views. First, we examine a temporal view and design a short-range gated convolution component from a short-term subview, and a long-range gated convolution component from a long-term subview. These two components effectively aggregate knowledge of the temporal domain at multiple granularities and mine patterns of node evolution across time steps. Meanwhile, in the spatial view, we design a dual-graph spatial learning module that captures fixed and dynamic spatial dependencies of nodes, as well as the evolution patterns of edges, from the static and dynamic graph subviews, respectively. In addition, we further design a spatial-temporal transformer to mine different levels of spatial-temporal features through multiview knowledge fusion. Extensive experiments on four real-world traffic datasets show that our method consistently outperforms the state-of-the-art baseline. The code of MVSTT is available at https://github.com/JianSoL/MVSTT.

Re-oligotrophication and warming stabilize phytoplankton networks

Phytoplankton taxa are strongly interconnected as a network, which could show temporal dynamics and non-linear responses to changes in drivers at both seasonal and long-term scale. Using a high quality dataset of 20 Danish lakes (1989–2008), we applied extended Local Similarity Analysis to construct temporal network of phytoplankton communities for each lake, obtained sub-network for each sampling month, and then measured indices of network complexity and stability for each sub-network. We assessed how lake re-oligotrophication, climate warming and grazers influenced the temporal dynamics on network complexity and stability of phytoplankton community covering three aspects: seasonal trends, long-term trends and detrended variability. We found strong seasonality for the complexity and stability of phytoplankton network, an increasing trend for the average degree, modularity, nestedness, persistence and robustness, and a decreasing trend for connectance, negative:positive interactions and vulnerability. Our study revealed a cascading effect of lake re-oligotrophication, climate warming and zooplankton grazers on phytoplankton network stability through changes in network complexity characterizing diversity, interactions and topography. Network stability of phytoplankton increased with average degree, modularity, nestedness and decreased with connectance and negative:positive interactions. Oligotrophication and warming stabilized the phytoplankton network (enhanced robustness, persistence and decreased vulnerability) by enhancing its average degree, modularity, nestedness and by reducing its connectance, while zooplankton richness promoted stability of phytoplankton network through increases in average degree and decreases in negative interactions. Our results further indicate that the stabilization effects might lead to more closed, compartmentalized and nested interconnections especially in the deeper lakes, in the warmer seasons and during bloom periods. From a temporal dynamic network view, our findings highlight stabilization of the phytoplankton community as an adaptive response to lake re-oligotrophication, climate warming and grazers.

Multi-layer representation for cross-view action recognition

Daily action recognition is an important application domain in computer vision. Considering the complex temporal structure and view changes of daily actions, we propose a multi-layer representation for cross-view action recognition (MRCAR) based on two-stage optimization using a training set and a validation set. In the first stage, a cross-view atomic dictionary learning model is constructed based on jointly sparse constraints, utilizing the original features of the training set. Subsequently, a shared atomic dictionary, containing representative view-independent information, is obtained through alternating optimization. In the second stage, a reconstruction function based on the obtained shared atomic dictionary is developed. This function is employed to acquire sparse codes for the training and validation sets, enhancing the representativeness of motion atoms and improving overall recognition performance. Finally, experimental results from the WVU (West Virginia University) dataset and the NTU (Nanyang Technological University) RGB-D 120 dataset demonstrate that the proposed MRCAR method achieves an accuracy improvement of 3%-20% over recent state-of-the-art methods.

Deep Reinforcement Learning for Voltage Control and Renewable Accommodation Using Spatial-Temporal Graph Information

Renewable energy resources (RERs) have been increasingly integrated into distribution networks (DNs) for decarbonization. However, the variable nature of RERs introduces uncertainties to DNs, frequently resulting in voltage fluctuations that threaten system security and hamper the further adoption of RERs. To incentivize more RER penetration, we propose a deep reinforcement learning (DRL)-based strategy to dynamically balance the trade-off between voltage fluctuation control and renewable accommodation. To further extract multi-time-scale spatial-temporal (ST) graphical information of a DN, our strategy draws on a multi-grained attention-based spatial-temporal graph convolution network (MG-ASTGCN), consisting of ST attention mechanism and ST convolution to explore the node correlations in the spatial and temporal views. The continuous decision-making process of balancing such a trade-off can be modeled as a Markov decision process optimized by the deep deterministic policy gradient (DDPG) algorithm with the help of the derived ST information. We validate our strategy on the modified IEEE 33, 69, and 118-bus radial distribution systems, with outcomes significantly outperforming the optimization-based benchmarks. Simulations also reveal that our developed MG-ASTGCN can to a great extent accelerate the convergence speed of DDPG and improve its performance in stabilizing node voltage in an RER-rich DN. Moreover, our method improves the DN's robustness in the presence of generator failures.

Sleep Stage Classification Via Multi-View Based Self-Supervised Contrastive Learning of EEG.

Self-supervised learning (SSL) is a challenging task in sleep stage classification (SSC) that is capable of mining valuable representations from unlabeled data. However, traditional SSL methods typically focus on single-view learning and do not fully exploit the interactions among information across multiple views. In this study, we focused on a multi-domain view of the same EEG signal and developed a self-supervised multi-view representation learning framework via time series and time-frequency contrasting (MV-TTFC). In the MV-TTFC framework, we built-in a cross-domain view contrastive learning prediction task to establish connections between the temporal view and time-frequency (TF) view, thereby enhancing the information exchange between multiple views. In addition, to improve the quality of the TF view inputs, we introduced an enhanced multisynchrosqueezing transform, which can create high energy concentration TF image views to compensate for the inaccurate representations in traditional TF processing techniques. Finally, integrating temporal, TF, and fusion space contrastive learning effectively captured the latent features in EEG signals. We evaluated MV-TTFC based on two real-world SSC datasets (SleepEDF-78 and SHHS) and compared it with baseline methods in downstream tasks. Our method exhibited state-of-the-art performance, achieving accuracies of 78.64% and 81.45% with SleepEDF-78 and SHHS, respectively, and macro F1-scores of 70.39% with SleepEDF-78 and 70.47% with SHHS.

Analyzing the Temporal Theme and View of Time in Li Jinfa's Poetry

As a pioneer of symbolism in modern Chinese poetry, Li Jinfa's works have received complex evaluations in literary history. In the first half of the twentieth century, Li Jinfa's poetry contrasted sharply with the realism and romanticism of the time, and his obscure style, though criticized, opened up new aesthetic paths in poetry. This paper delves into the theme of time and the view of time in Li Jinfa's poetry, revealing his deep feelings about the passage of time and his reflections on modernity through a close reading of Li Jinfa's poetry. Starting from the theme of time in Li Jinfa's poetry, this paper explains Li's attitude towards the past and the present moment, as well as Li's criticism of the negative effects of modernity. From a cross-section of the single image of "bee", the paper also examines the special position and meaning of "bee" in Li Jinfa's imagery system. In conclusion, this study illustrates Lee Goldfarb's reflection on linear time and the heterotopias and heterotopias he creates in his poetry, demonstrating a rebellion against the rhythms of modern life and an emphasis on momentary experience.

Analyzing temporal coherence for deepfake video detection

<abstract><p>Current facial image manipulation techniques have caused public concerns while achieving impressive quality. However, these techniques are mostly bound to a single frame for synthesized videos and pay little attention to the most discriminatory temporal frequency artifacts between various frames. Detecting deepfake videos using temporal modeling still poses a challenge. To address this issue, we present a novel deepfake video detection framework in this paper that consists of two levels: temporal modeling and coherence analysis. At the first level, to fully capture temporal coherence over the entire video, we devise an efficient temporal facial pattern (TFP) mechanism that explores the color variations of forgery-sensitive facial areas by providing global and local-successive temporal views. The second level presents a temporal coherence analyzing network (TCAN), which consists of novel global temporal self-attention characteristics, high-resolution fine and low-resolution coarse feature extraction, and aggregation mechanisms, with the aims of long-range relationship modeling from a local-successive temporal perspective within a TFP and capturing the vital dynamic incoherence for robust detection. Thorough experiments on large-scale datasets, including FaceForensics++, DeepFakeDetection, DeepFake Detection Challenge, CelebDF-V2, and DeeperForensics, reveal that our paradigm surpasses current approaches and stays effective when detecting unseen sorts of deepfake videos.</p></abstract>

A Temporal View of Business Analytics in Interorganizational Relationships: Enablers and Barriers to Value Creation

A Temporal View of Business Analytics in Interorganizational Relationships: Enablers and Barriers to Value Creation

Weakly Supervised Monocular 3D Object Detection by Spatial-Temporal View Consistency.

Monocular 3D object detection plays a crucial role In the field of self-driving cars, estimating the size and location of objects solely based on input images. However, a notable disparity exists between the training and inference of 3D object detectors. This discrepancy arises because during inference, monocular 3D detectors depend solely on images captured by cameras; while during training, these methods require 3D ground truths labeled on point cloud data, which is obtained using specialized devices like LiDAR. This discrepancy creates a break in the data loop, preventing the feedback data from production cars from being utilized to enhance the robustness of the detectors. To address this issue and establish a connection in the data loop, we present a weakly-supervised solution that trains monocular 3D object detectors solely using 2D labels, eliminating the requirement for 3D ground truths. Our approach considers two view consistency: spatial and temporal view consistency, which play a crucial role in regulating the prediction of 3D bounding boxes. Spatial view consistency is achieved by employing projection and multi-view consistency techniques to guide the optimization of the target's location and size. We leverage temporal viewpoint consistency to provide temporal multi-view image pairs, and we further introduce temporal movement consistency to tackle the challenge of dynamic scenes. With only 2D ground truths, our method achieves comparable performance to fully supervised methods. Additionally, our method can be employed as a pre-training method and achieves significant improvement when fine-tuned with a small proportion of fully supervised labels.

Gentrification and changing visual landscapes: a Google Street View analysis of residential upgrading and class aesthetics in Hamilton’s Lower City

Measuring the pace and spatial distribution of gentrification is important to developing policies to mitigate its negative consequences. Typically, this is done through an analysis of census data on demographic, socioeconomic or housing change. However, this approach has numerous shortcomings, including the homogenizing effect on differences within neighbourhoods and the infrequency of census data collection. Visual analysis, particularly when examining multiple temporal views of the same location, has the potential to render visible fine-grained detail about spatial, economic and cultural changes within the urban landscape. Google Street View (GSV) is emerging as a source of repeat photography data. In this article, we employ a GSV analysis within a number of neighbourhoods in Hamilton, Ontario. Coding and analysing GSV images between 2009 and 2021 reveals an array of specific home upgrades, as well as aesthetic changes that reflect middle-class tastes, values and lifestyles that suggest more upgrading than found within conventional statistics or dominant narratives about the city. Mapping these changes paints a complex, and fine-grained, block-by-block picture of gentrification that reveals why some areas are more conducive to gentrification than others. Our analysis is important for critical visual methodologies, theoretical discussions about gentrification and neighbourhood change theories and debates within planning and policymaking.

The Temporality of Project Success: Vindeby, the World’s First Offshore Wind Farm

This article advances a temporal understanding of project success through a process study of Vindeby, an exploratory project developing the world’s first offshore wind farm. Pursuing a situated temporal view, our findings reveal how actors constructed Vindeby’s success differently when seeing the project as future, present, and past and how these constructions mutually shaped each other. Adding to prior literature adopting an over-time or in-time perspective, we develop a through-time perspective of project success and a model explaining the interplay of the three perspectives. We discuss how projects may serve as temporal stepping stones toward sustainable futures in the green transition and propose ways for project managers and policymakers to nurture this potential.

Spatio-temporal fusion and contrastive learning for urban flow prediction

Urban flow prediction is critical for urban planning, management, and safety. However, owing to the inherent instability of urban flows, prediction accuracy requires the fusion of multi-view influencing factors. Current prediction methods are insensitive to periodic changes in urban flows, and rarely consider the implied spatial and temporal correlations between similar functional areas. Thus, we propose a method based on spatiotemporal fusion and contrastive learning. We construct an extraction module of spatial and temporal views based on contrastive learning in the spatial and temporal dimensions. With the temporal view extraction, we can obtain the distribution and change in the global urban flow periodicity. Using the spatial view extraction, we can obtain the implied flow variation relationship between similar regions. Therefore, our model can capture the high-level semantic features of urban flow changes from multiple views. We also design a multi-view fusion and prediction network to combine multi-view representations that impact urban flow. We experimentally evaluated our proposed approach using two real-world datasets and demonstrated state-of-the-art forecasting performance, as well as effectiveness in resource-limited environments.

On the consensus of synchronous temporal and spatial views: A novel multimodal deep learning method for social video prediction

The blowout development of video social platforms has spawned a wide range of social video prediction (SVP) tasks, such as video attractiveness prediction and video sentiment classification. In this paper, we propose to enhance SVP by making synchronous predictions based on temporal and spatial data perspectives and reconciling them to form a consistent predictive view. To this end, we develop a novel multimodal deep learning method named MATSC (modality-awareness- and temporal-spatial-consistency-based neural network). Specifically, MATSC first constructs the temporal predictive view by capturing valuable fine-grained data patterns and generating diverse multimodal representations via the modality-awareness learning strategy. Then, MATSC constructs the spatial predictive view by exploiting diverse modality-wise interactive patterns in fine-grained video clips. Third, MATSC reconciles the heterogeneous temporal and spatial predictive capabilities via a temporal-spatial-consistency learning objective. Empirical results based on three SVP datasets show the outperformance of MATSC over state-of-the-art benchmarks, demonstrating the enhancement effect of synergizing temporal and spatial data views for SVP tasks.