Integration Of Spatial Information Research Articles

HyperGCN: an effective deep representation learning framework for the integrative analysis of spatial transcriptomics data

BackgroundAdvances of spatial transcriptomics technologies enabled simultaneously profiling gene expression and spatial locations of cells from the same tissue. Computational tools and approaches for integration of transcriptomics data and spatial context information are urgently needed to comprehensively explore the underlying structure patterns. In this manuscript, we propose HyperGCN for the integrative analysis of gene expression and spatial information profiled from the same tissue. HyperGCN enables data visualization and clustering, and facilitates downstream analysis, including domain segmentation, the characterization of marker genes for the specific domain structure and GO enrichment analysis.ResultsExtensive experiments are implemented on four real datasets from different tissues (including human dorsolateral prefrontal cortex, human positive breast tumors, mouse brain, mouse olfactory bulb tissue and Zabrafish melanoma) and technologies (including 10X visium, osmFISH, seqFISH+, 10X Xenium and Stereo-seq) with different spatial resolutions. The results show that HyperGCN achieves superior clustering performance and produces good domain segmentation effects while identifies biologically meaningful spatial expression patterns. This study provides a flexible framework to analyze spatial transcriptomics data with high geometric complexity.ConclusionsHyperGCN is an unsupervised method based on hypergraph induced graph convolutional network, where it assumes that there existed disjoint tissues with high geometric complexity, and models the semantic relationship of cells through hypergraph, which better tackles the high-order interactions of cells and levels of noise in spatial transcriptomics data.

SpaNCMG: improving spatial domains identification of spatial transcriptomics using neighborhood-complementary mixed-view graph convolutional network.

The advancement of spatial transcriptomics (ST) technology contributes to a more profound comprehension of the spatial properties of gene expression within tissues. However, due to challenges of high dimensionality, pronounced noise and dynamic limitations in ST data, the integration of gene expression and spatial information to accurately identify spatial domains remains challenging. This paper proposes a SpaNCMG algorithm for the purpose of achieving precise spatial domain description and localization based on a neighborhood-complementary mixed-view graph convolutional network. The algorithm enables better adaptation to ST data at different resolutions by integrating the local information from KNN and the global structure from r-radius into a complementary neighborhood graph. It also introduces an attention mechanism to achieve adaptive fusion of different reconstructed expressions, and utilizes KPCA method for dimensionality reduction. The application of SpaNCMG on five datasets from four sequencing platforms demonstrates superior performance to eight existing advanced methods. Specifically, the algorithm achieved highest ARI accuracies of 0.63 and 0.52 on the datasets of the human dorsolateral prefrontal cortex and mouse somatosensory cortex, respectively. It accurately identified the spatial locations of marker genes in the mouse olfactory bulb tissue and inferred the biological functions of different regions. When handling larger datasets such as mouse embryos, the SpaNCMG not only identified the main tissue structures but also explored unlabeled domains. Overall, the good generalization ability and scalability of SpaNCMG make it an outstanding tool for understanding tissue structure and disease mechanisms. Our codes are available at https://github.com/ZhihaoSi/SpaNCMG.

Revealing the urban heat Island: Investigating spatiotemporal surface temperature dynamics, modeling, and interactions with controllable and non-controllable factors

The unchecked growth of urban populations has spurred rapid and unsustainable urban expansion, exacerbating environmental degradation both locally and globally. This research offers a comprehensive analysis of the Urban Heat Island (UHI) phenomenon in Bangalore and Hyderabad, India, focusing on its spatial and temporal distribution and its correlation with air pollutants. Conducted over the course of summer and winter seasons from 2001 to 2021, this study advances our understanding of UHI dynamics in rapidly urbanizing regions, shedding light on the complex interplay between urbanization, climate, and air quality. The primary objective of this study is to assess and compare the predictive performance of baseline Random Forest (RF) and Extreme Gradient Boosting (XGBoost) models for forecasting Land Surface Temperature (LST) in Hyderabad, India, while examining the impact of integrating spatial information into these models. Incorporating an extensive array of input variables, ranging from spectral indices to terrain features, land use/land cover (LULC) data, atmospheric parameters, and spatial data, the analysis reveals significant relationships between LST and various factors. Notably, Aerosol Optical Depth (AOD) and Normalized Difference Built-up Index (NDBI) exhibit strong positive correlations with LST, while Enhanced Vegetation Index (EVI) demonstrates a pronounced negative correlation. Similarly, Modified Normalized Difference Water Index (MNDWI) displays the strongest positive correlation with LST (r = 0.49), whereas EVI exhibits the most notable negative correlation (r = −0.67). Additionally, Modified Bare Soil Index (MBI) demonstrates a noteworthy negative correlation (r = −0.48) with LST for Bangalore. Atmospheric components such as CO, NO2, and O3 also show significant positive correlations with LST, with NO2 displaying the highest correlation during winter in urban areas. XGBoost emerges as the superior model, achieving high R2 values of 0.84 for Bangalore and 0.88 for Hyderabad. Integration of spatial information consistently enhances the performance of both RF and XGBoost models, resulting in reduced Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) values. However, predictions over water bodies tend to be underestimated. The study underscores the importance of incorporating spatial data for accurate LST prediction, with implications for environmental monitoring and management. Despite the advancements made, further research is needed to elucidate the specific mechanisms through which spatial information improves model performance, suggesting avenues for future investigation and refinement of modeling methodologies. At the intersection of rapid urban growth and environmental conservation, these findings are poised to inform policymakers, urban planners, and researchers, guiding them towards innovative and sustainable urban development strategies aimed at mitigating the adverse impacts of rapid urban expansion while promoting ecosystem well-being. The study contributes to spatial modeling strategies and refines LST prediction methodologies, offering valuable insights for addressing the challenges posed by urbanization and climate change in rapidly urbanizing regions.

Deep Fusion Module for Video Action Recognition

In video action recognition, effective spatiotemporal modeling is crucial. However, traditional two-stream methods face challenges in integrating spatial information from RGB images and temporary information from optical flow without long-range temporal modelling. To address these limitations, we propose the Deep Fusion Module (DFM), which focuses on the deep fusion of spatial and temporal information and consists of two components. First, we propose an Attention Fusion Module (AFM) to effectively fuse the shallow features obtained from a two-stream network, thereby facilitating the integration of spatial and temporal information. Next, we incorporate a SpatioTemporal Module (STM), comprising a ConvGRU and a 1×1 convolution, to model long-range temporal dependency and fuse spatial-temporal features. Experiments on the UCF101 dataset show that our method achieves 96.5% accuracy, outperforming baseline two-stream models by 0.3%.

BiLSTM Classifier Model for Land Cover Change Detection and Classification

Objective: This research proposes a change detection in the satellite images and land cover analysis using a Bidirectional Long Short-Term Memory (BiLSTM) model and vegetation index-based feature maps for future map generation. Change detection complements land cover analysis by identifying and quantifying alterations in land cover over time. Theoretical framework: Land cover analysis and land cover change detection plays crucial role in understanding and monitoring the Earth's surface.Despite its importance, land cover analysis and change detection face several challenges. One major challenge is obtaining accurate and up-to-date information on land cover and change detection across large areas can be complex and costly. Inconsistent data sources and limited access to historical records can hinder the accuracy and reliability of change detection analyses.These challenges requires a combination of technological advancements, improved data availability, refined algorithms, robust validation approaches, and interdisciplinary collaborations. Method: BiLSTM method is used for implementation which is powerful for land cover classification, as it allowing the integration of spatial and temporal information and capturing complex patterns in satellite imagery data. BiLSTM, tend to be more complex but they often offer higher accuracy and the ability to learn intricate patterns and representations. Results and conclusion: The six vegetation index-based feature maps are considered.Therefore, the resulting accuracy is also determined using the Flamingo-Hyena optimization (FHO), and the experimental outcomes disclosed that the proposed model is superior to the existing model in terms of accuracy with 0.95%, MSE with 0.05%, precision with 0.94%, Recall with 0.94%, and F1 measure with 0.94% respectively. vegetation index-based feature maps are essential for land cover analysis and change detection as they provide valuable information on vegetation dynamics, ecological processes, land management, and climate change impacts. Research implications: This process land cover analysis and change detection helps to detect deforestation, urban expansion, agricultural expansion, and other land cover changes. By harnessing these techniques, policy makers can address emerging issues such as deforestation, loss of biodiversity, and encroachment on natural habitats.

Conditioned preferences: Gated by experience, context, and endocrine systems

Central to the navigation of an ever-changing environment is the ability to form positive associations with places and conspecifics. The functions of location and social conditioned preferences are often studied independently, limiting our understanding of their interplay. Furthermore, a de-emphasis on natural functions of conditioned preferences has led to neurobiological interpretations separated from ecological context. By adopting a naturalistic and ethological perspective, we uncover complexities underlying the expression of conditioned preferences. Development of conditioned preferences is a combination of motivation, reward, associative learning, and context, including for social and spatial environments. Both social- and location-dependent reward-responsive behaviors and their conditioning rely on internal state-gating mechanisms that include neuroendocrine and hormone systems such as opioids, dopamine, testosterone, estradiol, and oxytocin. Such reinforced behavior emerges from mechanisms integrating past experience and current social and environmental conditions. Moreover, social context, environmental stimuli, and internal state gate and modulate motivation and learning via associative reward, shaping the conditioning process. We highlight research incorporating these concepts, focusing on the integration of social neuroendocrine mechanisms and behavioral conditioning. We explore three paradigms: 1) conditioned place preference, 2) conditioned social preference, and 3) social conditioned place preference. We highlight nonclassical species to emphasize the naturalistic applications of these conditioned preferences. To fully appreciate the complex integration of spatial and social information, future research must identify neural networks where endocrine systems exert influence on such behaviors. Such research promises to provide valuable insights into conditioned preferences within a broader naturalistic context.

Multi-scale spatial features and temporal attention mechanisms: advancing the accuracy of ENSO prediction

The exponential progression in oceanic observational technology has fostered the accumulation of substantial time series data pivotal for predictions in ocean meteorology. Foremost among the phenomena observed is El Niño-Southern Oscillation (ENSO), a critical determinant in the interplay of global ocean atmosphere interactions, with its severe manifestations inducing extreme meteorological conditions. Therefore, precisely predicting ENSO events carries immense gravitas. Historically, predictions hinged primarily on dynamic models and statistical approaches; however, the intricate and multifaceted spatiotemporal dynamics of ENSO events have often impeded the accuracy of these traditional methodologies. A notable lacuna in contemporary research is the insufficient exploration of long-term dependencies within oceanic data and the suboptimal integration of spatial information derived from spatiotemporal data. To address these limitations, this study introduces a forward-thinking ENSO prediction framework synergizing multiscale spatial features with temporal attention mechanisms. This innovation facilitates a more profound exploration of temporal and spatial domains, enhancing the retention of extensive-period data while optimizing the use of spatial information. Preliminary analyses executed on the global ocean data assimilation system dataset attest to the superior efficacy of the proposed method, underscoring a substantial improvement over established methods including SA-convolutional long short-term memory, particularly in facilitating long-term predictions.The source code and datasets are provided. The code is available at https://github.com/tse1998/ENSO-prediction.

Leveraging adaptive spatial shifts multi-layer perceptron with parameterized spatial offsets for hyperspectral image classification

ABSTRACT The architectures based on Multi-Layer Perceptron (MLP) have attracted great attention in hyperspectral image (HSI) classification recently, due to their simplified and efficient architectures. However, such architectures are qualified by the rigid positional relationships between weights and feature elements, inhibiting their capacity to effectively extract diversified features. To address these challenges, An adaptive spatial-shift MLP (AS2MLP) is presented to dynamically modify spatial features by parameterizing learnable spatial offsets. In this way, the AS2MLP can facilitate sample-specific spatial shifts, aligning spatial structures more effectively. Then, An innovative adaptive spatial-shift block (AS2block) is designed to adaptively shift spatial features along distinct spatial axes, enabling the extraction of diversified features separately. It also implements a re-weighting strategy to mitigate redundant features. Building on this foundation, the proposed adaptive spatial-shift network (AS2Net) is for HSI classification. The dual-path AS2Net employs AS2blocks and MLPs for channel mixing, facilitating an adaptive integration of dynamic spatial contextual information dispersed across a range of spectra. The effectiveness of this model is demonstrated using five widely used HSI datasets.

HEMAsNet: A Hemisphere Asymmetry Network Inspired by the Brain for Depression Recognition From Electroencephalogram Signals.

Depression is a prevalent mental disorder that affects a significant portion of the global population. Despite recent advancements in EEG-based depression recognition models rooted in machine learning and deep learning approaches, many lack comprehensive consideration of depression's pathogenesis, leading to limited neuroscientific interpretability. To address these issues, we propose a hemisphere asymmetry network (HEMAsNet) inspired by the brain for depression recognition from EEG signals. HEMAsNet employs a combination of multi-scale Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) blocks to extract temporal features from both hemispheres of the brain. Moreover, the model introduces a unique 'Callosum- like' block, inspired by the corpus callosum's pivotal role in facilitating inter-hemispheric information transfer within the brain. This block enhances information exchange between hemispheres, potentially improving depression recognition accuracy. To validate the performance of HEMAsNet, we first confirmed the asymmetric features of frontal lobe EEG in the MODMA dataset. Subsequently, our method achieved a depression recognition accuracy of 0.8067, indicating its effectiveness in increasing classification performance. Furthermore, we conducted a comprehensive investigation from spatial and frequency perspectives, demonstrating HEMAsNet's innovation in explaining model decisions. The advantages of HEMAsNet lie in its ability to achieve more accurate and interpretable recognition of depression through the simulation of physiological processes, integration of spatial information, and incorporation of the Callosum- like block.

Grounding End-to-End Pre-trained architectures for Semantic Role Labeling in multiple languages

Situated natural language interactions between humans and robots are strictly necessary for complex applications: communication here implies the reference to the environment shared between a user and the robot. This paper proposes a transformer-based architecture that supports the integration of spatial information (as logical representation) about a semantic map of the environment and the input utterances. The generated interpretation is a logical form of the command that makes references to the state of the world through a single end-to-end process, stimulated at each interaction by an explicit linguistic description of the environment. In this specific work, the end-to-end capability of the targeted transformer is studied in light of its multilingual applications where the robot can be queried in different natural languages. The obtained experimental results confirm the applicability of transformers to grounded human-robotic interaction, with benefits in terms of both portability of the approach across domains and effectiveness in terms of reachable accuracy. Moreover, language-specific processing chains are shown to be preferable to large-scale multilingual models for their better trade-off between accuracy and complexity. Overall, the proposed architecture outperforms previous approaches and paves the way for sustainable multilingual architectures.

AUTOMATIC GENERATION OF ROUTING GRAPHS FOR INDOOR-OUTDOOR TRANSITIONAL SPACE TO SUPPORT SEAMLESS NAVIGATION

Abstract. With the fast development of urbanization, the complexity of built environments has dramatically increased, driving a need for assistance in seamless indoor-outdoor navigation. This requires integration of spatial information of indoor and outdoor environments from heterogeneous data sources. While outdoor road network data is largely available from many sources (such as OpenStreetMap), indoor spatial information is either inexistent or is inconsistently represented using several different standards. Among these standards, IndoorGML is a well-developed standard with the focus on indoor location-based services. This standard has already been accepted by Open Geospatial Consortium (OGC) and is now under active development. Although in IndoorGML some mechanisms have been defined to enable integration of indoor and outdoor networks, there is still a lack of concrete guidelines for determination of indoor-outdoor connections. It also lacks solid scientific foundations and efficient tools to extract the connecting nodes and edges that link indoor and outdoor spaces. To address this gap, in this study we focus on the connection of indoor and outdoor spaces and aim to provide a tool, which can automatically construct navigation graphs of the indoor-outdoor transitional space to support seamless integration of indoor-outdoor navigation. To this end, voxel-based modeling approaches are used to model the connecting space between indoor and outdoor environments. Based on Python, we develop the intended tool, which can generate voxel models from point clouds, identify navigable space by taking into account the characteristics of agents (such as pedestrians, wheelchairs, and vehicles), and automatically build navigation graphs linking IndoorGML networks with outdoor street networks. It is expected that the methodology and tools developed from this project will benefit the IndoorGML ecosystem and greatly advance the capability of IndoorGML in representing navigable space to support location-based services.

The role of conflict processing in multisensory perception: behavioural and electroencephalography evidence.

To form coherent multisensory perceptual representations, the brain must solve a causal inference problem: to decide if two sensory cues originated from the same event and should be combined, or if they came from different events and should be processed independently. According to current models of multisensory integration, during this process, the integrated (common cause) and segregated (different causes) internal perceptual models are entertained. In the present study, we propose that the causal inference process involves competition between these alternative perceptual models that engages the brain mechanisms of conflict processing. To test this hypothesis, we conducted two experiments, measuring reaction times (RTs) and electroencephalography, using an audiovisual ventriloquist illusion paradigm with varying degrees of intersensory disparities. Consistent with our hypotheses, incongruent trials led to slower RTs and higher fronto-medial theta power, both indicative of conflict. We also predicted that intermediate disparities would yield slower RTs and higher theta power when compared to congruent stimuli and to large disparities, owing to the steeper competition between causal models. Although this prediction was only validated in the RT study, both experiments displayed the anticipated trend. In conclusion, our findings suggest a potential involvement of the conflict mechanisms in multisensory integration of spatial information. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Spatial-MGCN: a novel multi-view graph convolutional network for identifying spatial domains with attention mechanism.

Recent advances in spatial transcriptomics technologies have enabled gene expression profiles while preserving spatial context. Accurately identifying spatial domains is crucial for downstream analysis and it requires the effective integration of gene expression profiles and spatial information. While increasingly computational methods have been developed for spatial domain detection, most of them cannot adaptively learn the complex relationship between gene expression and spatial information, leading to sub-optimal performance. To overcome these challenges, we propose a novel deep learning method named Spatial-MGCN for identifying spatial domains, which is a Multi-view Graph Convolutional Network (GCN) with attention mechanism. We first construct two neighbor graphs using gene expression profiles and spatial information, respectively. Then, a multi-view GCN encoder is designed to extract unique embeddings from both the feature and spatial graphs, as well as their shared embeddings by combining both graphs. Finally, a zero-inflated negative binomial decoder is used to reconstruct the original expression matrix by capturing the global probability distribution of gene expression profiles. Moreover, Spatial-MGCN incorporates a spatial regularization constraint into the features learning to preserve spatial neighbor information in an end-to-end manner. The experimental results show that Spatial-MGCN outperforms state-of-the-art methods consistently in several tasks, including spatial clustering and trajectory inference.

Geostatistical analysis of the integration of spatial information on nitrate–N observations and agricultural land uses for establishing groundwater pollution zones

Geostatistical analysis of the integration of spatial information on nitrate–N observations and agricultural land uses for establishing groundwater pollution zones

Spatially aware self-representation learning for tissue structure characterization and spatial functional genes identification.

Spatially resolved transcriptomics (SRT) enable the comprehensive characterization of transcriptomic profiles in the context of tissue microenvironments. Unveiling spatial transcriptional heterogeneity needs to effectively incorporate spatial information accounting for the substantial spatial correlation of expression measurements. Here, we develop a computational method, SpaSRL (spatially aware self-representation learning), which flexibly enhances and decodes spatial transcriptional signals to simultaneously achieve spatial domain detection and spatial functional genes identification. This novel tunable spatially aware strategy of SpaSRL not only balances spatial and transcriptional coherence for the two tasks, but also can transfer spatial correlation constraint between them based on a unified model. In addition, this joint analysis by SpaSRL deciphers accurate and fine-grained tissue structures and ensures the effective extraction of biologically informative genes underlying spatial architecture. We verified the superiority of SpaSRL on spatial domain detection, spatial functional genes identification and data denoising using multiple SRT datasets obtained by different platforms and tissue sections. Our results illustrate SpaSRL's utility in flexible integration of spatial information and novel discovery of biological insights from spatial transcriptomic datasets.

Abstract 4621: Distinct spatial and molecular maps in the tumour microenvironment of resistant melanoma patients treated with checkpoint inhibitor immunotherapy

Abstract Background: Checkpoint-based immunotherapies can achieve durable clinical responses in a variety of advanced cancers, including melanoma. However, clinical responses are not universal and there is still a substantial gap in understanding the mechanistic insights of immunotherapy resistance. This study aims to better understand patients’ resistance to immune checkpoint therapies by dissecting and tracing individual tumour and immune cell clones. Methods: Melanoma tumour dissociates and lesion matched formalin-fixed paraffin-embedded tissue samples were collected from patients with advanced melanoma receiving ipilimumab in combination with nivolumab; 2 patients were long-term responders, 2 had innate resistance and 1 acquired resistance. The molecular characteristics and spatial cellular relations associated with tumour and immune subsets in response to treatment were identified using single-cell indexing of transcriptomes and epitopes by sequencing and CODEX (CO-Detection by indEXing) multiplexed tissue imaging. Integration of spatial and molecular information was performed by combining reference-based phenotyping, Louvain clustering, and a mutual nearest neighbours-based transfer learning algorithm. Results: Pre-treatment melanoma biopsies from 5 patients (3 responding and 2 non-responding) along with a subsequent acquired resistance biopsy from an initially responding patient were analysed. Comparisons of cellular phenotypes identified higher proportions of gamma delta T cells (1.98% versus 0.2%) and enriched memory T cell profiles (CD3E, CD44, CD45RO, CD8) in responding compared to non-responding pre-treatment biopsies. Fibroblast from non-responding lesions overexpressed stromal related cytokine, IL1A (log2 fold change (FC) = 3.3) and CXCL1 (log2 FC = 1.4), which were enrichment at the tumour margin compared to responding lesions. Comparisons of a patient’s pre-treatment biopsy that initially responded to a subsequent progressing biopsy with acquired resistance had altered expression of transcript factor, ZFX (log2 FC = -1.59; adjusted p = 0.0119), and MBD2 (log2 FC = -1.5; adjusted p = 0.0225) within the patient’s memory T-cells. Conclusion: The combination of high dimensional single cell transcriptomics and spatial tissue imaging are effective tools to identify specific changes in cell phenotypes, intracellular signalling, and spatial location. This analysis identified reduced memory T cells, increased fibroblast, and altered cellular signalling were associated with resistance to immunotherapies. Citation Format: Camelia Quek, Aditya Pratapa, Ines Pires da Silva, Ghamdan Al-Eryani, Aaron Mayer, Nenad Bartonicek, Kate Harvey, Oliver Braubach, Robyn P.M. Saw, Jonathan Stretch, Kerwin F. Shannon, Alexander M. Menzies, Richard A. Scolyer, Georgina V. Long, Alexander Swarbrick, James S. Wilmott. Distinct spatial and molecular maps in the tumour microenvironment of resistant melanoma patients treated with checkpoint inhibitor immunotherapy. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 4621.

Smart Archive Management: Application of IoT, BD and GIS to Infrastructure ArchivesManagement

In the management of infrastructure archives, there are three problems that need to be solved: the lack of smart infrastructure archives management paradigm with systematic and standardization, the fact that 2-dimensional paper-based infrastructure archives cannot accurately describe three-dimensional infrastructure archive information, and the problem of "information gap" caused by the lack of integration of infrastructure spatial information and archives. This paper proposed a new paradigm of smart infrastructure archive management. The Smart Infrastructure Archive Management Paradigm takes realizes the perception of infrastructure archive resources by combining the Internet of things technology, cloud computing and big data technology, as well as geographic information system. This paper proposes the Physical Archives Management Platform, the Archives Smart Repository Management Platform, the Infrastructure Archives Visual Management Platform. The research outputs have been applied to the Archives Management of Zhengzhou University and achieved good feedback.

The Summarization of Split Brain Study of Last Few Decades

At the end of last century, scientists began to explore cortical circuit of cognition and memory with split-brain surgery. Later, they tried to treat epilepsy with split-brain surgery and achieved certain results. Since then, scientists have tried to understand the functional differences between the left and right hemispheres and the effects of the split brain on behavioral cognition by working with different animals as well as patients with the split brain. The ongoing study of split-brain provides theoretical explanations for clinical brain imaging findings and continues to help us understand the mechanisms of cognition and memory. Split-brain patients are always valuable samples for scientists to study inter-hemispheric structural and functional connectivity, differences in the two hemispheres, integration of sensory, motor, and spatial information, and general behaviour. The following information summarizes split-brain related concepts and includes methods and experiments to find the results for the studies mentioned above to help readers understand how these achievements were reached.

An Efficient Recurrent Adversarial Framework for Unsupervised Real-Time Video Enhancement

Video enhancement is a challenging problem, more than that of stills, mainly due to high computational cost, larger data volumes and the difficulty of achieving consistency in the spatio-temporal domain. In practice, these challenges are often coupled with the lack of example pairs, which inhibits the application of supervised learning strategies. To address these challenges, we propose an efficient adversarial video enhancement framework that learns directly from unpaired video examples. In particular, our framework introduces new recurrent cells that consist of interleaved local and global modules for implicit integration of spatial and temporal information. The proposed design allows our recurrent cells to efficiently propagate spatio-temporal information across frames and reduces the need for high complexity networks. Our setting enables learning from unpaired videos in a cyclic adversarial manner, where the proposed recurrent units are employed in all architectures. Efficient training is accomplished by introducing one single discriminator that learns the joint distribution of source and target domain simultaneously. The enhancement results demonstrate clear superiority of the proposed video enhancer over the state-of-the-art methods, in all terms of visual quality, quantitative metrics, and inference speed. Notably, our video enhancer is capable of enhancing over 35 frames per second of FullHD video (1080x1920).

Can rats and ants exchange information between the horizontal and vertical domains?

Since traveling in nature involves encountering various vertical structures, integration of horizontal and vertical spatial information is required. One form of such integration is to use information acquired in one plane for spatial navigation in another plane. Here we tested whether rats and ants that learned a reward location in a horizontal maze could utilize this information when the maze was rotated to a vertical orientation and vice versa. Rats that were trained in a horizontal Y-maze required more time to reach the reward when the maze was vertically rotated, but they were more accurate in choosing the correct arm. In contrast, rats tested in a horizontal maze after being trained in a vertical maze were less accurate but reached the reward faster. Changes after maze rotation were moderate and non-significant in ants, perhaps since the number of ants arriving at the reward increased over trials, diminishing the effect of maze rotation in ants compared to rats. According to the notion that horizontal spatial information is encoded in more detail than vertical information, the slow performance of rats in the vertical domain could be due to a more physically demanding task whereas their accuracy was due to a preceding detailed horizontal encoding. In contrast, rats in the vertical maze could gather less detailed information and therefore were less accurate in subsequent horizontal trials, where the lower energy cost enabled them to swiftly correct wrong choices. Altogether, the present results provide an indication for transferring spatial information between horizontal and vertical dimensions.