Temporal Object Research Articles

Computational temporal ghost imaging based on complementary modulation

Abstract We report an experimental demonstration of temporal ghost imaging in which a digital micromirror device (DMD) and +1/−1 binary modulation have been combined to give an accurate reconstruction of a nonperiodic time object. Compared to the 0/1 modulation, the reconstruction signal can be improved greatly by +1/−1 binary modulation even with half of the measurements. Experimental results show that 0/1 binary temporal objects up to 4 kHz and sinusoidal time objects up to 1 kHz can be reconstructed by this method. The influences of modulation speed and array detector gray levels are also discussed.

LiDAR-Based 3D Temporal Object Detection via Motion-Aware LiDAR Feature Fusion.

Recently, the growing demand for autonomous driving in the industry has led to a lot of interest in 3D object detection, resulting in many excellent 3D object detection algorithms. However, most 3D object detectors focus only on a single set of LiDAR points, ignoring their potential ability to improve performance by leveraging the information provided by the consecutive set of LIDAR points. In this paper, we propose a novel 3D object detection method called temporal motion-aware 3D object detection (TM3DOD), which utilizes temporal LiDAR data. In the proposed TM3DOD method, we aggregate LiDAR voxels over time and the current BEV features by generating motion features using consecutive BEV feature maps. First, we present the temporal voxel encoder (TVE), which generates voxel representations by capturing the temporal relationships among the point sets within a voxel. Next, we design a motion-aware feature aggregation network (MFANet), which aims to enhance the current BEV feature representation by quantifying the temporal variation between two consecutive BEV feature maps. By analyzing the differences and changes in the BEV feature maps over time, MFANet captures motion information and integrates it into the current feature representation, enabling more robust and accurate detection of 3D objects. Experimental evaluations on the nuScenes benchmark dataset demonstrate that the proposed TM3DOD method achieved significant improvements in 3D detection performance compared with the baseline methods. Additionally, our method achieved comparable performance to state-of-the-art approaches.

The social lives of point-of-care tests in low- and middle-income countries: a meta-ethnography.

Point-of-care tests (POCTs) have become technological solutions for many global health challenges. This meta-ethnography examines what has been learned about the 'social lives' of POCTs from in-depth qualitative research, highlighting key social considerations for policymakers, funders, developers and users in the design, development and deployment of POCTs. We screened qualitative research examining POCTs in low- and middle-income countries and selected 13 papers for synthesis. The findings illuminate five value-based logics-technological autonomy, care, scalability, rapidity and certainty-shaping global health innovation ecosystems and their entanglement with health systems. Our meta-ethnography suggests that POCTs never achieve the technological autonomy often anticipated during design and development processes. Instead, they are both embedded in and constitutive of the dynamic relationships that make up health systems in practice. POCTs are often imagined as caring commodities; however, in use, notions of care inscribed in these devices are constantly negotiated and transformed in relation to multiple understandings of care. POCTs promise to standardize care across scale, yet our analysis indicates nonstandard processes, diagnoses and treatment pathways as essential to 'fluid technologies' rather than dangerous aberrations. The rapidity of POCTs is constructed and negotiated within multiple distinct temporal registers, and POCTs operate as temporal objects that can either speed up or slow down experiences of diagnosis and innovation. Finally, while often valued as epistemic tools that can dispel diagnostic uncertainty, these papers demonstrate that POCTs contribute to new forms of uncertainty. Together, these papers point to knowledge practices as multiple, and POCTs as contributing to, rather than reducing, this multiplicity. The values embedded in POCTs are fluid and contested, with important implications for the kind of care these tools can deliver. These findings can contribute to more reflexive approaches to global health innovation, which take into account limitations of established global health logics, and recognize the socio-technical complexity of health systems.

Selective activations and functional connectivities to the sight of faces, scenes, body parts and tools in visual and non-visual cortical regions leading to the human hippocampus

Connectivity maps are now available for the 360 cortical regions in the Human Connectome Project Multimodal Parcellation atlas. Here we add function to these maps by measuring selective fMRI activations and functional connectivity increases to stationary visual stimuli of faces, scenes, body parts and tools from 956 HCP participants. Faces activate regions in the ventrolateral visual cortical stream (FFC), in the superior temporal sulcus (STS) visual stream for face and head motion; and inferior parietal visual (PGi) and somatosensory (PF) regions. Scenes activate ventromedial visual stream VMV and PHA regions in the parahippocampal scene area; medial (7m) and lateral parietal (PGp) regions; and the reward-related medial orbitofrontal cortex. Body parts activate the inferior temporal cortex object regions (TE1p, TE2p); but also visual motion regions (MT, MST, FST); and the inferior parietal visual (PGi, PGs) and somatosensory (PF) regions; and the unpleasant-related lateral orbitofrontal cortex. Tools activate an intermediate ventral stream area (VMV3, VVC, PHA3); visual motion regions (FST); somatosensory (1, 2); and auditory (A4, A5) cortical regions. The findings add function to cortical connectivity maps; and show how stationary visual stimuli activate other cortical regions related to their associations, including visual motion, somatosensory, auditory, semantic, and orbitofrontal cortex value-related, regions.

Mid-infrared computational temporal ghost imaging

Ghost imaging in the time domain allows for reconstructing fast temporal objects using a slow photodetector. The technique involves correlating random or pre-programmed probing temporal intensity patterns with the integrated signal measured after modulation by the temporal object. However, the implementation of temporal ghost imaging necessitates ultrafast detectors or modulators for measuring or pre-programming the probing intensity patterns, which are not available in all spectral regions especially in the mid-infrared range. Here, we demonstrate a frequency downconversion temporal ghost imaging scheme that enables to extend the operation regime to arbitrary wavelengths regions where fast modulators and detectors are not available. The approach modulates a signal with temporal intensity patterns in the near-infrared and transfers the patterns to an idler via difference-frequency generation in a nonlinear crystal at a wavelength where the temporal object can be retrieved. As a proof-of-concept, we demonstrate computational temporal ghost imaging in the mid-infrared with operating wavelength that can be tuned from 3.2 to 4.3 μm. The scheme is flexible and can be extended to other regimes. Our results introduce new possibilities for scan-free pump-probe imaging and the study of ultrafast dynamics in spectral regions where ultrafast modulation or detection is challenging such as the mid-infrared and THz regions.

T-UNet: triplet UNet for change detection in high-resolution remote sensing images

ABSTRACT Remote sensing image change detection aims to identify differences between images acquired at different times in the same area, crucial for land management, environmental monitoring, and disaster assessment. Current change detection methods mainly use Siamese or early fusion structures. Siamese networks focus on object features at different times but lack attention to change information, leading to false alarms and missed detections. Early fusion structures focus on fused features but neglect temporal object features, hindering accurate change detection. To address these issues, we propose a novel network, Triplet UNet (T-UNet), employing a triplet encoder to simultaneously extract object features and change features between the pre- and post-time-phase images. To effectively interact features extracted from the three branches of triplet encoder, we propose a multi-branch spatial-spectral cross-attention module. In the decoder, we employ channel attention and spatial attention mechanisms to fully mine and integrate detailed texture and semantic localization information. The proposed T-UNet surpasses seven other state-of-the-art methods on three publicly available datasets. Extensive experiments verify the effectiveness of the proposed structure and modules as well as the superiority of the proposed T-UNet. The source code for the proposed T-UNet is accessible at https://github.com/Pl-2000/T-UNet.

A neurocomputational model of decision and confidence in object recognition task

How does the brain process natural visual stimuli to make a decision? Imagine driving through fog. An object looms ahead. What do you do? This decision requires not only identifying the object but also choosing an action based on your decision confidence. In this circumstance, confidence is making a bridge between seeing and believing. Our study unveils how the brain processes visual information to make such decisions with an assessment of confidence, using a model inspired by the visual cortex. To computationally model the process, this study uses a spiking neural network inspired by the hierarchy of the visual cortex in mammals to investigate the dynamics of feedforward object recognition and decision-making in the brain. The model consists of two modules: a temporal dynamic object representation module and an attractor neural network-based decision-making module. Unlike traditional models, ours captures the evolution of evidence within the visual cortex, mimicking how confidence forms in the brain. This offers a more biologically plausible approach to decision-making when encountering real-world stimuli. We conducted experiments using natural stimuli and measured accuracy, reaction time, and confidence. The model's estimated confidence aligns remarkably well with human-reported confidence. Furthermore, the model can simulate the human change-of-mind phenomenon, reflecting the ongoing evaluation of evidence in the brain. Also, this finding offers decision-making and confidence encoding share the same neural circuit.

Temporal crowding with central vision reveals the fragility of visual representations.

This study examined whether temporal crowding-the impaired object identification when distracting objects precede and succeed it-occurs at the fovea and if so whether its magnitude is reduced. We presented a central sequence of three oriented items separated by relatively long intervals (200/400 ms) and used an orientation estimation task with mixture-model analyses. We found clear evidence of temporal crowding with central vision, even with 400 ms intervals. Critically, reduced encoding precision surfaced as a robust and unique characteristic of temporal crowding. The magnitude of central and peripheral temporal crowding was similar suggesting the involvement of higher visual areas. Precision impairment emerged even when only the target contained orientation information, excluding "response competition" as the sole interference mechanism; yet it was larger when all items included orientation information, underscoring the importance of orientation-selective mechanisms. Overall, we show that even with central simple stimuli, the formation of a stable visual representation is surprisingly slow. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Knowledge graph representation method for semantic 3D modeling of Chinese grottoes

The integration of 3D geometric models with semantic information significantly improves the applicability and comprehensibility of cultural heritage. The semantic 3D modeling of Chinese grottoes poses challenges for individuals without expertise in cultural heritage due to gaps in domain knowledge and discrepancies in understanding. However, the existing domain ontology and knowledge graph provide an insufficient representation of the knowledge of Chinese grottoes. To overcome these obstacles, we propose a knowledge graph representation method to provide explicit knowledge for participants at different stages of semantic 3D modeling of Chinese grottoes, which includes schema layer construction and data layer construction. On the schema layer, we design a domain ontology named ChgOnto (Chinese Grottoes Ontology) that consists of four high-level concept classes: spatial object, informational object, digital device, and temporal object. Among the classes in the ChgOnto, the components (e.g., cliff wall, cave roof, cliff wall footing), elements (e.g., primary Buddha statue, pedestal, decoration), the properties (e.g., length, width, depth) of caves and niches in Chinese grottoes as well as the spatial relationships between them are all precisely defined. ChgOnto also reuse the classes from the renowned CIDOC CRM ontology in the cultural heritage field and GeoSPARQL in the geospatial domain, facilitating integration between the two subjects. Considering the schema layer as the conceptual data model, the data layer extracts knowledge from unstructured text through natural language processing tools to instantiate the abstract classes and fill the properties of the schema layer. Finally, the knowledge required for semantic 3D modeling of Chinese grottoes is expressed in the data layer by a knowledge graph in a fixed expression form. Dazu Rock Carvings, a World Heritage site in China, is selected as a case study to validate the practicality and effectiveness of the proposed method. The results reveal that our method offers a robust knowledge-sharing platform for the semantic 3D modeling of Chinese grottoes and demonstrates excellent scalability. The method proposed in this paper can also serve as an informative reference for other types of cultural heritage.

Quantifying agonistic interactions between group-housed animals to derive social hierarchies using computer vision: a case study with commercially group-housed rabbits

In recent years, computer vision has contributed significantly to the study of farm animal behavior. In complex environments such as commercial farms, however, the automated detection of social behavior and specific interactions between animals can be improved. The present study addresses the automated detection of agonistic interactions between caged animals in a complex environment, relying solely on computer vision. An automated pipeline including group-level temporal action segmentation, object detection, object tracking and rule-based action classification for the detection of agonistic interactions was developed and extensively validated at a level unique in the field. Comparing with observations made by human observers, our pipeline reaches 77% precision and 85% recall using a 5-min tolerance interval for the detection of agonistic interactions. Results obtained using this pipeline allow to construct time-dependent socio-matrices of a group of animals and derive metrics on the dominance hierarchy in a semi-automated manner. Group-housed breeding rabbits (does) with their litters in commercial farms are the main use-case in this work, but the idea is probably also applicable to other social farm animals.

Video Action Recognition by Combining Spatial-Temporal Cues with Graph Convolutional Networks

Video action recognition relies heavily on the way spatio-temporal cues are combined in order to enhance recognition accuracy. This issue can be addressed with explicit modeling of interactions among objects within or between videos, such as the graph neural network, which has been shown to accurately model and represent complicated spatial- temporal object relations for video action classification. However, the visual objects in the video are diversified, whereas the nodes in the graphs are fixed. This may result in information overload or loss if the visual objects are too redundant or insufficient for graph construction. Segment level graph convolutional networks (SLGCNs) are proposed as a method for recognizing actions in videos. The SLGCN consists of a segment-level spatial graph and a segment-level temporal graph, both of which are capable of simultaneously processing spatial and temporal information. Specifically, the segment-level spatial graph and the segment-level temporal graph are constructed using 2D and 3D CNNs to extract appearance and motion features from video segments. Graph convolutions are applied in order to obtain informative segment-level spatial-temporal features. A variety of challenging video datasets, such as EPIC-Kitchens, FCVID, HMDB51 and UCF101, are used to evaluate our method. In experiments, it is demonstrated that the SLGCN can achieve performance comparable to the state-of-the-art models in terms of obtaining spatial-temporal features.

Recency memory is altered in cocaine-withdrawn adolescent rats: Implication of cortical mTOR signaling

In humans, cocaine abuse during adolescence poses a significant risk for developing cognitive deficits later in life. Among the regions responsible for cognitive processes, the medial prefrontal cortex (mPFC) modulates temporal order information via mechanisms involving the mammalian-target of rapamycin (mTOR)-mediated pathway and protein synthesis regulation. Accordingly, our goal was to study the effect of repeated cocaine exposure during both adolescence and adulthood on temporal memory by studying the mTOR pathway in the mPFC. Adolescent or adult rats underwent repeated cocaine injections for 15 days and, after two weeks of withdrawal, engaged in the temporal order object recognition (TOOR) test.We found that repeated cocaine exposure during adolescence impaired TOOR performance, while control or adult-treated animals showed no impairments. Moreover, activation of the mTOR-S6-eEF2 pathway following the TOOR test was diminished only in the adolescent cocaine-treated group. Notably, inhibition of the mTOR-mediated pathway by rapamycin injection impaired TOOR performance in naïve adolescent and adult animals, revealing this pathway to be a critical component in regulating recency memory.Our data indicate that withdrawal from cocaine exposure impairs recency memory via the dysregulation of protein translation mechanisms, but only when cocaine is administered during adolescence.

Origin of biased noise in one-time readout temporal single pixel imaging

One-time readout temporal ghost imaging is a viable time-analog of single pixel imaging using a sluggish light receiver haunted by memory effects, which can be utilized to eliminate the need for steady watch and as such save computer clocks for data capturing and processing. Although the as-retrieved ghost image of a temporal object is modulated in proportion to the time-reversed impulse response function of the light receiver, it can be easily corrected by division. Nevertheless, inhomogeneously distributed noise plagues the images wildly, depending on the transmittance profile of an intensity-only object, which puzzles us. A coherent explanation is attempted for such apparently inconsistent behavior of the biased noise that develops even in simulation.

Time-to-space ghost imaging.

Temporal ghost imaging is based on the temporal correlations of two optical beams and aims at forming a temporal image of a temporal object with a resolution, fundamentally limited by the photodetector resolution time and reaching 55 ps in a recent experiment. For further improvement of the temporal resolution, it is suggested to form a spatial ghost image of a temporal object relying on strong temporal-spatial correlations of two optical beams. Such correlations are known to exist between two entangled beams generated in type-I parametric downconversion. It is shown that a sub-picosecond-scale temporal resolution is accessible with a realistic source of entangled photons.

3D Point Cloud Object Detection Algorithm Based on Temporal Information Fusion and Uncertainty Estimation

In autonomous driving, LiDAR (light detection and ranging) data are acquired over time. Most existing 3D object detection algorithms propose the object bounding box by processing each frame of data independently, which ignores the temporal sequence information. However, the temporal sequence information is usually helpful to detect the object with missing shape information due to long distance or occlusion. To address this problem, we propose a temporal sequence information fusion 3D point cloud object detection algorithm based on the Ada-GRU (adaptive gated recurrent unit). In this method, the feature of each frame for the LiDAR point cloud is extracted through the backbone network and is fed to the Ada-GRU together with the hidden features of the previous frames. Compared to the traditional GRU, the Ada-GRU can adjust the gating mechanism adaptively during the training process by introducing the adaptive activation function. The Ada-GRU outputs the temporal sequence fusion features to predict the 3D object in the current frame and transmits the hidden features of the current frame to the next frame. At the same time, the label uncertainty of the distant and occluded objects affects the training effect of the model. For this problem, this paper proposes a probability distribution model of 3D bounding box coordinates based on the Gaussian distribution function and designs the corresponding bounding box loss function to enable the model to learn and estimate the uncertainty of the positioning of the bounding box coordinates, so as to remove the bounding box with large positioning uncertainty in the post-processing stage to reduce the false positive rate. Finally, the experiments show that the methods proposed in this paper improve the accuracy of the object detection without significantly increasing the complexity of the algorithm.

Adult mice with noise-induced hearing loss exhibited temporal ordering memory deficits accompanied by microglia-associated neuroplastic changes in the medial prefrontal cortex

Acquired peripheral hearing loss in midlife is considered the primary modifiable risk factor for dementia, while the underlying pathological mechanism remains poorly understood. Excessive noise exposure is the most common cause of acquired peripheral hearing loss in modern society. This study was designed to investigate the impact of noise-induced hearing loss (NIHL) on cognition, with a focus on the medial prefrontal cortex (mPFC), a brain region that is involved in both auditory and cognitive processes and is highly affected in patients with cognitive impairment. Adult C57BL/6 J mice were randomly assigned to a control group and seven noise groups: 0HPN, 12HPN, 1DPN, 3DPN, 7DPN, 14DPN, and 28DPN, which were exposed to broadband noise at a 123 dB sound pressure level (SPL) for 2 h and sacrificed immediately (0 h), 12 h, or 1, 3, 7, 14, or 28 days post-noise exposure (HPN, DPN), respectively. Hearing assessment, behavioral tests, and neuromorphological studies in the mPFC were performed in control and 28DPN mice. All experimental animals were included in the time-course analysis of serum corticosterone (CORT) levels and mPFC microglial morphology. The results illustrated that noise exposure induced early-onset transient serum CORT elevation and permanent moderate-to-severe hearing loss in mice. 28DPN mice, in which permanent NIHL has been verified, exhibited impaired performance in temporal order object recognition tasks concomitant with reduced structural complexity of mPFC pyramidal neurons. The time-course immunohistochemical analysis in the mPFC revealed significantly higher morphological microglial activation at 14 and 28 DPN, preceded by a remarkably higher amount of microglial engulfed postsynaptic marker PSD95 at 7 DPN. Additionally, lipid accumulation in microglia was observed in 7DPN, 14DPN and 28DPN mice, suggesting a driving role of lipid handling deficits following excessive phagocytosis of synaptic elements in delayed and sustained microglial abnormalities. These findings provide fundamentally novel information concerning mPFC-related cognitive impairment in mice with NIHL and empirical evidence suggesting the involvement of microglial malfunction in the mPFC neurodegenerative consequences of NIHL.

Detection of a temporal salient object benefits from visual stimulus-specific adaptation in avian midbrain inhibitory nucleus.

Food and predators are the most noteworthy objects for the basic survival of wild animals, and both are often deviant in both spatial and temporal domains and quickly attract an animal's attention. Although stimulus-specific adaptation (SSA) is considered a potential neural basis of salient sound detection in the temporal domain, related research on visual SSA is limited and its relationship with temporal saliency is uncertain. The avian nucleus isthmi pars magnocellularis (Imc), which is central to midbrain selective attention network, is an ideal site to investigate the neural correlate of visual SSA and detection of a salient object in the time domain. Here, the constant order paradigm was applied to explore the visual SSA in the Imc of pigeons. The results showed that the firing rates of Imc neurons gradually decrease with repetitions of motion in the same direction, but recover when a motion in a deviant direction is presented, implying visual SSA to the direction of a moving object. Furthermore, enhanced response for an object moving in other directions that were not presented ever in the paradigm is also observed. To verify the neural mechanism underlying these phenomena, we introduced a neural computation model involving a recoverable synaptic change with a "center-surround" pattern to reproduce the visual SSA and temporal saliency for the moving object. These results suggest that the Imc produces visual SSA to motion direction, allowing temporal salient object detection, which may facilitate the detection of the sudden appearance of a predator.

Hornets, pelicans, bobcats, and identity: the problem of persistence of temporal abstract objects

Hornets, pelicans, bobcats, and identity: the problem of persistence of temporal abstract objects

Depletion of microglial BDNF increases susceptibility to the behavioral and synaptic effects of chronic unpredictable stress

In the medial prefrontal cortex (PFC), chronic stress reduces synaptic expression of glutamate receptors, leading to decreased excitatory signaling from layer V pyramidal neurons and working memory deficits. One key element driving these changes is a reduction in brain-derived neurotrophic factor (BDNF) signaling. BDNF is a potent mediator of synaptic growth and deficient BDNF signaling has been linked to stress susceptibility. Prior studies indicated that neurons are the primary source of BDNF, but more recent work suggests that microglia are also an important source of BDNF. Adding to this, our work showed that 14 days of chronic unpredictable stress (CUS) reduced Bdnf transcript in PFC microglia, evincing its relevance in the effects of stress. To explore this further, we utilized transgenic mice with microglia-specific depletion of BDNF (Cx3cr1Cre/+:Bdnffl/fl) and genotype controls (Cx3cr1Cre/+:Bdnf+/+). In the following experiments, mice were exposed to a shortened CUS paradigm (7 days) to determine if microglial Bdnf depletion promotes stress susceptibility. Analyses of PFC microglia revealed that Cx3cr1Cre/+:Bdnffl/fl mice had shifts in phenotypic markers and gene expression. In a separate cohort, synaptoneurosomes were collected from the PFC and western blotting was performed for synaptic markers. These experiments showed that Cx3cr1Cre/+:Bdnffl/fl mice had baseline deficits in GluN2B, and that 7 days of CUS additionally reduced GluN2A levels in Cx3cr1Cre/+:Bdnffl/fl mice, but not genotype controls. Behavioral and cognitive testing showed that this coincided with exacerbated stress effects on temporal object recognition in Cx3cr1Cre/+:Bdnffl/fl mice. These results indicate that microglial BDNF promotes glutamate receptor expression in the PFC. As such, mice with deficient microglial BDNF had increased susceptibility to the behavioral and cognitive consequences of stress.

Rethinking prediction alignment in one-stage object detection

Owing to their excellent performance and efficiency, one-stage detectors have been widely used in multimedia tasks, such as temporal action detection, object tracking, and video detection. However, misalignment between classification and regression branches limits the accuracy of the detector. Most existing works add an auxiliary branch or adopt a specific sample assignment strategy to alleviate this problem, but with little effect. In this paper, we attribute this to incomplete branch interactions and propose a comprehensive Predictive Aligned Object Detector (PAOD), which can better correlate two subtasks. Specifically, our proposed PAOD achieves a better trade-off between prediction-interactive and prediction-specific by adopting an Iterative Aggregation Module (IAM) and a Mutual Constraint Module (MCM). We also design an aligned label assignment with an adaptive metric and re-weighting mechanism to further narrow the misalignment between prediction heads. With negligible additional overhead, PAOD achieves 50.4 AP at single-model single-scale testing on the MS-COCO branch, which demonstrates the effectiveness of our proposal. Notably, PAOD consistently outperforms previous sota such as ATSS (47.7 AP), BorderDet (48.0 AP) and GFL (48.2 AP) by a large margin on COCO test-dev dataset, and achieves better performance than various dense detectors on Pascal VOC and CrowdHuman datasets. Code is available at https://github.com/JunruiXiao/PAOD.