Spatio-temporal Color Research Articles

Spatiotemporal Retention of Structural Color and Induced Stiffening in Crosslinked Hydroxypropyl Cellulose Beads.

Hydroxypropyl cellulose (HPC) is known for its ability to form cholesteric liquid crystalline phases displaying vivid structural colors. However, these vibrant colors tend to fade over time when the material dries. This issue is a major bottleneck to finding practical applications for these materials. Here this problem is overcome by producing free-standing, millimeter-sized HPC structurally colored beads with spatiotemporal color retention, facilitated by a glutaraldehyde crosslinker. By leveraging the well-known chemically induced stabilization of cholesteric liquid crystalline phases, stable structural colors are achieved for at least three weeks. The presence of glutaraldehyde significantly increases the mechanical stiffness, with Young's modulus rising from 0.3± 0.1GPa to 1.8± 0.2GPa. This integrated approach of creating free-standing photonic HPC beads offers a strategy for developing robust and durable photonic HPC materials with enhanced stability, advancing photonic material applications with spatiotemporal color stability.

Hierarchical Color Fusion Network (HCFN): Enhancing exemplar-based video colorization

We conducted a study on exemplar-based colorization for grayscale videos, aiming to enhance visual perception by transforming them into vibrant videos with plausible colors. Despite promising results from existing approaches, effectively transferring colors from a reference image to grayscale video frames while maintaining temporal consistency between frames remains a challenge. To tackle this challenge, we designed a Hierarchical Color Fusion Network (HCFN). For each grayscale video frame, HCFN initially uses global and local attention mechanisms to calculate pixel-level similarity with the reference image and spatio-temporal correspondence with its neighboring frame, respectively. Based on these relationships, the pixel-level color and spatio-temporal color of the grayscale video frame are generated. Additionally, the tone-based color for the given frame is obtained based on the color distribution of the reference image. Finally, the ambiguity among these three types of colors is eliminated through the proposed hierarchical fusion mechanism, and the final color of the grayscale video frame is produced. Experimental results on public databases show that our method outperforms state-of-the-art methods in visual quality, realism and temporal consistency by a large margin. The source code and pre-trained checkpoints for HCFN is publicly available at https://github.com/wangyins/HCFN.

Temporal and spatial reference frames in visual working memory are defined by ordinal and relational properties.

Natural environments provide a rich spatiotemporal context that allows for visual objects to be differentiated based on different types of information: their absolute or relative spatial or temporal coordinates, or their ordinal positions in a spatial or temporal sequence. Here, we investigated which spatial and temporal properties are incidentally encoded along with to-be-remembered features to provide reference frames in visual working memory (VWM). We tested the different possibilities in a spatiotemporal color change-detection task by transforming spatial and/or temporal structures of item presentation at retrieval relative to encoding. More precisely, spatial and/or temporal coordinates were (a) switched, changing the order of items in a spatial or temporal sequence (ordinal transformation); (b) multiplied by different factors, changing interitem distances (relational transformation); or (c) multiplied by a constant factor, expanding or shrinking the entire configuration (global transformation). Such transformations of the external reference frame at retrieval should only interfere with VWM if the internal reference frame relies on the spatial or temporal properties affected by the respective transformation. We found that ordinal and relational transformations of either the spatial or temporal structure impaired performance, whereas global transformations did not. Thus, reference frames appear to be primarily defined by interitem relations-including relative distances between items as well as their order-rather than absolute positions in space or time. These results corroborate and extend previous findings for the spatial domain, and highlight functional similarities of the spatial and temporal dimensions in VWM by revealing the same metrical properties for temporal reference frames. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Sensing Gas Mixtures by Analyzing the Spatiotemporal Optical Responses of Liquid Crystals Using 3D Convolutional Neural Networks.

We report how analysis of the spatial and temporal optical responses of liquid crystal (LC) films to targeted gases, when performed using a machine learning methodology, can advance the sensing of gas mixtures and provide important insights into the physical processes that underlie the sensor response. We develop the methodology using O3 and Cl2 mixtures (representative of an important class of analytes) and LCs supported on metal perchlorate-decorated surfaces as a model system. Although O3 and Cl2 both diffuse through LC films and undergo redox reactions with the supporting metal perchlorate surfaces to generate similar initial and final optical states of the LCs, we show that a three-dimensional convolutional neural network can extract feature information that is encoded in the spatiotemporal color patterns of the LCs to detect the presence of both O3 and Cl2 species in mixtures and to quantify their concentrations. Our analysis reveals that O3 detection is driven by the transition time over which the brightness of the LC changes, while Cl2 detection is driven by color fluctuations that develop late in the optical response of the LC. We also show that we can detect the presence of Cl2 even when the concentration of O3 is orders of magnitude greater than the Cl2 concentration. The proposed methodology is generalizable to a wide range of analytes, reactive surfaces, and LCs and has the potential to advance the design of portable LC monitoring devices (e.g., wearable devices) for analyzing gas mixtures using spatiotemporal color fluctuations.

Spatio-temporal colour variation of arthropods and their environment

Spatio-temporal colour variation of arthropods and their environment

Travelling colourful patterns in self-organized cellulose-based liquid crystalline structures

Cellulose-based systems are useful for many applications. However, the issue of self-organization under non-equilibrium conditions, which is ubiquitous in living matter, has scarcely been addressed in cellulose-based materials. Here, we show that quasi-2D preparations of a lyotropic cellulose-based cholesteric mesophase display travelling colourful patterns, which are generated by a chemical reaction-diffusion mechanism being simultaneous with the evaporation of solvents at the boundaries. These patterns involve spatial and temporal variation in the amplitude and sign of the helix´s pitch. We propose a simple model, based on a reaction-diffusion mechanism, which simulates the observed spatiotemporal colour behaviour.

Flexible Vertical Photogating Transistor Network with an Ultrashort Channel for In‐Sensor Visual Nociceptor

AbstractHumans can clearly perceive surroundings efficiently while consuming little energy because of human intelligence and powerful vision system. Thus, it has been a long‐sought dream for human beings to build such an energy‐efficient artificial intelligent vision system with emerging devices. Unfortunately, a wearable optoelectronic device for visual nociceptor systems, regarded as a key bionic function to protect the vision, remains to be developed so far. Herein, using the vertical coplanar‐multiterminal flexible transient photogating transistor network with a 3 nm ultrashort channel, a wearable artificial vision system with painful‐perceptual abilities is successfully demonstrated for flexible electronic‐skin (e‐skin) applications. The device not only has the ability of ultrafast transient physical disappearance of only 60 s for information security but also establishes a flexible optical in‐sensor visual nociceptor (ISVN) e‐skin. The optical transition from short‐time memory to long‐time memory of visual memory is educed by a strong photogating effect, and the higher‐level‐graded optical painful alarm‐sensing system is also demonstrated by this flexible artificial e‐skin. Moreover, the proposed devices will achieve painful light sensitization under different spatiotemporal color patterns to avoid external secondary injuries. It provides a good opportunity for future intelligent e‐skin taking advantage of its intriguing visual pain‐perceptual abilities.

Spatio-Temporal Color Differences Between Urban and Rural Populations of a Ground Beetle During the Last 100 Years

Increasing anthropogenic environmental impacts lead to rapid transitions of ecosystems and species. Species persisting in changing environments may respond to changes by altering phenotypic traits across space and/or time. Here we tested whether the frequencies of three color morphs in the ground beetle Harpalus affinis differed across spatial and temporal gradients. The gradients extended from urban to rural regions, and from the early 20th century until today, in the Berlin-Brandenburg area, Germany. Specimens comprised beetles from the entomological collection of the Museum fur Naturkunde, Berlin and recently collected material. As a result of differing environments, we expected to observe differences in color frequencies in beetles between habitats and across time, responding to different levels of urbanization. Our results revealed sexual dichromatism in H. affinis as well as some habitat dependent differences in trait frequency. Frequencies of color morphs remained generally constant in males across space and time. Females likewise showed no differences in color frequencies between habitats, urban and rural regions, and between different time periods in rural regions. In contrast color morph frequencies changed in urban regions over time in females: Bronze color decreased, whereas green color became more dominant over time. We assume that bronze color was selectively advantageous in times with high levels of soot pollution in the city, whereas green is more cryptic and thus advantageous in times with less polluted air. The color change of females thus could have been driven by natural selection. In contrast, the persistence of predominately green males through all times and habitats, more likely can be explained by sexual selection.

Three-layer structured soft particles to construct photonic paper exhibiting responsive spatio-temporal color patterns

Photonic crystals (PCs), which generate vivid colors from periodic physical structures, have attracted considerable attention. However, it is still highly challenging to produce large-scale PC films with high mechanical performance. To address the challenge, a new strategy to fabricate PC films from soft structured latex particles instead of widely used hard particles is reported. The soft latex particles are designed to have a three-layer structure: a responsive poly(acrylic acid) (PAA) corona, hard polystyrene (PSt) shell and soft poly(n-butyl acrylate) (PnBA) core. Such latex particles with easily-tunable and well-defined structures are synthesized from RAFT emulsion polymerization. Thanks to the softness, PCs can be easily achieved by latex casting on various substrates without pre-treatment, leading to a dense, crack-free, and mechanically robust PC film. The as-prepared PC films of the particles with PnBA around 55 wt% can generate vivid colors when the “inks” of water and hydrophilic liquids are applied. The full colors can be achieved by tuning particle size or pH value. Taking advantage of the responsive properties to hydrophilic liquids with different volatilities, both temporary and permanent patterns can be constructed on the PC films, leading to spatio-temporal colorful patterns. The method is simple, scalable-up, and versatile, showing great potentials for practical applications.

Optical properties of Forel-Ule water types deduced from 15 years of global satellite ocean color observations

The Forel-Ule (FU) color comparator scale is the oldest set of optical water types (OWTs). This scale was originally developed for visual comparison and generated an immense amount of data, with hundreds of thousands of observations being gathered from the last 130 years. Since recently, the FU scale is also applicable to remote sensing data. This has been possible thanks to an optical characterization of the 21 FU colors in terms of the (x,y) CIE standards and new algorithms that convert remote-sensing reflectances (Rrs) from satellite-borne ocean color sensors to FU. Rrs-derived hue angle and FU have been recently applied with success in the assessment of color variability of lakes and specific shelf areas, but an evaluation over global oceanic waters is still missing. By clustering global climatological ESA-OC-CCI v2.0 Rrs with the derived FU, we obtain a set of Rrs to be used as optical water types (OWTs). Diffuse attenuation coefficient, Secchi disk depth and chlorophyll concentration are also associated to the FU classes. The angular distances of a given Rrs to the two nearest FU classes are proposed as simple and robust membership indexes, adding up to one. We also evaluate the advantages and limitations of FU and the hue angle as monitoring tools over the full marine range, from the most oligotrophic areas to the turbid and productive coastal zones. The first 7 FU indexes cover 99% of global surface waters. Unlike the hue angle, that resolves all spatio-temporal color variations, the FU scale is coarse as a monitoring tool for oligotrophic waters as all the subtropical gyres saturate to FU = 1, while the color of other seas varies across 2, 3 or even 4 FU classes. We illustrate the introduction of a new “zero” FU class that increases monitoring resolution at the blue end of the color range. Finally, we show how optical diversity varies across the color range and compare several sets of OWTs from a color perspective. Overall, we provide a valuable and self-consistent dataset that enhances the usefulness of the FU scale by converting it to useful information for the oceanographic community. This OWT scheme keeps the advantages of other datasets, like being useful to study ocean color product quality and characterize the uncertainties, but also allows to continue to monitor long-term change in optical diversity over the global ocean color. Integration into the optical modules of ecosystem models can help verify past simulations that predate the satellite age, through comparisons with in-situ FU data collected at the time.

High strength dual-crosslinked hydrogels with photo-switchable color changing behavior

Herein, high strength dual-crosslinked hydrogels with photo-switchable color changing behavior were designed and fabricated. In the mixture solution of α-cyclodextrin (α-CD), α-cyclodextrin-spiropyran (α-CD-SP), and poly(ethylene glycol) diacrylate (PEGDA), the PEGDA chains first threaded into the cavity of α-CD/α-CD-SP and form inclusion complexes to generate the physical gel; then, the as-prepared physical gels were subjected to photopolymerization to form a covalently crosslinked network. With the contribution of both no-covalent and covalent bond, the dual-crosslinked hydrogels possess high tensile strength, which is about fifteen times that of the pure PEG hydrogel. And due to the sacrifice of no-covalent bond, hydrogels showed high energy dissipation ratio, up to 75.3% and high fatigue resistance during loading-unloading tests. While the ingenious introduction of SP moieties, enables the as-prepared gels photo-switchable spatiotemporal color changing behavior reversibly. Erasable patterns on hydrogels could be written using UV light and erased by Vis light for multiple cycles. This flexible hydrogel with good physical performance shows high potential in photochromic flexible wearable devices.

Spatiotemporal Colorization of Video Using 3D Steerable Pyramids

We propose a new technique for video colorization based on spatiotemporal color propagation in the 3D video volume, utilizing the dominant orientation response obtained from the steerable pyramid decomposition of the video. The volumetric color diffusion from the sources that are marked by scribbles occurs across spatiotemporally smooth regions, and the prevention of leakage is facilitated by the spatiotemporal discontinuities in the output of steerable filters, representing the object boundaries and motion boundaries. Unlike most existing methods, our approach dispenses with the need of motion vectors for interframe color transfer and provides a general framework for image and video colorization. The experimental results establish the effectiveness of the proposed approach in colorizing videos having different types of motion and visual content even in the presence of occlusion, in terms of accuracy and computational requirements.

Color correction for large-baseline multiview video

Color misalignment correction is an important, yet unsolved problem, especially for multiview video captured by large disparity camera setups. In this paper, we introduce a robust large-baseline color correction method that preserves the original manifold structure of the input video. The manifold structure is extracted by locally linear embedding (LLE), aimed at linearly representing each pixel based on its neighbors, assuming that they are all clustered in a high-dimensional feature space. Besides the proposed manifold structure preservation constraint, the proposed method enforces spatio-temporal color consistencies and gradient preservation. The multiview color correction solution is obtained by solving a global optimization problem. Thorough objective and subjective experimental results demonstrate that our proposed approach significantly and systematically outperforms the state-of-the-art color correction methods on large-baseline multiview video data.

A novel framework for CBCD using integrated color and acoustic features

Most studies in content-based video copy detection (CBCD) concentrate on visual signatures, while only very few efforts are made to exploit audio features. The audio data, if present, is an essential source of a video; hence, the integration of visual-acoustic fingerprints significantly improves the copy detection performance. Based on this aspect, we propose a new framework, which jointly employs color-based visual features and audio fingerprints for detecting the duplicate videos. The proposed framework incorporates three stages: First, a novel visual fingerprint based on spatio-temporal dominant color features is generated; Second, mel-frequency cepstral coefficients are extracted and compactly represented as acoustic signatures; Third, the resultant multimodal signatures are jointly used for the CBCD task, by employing combination rule and weighting strategies. The results of experiments on TRECVID 2008 and 2009 datasets, demonstrate the improved efficiency of the proposed framework compared to the reference methods against a wide range of video transformations.

Depth map compression and depth-aided view rendering for a three-dimensional video system

Three-dimensional (3D) video technologies are becoming increasingly popular, as they can provide high quality and immersive experience to end users, where depth maps are employed to generate the virtual views by depth-image-based rendering technique. However, how to reduce the compression and rendering complexities for depth maps while maintaining high rendering quality is still unresolved. In this study, a novel depth map compression and depth-aided view rendering method is proposed. In the proposed method, depth maps are represented with different layers and compressed with different macroblock-mode decision procedure, and several optimisation techniques, including spatio-temporal consistent warping, colour correction and temporal consistent hole filling are embedded into the view rendering framework. Experimental results show that compared with the traditional method, the proposed method can reduce more than 79% compression computational complexity and more than 45% rendering computational complexity, while maintaining high rendering quality.

65.2: A 120Hz Spatio‐Temporal Color Display Without Color Breakup

AbstractA color breakup reduction method is proposed for a 120Hz spatio‐temporal color display. It has a 1.5 ×∼3 × high transmission ratio and a 1.5 × high resolution compared to a normal liquid crystal display (LCD) with RGB (red, green, and blue) color filter. It can be implemented on a 120Hz LCD with two‐color filter and a local dimming RGB LED (light emitting diode) backlight. It algorithmically suppresses the color breakup by reducing color differences between fields with a local‐primary desaturation protocol in a segmented backlight.

DAG-based visual interfaces for navigation in indexed video content

Indexing and segmenting of video content by motion, color and texture has been intensively explored leading to a commonly used representation in a storyboard. In this paper, a novel method of visualization of video content is proposed. First of all, the content is segmented into shots, and then a spatio-temporal color signature of shots, based on color distribution in the frames, is proposed. This spatio-temporal color signature serves as a basis for graph clustering and graph visualization tools. Those, integrated in a platform for visualization of huge graphs, Tulip, supply an exciting graph-based navigation interface for multimedia content. The results obtained on feature documentaries are promising.

Sensitivity to spatiotemporal colour contrast in the peripheral visual field

Contrast detection thresholds for spatiotemporal colour modulation were determined at several retinal locations. Colour discrimination is studied for red-green modulation of a yellow field and for yellow-blue modulation of a white field. For constant target size colour vision deteriorates if the stimulus is moved away from the fovea, but if the shift is combined with a suitable enlargement of the target size colour discrimination at the periphery is comparable to that at the fovea. We did not find any retinal location along the horizontal meridian which was red-green or yellow-blue colour blind. Chromatic bars were detected in the nasal peripheral retina at eccentricities of up to at least 50, temporal colour contrast was perceptible at nasal eccentricities of up to at least 90.