Color Video Research Articles

CPNet: Continuity Preservation Network for infrared video colorization

Infrared video colorization can significantly improve perceptual quality by predicting reasonable colors and restoring vivid details, especially in harsh environments. However, as an inconspicuous computer vision task, there is no specialized method. Besides, directly applying current grayscale colorization methods may generate structurally obscure and temporally inconsistent frames. In this paper, we design an infrared video colorization network CPNet aims to generate visually plausible and spatial–temporal consistent colorized videos. To achieve this, a feature fusion module and a hierarchical colorizer are designed to learn the importance of each consecutive frame and the local and global correlation of the integrated features, respectively. In addition, to consolidate the temporal consistency at a fine-grained level, we further introduce a composite loss function to narrow the distance between high-level feature representations while retaining pixel-wise correspondence. Moreover, a new metric named Mean Temporal Variation Similarity (MTVS) is proposed for effectively evaluating the degree of video continuity. Comprehensive experiments conducted on the KAIST dataset demonstrate the superiority of CPNet to produce more authentic colorized videos than state-of-the-art colorization methods. In terms of quantitative comparison, CPNet achieves improvements of at least 0.89 dB on PSNR, 0.016 on SSIM, and a significant promotion on MTVS. In addition, experiments conducted on the DAVIS dataset also prove the applicability of CPNet for grayscale video colorization task.

Contributed Session I: Towards General Video Percepts Cone-by-Cone.

We aim to reprogram visual perception through an Adaptive Optics Scanning Laser Ophthalmoscope (AOSLO) Display, using a GPU renderer that rasterizes a target color image or video into cone-by-cone single-wavelength laser light pulses ("microdoses"). We imaged and tracked at ~ 2° eccentricity a 0.9° x 0.9° field of view of the retina in 840 nm. Stimulated in 543 nm, all resolved, spectrally classified cones receive microdoses of varying intensities. The renderer updates for each AOSLO frame (30 frames / sec) an underlying stimulation image buffer, encoding a desired color percept pattern that takes into account the cone locations, cone spectral sensitivity to the 543 nm stimulation light, and the corresponding color percept pixel values. Within one frame, the buffer gets pixelated strip-by-strip at 1 kHz into actual world-fixed microdose intensity values, each centered on a cone within that strip at that instant. The resulting frame of microdoses visually occupies the whole raster view. We showed multiple color percepts to a cone-classified subject, with logging data. The subject saw spatially-varying colors, e.g. a red box moving on a green canvas - these percepts validated the accuracy of the prototype. These initial prototyping experiments allude to the potential of presenting general percepts to a cone-classified subject, at cone-level accuracy in a fully programmable way. The technology allows us to probe neural plasticity and towards generation of novel percepts.

Holographic Encryption of Color Video Stream with 4k Resolution Using Phase Liquid Crystal Light Modulators

Holographic Encryption of Color Video Stream with 4k Resolution Using Phase Liquid Crystal Light Modulators

Deep Sensing for Compressive Video Acquisition.

A camera captures multidimensional information of the real world by convolving it into two dimensions using a sensing matrix. The original multidimensional information is then reconstructed from captured images. Traditionally, multidimensional information has been captured by uniform sampling, but by optimizing the sensing matrix, we can capture images more efficiently and reconstruct multidimensional information with high quality. Although compressive video sensing requires random sampling as a theoretical optimum, when designing the sensing matrix in practice, there are many hardware limitations (such as exposure and color filter patterns). Existing studies have found random sampling is not always the best solution for compressive sensing because the optimal sampling pattern is related to the scene context, and it is hard to manually design a sampling pattern and reconstruction algorithm. In this paper, we propose an end-to-end learning approach that jointly optimizes the sampling pattern as well as the reconstruction decoder. We applied this deep sensing approach to the video compressive sensing problem. We modeled the spatio-temporal sampling and color filter pattern using a convolutional neural network constrained by hardware limitations during network training. We demonstrated that the proposed method performs better than the manually designed method in gray-scale video and color video acquisitions.

Rapid single-photon color imaging of moving objects.

This paper outlines an experimental demonstration of a Bayesian image reconstruction approach to achieve rapid single-photon color imaging of moving objects. The capacity to extract the color of objects is important in a variety of target identification and computer vision applications. Nonetheless, it remains challenging to achieve high-speed color imaging of moving objects in low-photon flux environments. The low-photon regime presents particular challenges for efficient spectral separation and identification, while unsupervised image reconstruction algorithms are often slow and computationally expensive. In this paper, we address both of these difficulties using a combination of hardware and computational solutions. We demonstrate color imaging using a Single-Photon Avalanche Diode (SPAD) detector array for rapid, low-light-level data acquisition, with an integrated color filter array (CFA) for efficient spectral unmixing. High-speed image reconstruction is achieved using a bespoke Bayesian algorithm to produce high-fidelity color videos. The analysis is conducted first on simulated data allowing different pixel formats and photon flux scenarios to be investigated. Experiments are then performed using a plasmonic metasurface-based CFA, integrated with a 64 × 64 pixel format SPAD array. Passive imaging is conducted using white-light illumination of multi-colored, moving targets. Intensity information is recorded in a series of 2D photon-counting SPAD frames, from which accurate color information is extracted using the fast Bayesian method introduced herein. The per-frame reconstruction rate proves to be hundreds of times faster than the previous computational method. Furthermore, this approach yields additional information in the form of uncertainty measures, which can be used to assist with imaging system optimization and decision-making in real-world applications. The techniques demonstrated point the way towards rapid video-rate single-photon color imaging. The developed Bayesian algorithm, along with more advanced SPAD technology and utilization of time-correlated single-photon counting (TCSPC) will permit live 3D, color videography in extremely low-photon flux environments.

Superpixel-based robust tensor low-rank approximation for multimedia data recovery

Recently, the low-rank matrix/tensor-based methods have attracted increasing attention for multi-dimensional multimedia data (e.g., image and video) recovery, which assumes the holistic data is low-rank. For most multimedia data, the low-rank assumption is usually violated due to their spatially diverse local similarity. The cube-based method, which breaks the holistic image into small local cubes with a fixed size, can alleviate this issue to some extent. However, cubes with a fixed size are not flexible to cover the regions of local similarity at different scales. Inspired by the superpixel, we suggest the size-adaptive super-tensor as the generic unit instead of the cube with the fixed spatial size, which allows us to flexibly exploit the local similarity at different scales. The super-tensor enjoys significant low-rankness and avoids excessive overlapped pixels as compared with cubes of fixed sizes. Empowering with the super-tensor, we propose a low-rank super-tensor approximation (LRSTA) model for multi-dimensional data recovery, which can fully exploit the local similarity and low-rankness at different scales. Moreover, we develop an efficient alternating direction method of multipliers (ADMM) algorithm to solve the proposed model. Extensive experiments on multimedia data, including color images, multispectral images (MSIs), and videos, verify that the proposed method outperforms the other competing methods.

Socio-economic correlation analysis and hybrid artificial neural network model development for provincial waste electrical and electronic equipment generation forecasting in China

Ability to forecast the waste electrical and electronic equipment (WEEE) generation can help formulate a robust future WEEE management system. Previous studies applied forecasting models, such as the grey model and artificial neural networks, to predict WEEE generation from a country perspective, leading to less accurate forecasts due to huge socio-economic differences in rural and urban areas. Additionally, there has been the incompatibility of a single forecasting model for all WEEE types, and this remained a research gap. Taking advantage of respective forecasting models, this study presents a hybrid model, Grey Artificial Neural Network, to forecast the WEEE generation of 31 province-level regions in China while evaluating the socio-economic analysis of seven WEEE types via Pearson correlation analysis. More than 70% of WEEE from province-level regions strongly correlates (R > ±0.8) with the gross domestic product and the population, whereas some top WEEE-generating province-level regions (i.e., Tianjin and Shanghai) correlate weakly to moderately. The root mean square error and mean absolute error of the developed Grey Artificial Neural Network hybrid model are the lowest at 8.29 and 6.48, compared to the grey model (13.53 and 11.13) and back-propagation neural network (9.21 and 7.22). Though the Grey Artificial Neural Network hybrid model has the lowest error, posterior mean square deviation ratio analysis indicates that this hybrid model is only suitable for washing machines, refrigerators, color televisions, and personal computers (urban area), while the back-propagation neural network is suitable for monochrome televisions, air conditioners, and personal computers (rural area). Compared to 2019, it is projected that an additional 32.92 M units of WEEE will be generated by 2025, suggesting that China should build at least 15 extra recycling centers (14% more based on 2016) to handle the increased WEEE generation. This study provides policy implications for effective WEEE monitoring and collection systems to build resilient WEEE management.

Improvements to the CIELUV color space

AbstractThree new approximately uniform color spaces named MLUV, MLUV1, and MLUV2 were developed by modifying CIELUV lightness, chromaticity coordinates, and color coordinates, respectively. Performance tests using the combined and four individual datasets employed at CIEDE2000 development showed that MLUV, MLUV1 and MLUV2 were significantly better than CIELUV. Using values of Standardized Residual Sum of Squares (CIE 217:2016) for predictions of the four individual datasets the ranking (from best to worst) was MLUV, MLUV2 and MLUV1, but for predictions of two ellipses datasets the ranking (from best to worst) was MLUV2, MLUV, and MLUV1. Overall, the MLUV2 space was found to be the best. Hence, it is expected that MLUV2 could be used for color specification and color difference evaluations, especially in industrial applications that depend on additive light mixing, such as color TV sets, video monitors, and lighting.

ANALISIS QUALITY OF SERVICE PADA SISTEM MONITORING KULKAS DUA PINTU MENGGUNAKAN KAMERA USBWEBCAM BERBASIS ANDROID

The high level of public consumption in the field of technology encourages innovations made in the refrigerator so that it can be monitored remotely. This study aims to monitor the inside of the refrigerator by streaming and with color video quality and measuring network performance from the motion application on the Raspberry pi to the client. This system uses one camera usbwebcam and one lamp with an input voltage of 5 volts DC connected to the Raspberry Pi 3B. The software used to configure the camera is motion while controlling the lights uses the php language stored in the Raspberry Pi 3B. The application on Android has a display of both camera data, buttons light control, and graphical information from the MQ135, MQ136, and MQ137 sensors. Network quality based on 4 QoS parameters, namely throughput, delay, jitter, and packet loss in 3 conditions can be categorized as good based on the TIPHON standard. Thevalue is throughput 119kbps to 200kbps, thevalue is delay 25.56ms to 39.26ms, thevalue is jitter 50.36ms to 77.46ms and the packet loss value is 0% to 11.1%. Thevalues are throughput, delay, and jitter affected by the light captured by the camera, while packet loss is not affected by light and temperature in the refrigerator.

Sequential RGB color imaging with amillimeter-scale monochrome camera with a rolling shutter.

Applications are growing for ultracompact millimeter-scale cameras. For color images, these sensors commonly utilize a Bayer mask, which can negatively and perceptibly have an impact on image resolution and quality, especially for low pixel-count submillimeter sensors. To alleviate this, we built a time-multiplexed RGB LED illumination system synchronized to the rolling shutter of a monochrome camera. The sequential images are processed and displayed as near real-time color video. Experimental comparison with an identical sensor with a Bayer color mask showed significant improvement in the MTF curves and to perceived image clarity. Trade-offs with respect to system complexity and color motion artifacts are discussed.

Reference-Based Deep Line Art Video Colorization.

Coloring line art images based on the colors of reference images is a crucial stage in animation production, which is time-consuming and tedious. This paper proposes a deep architecture to automatically color line art videos with the same color style as the given reference images. Our framework consists of a color transform network and a temporal refinement network based on 3U-net. The color transform network takes the target line art images as well as the line art and color images of the reference images as input and generates corresponding target color images. To cope with the large differences between each target line art image and the reference color images, we propose a distance attention layer that utilizes non-local similarity matching to determine the region correspondences between the target image and the reference images and transforms the local color information from the references to the target. To ensure global color style consistency, we further incorporate Adaptive Instance Normalization (AdaIN) with the transformation parameters obtained from a multiple-layer AdaIN that describes the global color style of the references extracted by an embedder network. The temporal refinement network learns spatiotemporal features through 3D convolutions to ensure the temporal color consistency of the results. Our model can achieve even better coloring results by fine-tuning the parameters with only a small number of samples when dealing with an animation of a new style. To evaluate our method, we build a line art coloring dataset. Experiments show that our method achieves the best performance on line art video coloring compared to the current state-of-the-art methods.

Stability Study of Multi-Level Grayscales Based on Driving Waveforms for Electrowetting Displays.

Electrowetting Display (EWD) is a new reflective display with an outstanding performance of color video playback. However, some problems still exist and affect its performance. For instance, oil backflow, oil splitting, and charge trapping phenomena may occur during the driving process of EWDs, which would decrease its stability of multi-level grayscales. Therefore, an efficient driving waveform was proposed to solve these disadvantages. It consisted of a driving stage and a stabilizing stage. First, an exponential function waveform was used in the driving stage for driving the EWDs quickly. Then, an alternating current (AC) pulse signal waveform was used in the stabilizing stage to release the trapped positive charges of the insulating layer to improve display stability. A set of four level grayscale driving waveforms were designed by using the proposed method, and it was used in comparative experiments. The experiments showed that the proposed driving waveform could mitigate oil backflow and splitting effects. Compared to a traditional driving waveform, the luminance stability was increased by 8.9%, 5.9%, 10.9%, and 11.6% for the four level grayscales after 12 s, respectively.

Survey on Video Colorization: Concepts, Methods and Applications

Survey on Video Colorization: Concepts, Methods and Applications

Video colorization with semantic preservation and consistency

Video colorization is greatly useful for reviving historical moments recorded by black-and-white media or sketch, portrait, and the semantic preservation within colorized frames is challenging. In the paper an text-driven video colorization method is proposed, which focuses on the hybrid deep network handling image object segmentation, foreground object colorization, background completion, text to image object linkage, and spatiotemporal preservation. The deliberately designed network is composed of two-layer LSTM and mask RCNN. The flexible training strategy avoids the overcost of separate training, while holds the advantage of text instruction, foreground object recognition and location. Experiments demonstrated that the quantitative criteria is better than the state-of-the-art method, and the user study shows that the color naturalness, saturation and richness is well preserved within the colorized video.

Tensor form of GPBiCG algorithm for solving the generalized Sylvester quaternion tensor equations

Sylvester quaternion tensor equations have a wide range of applications in image processing and system and control theory. In this paper, by the Kronecker product and vectorization operator and the properties of quaternion tensors, we focus mainly on proposing the tensor form of the generalized product-type biconjugate gradient method for solving generalized Sylvester quaternion tensor equations. As an application, we apply the proposed method to restore a blurred and noisy-free color video. The obtained numerical results illustrate the effectiveness of our method compared with some existing methods.

A Novel Method of Invisible Video Watermarking Based on Index Mapping and Hybrid DWT-DCT

Watermarking is widely used in multimedia preservation and communication, which comprises, and is therefore not limited to, data security and validation. Security, readability, imperceptibility, and resilience are some of the key advantages of this technology. However, there are still certain issues that need to be addressed, such as the ability to withstand a variety of assaults without substantially affecting the quality and value of embedded data. Based on its operational domain, the watermarking technology may be divided into two groups: spatial and frequency watermarking. An index mapping-based watermarking approach for copyright protection of multi-media color videos is presented in this paper. We offer a hybrid discrete wavelet transform and discrete cosine transform (DCT) watermarking algorithm for digital video watermarking of a color video watermark. Peak signal to noise ratio (PSNR), similarity structure index measure (SSIM), have been used to analyze the distortion produced by watermarking. It is suggested that the proposed video watermarking method provides greater imperceptibility in conjunction with each other with the human visual system and it offers higher robustness against different attacks.

Tensor Robust Principal Component Analysis via Tensor Fibered Rank and \({\boldsymbol{{l_p}}}\) Minimization

Tensor robust principal component analysis (TRPCA) is an important method to handle high-dimensional data and has been widely used in many areas. In this paper, we mainly focus on the TRPCA problem based on tensor fibered rank for sparse noise removal, which aims to recover the low-fibered-rank tensor from grossly corrupted observations. Usually, the -norm is used as a convex approximation of tensor rank, but it is essentially biased and fails to achieve the best estimation performance. Therefore, we first propose a novel nonconvex model named , in which the norm is adopted to approximate tensor fibered rank and measure sparsity. Then, an error bound of the estimator of is established and this error bound can be better than those of similar models based on Tucker rank or tubal rank. Further, we use the alternating direction method of multipliers to solve and provide convergence guarantee. Finally, extensive experiments on color images, videos, and hyperspectral images demonstrate the effectiveness of the proposed method.

3D Video Tracking Technology in the Assessment of Orofacial Impairments in Neurological Disease: Clinical Validation.

This study sought to determine whether clinically interpretable kinematic features extracted automatically from three-dimensional (3D) videos were correlated with corresponding perceptual clinical orofacial ratings in individuals with orofacial impairments due to neurological disorders. 45 participants (19 diagnosed with motor neuron diseases [MNDs] and 26 poststroke) performed two nonspeech tasks (mouth opening and lip spreading) and one speech task (repetition of a sentence "Buy Bobby a Puppy") while being video-recorded in a standardized lab setting. The color video recordings of participants were assessed by an expert clinician-a speech language pathologist-on the severity of three orofacial measures: symmetry, range of motion (ROM), and speed. Clinically interpretable 3D kinematic features, linked to symmetry, ROM, and speed, were automatically extracted from video recordings, using a deep facial landmark detection and tracking algorithm for each of the three tasks. Spearman correlations were used to identify features that were significantly correlated (p value < .05) with their corresponding clinical scores. Clinically significant kinematic features were then used in the subsequent multivariate regression models to predict the overall orofacial impairment severity score. Several kinematic features extracted from 3D video recordings were associated with their corresponding perceptual clinical scores, indicating clinical validity of these automatically derived measures. Different patterns of significant features were observed between MND and poststroke groups; these differences were aligned with clinical expectations in both cases. The results show that kinematic features extracted automatically from simple clinical tasks can capture characteristics used by clinicians during assessments. These findings support the clinical validity of video-based automatic extraction of kinematic features.

Video Game Aesthetics in Squid Game

The South Korean TV-show Squid Game (2021) recently became a major cultural phenomenon. In this video essay, we argue that it has been especially impactful because it draws heavily on colorful video game aesthetics and contrasts them with a bleak, hypercapitalistic South Korean reality. In that way, the show is a metaphor for modern-day escapism.

A Review on Approval Process and Regulation of Medical Devices as per US FDA and CDSCO

Regulatory affairs play a significant role in the pharmaceutical industry, it is “the process of gathering, reviewing, and monitoring the existing regulatory environment and creating regulatory plans and gaining a competitive edge for securing regulatory approvals required more than just acquiring facts”. A medical device is a treatment instrument that "does not work on or in the human body in a way that is primarily intended by pharmacological, immunological, or metabolic mechanisms”. In other words, a medical device is essentially any medical product, not a medicine or biological product. By the Drugs and Cosmetics Act, Central Drugs Standard Control Organization (CDSCO) oversees approving drugs, conducting clinical trials, setting standards for drugs, monitoring the quality of drugs imported into the nation, and coordinating the efforts of state drug control organizations by offering professional advice to ensure uniformity in the application of the Drugs and Cosmetics Act. The US Food and Drug Administration (FDA), a single agency, oversees the regulation of a trillion dollars’ worth of goods, or about 25 cents of every dollar spent, including 80 percent of the food supply in the US, all medical devices, and prescription drugs, animal products, cosmetics, and even the production of tobacco products. The FDA's Center for Devices and Radiological Health regulates companies that produce, repackage, re-label, and/or import medical devices sold in the United States (CDRH). Also, CDRH is in charge of overseeing the regulation of lasers, x-ray machines, ultrasound equipment, microwave ovens, and color televisions, among other electronic gadgets that generate radiation for both medical and non-medical purposes. The IMDR, which was published in January 2017 and became effective in January 2018, was produced by the Medical Devices and Diagnostics Division of CDSCO and is a set of structured regulations for medical devices. In February 2020, the "Medical Devices (Amendment) Regulations, 2020" were established, and they became effective in April 2020. The 2020 amendment now includes a new clause called "registration of specific medical equipment."