Color Video Research Articles

LVCD: Reference-based Lineart Video Colorization with Diffusion Models

We propose the first video diffusion framework for reference-based lineart video colorization. Unlike previous works that rely solely on image generative models to colorize lineart frame by frame, our approach leverages a large-scale pretrained video diffusion model to generate colorized animation videos. This approach leads to more temporally consistent results and is better equipped to handle large motions. Firstly, we introduce Sketch-guided ControlNet which provides additional control to finetune an image-to-video diffusion model for controllable video synthesis, enabling the generation of animation videos conditioned on lineart. We then propose Reference Attention to facilitate the transfer of colors from the reference frame to other frames containing fast and expansive motions. Finally, we present a novel scheme for sequential sampling, incorporating the Overlapped Blending Module and Prev-Reference Attention , to extend the video diffusion model beyond its original fixed-length limitation for long video colorization. Both qualitative and quantitative results demonstrate that our method significantly outperforms state-of-the-art techniques in terms of frame and video quality, as well as temporal consistency. Moreover, our method is capable of generating high-quality, long temporal-consistent animation videos with large motions, which is not achievable in previous works. Our code and model are available at https://luckyhzt.github.io/lvcd.

A System of Tensor Equations over the Dual Split Quaternion Algebra with an Application

In this paper, we propose a definition of block tensors and the real representation of tensors. Equipped with the simplification method, i.e., the real representation along with the M-P inverse, we demonstrate the conditions that are necessary and sufficient for the system of dual split quaternion tensor equations (A∗NX,X∗SC)=(B,D), when its solution exists. Furthermore, the general expression of the solution is also provided when the solution of the system exists, and we use a numerical example to validate it in the last section. To the best of our knowledge, this is the first time that the aforementioned tensor system has been examined on dual split quaternion algebra. Additionally, we provide its equivalent conditions when its Hermitian solution X=X∗ and η-Hermitian solutions X=Xη∗ exist. Subsequently, we discuss two special dual split quaternion tensor equations. Last but not least, we propose an application for encrypting and decrypting two color videos, and we validate this algorithm through a specific example.

Computation of Outer Inverse of Tensors Based on t‐Product

ABSTRACTTensor computations play an essential role in various fields of science and engineering, including multiway data analysis. In this study, we established a few basic properties of the range and null space of a tensor by using block circulant matrices and a discrete Fourier matrix. We then discuss the outer inverse of the tensors based on ‐product with a prescribed range and kernel of third‐order tensors. We address the relation of this outer inverse with other generalized inverses, such as the Moore–Penrose inverse, group inverse, and Drazin inverse. In addition, we present a few algorithms for computing the outer inverses of the tensors. In particular, a ‐QR decomposition based algorithm was developed to compute outer inverses. It is well known that the confidentiality of information transmitted through the virtual world grows exponentially, and color image and video security have become a significant concern when communicating over the internet. As an application, a ‐QR decomposition based algorithm was demonstrated for concealing secret color images and videos.

Low-rank quaternion tensor completion for color video inpainting via a novel factorization strategy

Recently, a quaternion tensor product named Qt-product was proposed, and then the singular value decomposition and the rank of a third-order quaternion tensor were given. From a more applicable perspective, we extend the Qt-product and propose a novel multiplication principle for third-order quaternion tensor named gQt-product. With the gQt-product, we introduce a brand-new singular value decomposition for third-order quaternion tensors named gQt-SVD and then define gQt-rank and multi-gQt-rank. We prove that the optimal low-rank approximation of a third-order quaternion tensor exists and some numerical experiments demonstrate the low-rankness of color videos. So, we apply the low-rank quaternion tensor completion to color video inpainting problems and present alternating least-square algorithms to solve the proposed low gQt-rank and multi-gQt-rank quaternion tensor completion models. The convergence analyses of the proposed algorithms are established and some numerical experiments on various color video datasets show the high recovery accuracy and computational efficiency of our methods.

Developing Canva-Based Sitekol on Collage Topic for First-Grade Public Elementary School Students

The study aims to create a learning media product using Canva for elementary school students to alleviate the lack of learning materials, which has been negatively impacting student academic performance. The research also seeks to evaluate the practicality and effectiveness of the learning media product. This study uses research and development with ADDIE design. The results of the analysis revealed that some students do not meet the minimum standards and require learning tools to improve their skills in cutting and sticking when studying collage topics. This research developed Sitekol media on collage topics in the design stage using Canva and Wordwall applications. The validity test results showed that Sitekol media received a 90% score from media experts for color appearance, image combinations, fonts, media layout, videos, and practical use. Material experts rated it at 95.3% for material aspects, levels of thinking, and supporting learning content. Students also found Canva-based Sitekol learning media to be beneficial as a teaching tool due to its accessibility, portability, convenience, interactive features, and detailed learning media instructions. The statistical tests conducted resulted in significant findings with high average values. Students responded very positively to the practicality of the media, giving an average score of 4.9 out of 5, indicating 98% of the ideal score. It can be concluded that using Canva-based Sitecol learning on grade 1 collage material exhibits practicality and validity as a learning medium.

SwinTExCo: Exemplar-based video colorization using Swin Transformer

Video colorization represents a compelling domain within the field of Computer Vision. The traditional approach in this field relies on Convolutional Neural Networks (CNNs) to extract features from each video frame and employs a recurrent network to learn information between video frames. While demonstrating considerable success in colorization, most traditional CNNs suffer from a limited receptive field size, capturing local information within a fixed-sized window. Consequently, they struggle to directly grasp long-range dependencies or pixel relationships that span large image or video frame areas. To address this limitation, recent advancements in the field have leveraged Vision Transformer (ViT) and their variants to enhance performance. This article introduces Swin Transformer Exemplar-based Video Colorization (SwinTExCo), an end-to-end model for the video colorization process that incorporates the Swin Transformer architecture as the backbone. The experimental results demonstrate that our proposed method outperforms many other state-of-the-art methods in both quantitative and qualitative metrics. The achievements of this research have significant implications for the domain of documentary and history video restoration, contributing to the broader goal of preserving cultural heritage and facilitating a deeper understanding of historical events through enhanced audiovisual materials.

Copy-Right protection of color videos using robust watermarking based geometrically invariant moments of fractional orders and logistic sine cosine chaotic map

Copy-Right protection of color videos using robust watermarking based geometrically invariant moments of fractional orders and logistic sine cosine chaotic map

The BiCG Algorithm for Solving the Minimal Frobenius Norm Solution of Generalized Sylvester Tensor Equation over the Quaternions

In this paper, we develop an effective iterative algorithm to solve a generalized Sylvester tensor equation over quaternions which includes several well-studied matrix/tensor equations as special cases. We discuss the convergence of this algorithm within a finite number of iterations, assuming negligible round-off errors for any initial tensor. Moreover, we demonstrate the unique minimal Frobenius norm solution achievable by selecting specific types of initial tensors. Additionally, numerical examples are presented to illustrate the practicality and validity of our proposed algorithm. These examples include demonstrating the algorithm’s effectiveness in addressing three-dimensional microscopic heat transport and color video restoration problems.

Krylov subspace methods based quaternion tensor form for generalized Sylvester quaternion tensor equation with application to color video restoration

In this paper, we consider Krylov subspace methods based quaternion tensor form for a class of generalized Sylvester quaternion tensor equation. A novel quaternion tensor product and a global quaternion Arnoldi process based on the tensor form are proposed. Then, the quaternion tensor Krylov subspaces and their orthogonal bases are constructed via the proposed global quaternion Arnoldi process. The quaternion full orthogonalization method based on tensor format (denoted by QFOM−BTF) and the quaternion generalized minimal residual method based on tensor format (denoted by QGMRES−BTF) are established to solve the quaternion tensor equation. The theoretically analyze results of the proposals are discussed on the quaternion tensor form. Finally, we demonstrate the feasibility of QFOM−BTF and QGMRES−BTF methods on some numerical examples including the color video restoration.

Video Color OCT Angiography for Myopic Choroidal Neovascularization

To investigate the myopic macular neovascularization (mMNV) features on dynamic video-color optical coherence tomography (OCT) angiography (OCTA) and the diagnostic rate versus the static, four-segmentations visualization mode. Retrospective cohort study PARTICIPANTS: Fifty-four patients with mMNV METHODS: Sixty-two eyes with high myopia complicated by mMNV were included. Clinical charts, fluorescein angiography and structural OCT were used as standard reference to assess lesion activity. Static and video-color OCTA were then analysed and compared by two independent reviewers. Morphology description of mMNV on video-color OCTA and differences in the proportion of diagnosis between video-colour and static OCTA. 62 eyes from 54 patients (mean age 63,22 years) were enrolled. Thirty-four (55%) mMNV were active and 28 (45%) inactive. Twenty-two (65%) active mMNV presented on video-color OCTA as an interlacing vascular network in the outer retina and the choriocapillaris. A tapered form was the prevalent size (72,7%). In 3 eyes (9%) an abnormal and irregular vascular network (AVN) was disclosed and in 5 (15%) only some blood flow alteration. All the lesions extended both in the outer retinal and the choriocapillaris. Eleven (39%) inactive mMNV presented on video-color OCTA as an interlacing vascular network too, in the outer retina and the choriocapillaris. Eight (29%) had some AVN and 6 (21%) only some blood flow alteration. The diagnostic rate of video-color vs static OCTA was 95% (IC 95% 86% to 99%) vs 77% (IC 95% 86% to 99%, p= 0.0009), and shows an advantage in favour of video-colour OCTA of 15% (CI 95%, 3%-27%) and 22% (CI 95%, 7%-38%) in active and inactive lesions, respectively (p<0.026). Lesion extension within both the outer retina and the choriocapillaris was present in 90% and 69% of cases on dynamic OCTA and static OCTA, respectively, with a proportion difference of 20% (CI 95%, 10%-31%, p= 0.0005). Concordance between the two examiners was high: 0.95 (95%, CI 0.88 to 1.00) and 0.96 (0.91 to 1.00) for active and inactive lesions, respectively. Video color-enhanced OCTA may help in diagnosing mMNV and should be considered by clinicians in addition to structural OCT and static OCTA.

BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-Based Video Colorization.

How to effectively explore the colors of exemplars and propagate them to colorize each frame is vital for exemplar-based video colorization. In this article, we present a BiSTNet to explore colors of exemplars and utilize them to help video colorization by a bidirectional temporal feature fusion with the guidance of semantic image prior. We first establish the semantic correspondence between each frame and the exemplars in deep feature space to explore color information from exemplars. Then, we develop a simple yet effective bidirectional temporal feature fusion module to propagate the colors of exemplars into each frame and avoid inaccurate alignment. We note that there usually exist color-bleeding artifacts around the boundaries of important objects in videos. To overcome this problem, we develop a mixed expert block to extract semantic information for modeling the object boundaries of frames so that the semantic image prior can better guide the colorization process. In addition, we develop a multi-scale refinement block to progressively colorize frames in a coarse-to-fine manner. Extensive experimental results demonstrate that the proposed BiSTNet performs favorably against state-of-the-art methods on the benchmark datasets and real-world scenes. Moreover, the BiSTNet obtains one champion in NTIRE 2023 video colorization challenge (Kang et al. 2023).

Robust tensor recovery via a nonconvex approach with ket augmentation and auto‐weighted strategy

AbstractIn this article, we introduce a nonconvex tensor recovery approach, which employs the powerful ket augmentation technique to expand a low order tensor into a high‐order one so that we can exploit the advantage of tensor train (TT) decomposition tailored for high‐order tensors. Moreover, we define a new nonconvex surrogate function to approximate the tensor rank, and develop an auto‐weighted mechanism to adjust the weights of the resulting high‐order tensor's TT ranks. To make our approach robust, we add two mode‐unfolding regularization terms to enhance the model for the purpose of exploring spatio‐temporal continuity and self‐similarity of the underlying tensors. Also, we propose an implementable algorithm to solve the proposed optimization model in the sense that each subproblem enjoys a closed‐form solution. A series of numerical results demonstrate that our approach works well on recovering color images and videos.

TIHN: Tensor Improved Huber Norm for low-rank tensor recovery

Tensor Robust Principal Component Analysis (TRPCA) has received much attention in many real-world applications which aims to recover a low-rank tensor corrupted by sparse noise. Most existing TRPCA methods usually regularize the low-rank component by minimizing its Tensor Nuclear Norm (TNN). However, the original TNN shrinks all of the singular values with the same penalty which generally leads to suboptimal results. The reason is that the large singular values should be less penalized since they usually correspond to salient information of the real-world tensor. In this work, we develop a Tensor Improved Huber Norm (TIHN) which considers the difference between the singular values of tensor. The TIHN can achieve better performance by recovering the large and significant singular values exactly. We also establish the Huber TRPCA (HTRPCA) method by utilizing the proposed TIHN. In addition, an efficient optimization algorithm based on the half-quadratic theory and Alternating Direction Method of Multipliers (ADMM) framework is designed to implement the HTRPCA. Finally, experiments on color image recovery and video recovery validate the effectiveness of the proposed method.

Estimation of human body 3D pose for parent-infant interaction settings using azure Kinect and OpenPose

Automatic pose estimation has become a valuable tool for the study of human behavior, including dyadic interactions. It allows researchers to analyze the nuanced dynamics of interactions more effectively, and facilitates the integration of behavioral data with other modalities (EEG, etc.). However, many technical difficulties remain. Particularly, for parent-infant interactions, automatic pose estimation for infants is unpredictable; the immature proportions and smaller bodies of children may cause misdetections. OpenPose is one tool that has shown high performance in pose tracking from video, even in infants. However, OpenPose is limited to 2D (i.e., coordinates relative to the image space). This may be undesirable in a multitude of paradigms (e.g., naturalistic settings). We developed a method for expanding the functionality of OpenPose to 3D, tailored to parent-infant interaction paradigms. This method merges the estimations from OpenPose with the depth information from a depth camera to obtain a 3D pose that works even for young infants.•Video recordings of interactions of parents and infants are taken using a dual color-depth camera.•2D-positions of parents and their infants are estimated from the color video.•Using the depth camera, we transform the 2D estimations into real-world 3D positions, allowing movement analysis in full-3D space.

A nonlinear high-order transformations-based method for high-order tensor completion

The high-order tensor nuclear norm model (HTNN) has recently shown promising results in tensor completion problems. The HTNN-based approaches rely on the low-rank structure of tensor slices during reversible transformations. However, under reversible transformations, the low-rank configuration within the slice-wise modality of the tensor’s structure is not markedly pronounced. In order to more effectively describe the low-rank characteristics, we propose a model based on the nonlinear high-order transform-based tensor nuclear norm (NHTNN). Specifically, our framework consists of a linearly semi-orthogonal transformation along the high-dimensional modality and an element-wise nonlinear transformation. We introduce a model for tensor completion, grounded in the suggested measure of tensor low-rank, i.e., NHTNN. Utilizing this non-convex nonlinear model, we formulate a proximal alternating minimization (PAM) algorithm, establishing its convergence through a rigorous proof. In experiments on datasets such as hyperspectral videos (HSVs) and color videos (CVs), our approach demonstrates superior quantitative numerical results and qualitative visual effects compared to cutting-edge tensor completion techniques.

Estrategias publicitarias para fortalecer la identidad y presencia de TV color en el mercado de comunicación de Latacunga

Marketing has evolved with the use of technology and digital media. This research is aimed at optimizing TV Color strategies to strengthen the presence and identity of TV Color in the Latacunga communication market. For this purpose, it was necessary to apply a methodology that begins with a field investigation, with a mixed qualitative and quantitative approach, of a descriptive nature. The study population corresponds to the population between 15 and 75 years old, through a representative sample segmentedby urban and rural sector, parishes, gender and age group. The results reflect weaknesses in the channel's current marketing and little presence in the media, the demand focuses on adults. For this reason, a strategy optimization plan was proposed based on the Inbound Marketing methodology, aimed at capturing the young audience.

Low-Rank Tensor Completion Based on Self-Adaptive Learnable Transforms.

The tensor nuclear norm (TNN), defined as the sum of nuclear norms of frontal slices of the tensor in a frequency domain, has been found useful in solving low-rank tensor recovery problems. Existing TNN-based methods use either fixed or data-independent transformations, which may not be the optimal choices for the given tensors. As the consequence, these methods cannot exploit the potential low-rank structure of tensor data adaptively. In this article, we propose a framework called self-adaptive learnable transform (SALT) to learn a transformation matrix from the given tensor. Specifically, SALT aims to learn a lossless transformation that induces a lower average-rank tensor, where the Schatten- p quasi-norm is used as the rank proxy. Then, because SALT is less sensitive to the orientation, we generalize SALT to other dimensions of tensor (SALTS), namely, learning three self-adaptive transformation matrices simultaneously from given tensor. SALTS is able to adaptively exploit the potential low-rank structures in all directions. We provide a unified optimization framework based on alternating direction multiplier method for SALTS model and theoretically prove the weak convergence property of the proposed algorithm. Experimental results in hyperspectral image (HSI), color video, magnetic resonance imaging (MRI), and COIL-20 datasets show that SALTS is much more accurate in tensor completion than existing methods. The demo code can be found at https://faculty.uestc.edu.cn/gaobin/zh_CN/lwcg/153392/list/index.htm.

Experimental Combustion of Different Biomass Wastes, Coals and Two Fuel Mixtures on a Fire Bench

When designing settlements according to the “Green Building” principle, it is necessary to develop a heating system based on climatic conditions. For example, in areas with a sharply continental climate (cold and prolonged winters), it is sometimes necessary to use solid fuel boilers (in the absence of gas). However, to use these, it is necessary to use biomass or biomass-coal blends as fuel to increase their combustion heat. The addition of biomass waste to coal can be aimed at achieving various objectives: utilization of biomass waste; reduction of solid fossil fuel consumption; improvement of environmental performance at coal-fired boiler houses; improvement of the reactivity of coals or to improve the technical and economic performance of heat-generating plants due to the fact that biomass is a waste from various types of production, and its cost depends only on the distance of its transportation to the boiler house. In this work, combustion of various biomass wastes, including sewage sludge, was carried out on a fire bench emulating the operation of a boiler furnace. Fuel particles were ignited by convective heat transfer in a stream of hot air at a velocity of 5 m/s in the temperature range of 500–800 °C, and the experimental process was recorded on a high-speed, color video camera. The obtained values were compared with the characteristics of different coals used in thermal power generation (lignite and bituminous coal). The aim of the work is to determine the reactivity of various types of biomass, including fuel mixtures based on coal and food waste. The work presents the results of technical and elemental analysis of the researched fuels. Scanning electron microscopy was used to analyze the fuel particle surfaces for the presence of pores, cracks and channels. It was found that the lowest ignition delay is characteristic of cedar needles and hydrolyzed lignin; it is four times less than that of lignite coal and nine times less than that of bituminous coal. The addition of hydrolysis lignin to coal improves its combustion characteristics, while the addition of brewer’s spent grain, on the contrary, reduces it, increasing the ignition time delay due to the high moisture content of the fuel particles.

A Visually Meaningful Color-Video Encryption Scheme That Combines Frame Channel Fusion and a Chaotic System

Compared to text and images, video can show information more vividly and intuitively via a moving picture; therefore, video is widely used in all walks of life. However, videos uploaded or stored in various video applications have not been treated with any protection, and these videos contain a lot of sensitive information that is more likely to be compromised. To solve this problem, video encryption schemes have been proposed. However, the main concern with existing video encryption schemes is that the private information in the encrypted video should be effectively protected, and, thus, the pixel distribution of the original video can be greatly damaged in the process of encryption, resulting in no or poor visual usability of the encrypted video. To this end, a novel color-video encryption scheme is proposed, which can effectively protect video privacy information while retaining certain visual information, thus enhancing the usability of encrypted videos. Firstly, the R, G, and B channels of the original color video are viewed as a whole for splitting. The dimensions of the blocks are three-dimensional, and permutation encryption is performed in three-dimensional blocks, which eliminates the redundancy of information between the video frame channels. Secondly, after permutation encryption, the channels of the video frame are separated, and then each channel is divided into blocks. The shape of the blocks is a square, and substitution encryption and permutation encryption operations are performed in turn. The whole encryption process is combined with the 2D-LSM chaotic system to improve the security of the scheme, as well as to reduce the time. Extensive experiments have been carried out, and the results show that the proposed scheme allows the encrypted video to retain rough visual information and, at the same time, effectively protects privacy, achieving the goal of encrypted video security and usability.

“Sparse + Low-Rank” tensor completion approach for recovering images and videos

Recovering color images and videos from highly undersampled data is a fundamental and challenging task in face recognition and computer vision. By the multi-dimensional nature of color images and videos, in this paper, we propose a novel tensor completion approach, which is able to efficiently explore the sparsity of tensor data under the discrete cosine transform (DCT). Specifically, we introduce two “sparse + low-rank” tensor completion models as well as two implementable algorithms for finding their solutions. The first one is a DCT-based sparse plus weighted nuclear norm induced low-rank minimization model. The second one is a DCT-based sparse plus p-shrinking mapping induced low-rank optimization model. Moreover, we accordingly propose two implementable augmented Lagrangian-based algorithms for solving the underlying optimization models. A series of numerical experiments including color image inpainting and video data recovery demonstrate that our proposed approach performs better than many existing state-of-the-art tensor completion methods, especially for the case when the ratio of missing data is high.