Video Editing System Research Articles

Meetor: A Human-Centered Automatic Video Editing System for Meeting Recordings

Widely adopted digital cameras and smartphones have generated a large number of videos, which have brought a tremendous workload to video editors. Recently, a variety of automatic/semi-automatic video editing methods have been proposed to tackle these issues in some specific areas. However, for the production of meeting recordings, the existing studies highly depend on extra equipment in the conference venues, such as the infrared camera or special microphone, which are not practical. In this paper, we design and implement Meetor, a human-centered automatic video editing system for meeting recordings. The Meetor mainly contains three parts: an audio-based video synchronization algorithm, human-centered video content flaw detection algorithms, and an automatic video editing algorithm. Two main experiments are conducted from both objective and subjective aspects to evaluate the performance of the Meetor. The experimental results on a testbed illustrate that the proposed algorithms could achieve state-of-the-art (SOTA) performance in video content flaw detection. On the other hand, the conducted user study demonstrates that Meetor could generate meeting recordings with a satisfactory quality compared with professional video editors. Moreover, we also present a practical application of the Meetor in a university campus prototype, in which the Meetor is applied in the automatic editing of lecture recordings. All in all, the proposed Meetor can be utilized in practical applications to release the workload of professional video editors.

Audio and video editing system design based on OpenCV

With the rapid development of the Internet, a new carrier for people to perceive the world and communicate with each other - audio and video - is gradually being favoured by the public. The development of multimedia technology and artificial intelligence technology has provided a milestone for the maturity of audio and video technology. In particular, short video platforms have slowly become a new network position for various media promotions. Especially at the moment of the epidemic, the channel of understanding the world through audio and video is increasingly valued. The public has put forward higher demands on the content and presentation of audio and video. Therefore, it is particularly important to produce quality audio-video that meets the requirements of the times, which cannot be achieved without a feasible audio-video editing system. In addition, after previous research and practice, the application of artificial intelligence technology in the field of imaging has also become mature, including some applications in the direction of entertainment. Applying AI technology to the video editing process can improve the efficiency of video editing, increase the interest of video content, and allow video creators to focus on content creation without spending too much time and energy on video editing operations, thus creating better quality videos. This design is the main technology of OpenCV and front-end technology stack, such as JavaScript, React and Electron, to implement basic video editing, video filters, in addition to the development of a friendly interactive interface. The implementation of basic video editing module and video filter module are both based on OpenCV implementation. In this design, the basic video editing implements pan, zoom and rotate operations on the video, and the video filter module is implemented by changing the RGB channel values of the image. The operations on the video can be broken down into operations on each frame of the video, and OpenCV provides a way to implement these operations. The paper concludes with a summary of the shortcomings and flaws in the design, and an outlook on the next steps and future directions. This design is the main technology of OpenCV and front-end technology stack, such as JavaScript, React and Electron, to implement basic video editing, video filters, in addition to the development of a friendly interactive interface. The implementation of basic video editing module and video filter module are both based on OpenCV implementation. In this design, the basic video editing implements pan, zoom and rotate operations on the video, and the video filter module is implemented by changing the RGB channel values of the image. The operations on the video can be broken down into operations on each frame of the video, and OpenCV provides a way to implement these operations. The paper concludes with a summary of the shortcomings and flaws in the design, and an outlook on the next steps and future directions.

Selective video enhancement in the Laguerre–Gauss domain

Traditional image and video enhancement techniques do not consider the subjective preferences of the user, who might be interested in modifying or highlighting specific elements of a scene. In fact, most image or video post-processing techniques are applied to the entire image or frame. In this work, a framework for selective video enhancement is presented. It is based on an adaptive multi-resolution edge enhancement technique performed in the Laguerre–Gauss complex wavelet domain. In the proposed scheme, edges are selectively enhanced or attenuated according to the inputs of the user, taking into account the characteristics of the content, the response of the human visual system, and the masking effects induced by textured background. Two scenarios have been implemented and tested: (i) an interactive video editing system, where the user selects the objects to enhance through a graphical interface and the system automatically propagates the selection through the following video frames belonging to the same shot, (ii) a perceptually-driven technique to improve the quality of RGB plus Depth videos, which uses depth maps as additional features to guide the enhancement.

StylePortraitVideo: Editing Portrait Videos with Expression Optimization

AbstractHigh‐quality portrait image editing has been made easier by recent advances in GANs (e.g., StyleGAN) and GAN inversion methods that project images onto a pre‐trained GAN's latent space. However, extending the existing image editing methods, it is hard to edit videos to produce temporally coherent and natural‐looking videos. We find challenges in reproducing diverse video frames and preserving the natural motion after editing. In this work, we propose solutions for these challenges. First, we propose a video adaptation method that enables the generator to reconstruct the original input identity, unusual poses, and expressions in the video. Second, we propose an expression dynamics optimization that tweaks the latent codes to maintain the meaningful motion in the original video. Based on these methods, we build a StyleGAN‐based high‐quality portrait video editing system that can edit videos in the wild in a temporally coherent way at up to 4K resolution.

Storyboard-based accurate automatic summary video editing system

The recent popularity of smart mobile devices has led to a significant increase in the needs of multimedia services. Finding new more efficient methods for automatic classification and retrieval of a large number of multimedia files will significantly reduce manpower costs. However, most current video content analysis methods adopt low-level features to analyze video frame by frame, and need to improve high-level semantic analysis on a number of issues. Hence, this study presents a storyboard-based accurate automatic summary video editing system that uses storyboard information, such as character dialogue, narration, caption, background music and shot changes, to enable accurate video content retrieval and automatic render summary videos. The proposed system can be applied to the course video trailer and the commercial video trailer for quick preview video content or suitable viewing configuration for smart mobile devices. Consequently, the audience can quickly understand the whole video story and the video editors can substantially reduce the time taken to publish videos.

Forensic CG Video Generation with Augmented Reality Technology

This paper presents a technology for supporting in-house forensic CG video generation by police and prosecution. For confidentiality, it is usually not allowed to generate criminal CG video through subcontracting, whereas existing CG modeling and video editing systems are too complicated to use for police or prosecution. The proposed technique solves this problem with Augmented Reality (AR) technology. We propose to deploy multiple cameras and AR markers on criminal site. While a user of the police or the prosecution plays a role of the criminal wearing a video see-through head mount display, video is automatically generated. CG objects, such as the virtual victim, are displayed on the head mount display and rendered in the video. Our main contributions in this research are concurrent displaying of first-person and third-person views for visualizing spatial relationship between the user and its surroundings, and complementary use of marker information from multiple cameras for solving the occlusion problem inherently caused by operating at the actual criminal site.

Digital Cinema and Super-High-Definition Content Distribution on Optical High-Speed Networks

Digital cinema is a promising application that utilizes high-speed optical networks to transfer super-high-definition (SHD) images. The networks are primarily used for distributing digital cinema contents in packet data form, and are also used to support new services such as the live streaming of musicals and sport games to movie theaters. While current transfer services offer high-definition (HD) quality video, live-streaming applications will soon shift to providing cinema quality 4K content to both business and movie theaters users. The extra-high-quality 4K format enables a realistic telepresence, and will be combined with special tools such as video editing systems to realize effective remote collaboration for business workspaces. This paper introduces successive research on SHD image transmission and its application, especially in digital cinema and associated application fields.

Engaging Viewers Through the Connected Studio: Virtual Participation in TV Programs

In recent years, since broadband communication networks have become widely available, it is possible for ordinary viewers to access video content easily at any time or place with the use of services that deliver video content in a wide diversity of genres. Content delivery services that use user-generated video content have also become popular [1][3]. As technology supports the creation of video content, video editing systems [4], computer graphics (CG) software [5], and the like are being put to practical use. However, technical expertise is needed to use these tools. Furthermore, these tools expect the content creators to take care of key directorial and production roles, such as deciding how the video should be shot, what sort of composition to use, and how to present this content to the viewers. Specialized skills and experiences are needed for these tasks. For ordinary people with no special skills or experiences, the creation of video content can be difficult.

Planning for Reasoning with Multiple Common Sense Knowledge Bases

Intelligent user interfaces require common sense knowledge to bridge the gap between the functionality of applications and the user’s goals. While current reasoning methods have been used to provide contextual information for interface agents, the quality of their reasoning results is limited by the coverage of their underlying knowledge bases. This article presents reasoning composition , a planning-based approach to integrating reasoning methods from multiple common sense knowledge bases to answer queries. The reasoning results of one reasoning method are passed to other reasoning methods to form a reasoning chain to the target context of a query. By leveraging different weak reasoning methods, we are able to find answers to queries that cannot be directly answered by querying a single common sense knowledge base. By conducting experiments on ConceptNet and WordNet, we compare the reasoning results of reasoning composition, directly querying merged knowledge bases, and spreading activation. The results show an 11.03% improvement in coverage over directly querying merged knowledge bases and a 49.7% improvement in accuracy over spreading activation. Two case studies are presented, showing how reasoning composition can improve performance of retrieval in a video editing system and a dialogue assistant.

Location based abstraction of user generated mobile videos

Demand for efficient ways to represent vast amount of video data has grown rapidly in recent years. The advances in positioning services have led to new possibilities in combining location information to video content. In this paper we present an automatic video editing system for geotagged mobile videos. In our solution the system creates automatically a video summary from a set of unedited video clips. Location information and timestamps are used to group video clips with the same context properties. The groups are used to create a video summary where subshots from same context group are represented as scenes. The novelty in our solution lies in combining geotags with low level content analysis tools in video abstraction. We have evaluated the created video summaries with a group of users and the system usability for service creation by building a semi-automatic web-based video editing service. The evaluations prove that our concept is useful as it improves coherence and enjoyability of the automatic video summaries.

Refilming with Depth-Inferred Videos

Compared to still image editing, content-based video editing faces the additional challenges of maintaining the spatiotemporal consistency with respect to geometry. This brings up difficulties of seamlessly modifying video content, for instance, inserting or removing an object. In this paper, we present a new video editing system for creating spatiotemporally consistent and visually appealing refilming effects. Unlike the typical filming practice, our system requires no labor-intensive construction of 3D models/surfaces mimicking the real scene. Instead, it is based on an unsupervised inference of view-dependent depth maps for all video frames. We provide interactive tools requiring only a small amount of user input to perform elementary video content editing, such as separating video layers, completing background scene, and extracting moving objects. These tools can be utilized to produce a variety of visual effects in our system, including but not limited to video composition, "predator" effect, bullet-time, depth-of-field, and fog synthesis. Some of the effects can be achieved in real time.

Play-it-by-eye! Collect movies and improvise perspectives with tangible video objects

AbstractWe present an alternative video-making framework for children with tools that integrate video capture with movie production. We propose different forms of interaction with physical artifacts to capture storytelling. Play interactions as input to video editing systems assuage the interface complexities of film construction in commercial software. We aim to motivate young users in telling their stories, extracting meaning from their experiences by capturing supporting video to accompany their stories, and driving reflection on the outcomes of their movies. We report on our design process over the course of four research projects that span from a graphical user interface to a physical instantiation of video. We interface the digital and physical realms using tangible metaphors for digital data, providing a spontaneous and collaborative approach to video composition. We evaluate our systems during observations with 4- to 14-year-old users and analyze their different approaches to capturing, collecting, editing, and performing visual and sound clips.

New MPEG-4-AVC/H.264 Profiles for Professional and High-Quality Video Applications

To respond to strong demands for compression technology appropriate to high-quality video applications, the Joint Video Team (JVT) of MPEG and ITU-T has recently completed the standardization of five additional profiles for the MPEG-4 Part 10/H.264 video coding standard (ISO/IEC 14496–10ITU-T H.264) (also known as MPEG-4-AVC (for Advanced Video Coding). These new profiles have been developed to serve applications in the professional and semi-professional domains with enhanced compression capabilities for color sampling structure ranging up to 4:4:4 and bit depth dynamic ranges up to 14 bits per sample. Four of the five new profiles support “intra-only” applications with demands for extreme low-latency operation, simple editing, and frame random-access functionality. Potential applications of these new profiles include studio/cinematography camera and camcorders, digital video recording and editing systems, studio production and post-production systems, content contribution systems, and high-resolution display systems. This paper discusses the design, coding structure and tools, operation modes, profile definitions, and target application environments of these new profiles. Their improved coding performance will also be demonstrated by the rate distortion curves comparing these new profiles with some of the existing state-of-the-art compression solutions.

M-LTW: A fast and efficient intra video codec

Intra-video coding is a common way to process video material for applications like professional video editing systems, digital cinema, video surveillance applications, multi-spectral satellite imaging, HQ video delivery, etc. Most practical intra-coding systems employ JPEG encoders due to their simplicity, low coding delay and low memory requirements. JPEG2000 is the main candidate to replace JPEG in this kind of application due to its excellent rate/distortion (R/D) performance and high coding flexibility. However, its complexity and computational resource requirements for proper operation could be a limitation for certain applications. In this work, we propose an intra-video codec, M-LTW, which is able to obtain very good R/D performance results, as good as JPEG2000 or H.264 INTRA, with faster processing and lower memory usage.

Video Structure Analysis

Video structure analysis is a basic requirement for most content-based video editing and processing systems. This paper presents a fast video structure analysis method based on image segmentation in each frame, with region matching between frames. The structure analysis decomposes the video into several moving objects, including information about their colors, positions, shapes, movements, and lifetimes. The method also supports user interactions to improve the results. The result shows that this method is fast and stable and can complete video analyzing interactively.

RotoTexture: Automated Tools for Texturing Raw Video

We propose a video editing system that allows a user to apply a time-coherent texture to a surface depicted in the raw video from a single uncalibrated camera, including the surface texture mapping of a texture image and the surface texture synthesis from a texture swatch. Our system avoids the construction of a 3D shape model and instead uses the recovered normal field to deform the texture so that it plausibly adheres to the undulations of the depicted surface. The texture mapping method uses the nonlinear least-squares optimization of a spring model to control the behavior of the texture image as it is deformed to match the evolving normal field through the video. The texture synthesis method uses a coarse optical flow to advect clusters of pixels corresponding to patches of similarly oriented surface points. These clusters are organized into a minimum advection tree to account for the dynamic visibility of clusters. We take a rather crude approach to normal recovering and optical flow estimation, yet the results are robust and plausible for nearly diffuse surfaces such as faces and t-shirts.

Optimization-Based Automated Home Video Editing System

In this paper, we present an optimization-based system that automates home video editing. This system automatically selects suitable or desirable highlight segments from a set of raw home videos and aligns them with a given piece of incidental music to create an edited video segment to a desired length based on the content of the video and incidental music. We developed an approach for extracting temporal structure and determining the importance of a video segment in order to facilitate the selection of highlight segments. Additionally we extract a temporal structure, beats, and tempos from the incidental music. In order to create more professional-looking results, the selected highlight segments satisfy a set of editing rules and are matched to the content of the incidental music. This task is formulated as a nonlinear 0-1 programming problem and the rules, which are adjustable and increasable, are embedded as constraints. The output video is rendered by connecting the selected highlight video segments with transition effects and the incidental music. Under this framework, we can choose the best-matched music for a given video and support different output styles.

Preparing a digital video examination of urinary sediments.

1. Robert J Sullivan, PhD MT(ASCP)[⇑][1] 1. is in the Department of Medical Laboratory Sciences, Marist College, Poughkeepsie NY 2. Joey Petrella Wall, MS 1. is Director, Media and Instructional Technology, Marist College, Poughkeepsie NY 1. Address for correspondence: Robert J Sullivan PhD MT(ASCP), Department of Medical Laboratory Sciences, Marist College, 3399 North Road, Poughkeepsie NY 12601. (845) 575-3000, x 2177, (845) 471-6213 (fax). Robert.Sullivan{at}Marist.edu Teaching the microscopic examination of urine sediments with patient samples is limited to the availability of appropriate, labile material. The alternative is to use static, flat, single views in 35 mm slides or on CD-ROMS. This paper discusses the preparation of material that mimics photographically what is seen through a microscope and presents a dynamic view of samples. Sediments were viewed through a binocular, light microscope using a TeachCam. The video feed was transmitted to a SVHS recorder for maximum resolution. The videotapes were digitized, and processed through a computer based non-linear video editing system. The final formats produced are on VHS videotape and DVD. Each specimen is presented with changes in focus and fields. Each sample is presented unstained and stained, and appears under low and high power. Six discrete segments allow viewing of selected elements. The video is accompanied by a narrative describing the identification and clinical significance of each formed element. The program has been well accepted by students. Many medical conditions and aspects of the body's biochemistry can be identified through the simple means of examining a patient's urine for metabolic waste products, therapeutic or recreational chemicals and their metabolites, cells, and other products of the urinary tract. The ability to identify the formed elements of a urine sample is a subjective process that can be difficult to teach and to perform. Done properly, the identification of these elements may be the difference between an accurate diagnosis and a faulty one. A limiting factor in the… [1]: #corresp-1

PIV Web Visualization System toward PIV visualization grid

To visualize the simulation and experimental data, the fluid dynamics scientists and engineers have to possess a powerful visualization engine with CG package in their laboratories. We developed a Problem Solving Environment (PSE) system called "PIV Web Visualization System" to solve this obstacle, using computational grid common service tool "Globus", programming language Java and Java Servlet. This system has "Grid Portal" which enables us to view the visualization resources on the networks as a unified whole and can be accessed around the world through the Internet, and it combines PIV engine "KMU-PIV" on visualization server in Korea Maritime University with a nonlinear video editing system on Kanazawa University as a real application.

画像処理 映像文法に基づいた映像編集支援システムのための使用可能なショット区間の自動抽出

Video editing is used to produce a final version with a specific duration by finding and selecting appropriate shots from the raw video material and connecting them. Video editing process is generally conducted according to the special rules called “video grammar” in order to produce excellent and intelligible videos for broadcasting. However, this editing consumes a lot of human editor's working time. To solve this problem, an efficient and new video editing technique or system is required. The goal of this study is to develop an intelligent support system for video editing based on video grammar. This paper proposes a method of using camerawork density, camerawork instability, and cut point parameters to automatically segment the raw video materials into useful sections and useless sections. The method is based on video grammar as a part of the video editing support system.