Editing Operations Research Articles

Audio and video editing system design based on OpenCV

With the rapid development of the Internet, a new carrier for people to perceive the world and communicate with each other - audio and video - is gradually being favoured by the public. The development of multimedia technology and artificial intelligence technology has provided a milestone for the maturity of audio and video technology. In particular, short video platforms have slowly become a new network position for various media promotions. Especially at the moment of the epidemic, the channel of understanding the world through audio and video is increasingly valued. The public has put forward higher demands on the content and presentation of audio and video. Therefore, it is particularly important to produce quality audio-video that meets the requirements of the times, which cannot be achieved without a feasible audio-video editing system. In addition, after previous research and practice, the application of artificial intelligence technology in the field of imaging has also become mature, including some applications in the direction of entertainment. Applying AI technology to the video editing process can improve the efficiency of video editing, increase the interest of video content, and allow video creators to focus on content creation without spending too much time and energy on video editing operations, thus creating better quality videos. This design is the main technology of OpenCV and front-end technology stack, such as JavaScript, React and Electron, to implement basic video editing, video filters, in addition to the development of a friendly interactive interface. The implementation of basic video editing module and video filter module are both based on OpenCV implementation. In this design, the basic video editing implements pan, zoom and rotate operations on the video, and the video filter module is implemented by changing the RGB channel values of the image. The operations on the video can be broken down into operations on each frame of the video, and OpenCV provides a way to implement these operations. The paper concludes with a summary of the shortcomings and flaws in the design, and an outlook on the next steps and future directions. This design is the main technology of OpenCV and front-end technology stack, such as JavaScript, React and Electron, to implement basic video editing, video filters, in addition to the development of a friendly interactive interface. The implementation of basic video editing module and video filter module are both based on OpenCV implementation. In this design, the basic video editing implements pan, zoom and rotate operations on the video, and the video filter module is implemented by changing the RGB channel values of the image. The operations on the video can be broken down into operations on each frame of the video, and OpenCV provides a way to implement these operations. The paper concludes with a summary of the shortcomings and flaws in the design, and an outlook on the next steps and future directions.

Visual notations for viewing RDF constraints with UnSHACLed

The quality of knowledge graphs can be assessed by a validation against specified constraints, typically use-case specific and modeled by human users in a manual fashion. Visualizations can improve the modeling process as they are specifically designed for human information processing, possibly leading to more accurate constraints, and in turn higher quality knowledge graphs. However, it is currently unknown how such visualizations support users when viewing RDF constraints as no scientific evidence for the visualizations’ effectiveness is provided. Furthermore, some of the existing tools are likely suboptimal, as they lack support for edit operations or common constraints types. To establish a baseline, we have defined visual notations to represent RDF constraints and implemented them in UnSHACLed, a tool that is independent of a concrete RDF constraint language. In this paper, we (i) present two visual notations that support all SHACL core constraints, built upon the commonly used visualizations VOWL and UML, (ii) analyze both notations based on cognitive effective design principles, (iii) perform a comparative user study between both visual notations, and (iv) present our open source tool UnSHACLed incorporating our efforts. Users were presented RDF constraints in both visual notations and had to answer questions based on visualization task taxonomies. Although no statistical significant difference in mean error rates was observed, all study participants preferred ShapeVOWL in a self assessment to answer RDF constraint-related questions. Furthermore, ShapeVOWL adheres to more cognitive effective design principles according to our performed comparison. Study participants argued that the increased visual features of ShapeVOWL made it easier to spot constraints, but a list of constraints – as in ShapeUML – is easier to read. However, also that more deviations from the strict UML specification and introduction of more visual features can improve ShapeUML. From these findings we conclude that ShapeVOWL has a higher potential to represent RDF constraints more effective compared to ShapeUML. But also that the clear and efficient text encoding of ShapeUML can be improved with visual features. A one-size-fits-all approach to RDF constraint visualization and editing will be insufficient. Therefore, to support different audiences and use cases, user interfaces of RDF constraint editors need to support different visual notations.

Classification and Recognition of Bilingual Text Using Graph Edit Distance Based Degree of Similarity

Objectives: Graph Edit distance-based classification and recognition method is introduced in this study for bilingual characters. Specifically, this method aims to classify characters first and then recognize them in the 2nd level. Methods: This study combines both exact graph matching and inexact graph matching techniques to achieve better Recognition. The exact graph matching technique classifies characters by considering the number of vertices and edges as features to classify. Inexact graph matching uses an algorithmic model to measure the degree of similarity in the length of edit operations. Findings: The proposed model can compute, if not optimal, sequences all the time, at least near-optimal edit sequence. The proposed model is based on dynamic programming. Edit distance is estimated by comparing preorder sequences of the respective binary trees. Finally, this model uses the loss function to recognize text accurately. To bring out the practicality of the proposed model, we present a case study on the classification of English alphabets and Kannada conjunct consonants. Novelty: (i) A hybrid approach for converting a large class classification problem into a small classification problem that takes advantage of both exact and inexact graph matching techniques. (ii) An algorithmic model is designed to represent each character as a unique string. (iii) Uses dynamic programming in the right places to get the optimal tree, and (iv) to convert the graph matching problem to a string matching problem reduces the time complexity of the problem. Keywords: Bilingual text; Character Recognition; Degree of Similarity; Graph Edit Distance; Graph Matching

DeepFaceVideoEditing

Sketches, which are simple and concise, have been used in recent deep image synthesis methods to allow intuitive generation and editing of facial images. However, it is nontrivial to extend such methods to video editing due to various challenges, ranging from appropriate manipulation propagation and fusion of multiple editing operations to ensure temporal coherence and visual quality. To address these issues, we propose a novel sketch-based facial video editing framework, in which we represent editing manipulations in latent space and propose specific propagation and fusion modules to generate high-quality video editing results based on StyleGAN3. Specifically, we first design an optimization approach to represent sketch editing manipulations by editing vectors, which are propagated to the whole video sequence using a proper strategy to cope with different editing needs. Specifically, input editing operations are classified into two categories: temporally consistent editing and temporally variant editing. The former (e.g., change of face shape) is applied to the whole video sequence directly, while the latter (e.g., change of facial expression or dynamics) is propagated with the guidance of expression or only affects adjacent frames in a given time window. Since users often perform different editing operations in multiple frames, we further present a region-aware fusion approach to fuse diverse editing effects. Our method supports video editing on facial structure and expression movement by sketch, which cannot be achieved by previous works. Both qualitative and quantitative evaluations show the superior editing ability of our system to existing and alternative solutions.

Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation

The task of Grammatical Error Correction (GEC) has received remarkable attention with wide applications in Natural Language Processing (NLP) in recent years. While one of the key principles of GEC is to keep the correct parts unchanged and avoid over-correction, previous sequence-to-sequence (seq2seq) models generate results from scratch, which are not guaranteed to follow the original sentence structure and may suffer from the over-correction problem. In the meantime, the recently proposed sequence tagging models can overcome the over-correction problem by only generating edit operations, but are conditioned on human designed language-specific tagging labels. In this paper, we combine the pros and alleviate the cons of both models by proposing a novel Sequence-to-Action (S2A) module. The S2A module jointly takes the source and target sentences as input, and is able to automatically generate a token-level action sequence before predicting each token, where each action is generated from three choices named SKIP, COPY and GENerate. Then the actions are fused with the basic seq2seq framework to provide final predictions. We conduct experiments on the benchmark datasets of both English and Chinese GEC tasks. Our model consistently outperforms the seq2seq baselines, while being able to significantly alleviate the over-correction problem as well as holding better generality and diversity in the generation results compared to the sequence tagging models.

TCB-spline-based Image Vectorization

Vector image representation methods that can faithfully reconstruct objects and color variations in a raster image are desired in many practical applications. This article presents triangular configuration B-spline (referred to as TCB-spline)-based vector graphics for raster image vectorization. Based on this new representation, an automatic raster image vectorization paradigm is proposed. The proposed framework first detects sharp curvilinear features in the image and constructs knot meshes based on the detected feature lines. It iteratively optimizes color and position of control points and updates the knot meshes. By using collinear knots at feature lines, both smooth and discontinuous color variations can be efficiently modeled by the same set of quadratic TCB-splines. A variational knot mesh generation method is designed to adaptively introduce knots and update their connectivity to satisfy the local reconstruction quality. Experiments and comparisons show that our framework outperforms the existing state-of-the-art methods in providing more faithful reconstruction results. In particular, our method is able to model undetected features and subtle or complicated color variations in-between features, which the previous methods cannot handle efficiently. Our vectorization representation also facilitates a variety of editing operations performed directly over vector images.

Golden Mutator Recommendation Based on Mutation Pattern Mining

Mutation testing is widely used in the research of evaluation and optimization of test set quality, and has been paid attention to the study of bug localization and fixing. But one inherent problem of mutation testing is huge computation cost. Selective mutation is an important method to reduce mutation testing cost. However, the existing selective mutation researches indicate that there is no universal selection strategy. This paper proposes a method which integrates with historical software data mining to recommend suitable mutator for program under test. The basic idea of the method is in the software version control system, any pair of nonfixed and post-fixed programs can be original program and mutant of each other. The edit performed during bug fixing contains mutators, and such mutator can guide buggy program to correct program, hence it is called Golden Mutator. The basic method is to compare the buggy files and the corresponding fixed files in the version control system, obtain historical faulty statements and their fix editing operations, thereby accumulating the bug-fix instance base, and then mining mutation patterns from it. Before mutating the target statement, we first find out faulty instances similar to the target statement, and use their fixing edits to match with the mutation pattern, so as to get the golden mutator applicable to the target statement. This paper uses Defects4j dataset in the test, verifies the accuracy of the proposed recommendation method, and further uses the method in bug localization. Compared to fixedly selected mutators, when applying the golden mutator recommended by using the proposed method, the average accuracy of bug localization is higher.

A Contrastive Learning Pre-Training Method for Motif Occupancy Identification.

Motif occupancy identification is a binary classification task predicting the binding of DNA motif instances to transcription factors, for which several sequence-based methods have been proposed. However, through direct training, these end-to-end methods are lack of biological interpretability within their sequence representations. In this work, we propose a contrastive learning method to pre-train interpretable and robust DNA encoding for motif occupancy identification. We construct two alternative models to pre-train DNA sequential encoder, respectively: a self-supervised model and a supervised model. We augment the original sequences for contrastive learning with edit operations defined in edit distance. Specifically, we propose a sequence similarity criterion based on the Needleman–Wunsch algorithm to discriminate positive and negative sample pairs in self-supervised learning. Finally, a DNN classifier is fine-tuned along with the pre-trained encoder to predict the results of motif occupancy identification. Both proposed contrastive learning models outperform the baseline end-to-end CNN model and SimCLR method, reaching AUC of 0.811 and 0.823, respectively. Compared with the baseline method, our models show better robustness for small samples. Specifically, the self-supervised model is proved to be practicable in transfer learning.

A Linear Time Solution to the Labeled Robinson-Foulds Distance Problem.

A large variety of pairwise measures of similarity or dissimilarity have been developed for comparing phylogenetic trees, for example, species trees or gene trees. Due to its intuitive definition in terms of tree clades and bipartitions and its computational efficiency, the Robinson–Foulds (RF) distance is the most widely used for trees with unweighted edges and labels restricted to leaves (representing the genetic elements being compared). However, in the case of gene trees, an important information revealing the nature of the homologous relation between gene pairs (orthologs, paralogs, and xenologs) is the type of event associated to each internal node of the tree, typically speciations or duplications, but other types of events may also be considered, such as horizontal gene transfers. This labeling of internal nodes is usually inferred from a gene tree/species tree reconciliation method. Here, we address the problem of comparing such event-labeled trees. The problem differs from the classical problem of comparing uniformly labeled trees (all labels belonging to the same alphabet) that may be done using the Tree Edit Distance (TED) mainly due to the fact that, in our case, two different alphabets are considered for the leaves and internal nodes of the tree, and leaves are not affected by edit operations. We propose an extension of the RF distance to event-labeled trees, based on edit operations comparable to those considered for TED: node insertion, node deletion, and label substitution. We show that this new Labeled Robinson–Foulds (LRF) distance can be computed in linear time, in addition of maintaining other desirable properties: being a metric, reducing to RF for trees with no labels on internal nodes and maintaining an intuitive interpretation. The algorithm for computing the LRF distance enables novel analyses on event-label trees such as reconciled gene trees. Here, we use it to study the impact of taxon sampling on labeled gene tree inference and conclude that denser taxon sampling yields trees with better topology but worse labeling. [Algorithms; combinatorics; gene trees; phylogenetics; Robinson–Foulds; tree distance.]

Towards More Direct Text Editing With Handwriting Interfaces

The conventional handwriting interface on pen computers uses two parts of the screen. The user must put a text cursor where they want to insert the text and then write on a separate pop-up window that recognizes ink. The separation of the two spaces may cause user fatigue and discomfort because it requires additional gaze and hand movement, which may cause a decrease in text editing efficiency and additional cognitive burden. As an alternative to alleviate these potential problems, we revisited and explored the concept of direct-writing interfaces as used in the past. In this study, we designed and implemented a direct- and indirect-writing text editor. Subsequently, we conducted an experiment to compare the usability of both text editors. The results showed that users’ gaze movement distance and frequency could be reduced, and constant time could be saved for each text editing operation using the direct-writing interface. In addition, participants preferred the direct-writing interface when simple and frequent text editing was required. However, the overall throughput of the text editing task was not affected. Lastly, we provided guidelines for further development of a direct-writing interface by combining empirical results, comments, and observations.

Input-driven pushdown automata for edit distance neighborhood

Edit distance ℓ-neighborhood of a language L is the set of all strings that can be obtained by at most ℓ elementary edit operations—deleting or inserting one symbol in the string—from some string in L. We shall show that if L is recognized by a nondeterministic input-driven pushdown automaton (pda) with |Γ| pushdown symbols and |Q| states, then its edit distance ℓ-neighborhood can be recognized by a nondeterministic input-driven pda with 2⋅|Γ|+1 pushdown symbols and O(|Q|⋅|Γ|ℓ) states. We shall also prove the corresponding lower bound: if |Q|>|Γ|≥2, there exists a language L that is recognized by a deterministic input-driven pda using |Γ| pushdown symbols and |Q| states but any nondeterministic input-driven pda for its edit distance ℓ-neighborhood must use at least (|Q|−1)⋅|Γ|ℓ states. This improves the known upper bound from literature and gives the bound that is asymptotically the best possible. If, beside deleting or inserting one symbol in the string, the measure of edit distance includes the operation of rewriting one symbol with another, the edit distance ℓ-neighborhood can be recognized with 2⋅|Γ|+1 pushdown symbols and O(|Q|⋅|Γ|2⋅ℓ) states.

An interactive sketch-based CAD interface realizing geometrical and topological editing across multiple objects based on fuzzy logic

With the widespread use of digital pen tablets and touch displays, sketch-based interactive computer-aided design (CAD) user interfaces have attracted considerable attention. In an existing interface called the Sketch Input Tracer (SKIT), however, the editing operation performed via an over-sketching stroke can target only a single geometric object at any given time. To overcome this limitation, in this study, we propose a new general-purpose sketch-based interface developed for use in two-dimensional CAD systems. In the proposed interface, a sketch-based editing operation can simultaneously modify the geometry and topology of multiple geometric objects via over-sketching. The interface was developed based on the fuzzy logic-based strategy of SKIT. With this interface, a user can draw in a creative manner; for example, they can begin with a rough sketch and progressively create a detailed design through repeated over-sketches. The results of experiments conducted demonstrate that experts can control the proposed interactive drawing interface and beginners can habituate to controlling it.

Semantics to the rescue of document‐based XML diff: A JATS case study

ABSTRACTThe writing of digital text documents has become a longer process that usually goes through revision rounds. Document comparison is important for the human reader interested in changes made by the authors. These documents contain structural data using text‐centric XML as one of their main storage systems. Current XML diff algorithms are able to represent differences with a limited number of edit operations: insert, delete, move and update. This approach does not fit the scope of digital text document comparison where the human reader needs to understand actual modifications made by the author. With JATS being a text‐centric XML vocabulary, we propose within this paper a new XML diff algorithm called jats‐diff, able to support bijection between higher‐level modifications made by the authors, such as structural changes and restyling, and the changes detected between XML documents. In addition, jats‐diff provides similarity information between different nodes in order to measure the impact of the text changes on the XML tree.

MSRD-CNN: Multi-Scale Residual Deep CNN for General-Purpose Image Manipulation Detection

The authenticity of digital images is a major concern in multimedia forensics due to the availability of advanced photo editing tools/devices. In the literature, several image forensic methods are available to detect specific image processing or editing operations. However, it remains a challenging task to design a universal forensic method that can detect multiple image editing operations. In this paper, a novel Multi-Scale Residual Deep CNN (MSRD-CNN) is designed to learn the image manipulation features adaptively for multiple image manipulation detection. Our network comprises of three stages: pre-processing, hierarchical high-level feature extraction, and classification. Firstly, a multi-scale residual module is employed in pre-processing stage to extract the prediction error or noise features adaptively. Afterwards, the obtained noise features are processed by feature extraction network having multiple Feature Extraction Blocks (FEBs) for the extraction of high-level image tampering features. Lastly, the resultant feature map is provided to the fully-connected dense layer for classification. The experiment results show that our model surpasses the existing schemes even under anti-forensic attacks, when evaluated on large-scale datasets by considering multiple image processing operations. The proposed network provides overall classification accuracies of 97.07% and 97.48% for BOSSBase and Dresden datasets, respectively.

VILS: A Verifiable Image Licensing System

Image licensing regulates the scope, type, and limitations of using an image through an agreement. However, it is challenging to verify whether an agreement has been fulfilled honestly. Existing techniques, such as watermarking and perceptual hashing, help check image originality and editing operations specified in the agreement, but fail to achieve editor designation. In this paper, we propose a verifiable image licensing system (VILS) which provides an effective solution to verify if a received image is used legally according to its licensing agreement. The core building block of our design is a new kind of cryptographic primitive, called accumulator with a designated entity. The new accumulator helps achieve not only editing restriction, but also editor designation in image authentication. Our VILS has the following two appealing features: (1) Authorization: Only an authorized licensee who edits an image with operations declared in a licensing agreement can produce valid images; (2) Efficiency: The verification of VILS is efficient and independent of the number of operations or image size. Compared with the most relevant schemes from the state-of-the-art, the new design enriches the functionality of image authentication but reduces the verification time by 40%.

SIMULTANEOUS GENOMIC KNOCKOUT OF THE PRO-APOPTOTIC BAX, BAK1 GENES AND OVEREXPRESSION OF THE ANTI-APOPTOTIC PROTEIN BCL-2 AND THE AUTOPHAGY INDUCER BECLIN-1 GREATLY INCREASE THE INTEGREL CELL DENSITY AND LIFESPAN IN CHO S SUSPENSION CELL CULTURE

Using two genome editing operations, an CHO A4 cell line was obtained with homozygous knockouts of the bak1, bax, dhfr, glul genes and insertion of the additional copies of bcl-2, beclin-1 genes. This cell line is capable of long- term growth without replacing the culture medium and has an increased integral cell density by 5 times in this mode of cultivation.

Textual Entailment for Modern Standard Arabic

This paper summarizes the Doctoral Thesis that examines various techniques to recognizing Arabic textual entailment, deciding whether one fragment of text entails another, where there is an exceptional level of structural and lexical ambiguities. As far as we know, the current work is the first study to apply this task for Arabic. For this purpose, we firstly describe a semi-automatic method for constructing a first Arabic textual entailment dataset. Then, we have investigated various system combination techniques for improving tagging and parsing depending on having accurate linguistic analyses. Finally, we have improved the standard tree edit distance (TED) algorithm. This extended version of TED, ETED, calculates the distance between two trees by applying operations on subtrees and single nodes. The current work also uses the artificial bee colony (ABC) algorithm to automatically guess the edit operations cost for both subtrees and single nodes and to decide thresholds. The current findings were encouraging for Arabic and English RTE-2 test sets.

Analysis of Edit Operations for Post-editing Systems

Post-editing has become an important part not only of translation research but also in the global translation industry. While computer-aided translation tools, such as translation memories, are considered to be part of a translator's work, lately, machine translation (MT) systems have also been accepted by human translators. However, many human translators are still adopting the changes brought by translation technologies to the translation industry. This paper introduces a novel approach for seeking suitable pairs of n-grams when recommending n-grams (corresponding n-grams between MT and post-edited MT) based on the type of text (manual or administrative) and MT system used for machine translation. A tool that recommends and speeds up the correction of MT was developed to help the post-editors with their work. It is based on the analysis of words with the same lemmas and analysis of n-gram recommendations. These recommendations are extracted from sequence patterns of the mismatched words (MisMatch) between MT output and post-edited MT output. The paper aims to show the usage of morphological analysis for recommending the post-edit operations. It describes the usage of mismatched words in the n-gram recommendations for the post-edited MT output. The contribution consists of the methodology for seeking suitable pairs of words, n-grams and additionally the importance of taking into account metadata (the type of the text and/or style and MT system) when recommending post-edited operations.

Detecting Non-Aligned Double JPEG Compression Based on Amplitude-Angle Feature

Due to the popularity of JPEG format images in recent years, JPEG images will inevitably involve image editing operation. Thus, some tramped images will leave tracks of Non-aligned double JPEG ( NA-DJPEG ) compression. By detecting the presence of NA-DJPEG compression, one can verify whether a given JPEG image has been tampered with. However, only few methods can identify NA-DJPEG compressed images in the case that the primary quality factor is greater than the secondary quality factor. To address this challenging task, this article proposes a novel feature extraction scheme based optimized pixel difference ( OPD ), which is a new measure for blocking artifacts. Firstly, three color channels (RGB) of a reconstructed image generated by decompressing a given JPEG color image are mapped into spherical coordinates to calculate amplitude and two angles (azimuth and zenith). Then, 16 histograms of OPD along the horizontal and vertical directions are calculated in the amplitude and two angles, respectively. Finally, a set of features formed by arranging the bin values of these histograms is used for binary classification. Experiments demonstrate the effectiveness of the proposed method, and the results show that it significantly outperforms the existing typical methods in the mentioned task.

Efficient approximate approach for graph edit distance problem

• Near optimal results, with a less than 5% deviation from optimal results of a DFS algorithm. • Fast execution time. • Scalable since it can deal with large data-sets in small execution time. • report tight upper-bounds for large data-sets. Graph Edit Distance (GED) problem is a well-known tool used to measure the similarity/dissimilarity between two graphs. It searches for the best set of edit operations (in terms of cost) that transforms one graph into another. Due to the NP-hardness nature of the problem, the search space increases exponentially making exact approaches impossible to use for large graphs. In this context, there is a huge need for approaches that give near-optimal results in reasonable time. In this paper, we propose a tree-based approximate approach for dealing with GED problem. It operates on a search tree that models all possible solutions of the problem. Since exploring the whole tree is impractical; this approach keeps only the best k nodes at each level of the tree for further exploration. This reduces enormously the execution time without scarifying the solution quality. Experiments using small and medium size data-sets show the low deviation of our results as compared to the optimal results of a Depth First Search algorithm. Moreover, our approach show a strong scalability potential by dealing with large data-sets in low execution time.