Editing Operations Research Articles

Voice and Touch Based Error-tolerant Multimodal Text Editing and Correction for Smartphones.

Editing operations such as cut, copy, paste, and correcting errors in typed text are often tedious and challenging to perform on smartphones. In this paper, we present VT, a voice and touch-based multi-modal text editing and correction method for smartphones. To edit text with VT, the user glides over a text fragment with a finger and dictates a command, such as “bold” to change the format of the fragment, or the user can tap inside a text area and speak a command such as “highlight this paragraph” to edit the text. For text correcting, the user taps approximately at the area of erroneous text fragment and dictates the new content for substitution or insertion. VT combines touch and voice inputs with language context such as language model and phrase similarity to infer a user’s editing intention, which can handle ambiguities and noisy input signals. It is a great advantage over the existing error correction methods (e.g., iOS’s Voice Control) which require precise cursor control or text selection. Our evaluation shows that VT significantly improves the efficiency of text editing and text correcting on smartphones over the touch-only method and the iOS’s Voice Control method. Our user studies showed that VT reduced the text editing time by 30.80%, and text correcting time by 29.97% over the touch-only method. VT reduced the text editing time by 30.81%, and text correcting time by 47.96% over the iOS’s Voice Control method.

TaGSim

Computing similarity between graphs is a fundamental and critical problem in graph-based applications, and one of the most commonly used graph similarity measures is graph edit distance (GED), defined as the minimum number of graph edit operations that transform one graph to another. Existing GED solutions suffer from severe performance issues due in particular to the NP-hardness of exact GED computation. Recently, deep learning has shown early promise for GED approximation with high accuracy and low computational cost. However, existing methods treat GED as a global, coarse-grained graph similarity value, while neglecting the type-specific transformative impacts incurred by different types of graph edit operations, including node insertion/deletion, node relabeling, edge insertion/deletion, and edge relabeling. In this paper, we propose a type-aware graph similarity learning and computation framework, TaGSim (T ype -a ware G raph Sim ilarity), that estimates GED in a fine-grained approach w.r.t. different graph edit types. Specifically, for each type of graph edit operations, TaGSim models its unique transformative impacts upon graphs, and encodes them into high-quality, type-aware graph embeddings, which are further fed into type-aware neural networks for accurate GED estimation. Extensive experiments on five real-world datasets demonstrate the effectiveness and efficiency of TaGSim, which significantly outperforms state-of-the-art GED solutions.

Redefining the Graph Edit Distance

Graph edit distance has been used since 1983 to compare objects in machine learning when these objects are represented by attributed graphs instead of vectors. In these cases, the graph edit distance is usually applied to deduce a distance between attributed graphs. This distance is defined as the minimum amount of edit operations (deletion, insertion and substitution of nodes and edges) needed to transform a graph into another. Since now, it has been stated that the distance properties have to be applied [(1) non-negativity (2) symmetry (3) identity and (4) triangle inequality] to the involved edit operations in the process of computing the graph edit distance to make the graph edit distance a metric. In this paper, we show that there is no need to impose the triangle inequality in each edit operation. This is an important finding since in pattern recognition applications, the classification ratio usually maximizes in the edit operation combinations (deletion, insertion and substitution of nodes and edges) that the triangle inequality is not fulfilled.

Image editing with varying intensities of processing

Current image editing algorithms prevailingly involve specific processing intensities. After the network training phase, the definite mapping is carried on testing images, which leads to under- or over-editing in certain cases. However, for many real problems, having access to diverse intensities of processing of the output is preferable. The hypothesis is that the part of parameters that correspond to the intensity of processing of the network are implicitly embedded in the whole parameter set and can be explicitly extracted and utilized, even with limited supervision. In this paper, we introduce the concept of Intensity of Processing, termed as IP, and propose image editing with Varying Intensities of Processing, termed as VIP. Given the input and corresponding fully-edited output, we would like to not only figure out the particular mapping from input to output, but also be able to obtain multiple continuous intermediate states with in-between intensities of processing. Refer to the network that deals with the original image editing operation as Base Network. By adding an extra module, which we call Condition Encoder, to condition and regulate the Base Network, we make the whole image editing process controllable. Moreover, we develop a triplet loss and construct the sequentiality of generated intermediate states with limited supervision. A significant advantage of our method is that it does not require groundtruth images of the middle states during training, which substantially relaxes the range of applicable problems. Extensive experiments demonstrate the effectiveness and potentiality of the proposed framework; discussion and future work are also provided.

Knit sketching

We present a novel workflow to design and program knitted garments for industrial whole-garment knitting machines. Inspired by traditional garment making based on cutting and sewing, we propose a sketch representation with additional annotations necessary to model the knitting process. Our system bypasses complex editing operations in 3D space, which allows us to achieve interactive editing of both the garment shape and its underlying time process. We provide control of the local knitting direction, the location of important course interfaces, as well as the placement of stitch irregularities that form seams in the final garment. After solving for the constrained knitting time process, the garment sketches are automatically segmented into a minimal set of simple regions that can be knitted using simple knitting procedures. Finally, our system optimizes a stitch graph hierarchically while providing control over the tradeoff between accuracy and simplicity. We showcase different garments created with our web interface.

Reactive Auto-Completion of Modeling Activities

Assisting and automating software engineering tasks is a state-of-the-art way to support stakeholders of development projects. A common assistance function of IDEs is the auto-completion of source code. Assistance functions, such as auto-completion, are almost entirely missing in modeling tools though auto-completion in general gains continuously more importance in software development. We analyze a user’s performed editing operations in order to anticipate modeling activities and to recommend appropriate auto-completions for them. Editing operations are captured as events and modeling activities are defined as complex event patterns, facilitating the matching by complex-event-processing. The approach provides adapted auto-completions reactively upon each editing operation of the user. We implemented the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RapMOD</i> prototype as add-in for the modeling tool Sparx Enterprise Architect™ . A controlled user experiment with 37 participants performing modeling tasks demonstrated the approach’s potential to reduce modeling effort significantly. Users having auto-completions available for a modeling scenario performed the task 27 percent faster, needed to perform 56 percent less actions, and perceived the task 29 percent less difficult.

Development of a Linguistic Approach to the Problem of the Computer Electrocardiogram’s Classifications

Introduction. The linguistic approach, based on the transition from the observed cyclic signal to a sequence of symbols (codeword), which characterize the dynamics of indicators from cycle to cycle, makes it possible to use the procedures of mathematical linguistics to increase the reliability of decisions. The purpose of the article is to expand the diagnostic capabilities of the linguistic approach to the analysis and interpretation of electrocardiograms (ECG). Methods. Each ECG cycle is encoded with one of four symbols characterizing changes in two indicators: traditional (cycle duration) and original (symmetry of the repolarization area). Results. Based on the processing of real clinical data of verified patients and healthy volunteers, standards of patients with chronic coronary artery disease (СAD) and healthy patients. The standards are developed using computational procedures of mathematical linguistics – the Levenshtein distance, which is the minimum number of editing operations (insertion, deletion and replacement of a character), ensuring the transition from one word to another and the frequency of occurrence of substrings in the analyzed word. On the basis of these procedures, decision rules that have been developed allow making diagnostic decisions based on the Levenshtein distance to the standards and the frequency of occurrence of one-, two- and three-symbol patterns in code words. It was found that the combination of these two methods expands the diagnostic capabilities of the linguistic approach to the analysis and interpretation of the ECG. Conclusions. It has been shown that using of the developed decision rules makes it possible to increase the sensitivity and specificity of diagnostics even in cases when the ECG does not show traditional electrocardiological signs of myocardial ischemia.

Camera Obscura: Exploiting in-camera processing for image counter forensics

For the last two decades Image Forensics has been providing an arsenal of forensic tools to detect tampered images. At the same time, anti-forensics technologies kept evolving to mislead forensic detectors. Such attacks are generally designed to affect a single forensic trace (e.g. JPEG compression, sensor patter noise, histogram statistics) without considering that the image alteration can negatively affect other traces, thus making harder to mimic multiple statistics of natural images. Nevertheless, performing of several attacks can ruin the image quality. In this paper we introduce Camera Obscura, a method that, given a tampered picture, generates a pristine image with native camera metadata, JPEG structure, Quantization Tables, Preview, Thumbnails, their corresponding Quantization Tables, single compression statistics and any in-camera (even proprietary) processing. The attack is performed by exploiting and extending Magic Lantern, an alternative camera firmware, to perform in-camera image buffer substitution. As opposed to available image anti-forensic techniques, Camera Obscura accurately reproduces multiple image statistics in one attack with a limited effect on global image quality. With these characteristics, it can be applied for many different applications such as injecting reference image statistics into a computer generated image, exchange the supposed source of a given image or hide any signal-based editing operation.

Learning Multimodal Affinities for Textual Editing in Images

Nowadays, as cameras are rapidly adopted in our daily routine, images of documents are becoming both abundant and prevalent. Unlike natural images that capture physical objects, document-images contain a significant amount of text with critical semantics and complicated layouts. In this work, we devise a generic unsupervised technique to learn multimodal affinities between textual entities in a document-image, considering their visual style, the content of their underlying text, and their geometric context within the image. We then use these learned affinities to automatically cluster the textual entities in the image into different semantic groups. The core of our approach is a deep optimization scheme dedicated for an image provided by the user that detects and leverages reliable pairwise connections in the multimodal representation of the textual elements to properly learn the affinities. We show that our technique can operate on highly varying images spanning a wide range of documents and demonstrate its applicability for various editing operations manipulating the content, appearance, and geometry of the image.

6DIVE: 6 Degrees-of-Freedom Immersive Video Editor

Editing 6DoF videos using standard video editing tools is challenging, especially for non-expert users. There is a large gap between the 2D interface used for traditional video editing and the immersive VR environment used for replay. In this paper, we present 6DIVE, a 6 degrees-of-freedom (DoF) immersive video editor. 6DIVE allows users to edit these 6DoF videos directly in an immersive VR environment. In this work, we explored options for a timeline representation as well as UI placement suitable for immersive video editing. We used our prototypical implementation of an immersive video editor to conduct a user study to analyze the feasibility and usability of immersive 6DoF editing. We compared 6DIVE to a desktop-based implementation of a VR video editor. Our initial results suggest that 6DIVE allows even non-expert users to perform basic editing operations on videos in VR. While we did not find any statistically significant differences for the workload between the VR and the desktop interface, we found a statistically significant difference in user preference, with a preference for the VR interface. We also found higher ratings for the user experience metrics in VR captured by the user experience questionnaire.

Codes Over Trees

In graph theory, a tree is one of the more popular families of graphs with a wide range of applications in computer science as well as many other related fields. While there are several distance measures over the set of all trees, we consider here the one which defines the so-called tree distance, defined by the minimum number of edit operations, of removing and adding edges, in order to change one tree into another. From a coding theoretic perspective, codes over the tree distance are used for the correction of edge erasures and errors. However, studying this distance measure is important for many other applications that use trees and properties on their locality and the number of neighbor trees. Under this paradigm, the largest size of code over trees with a prescribed minimum tree distance is investigated. Upper bounds on these codes as well as code constructions are presented. A significant part of our study is dedicated to the problem of calculating the size of the ball of trees of a given radius. These balls are not regular and thus we show that while the star tree has asymptotically the smallest size of the ball, the maximum is achieved for the path tree.

A Model with Iterative Trials for Correcting Logic Errors in Source Code

It is difficult for students and teachers to detect and correct logic errors in source code. Compilers and integrated development environments (IDEs) have the ability to detect and correct syntax errors but it is also difficult for them to detect and correct logic errors. Although many machine learning approaches have been proposed that can show correction candidates for logic errors, they do not provide guidance concerning how the user should fix them. In this paper, we propose a model for correcting logic errors in a given source code. The proposed model realizes debugging of multiple logic errors in the source code by iterative trials of identifying the errors, correcting the errors, and testing the source code. In this model, in the first stage, a list of correction candidates is provided by a deep learning model, and then the list is given to an editing operation predictor that predicts the editing operation for the correction candidate. To learn the internal parameters of the proposed model, we use a set of solution codes created to solve the corresponding programming tasks in a real e-learning system. To verify the usefulness of the proposed model, we apply it to 32 programming tasks. Experimental results show that the correction accuracy is, on average, 58.64% higher than that of the conventional model without iterative trials.

Visual Comfort Aware-Reinforcement Learning for Depth Adjustment of Stereoscopic 3D Images

Depth adjustment aims to enhance the visual experience of stereoscopic 3D (S3D) images, which accompanied with improving visual comfort and depth perception. For a human expert, the depth adjustment procedure is a sequence of iterative decision making. The human expert iteratively adjusted the depth until he is satisfied with the both levels of visual comfort and the perceived depth. In this work, we present a novel deep reinforcement learning (DRL)-based approach for depth adjustment named VCA-RL (Visual Comfort Aware Reinforcement Learning) to explicitly model human sequential decision making in depth editing operations. We formulate the depth adjustment process as a Markov decision process where actions are defined as camera movement operations to control the distance between the left and right cameras. Our agent is trained based on the guidance of an objective visual comfort assessment metric to learn the optimal sequence of camera movement actions in terms of perceptual aspects in stereoscopic viewing. With extensive experiments and user studies, we show the effectiveness of our VCA-RL model on three different S3D databases.

On the complexity of approximately matching a string to a directed graph

The problem of matching a query string to a directed graph, whose vertices are labeled by strings, has application in different fields. In this paper we present results on the complexity of the approximate matching problem, where edit operations are symbol substitutions and are allowed only to the graph labels or both to the graph labels and the query string. We show that deciding if there exists a path in a graph that represents a query string with edit operations to vertex labels is an NP-complete problem and it is fixed-parameter tractable when parameterized by the length of the input string. We show that the variants of approximate string matching we consider are not fixed-parameter tractable, when the parameter is (1) the length of the query string and (2) the number of edit operations. Moreover, we provide inapproximability results for these variants of the of approximate string matching problem.

The Technology of Distortion-Free Quenching in a Magnetic Field of Thin-Walled Details of the Ring Form

During heat treatment of machine parts and tools, besides the usual task of ensuring a high complex of mechanical and operational properties, there is a problem of distortion of products in the process of heat treatment and the need for editing operations (which are carried out manually and require significant labor costs). The known methods do not solve the problem of removing distortion for thin-walled parts of the ring shape completely. This paper shows the technical possibility of using the energy of a constant magnetic field for the "internal" straightening of products during heat treatment in the temperature range of super-plasticity of transformation. The use of special equipment makes it possible to eliminate virtually the distortion of thin-walled parts of the ring shape and to improve their mechanical properties.

On-line learning the graph edit distance costs

Abstract This paper presents the first on-line learning method to automatically deduce the insertion, deletion and substitution edit costs of the graph edit distance. The learning method is based on embedding the substitution and deletion operations into a Euclidean space. The points in this space are classified into the ones that represent substitution edit operations and the ones that represent deletion edit operations. Thus, the learning strategy is based on deducing the hyper-plane in this space that best splits these two types of points. Any linear classifier can be used to deduce this hyper-plane, for instance LDA or SVM. The on-line method has the advantage that learning the edit costs and computing the graph edit distance with the new updated costs can be done simultaneously. Experimental validation shows that the matching accuracy is competitive with the off-line methods but without the need of the whole learning set.

Genetic programming hyperheuristic parameter configuration using fitness landscape analysis

Fitness landscape analysis is a tool that can help us gain insight into a problem, determine how hard it is to solve a problem using a given algorithm, choose an algorithm for solving a given problem, or choose good algorithm parameters for solving the problem. In this paper, fitness landscape analysis of hyperheuristics is used for clustering instances of three scheduling problems. After that, good parameters for tree-based genetic programming that can solve a given scheduling problem are calculated automatically for every cluster. Additionally, we introduce tree editing operators which help in the calculation of fitness landscape features in tree based genetic programming. A heuristic is proposed based on introduced operators, and it calculates the distance between any two trees. The results show that the proposed approach can obtain parameters that offer better performance compared to manual parameter selection.

On the Application of Computer Combined with Multimedia Technology in Post-production of Film and Television

Since entering the 21st century, with the rapid development of economy, the people’s material living standard has been improved, the level of spiritual civilization has been improved, and the demand for leisure and entertainment has become more and more diversified, thus the demand for film and television works has become higher and higher. The relationship between film and television works and computer multimedia works is becoming more and more close. Computer multimedia technology has even become its core technology and has been widely used in film and television works, especially in post-production. The application of computer combined with multimedia technology has brought a good development opportunity for film and television post-production, met the requirements of film and television works in the process of post-production, and improved the quality and presentation effect of film and television works. Thus, the production efficiency of film and television works has been improved. The computer combines the multimedia technology to realize the simplification and flexibility of the editing operation, which makes the editing time greatly shortened, and even if the error occurs in the editing process, it can be scientifically restored at any time. What role does computer multimedia technology play in film and television works and how to further use multimedia technology to make the film and television culture develop at a higher level is a topic that has attracted the attention of the industry. This paper briefly discusses the role and methods of computer multimedia technology in film and television post-production[1].

Blended graphical and textual modelling for UML profiles: A proof-of-concept implementation and experiment

Domain-specific modelling languages defined by extending or constraining the Unified Modelling Language (UML) through the profiling mechanism have historically relied on graphical notations to maximise human understanding and facilitate communication among stakeholders. Other notations, such as text-, form-, or table-based are, however, often preferred for specific modelling purposes, due to the nature of a specific domain or the available tooling, or for personal preference. Currently, the state of the art support for UML-based languages provides an almost completely detached, or even entirely mutually exclusive, use of graphical and textual modelling. This becomes inadequate when dealing with the development of modern systems carried out by heterogeneous stakeholders. Our intuition is that a modelling framework based on seamless blended multi-notations can disclose several benefits, among which: flexible separation of concerns, multi-view modelling based on multiple notations, convenient text-based editing operations (inside and outside the modelling environment), and eventually faster modelling activities.In this paper we report on: (i) a proof-of-concept implementation of a framework for UML and profiles modelling using blended textual and graphical notations, and (ii) an experiment on the framework, which eventually shows that blended multi-notation modelling performs better than standard single-notation modelling.

History-based Model Repair Recommendations

Models in Model-driven Engineering are primary development artifacts that are heavily edited in all stages of software development and that can become temporarily inconsistent during editing. In general, there are many alternatives to resolve an inconsistency, and which one is the most suitable depends on a variety of factors. As also proposed by recent approaches to model repair, it is reasonable to leave the actual choice and approval of a repair alternative to the discretion of the developer. Model repair tools can support developers by proposing a list of the most promising repairs. Such repair recommendations will be only accepted in practice if the generated proposals are plausible and understandable, and if the set as a whole is manageable. Current approaches, which mostly focus on exhaustive search strategies, exploring all possible model repairs without considering the intention of historic changes, fail in meeting these requirements. In this article, we present a new approach to generate repair proposals that aims at inconsistencies that have been introduced by past incomplete edit steps that can be located in the version history of a model. Such an incomplete edit step is either undone or it is extended to a full execution of a consistency-preserving edit operation. The history-based analysis of inconsistencies as well as the generation of repair recommendations are fully automated, and all interactive selection steps are supported by our repair tool called R E V ISION . We evaluate our approach using histories of real-world models obtained from popular open-source modeling projects hosted in the Eclipse Git repository, including the evolution of the entire UML meta-model. Our experimental results confirm our hypothesis that most of the inconsistencies, namely, 93.4, can be resolved by complementing incomplete edits. 92.6% of the generated repair proposals are relevant in the sense that their effect can be observed in the models’ histories. 94.9% of the relevant repair proposals are ranked at the topmost position.