Hand-drawn Diagrams Research Articles

Intelligent System of Constructing Vector Diagrams of Electrical Circuits

Phasor diagrams are a powerful tool for visualizing and understanding the distribution of current, voltage, and power in electrical systems. During Russia's war against Ukraine, our energy industry has become very vulnerable to enemy attacks, and therefore needs a quick and effective recovery. Energy specialists lack software tools for working with the power system, and in the period of development of artificial intelligence, creating such tools is not so difficult. For example, familiar specialists often have to build vector diagrams - tools for visualizing and understanding the distribution of current, voltage and power in electrical systems. Using a combination of frameworks for working with artificial intelligence and creating a graphic shell, you can achieve the desired result in a few months, and at the same time do a useful thing for our victory. The YOLO model (based on the PyTorch framework) and the QT framework are considered among the proposed tools for creating an intelligent vector diagram construction system. The role of artificial intelligence is to recognize electrical elements in circuits and their connections to each other. Creating a user interface is an equally important thing, which is already implemented in many places with the help of QT. As of today, there is no specialized software tool for solving the problems of manual construction of vector diagrams, but the proposed approaches are already used for its creation. The system uses the Yolov5 model to recognize electrical elements on the circuit. The model is trained on more than 150 images and is able to recognize hand-drawn diagrams. Recognition is run as a separate process from the main program written in C++. This part of the system processes the input data from Yolo, saves it in a convenient format, creates a user interface, and displays the result as a vector diagram.

DrawnNet: Offline Hand-Drawn Diagram Recognition Based on Keypoint Prediction of Aggregating Geometric Characteristics.

Offline hand-drawn diagram recognition is concerned with digitizing diagrams sketched on paper or whiteboard to enable further editing. Some existing models can identify the individual objects like arrows and symbols, but they become involved in the dilemma of being unable to understand a diagram’s structure. Such a shortage may be inconvenient to digitalization or reconstruction of a diagram from its hand-drawn version. Other methods can accomplish this goal, but they live on stroke temporary information and time-consuming post-processing, which somehow hinders the practicability of these methods. Recently, Convolutional Neural Networks (CNN) have been proved that they perform the state-of-the-art across many visual tasks. In this paper, we propose DrawnNet, a unified CNN-based keypoint-based detector, for recognizing individual symbols and understanding the structure of offline hand-drawn diagrams. DrawnNet is designed upon CornerNet with extensions of two novel keypoint pooling modules which serve to extract and aggregate geometric characteristics existing in polygonal contours such as rectangle, square, and diamond within hand-drawn diagrams, and an arrow orientation prediction branch which aims to predict which direction an arrow points to through predicting arrow keypoints. We conducted wide experiments on public diagram benchmarks to evaluate our proposed method. Results show that DrawnNet achieves , , and recognition rate improvements compared with the state-of-the-art methods across benchmarks of FC-A, FC-B, and FA, respectively, outperforming existing diagram recognition systems on each metric. Ablation study reveals that our proposed method can effectively enable hand-drawn diagram recognition.

Smartphone Handwritten Circuits Solver Using Augmented Reality and Capsule Deep Networks for Engineering Education

Resolving circuit diagrams is a regular part of learning for school and university students from engineering backgrounds. Simulating circuits is usually done manually by creating circuit diagrams on circuit tools, which is a time-consuming and tedious process. We propose an innovative method of simulating circuits from hand-drawn diagrams using smartphones through an image recognition system. This method allows students to use their smartphones to capture images instead of creating circuit diagrams before simulation. Our contribution lies in building a circuit recognition system using a deep learning capsule networks algorithm. The developed system receives an image captured by a smartphone that undergoes preprocessing, region proposal, classification, and node detection to get a Netlist and exports it to a circuit simulator program for simulation. We aim to improve engineering education using smartphones by (1) achieving higher accuracy using less training data with capsule networks and (2) developing a comprehensive system that captures hand-drawn circuit diagrams and produces circuit simulation results. We use 400 samples per class and report an accuracy of 96% for stratified 5-fold cross-validation. Through testing, we identify the optimum distance for taking circuit images to be 10 to 20 cm. Our proposed model can identify components of different scales and rotations.

Multi-Domain Recognition of Hand-Drawn Diagrams Using Hierarchical Parsing

This paper presents an approach for the recognition of multi-domain hand-drawn diagrams, which exploits Sketch Grammars (SkGs) to model the symbols’ shape and the abstract syntax of diagrammatic notations. The recognition systems automatically generated from SkGs process the input sketches according to the following phases: the user’ strokes are first segmented and interpreted as primitive shapes, then by exploiting the domain context they are clustered into symbols of the domain and, finally, an interpretation of whole diagram is given. The main contribution of this paper is an efficient model of parsing suitable for both interactive and non-interactive sketch-based interfaces, configurable to different domains, and able to exploit contextual information for improving recognition accuracy and solving interpretation ambiguities. The proposed approach was evaluated in the domain of UML class diagrams obtaining good results in terms of recognition accuracy and usability.

Single object detection to support requirements modeling using faster R-CNN

Requirements engineering (RE) is one of the most important phases of a software engineering project in which the foundation of a software product is laid, objectives and assumptions, functional and non-functional needs are analyzed and consolidated. Many modeling notations and tools are developed to model the information gathered in the RE process, one popular framework is the iStar 2.0. Despite the frameworks and notations that are introduced, many engineers still find that drawing the diagrams is easier done manually by hand. Problem arises when the corresponding diagram needs to be updated as requirements evolve. This research aims to kickstart the development of a modeling tool using Faster Region-based Convolutional Neural Network for single object detection and recognition of hand-drawn iStar 2.0 objects, Gleam grayscale, and Salt and Pepper noise to digitalize hand-drawn diagrams. The single object detection and recognition tool is evaluated and displays promising results of an overall accuracy and precision of 95%, 100% for recall, and 97.2% for the F-1 score.

An Empirical Investigation of the Effectiveness of Optical Recognition of Hand-Drawn Business Process Elements by Applying Machine Learning

Business process diagrams represent an essential part of business process management activities, including process analysis, process-related communication, and process automation. While the majority of these artifacts are produced with modeling tools, there are still cases where hand-drawn diagrams are created, especially in the initial phases of process discovery and process innovation activities. The transformation of a hand-drawn diagram to digital format often presents several challenges. Thus, our efforts are directed towards investigating the effectiveness of an automatic transformation of hand-drawn diagrams into digital artifacts utilizing optical character recognition. To this aim, we performed empirical research in which subjects were instructed to redraw standardized process elements, which were afterward used for the training and testing set of machine-learning-based character recognition application. The findings obtained in the analysis showed that TensorFlow based and trained solution is capable of identifying hand-drawn process diagrams elements on different levels of accuracy. These insights may be considered when specifying or adapting the visual vocabularies of notations assuring appropriate visual distances between depictions of individual elements.

SKETRACK: Stroke-Based Recognition of Online Hand-Drawn Sketches of Arrow-Connected Diagrams and Digital Logic Circuit Diagrams

Digitalization of handwritten documents has created a greater need for accurate online recognition of hand-drawn sketches. However, the online recognition of hand-drawn diagrams is an enduring challenge in human-computer interaction due to the complexity in extracting and recognizing the visual objects reliably from a continuous stroke stream. This paper focuses on the design and development of a new, efficient stroke-based online hand-drawn sketch recognition scheme named SKETRACK for hand-drawn arrow diagrams and digital logic circuit diagrams. The fundamental parts of this model are text separation, symbol segmentation, feature extraction, classification, and structural analysis. The proposed scheme utilizes the concepts of normalization and segmentation to isolate the text from the sketches. Then, the features are extracted to model different structural variations of the strokes that are categorized into the arrows/lines and the symbols for effective processing. The strokes are clustered using the spectral clustering algorithm based on p-distance and Euclidean distance to compute the similarity between the features and minimize the feature dimensionality by grouping similar features. Then, the symbol recognition is performed using modified support vector machine (MSVM) classifier in which a hybrid kernel function with a lion optimized tuning parameter of SVM is utilized. Structural analysis is performed with lion-based task optimization for recognizing the symbol candidates to form the final diagram representations. This proposed recognition model is suitable for simpler structures such as flowcharts, finite automata, and the logic circuit diagrams. Through the experiments, the performance of the proposed SKETRACK scheme is evaluated on three domains of databases and the results are compared with the state-of-the-art methods to validate its superior efficiency.

Computer vision for LCA foreground modelling\u2014an initial pipeline and proof of concept software, lcopt-cv

PurposeThe majority of LCA studies begin with the drawing of a process flow diagram, which then needs to be translated manually into an LCA model. This study presents an initial image processing pipeline, implemented in an open-source software package, called lcopt-cv, which can be used to identify the boxes and links in a photograph of a hand-drawn process flow diagram and automatically create an LCA foreground model.MethodsThe computer vision pipeline consists of a total of 15 steps, beginning with loading the image file and conversion to greyscale. The background is equalised, then the foreground of the image is extracted from the background using thresholding. The lines are then dilated and closed to account for drawing errors. Contours in the image are detected and simplified, and rectangles (contours with four corners) are identified from the simplified contours as ‘boxes’. Links between these boxes are identified using a flood-filling technique. Heuristic processing, based on knowledge of common practice in drawing of process flow diagrams, is then performed to more accurately identify the typology of the identified boxes and the direction of the links between them.Results and discussionThe performance of the image processing pipeline was tested on four flow diagrams of increasing difficulty: one simple computer drawn diagram and three photographs of hand-drawn diagrams (a simple diagram, a complex diagram and a diagram with merged lines). A set of default values for the variables which define the pipeline was developed through trial and error. For the two simple flow charts, all boxes and links were identified using the default settings. The complex diagram required minor tweaks to the default values to detect all boxes and links. An ‘unstacking’ heuristic allowed the diagram with merged lines to be correctly processed. After some manual reclassification of link directions and process types, the diagrams were turned into LCA models and exported to open-source LCA software packages (lcopt and Brightway) to be verified and analysed.ConclusionsThis study demonstrates that it is possible to generate a fully functional LCA model from a picture of a flow chart. This has potentially important implications not only for LCA practitioners as a whole, but in particular for the teaching of LCA. Skipping the steep learning curve required by most LCA software packages allows teachers to focus on important LCA concepts, while participants maintain the benefits of experiential learning by doing a ‘real’ LCA.

A novel shape descriptor based on salient keypoints detection for binary image matching and retrieval

We introduce a shape descriptor that extracts keypoints from binary images and automatically detects the salient ones among them. The proposed descriptor operates as follows: First, the contours of the image are detected and an image transformation is used to generate background information. Next, pixels of the transformed image that have specific characteristics in their local areas are used to extract keypoints. Afterwards, the most salient keypoints are automatically detected by filtering out redundant and sensitive ones. Finally, a feature vector is calculated for each keypoint by using the distribution of contour points in its local area. The proposed descriptor is evaluated using public datasets of silhouette images, handwritten math expressions, hand-drawn diagram sketches, and noisy scanned logos. Experimental results show that the proposed descriptor compares strongly against state of the art methods, and that it is reliable when applied on challenging images such as fluctuated handwriting and noisy scanned images. Furthermore, we integrate our descriptor in a content-based document image retrieval system using sketch queries as a step for query and candidate occurrence matching, and we show that it leads to a significant boost in retrieval performances.

Commentary: In defense of Crazy Ideas

Commentary: In defense of Crazy Ideas

Retrieving geometric information from images: the case of hand-drawn diagrams

This paper addresses the problem of retrieving meaningful geometric information implied in image data. We outline a general algorithmic scheme to solve the problem in any geometric domain. The scheme, which depends on the domain, may lead to concrete algorithms when the domain is properly and formally specified. Taking plane Euclidean geometry $${\mathbb {E}}$$E as an example of the domain, we show how to formally specify $${\mathbb {E}}$$E and how to concretize the scheme to yield algorithms for the retrieval of meaningful geometric information in $${\mathbb {E}}$$E. For images of hand-drawn diagrams in $${\mathbb {E}}$$E, we present concrete algorithms to retrieve typical geometric objects and geometric relations, as well as their labels, and demonstrate the feasibility of our algorithms with experiments. An example is presented to illustrate how nontrivial geometric theorems can be generated from retrieved geometric objects and relations and thus how implied geometric knowledge may be discovered automatically from images.

Measuring the operational impact of digitized hospital records: a mixed methods study.

BackgroundDigitized (scanned) medical records have been seen as a means for hospitals to reduce costs and improve access to records. However, clinical usability of digitized records can potentially have negative effects on productivity.MethodsData were collected during follow-up outpatient consultations in two NHS hospitals by non-clinical observers using a work sampling approach in which pre-defined categories of clinician time usage were specified. Quantitative data was analysed using two-way ANOVA models and the Mann-Whitney U test. A focus group was held with clinicians to qualitatively explore their experiences using digitized medical records. The quantitative and qualitative results were synthesized.ResultsFour hundred six consultations were observed. Using paper records, there was a significant difference in consultation times between hospitals (p = 0.016) and a significant difference in consultation times between specialties within hospitals (p = 0.003). Using digitized records there was a significant difference in consultation times between specialties within a hospital (p = 0.001). Excluding outliers, there was no significant difference between consultation times using digitized records compared with consultations using paper records in the same hospital, either at site (p > =0.285) or specialty level (p > =0.122). With digitized records at site A, two out of three specialties showed a significant increase in time spent searching computer records (p < =0.010, Δ = 01:50–07:10) and one specialty had a corresponding reduction in time spent searching paper records (p = 0.015, Δ = −00:28). Site B showed a notable increase in direct patient care (p < 0.001, Δ = 04:20–06:00) and time spent searching computer records (p < =0.043, Δ = 00:10–01:40) and reductions in the other time categories.The focus group confirmed that the most recent clinical letter was a vital document in the patient record, often containing most of the required information. Concerns were expressed about consistency of scanning practice, causing uncertainty about what could be relied upon to exist in the digitized record. Benefits of digitized records included: access from multiple locations, better prepared ward rounds, improved inpatient handovers and an improved timeline of patient events. Limitations of digitized records included: increased complexity of creating a patient summary, display of specialised content such as hand-drawn diagrams, inability to quickly flick through the pages to find relevant content.ConclusionsDigitized medical records can be implemented without detrimental operational impact. Inherent differences between specialties can outweigh the differences between paper and digitized records. Clear and consistent operational processes are vital for the reliability and usability of digitized medical records. Divergent views about usability (such as whether patient summary information is better or worse) may reflect familiarity with features of the digitized record.

Hand-drawn Digital Logic Circuit Component Recognition using SVM

Hand-drawn circuit diagrams are widely used in engineering fields, especially for the early design phases because sketch is a convenient tool to understand rough ideas of system development. Using such hand-drawn diagrams designers can focus more on the critical issues rather than on the intricate details. Electronic system design is one of the application areas in this context. To speed up the development process of such systems, hand-drawn circuits design need be transformed into the computers. The problem is that although it seems very easy for humans to recognize sketches, it is really a great challenge for the computers. This paper proposed a method to recognize components in hand drawn digital logic circuit diagram. This system used region based segmentation method to segment circuit sketch, and classified each component using Support Vector Machine that uses Fourier descriptor as the feature vector. An average of 83% circuit recognition accuracy is achieved.

Online recognition of sketched arrow-connected diagrams

We introduce a new, online, stroke-based recognition system for hand-drawn diagrams which belong to a group of documents with an explicit structure obvious to humans but only loosely defined from the machine point of view. We propose a model for recognition by selection of symbol candidates, based on evaluation of relations between candidates using a set of predicates. It is suitable for simpler structures where the relations are explicitly given by symbols, arrows in the case of diagrams. Knowledge of a specific diagram domain is used--the two domains are flowcharts and finite automata. Although the individual pipeline steps are tailored for these, the system can readily be adapted for other domains. Our entire diagram recognition pipeline is outlined. Its core parts are text/non-text separation, symbol segmentation, their classification and structural analysis. Individual parts have been published by the authors previously and so are described briefly and referenced. Thorough evaluation on benchmark databases shows the accuracy of the system reaches the state of the art and is ready for practical use. The paper brings several contributions: (a) the entire system and its state-of-the-art performance; (b) the methodology exploring document structure when it is loosely defined; (c) the thorough experimental evaluation; (d) the new annotated database for online sketched flowcharts and finite automata diagrams.

Transgressive Truth Telling

Transgressive Truth Telling

The globins: classic structures and new insights

The oxygen-binding proteins myoglobin and haemoglobin have a special place in the history of structural biology, and, by implication, of molecular biology and molecular medicine. When the first structures of these proteins were published, in 19581 and 19602, they were the only protein structures known. And even these structures bore very little resemblance to what we now think of as ‘protein structures’. The structures were far too crude – low resolution, as structural biologists say – to reveal the positions of any of the atoms. The beautiful and meticulous hand-drawn diagrams of the myoglobin monomer and the haemoglobin tetramer in these proteins show fat sausage-like shapes representing the eight -helices of the now well-known globin fold, with the haem group nestling in between. It took the genius of the principal scientists involved in that work, Max Perutz and John Kendrew, to imagine the potential of these rather unpromising looking structures. Perutz and Kendrew were jointly awarded the Nobel Prize for Chemistry in 1962, which was a particularly good year for structural biology; Watson, Crick and Wilkins were awarded the Physiology or Medicine Nobel for the DNA structure then too.

Grouping Strokes into Shapes in Hand-Drawn Diagrams

Objects in freely-drawn sketches often have no spatial or temporal separation, making object recognition difficult. We present a two-step stroke-grouping algorithm that first classifies individual strokes according to the type of object to which they belong, then groups strokes with like classifications into clusters representing individual objects. The first step facilitates clustering by naturally separating the strokes, and both steps fluidly integrate spatial and temporal information. Our approach to grouping is unique in its formulation as an efficient classification task rather than, for example, an expensive search task. Our single-stroke classifier performs at least as well as existing single-stroke classifiers on text vs. nontext classification, and we present the first three-way single-stroke classification results. Our stroke grouping results are the first reported of their kind; our grouping algorithm correctly groups between 86% and 91% of the ink in diagrams from two domains, with between 69% and 79% of shapes being perfectly clustered.

The Texas Rangers: Wearing the Cinco Peso, 1821-1900 (review)

126Southwestern Historical QuarterlyJuly Despite their reluctance to serve in the Department of New Mexico, all evidence indicates that Wallen and Evans carried out their duties competently. Indeed, both men produced detailed reports that provide an interesting portrait of life at frontier military outposts. Because of the nature of the inspectors' duties, the reports document the true conditions at the forts and obviously make no attempt to hide any problems. The documents note the inspectors' concerns with specific officers and soldiers, discipline, equipment and supplies, living conditions , payroll information, and include the inspectors' suggestions for improvements . The reports published in this volume pertain to Forts Marcy, Union, Craig, Sumner, West, McRae, and Stanton, and posts at Albuquerque, Los Pinos, and Mesilla in New Mexico Territory; Fort Garland in Colorado Territory; and Franklin, Texas. The book is comprised of thirteen chapters, most of which include an inspector 's report for one fort or post. Each chapter begins with a brief history of the installation in question, and illustrations include maps and several hand-drawn diagrams of the forts. Thompson's endnotes reflect extensive research in National Archives records and select published sources. The book will appeal to individuals interested in the Civil War period in the Trans-Mississippi West, particularly the Far West, and those interested in first-hand accounts of daily life at nineteenth-century frontier military posts. East Central UniversityBradley R. Clampitt The Texas Rangers: Wearing the Cinco Peso, 1821-içoo. By Mike Cox. (New York: Forge, 2008. Pp. 492. Illustrations, notes, bibliography, index. ISBN 9780312873868, $25.95 cloth.) Mike Cox, storyteller and author of the entertaining and anecdotal Texas Ranger Tales and Texas Ranger Tales II, delivers a likewise readable but much more broadly historical perspective of the legendary frontier law enforcement agency from its beginnings and throughout the nineteenth century. While doing so, Cox considers the tangle of myth and fact—"as closely interwoven as a fine horsehair quirt" (15)—and concludes that it is difficult to separate the two strands from one another. He does confront the revisionist historians who have relentlessly attacked the Ranger lore of late, admitting that there were certainly acts committed above the law but that those instances were "rare and often exaggerated," and that in the final accounting the Rangers managed against great odds to keep the peace on a dangerous frontier. Cox divides his treatise into two large sections, the early period (1821-1873) of the Rangers as Indian fighters, and the later period (1874-1900) as peace officers with increasing responsibilities to track and capture criminals and fugitives across Texas. The Rangers were the only agency given authority to cross county lines in their pursuits of the lawless and, as the Indian threat subsided following Reconstruction, their response to the needs of civil and state agencies increased. During the formative period of Texas's history from Anglo colonization 20ogBook Reviews127 through the era of the Republic, the Texas Rangers evolved through several stages of incubation. There is a contingent of early Texas historians who continue to debate exacdy when and where and who the "first" Texas Rangers may have been—some suggest 1823, 1836, or 1846 or even 1874; here Cox is refreshingly able to include all of them in the narrative without having to decide on one or the other as the "real" date. And it is forgivable for Cox to quote extensively from Wilbarger's old-fashioned Indian Depredations in Texas early on; Wilbarger's book is, after all, good storytelling if not the best of accurate, objective history (where Indians and Mexicans were considered the intruders). In 1874 the Texas legislature created what became known as the Frontier Battalion of the Texas Rangers. Its broader responsibilities necessitated a larger force of men who were well organized and vested with nearly unimpeachable authority to enforce the law as they saw fit. A budget leveled at a remarkable (for that period) $300,000 provided die material support for a more permanent agency than had existed heretofore. Cox enthralls the reader with a dashing narrative through late nineteenthcentury Texas expansion, as MajorJohn B. Jones—who still does not have a biography worthy of his exploits—and the Frontier Battalion...

Towards realism in drawing areas of interest on architecture diagrams

Areas of interest (AOIs) are defined as groups of elements of system architecture diagrams that share some common property. Visualizing AOIs is a useful addition to plain diagrams, such as UML diagrams. Some methods have been proposed to automatically draw AOIs on UML diagrams. However, it is not clear whether actual users perceive the results of such methods to be better or worse as compared to human-drawn AOI, and what needs to be improved. We present here a process of studying and improving the perceived quality of computer-drawn AOI. For this, we conducted a qualitative evaluation that delivered insight in how users perceive the quality of computer-drawn AOIs as compared to hand-drawn diagrams. Following these results, we derived and implemented several improvements to an existing algorithm for computer-drawn AOIs. Next, we designed a distance metric to quantitatively compare different AOI drawings, and used this metric to show that our improved rendering algorithm creates drawings which are closer to (good) human drawings than the original rendering algorithm. We present here the results of the user evaluation, our improved algorithm for drawing AOIs, and the quantitative analysis performed to compare different drawings. The combined user evaluation, algorithmic improvements, and quantitative comparison method support our claim of having improved the perceived quality and understandability of AOI rendered on architecture diagrams.

Comments on “A Simple Model for Simulating Tornado Damage in Forests”

Corresponding author address: Dr. Nikolai Dotzek, DLR-Institut für Physik der Atmosphäre, Oberpfaffenhofen, 82234 Wessling, Germany. Email: nikolai.dotzek@dlr.de