Layout Knowledge Research Articles

Pose estimation and robotic insertion tasks based on YOLO and layout features

In this study, we proposed a practical scheme for the challenging robotic cable insertion task. In general, the applied architecture is based on two YOLOs, vision-based pose estimation and admittance control. A precise and effective method was developed to estimate the pose of a manipulated connector. Our method uses a deep convolutional neural network to detect the relevant regions in the image. The characteristics of these relevant regions along with the pins’ layout manifold are combined to conduct the estimation. Practical problems such as error examination and time efficiency were considered in the proposed method for real applications. An admittance controller is introduced to experimentally validate the performance of pose estimation and provide compliant insertion by the proposed architecture. Our method is only based on less prior layout knowledge and does not require a precise model, which facilitates modeling and deployment. In addition, our method is robust to image quality and has low computational complexity, which makes it highly suitable for online manipulation. Besides, our method can handle multiple connector types which can cover most cases in aeronautical manufacturing and guide the design of connectors in the production process. The advantages of our method were demonstrated by extensive testing using both synthetic data and experiments. We also designed an insertion controller and realized a complete insertion task using a PC with a 12 GB RTX 3060 GPU and 32 GB RAM. These experimental results show that our method can achieve precise and reliable estimation with mean absolute errors less than 0.44 deg and 0.36 mm, an estimation accuracy of over 99%, and a successful manipulation rate of over 94%. This reveals that the proposed method displays potential for the challenging robotic cable insertion task.

Layout Aware Semantic Element Extraction for Sustainable Science & Technology Decision Support

New scientific and technological (S&T) knowledge is being introduced rapidly, and hence, analysis efforts to understand and analyze new published S&T documents are increasing daily. Automated text mining and vision recognition techniques alleviate the burden somewhat, but the various document layout formats and knowledge content granularities across the S&T field make it challenging. Therefore, this paper proposes LA-SEE (LAME and Vi-SEE), a knowledge graph construction framework that simultaneously extracts meta-information and useful image objects from S&T documents in various layout formats. We adopt Layout-aware Metadata Extraction (LAME), which can accurately extract metadata from various layout formats, and implement a transformer-based instance segmentation (i.e., Vision based Semantic Elements Extraction (Vi-SEE)) to maximize the vision-based semantic element recognition. Moreover, to constructing a scientific knowledge graph consisting of multiple S&T documents, we newly defined an extensible Semantic Elements Knowledge Graph (SEKG) structure. For now, we succeeded in extracting about 6 million semantic elements from 49,649 PDFs. In addition, to illustrate the potential power of our SEKG, we provide two promising application scenarios, such as a scientific knowledge guide across multiple S&T documents and questions and answering over scientific tables.

Overview of the modified magnetoelastic method applicability

<p class="Abstract">A requirement of axial force determination in important structural elements of a building or engineering structure during its construction or operational state is very frequent in technical practice. In civil engineering practice, five experimental techniques are usually used for evaluation of axial tensile forces in these elements. Each of them has its advantages and disadvantages. One of these methods is the magnetoelastic method, that can be used, for example, on engineering structures for experimental determination of the axial forces in prestressed structural elements made of ferromagnetic materials, e.g., prestressed bars, wires and strands. The article presents general principles of the magnetoelastic method, the magnetoelastic sensor layout and actual information and knowledge about practical application of the new approach based on the magnetoelastic principle on prestressed concrete structures. Subsequently, recent results of the experimental verification and the in-situ application of the method are described in the text. The described experimental approach is usable not only for newly built structures but in particular for existing ones. Furthermore, this approach is the only one effectively usable experimental method for determination of the prestressed force on existing prestressed concrete structures in many cases in the technical practice.</p>

Contextual Stroke Classification in Online Handwritten Documents with Edge Graph Attention Networks

The task of grouping strokes into different categories is an essential processing step in the automatic analysis of online handwritten documents. The technical challenge originates from the variation of the handwriting style, content heterogeneity and lack of prior layout knowledge. In this work, we propose the edge graph attention network (EGAT) to address the stroke classification problem. In this framework, the stroke classification problem is formulated as a node classification problem in a relational graph, which is constructed based on the temporal and spatial relationship of strokes. Then distributed node and edge features for classification are learned by stacking of multiple edge graph attention layers, in which various attention mechanisms are exploited to aggregate information between neighborhood nodes. In the task of text/nontext classification, the proposed model achieves accuracies 98.65% and 98.90% on the IAMOnDo and Kondate datasets, respectively. In the task of multi-class classification, the achieved accuracies are 95.81%, 97.36% and 99.05% on the IAMOnDo, FC and FA datasets, respectively. In addition, we conduct ablation experiments to quantitatively and qualitatively evaluate the key modules of our model.

Knowledge component-based intelligent method for fixture design

Aiming at providing an effective mechanism for representation, organization, and reuse of fixture design knowledge, this paper puts forward an intelligent methodology for fixture design based on knowledge component technology. First, the ontology models of fixture elements, the workpiece, and the fixture design case are constructed to organize the statical design knowledge related to each of them. Then, the fixture design process-related knowledge is modeled by knowledge components corresponding to fixture layout planning process and fixture structure design process. Hence, the hierarchy structure of the fixture design knowledge reuse is formed. Under the framework of the fixture layout knowledge component, the association analysis method is used for generating reasoning rules and then the rule-based reasoning (RBR) method is used for the determination of the workpiece-locating mode and the initiative candidate of the locating datum. Further, an entropy weight fuzzy comprehensive evaluation method is employed to select the most suitable locating datum from the candidates. Under the framework of the fixture structure design knowledge component, the selection of fixture elements, element dimension drive, and auto-assemble are fulfilled by the combination of RBR and case-based reasoning (CBR). The method realizes the effective organization and efficient use of fixture design knowledge and improves the efficiency of fixture design. It has been verified by a case of airplane part machining fixture design.

A Method of the Post-Processing of Printed Formula Extraction

As an important step of printed formula recognition system, formula extraction locates the formula fields on the layout images of printed documents, which influences the performance of formula recognition to a great extent. However, the errors of automatic formula extraction occur inevitably because of the complexity of formulas themselves and the layouts which the formulas situated. To solve this problem, this paper designed a post-processing method to correct the errors existing in the results of formula extraction algorithm according to relative layout knowledge. First of all, the geometrical features of various layout fields were employed to correct the extraction errors. Then, the syntax rules were used to check the boundary components of different kinds of areas in layouts to identify which field it should belong to. Finally, the formula area was adjusted according to above mentioned information.

Spatial Learning Using Locomotion Interface to Virtual Environment

The inability to navigate independently and interact with the wider world is one of the most significant handicaps that can be caused by blindness, second only to the inability to communicate through reading and writing. Many difficulties are encountered when visually impaired people (VIP) need to visit new and unknown places. Current speech or haptics technology does not provide a good solution. Our approach is to use treadmill-style locomotion interface, unconstrained walking plane (UWP), to allow a richer and more immersive form of virtual environment (VE) exploration to enable VIP to create cognitive maps efficiently and thereby to enhance their mobility. An experimental study is reported that tests design of UWP for both straight walking and turning motions. Two groups of participants, blind-folded-sighted and blind, learned spatial layout in VE using two exploration modes: guided (training phase) and unguided (testing phase). Spatial layout knowledge was assessed by asking participants to perform object-localization task and target-object task. Our results showed a significant decrease in time and helps taken to complete tasks, subjective workload, and errors in a post-training trial as compared to a partial-training trial. UWP has been found to significantly improve interaction with VE with visualizations such as spatial information.

Learning Building Layouts with Non-Geometric Visual Information: The Effects of Visual Impairment and Age

Previous studies suggest that humans rely on geometric visual information (hallway structure) rather than non-geometric visual information (eg doors, signs, and lighting) for acquiring cognitive maps of novel indoor layouts. In this study we asked whether visual impairment and age affect reliance on non-geometric visual information for layout learning. We tested three groups of participants-younger (<50 years of age) normally sighted; older (50-70 years of age) normally sighted; and low-vision (people with heterogeneous forms of visual impairment ranging in age from 18 to 67 years). Participants learned target locations in building layouts using four presentation modes: a desktop virtual environment (VE) displaying only geometric cues (sparse VE); a VE displaying both geometric and non-geometric cues (photorealistic VE); a map; and a real building. Layout knowledge was assessed by map drawing and by asking participants to walk to specified targets in the real space. Results indicate that low-vision and older normally sighted participants relied on additional non-geometric information to accurately learn layouts. In conclusion, visual impairment and age may result in reduced perceptual and/or memory processing that makes it difficult to learn layouts without non-geometric visual information.

A layout-based schematic method for very high-speed CMOS cell design

In very high-speed CMOS cell design, the result of schematic simulation is inaccurate because of missing parasitic components, such as diodes and parasitic capacitances. Designer cannot pass enough information to the simulator by conventional transistor symbols, therefore, simulation error occurs. In this paper, we address a layout-based schematic (LBS) method for high-speed CMOS cell design. In this method, we introduce several types of MOS transistors and estimate parasitic wire capacitances by using layout knowledge. The simulation results show that the difference between LBS and real layout is much smaller, less than 3% in rise time, compared to in the worst case of up to 65% in the original schematic. This method can be applied to both digital and analog circuits and it is helpful for layout automation. Time and cost will be reduced in high-speed circuit design.

A neuro-based expert system for facility layout construction

Motivated by the success of implementing expert systems (ESs) based on artificial neural networks (ANNs) to improved classical rule-based expert systems (RBESs), this paper reports on the development of a neuro-based expert system (NBES) for facility layout construction in a manufacturing system. In an artificial intelligence (AI) technique such as the NBES, the semantic structure of If-Then rules is preserved, while incorporating the learning capability of ANNs into the inference mechanism. Unlike implementing a popular back propagation network (BPN) as an ES, the proposed BAMFLO (Bidirectional Associative Memories for Facility LayOut) system is an intelligent layout consultant system consisting of pipeline BAM neural networks with simple, fast incremental learning and multiple bidirectional generalization characteristics. This incrementability makes BAMFLO effective at acquiring, adding or adapting learned layout knowledge; thus it is possible to memorize newly extended If-Then layout rules without retraining old ones. The multi-bidirectionality gives BAMFLO the ability to quickly and reliably generalize a layout solution, and to further infer unknown facts from known facts through a complex knowledge base (memorization) without losing information. The solution process of BAMFLO contains three essential steps: training example generation, incremental learning and solution generalization. The examples (layout knowledge) can be generated from practical experience and/or classical layout software solutions for incrementally training BAMFLO; the process then derives multiply bidirectionally generalized construction layout solutions. The experimental results show that the BAMFLO scheme outperforms five classical layout methods used to generate training examples.

Characteristics of Design-Relevant Constructability Knowledge

Constructability should be an important objective in all phases of a construction project, and designers play an important role in achieving superior constructability. Most projects, however, do not receive constructability input though prior research has demonstrated the benefits of such input. One reason for this lack of constructability input, we argue, is the lack of formal, explicit constructability knowledge bases that link constructability issues to design decisions and that can be made available on-line to interested parties. This paper describes research that compiled and formalized constructability knowledge related to reinforced concrete structures. To ensure appropriate and specific constructability input, we classified the knowledge by construction methods and structural elements. To make this specific knowledge available to designers at the right time during design development, we further divided it into the five groups: application heuristics, layout knowledge, dimensioning knowledge, detailing knowledge, and exogenous knowledge. This classification leads to conclusions regarding the ability to collect and formalize this knowledge, and to make it readily available to designers to improve project performance.

Dimensions of Incompetency: A Factor Analytic Study of the Georgia Court Competency Test

The Georgia Court Competency Test (GCCT) was developed to systematically evaluate knowledge and skills needed to be competent to stand trial. Previous research has identified three factors measured by the GCCT: General Legal Knowledge, Courtroom Layout, and Specific Legal Knowledge. In this study, 125 mentally disordered offenders were examined using the GCCT. Confirmatory factor analysis failed to confirm the previously identified model. As a result, the authors advocate modifying the clinical use of the GCCT. Specifically, they suggest employing the GCCT as a checklist to identify potential deficits requiring more comprehensive evaluation. Language: en

Layout recognition of multi-kinds of table-form documents

Many approaches have reported that knowledge-based layout recognition methods are very successful in classifying the meaningful data from document images automatically. However, these approaches are applicable to only the same kind of documents because they are based on the paradigm that specifies the structure definition information in advance so as to be able to analyze a particular class of documents intelligently. In this paper, the authors propose a method to recognize the layout structures of multi-kinds of table-form document images. For this purpose, the authors introduce a classification tree to manage the relationships among different classes of layout structures. The authors' recognition system has two modes: layout knowledge acquisition and layout structure recognition. In the layout knowledge acquisition mode, table-form document images are distinguished according to this. Classification tree and then the structure description trees which specify the logical structures of table-form documents are generated automatically. While, in the layout structure recognition mode, individual item fields in the table-form document images are extracted and classified successfully by searching the classification tree and interpreting the structure description tree.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Artificial Intelligence as applied to Optical Image Filing

This paper presents an image of a futuristic document filing system, or an advanced filing system, based on an analysis of the filing activity. Then, the techniques in artificial intelligence required to realize this system are discussed. The approaches proposed here are document understanding and knowledge-based filing. The document understanding technology may automate document image analysis and information extraction using character recognition. A method of image segmentation using layout knowledge is described. Knowledge-based filing, on the other hand, can ease the user to register and retrieve documents with the help of inference applied to domain specific knowledge. Software simulations and experiments on a prototype system are also described.