Annotation Model Research Articles

Data equilibrium based automatic image annotation by fusing deep model and semantic propagation

Automatic image annotation is a challenging research problem that includes a large number of tags and various features. Traditional shallow machine learning algorithms lack generalization performance when dealing with complex classification problems. Automatic image annotation based on a stacked auto-encoder (SAE) is proposed to enhance the annotation generalization performance. In this paper, two kinds of strategies, the annotation model and the annotation process, are proposed to solve the main problem of unbalanced data in image annotation. 1) For the annotation model itself, to improve the annotation effect of low frequency tags, we propose a balanced and stacked auto-encoder (BSAE) that can enhance training for low frequency tags. On the basis of this model, a robust BSAE (RBSAE) algorithm which enhances training for sub BSAE model by group is proposed to enhance the annotation stability. This strategy ensures that the model itself has a strong ability to deal with the problem of unbalanced data. 2) For the annotation process, we propose a framework of attribute discrimination annotation (ADA). We first take an unknown image. Then we construct a local equilibrium dataset based on the unknown image and discriminate the high- and low-frequency attribute of the image to determine the corresponding annotation process. One process called the local semantic propagation (LDE-SP) algorithm annotates the low frequency image and the RBSAE algorithm annotates the high frequency image. This strategy improves the overall image annotation effect and ensures that the annotation process has a strong ability to deal with the problem of unbalanced data. For each SAE (including BSAE and RBSAE) annotation model, we propose two kinds of optimization methods, namely, one that is based on non-linear optimization and one on linear optimization. Experimental results on three benchmark datasets show that the proposed model outperforms the previous models in many performance indices.

Deep learning based feature representation for automated skin histopathological image annotation

Automated annotation of skin biopsy histopathological images provides valuable information and supports for diagnosis, especially for the discrimination between malignant and benign lesions. Currently, computer-aid analysis of skin biopsy images mostly relied on some human-designed features, which requires expensive human efforts and experiences in problem domains. In this study, we propose an annotation framework for automated skin biopsy image analysis which makes use of a deep model for image feature representation. A convolutional neural network (CNN) is designed for local regions of skin biopsy images which learns potential high-level features automatically from input raw pixels. The annotation model is constructed in the multiple-instance multiple-label (MIML) learning framework with the features learned through the network. We achieve significant improvement of the model performance on a real world clinical skin biopsy image dataset and a benchmark dataset. Moreover, our study indicates that deep learning based model could achieve better performance than human designed features.

Automated Fault Tree Analysis from AADL Models

Cyber-physical systems, used in domains such as avionics or medical devices, perform critical functions where a fault might have catastrophic consequences (mission failure, severe injuries, etc.). Their development is guided by rigorous practice standards that prescribe safety analysis methods in order to verify that failure have been correctly evaluated and/or mitigated. This laborintensive practice typically focuses system safety analysis on system engineering activities. As reliance on software for system operation grows, embedded software systems have become a major source of hazard contributors. Studies show that late discovery of errors in embedded software system have resulted in costly rework, making up as much as 50% of the total software system cost. Automation of the safety analysis process is key to extending safety analysis to the software system and to accommodate system evolution. In this paper we discuss three elements that are key to safety analysis automation in the context of fault tree analysis (FTA). First, generation of fault trees from annotated architecture models consistently reflects architecture changes in safety analysis results. Second, use of a taxonomy of failure effects ensures coverage of potential hazard contributors is achieved. Third, common cause failures are identified based on architecture information and reflected appropriately in probabilistic fault tree analysis. The approach utilizes the SAE Architecture Analysis & Design Language (AADL) standard and the recently published revised Error Model Annex V2 (EMV2) standard to represent annotated architecture models of systems and embedded software systems. The approach takes into account error sources specified with an EMV2 error propagation type taxonomy and occurrence probabilities as well as direct and indirect propagation paths between system components identified in the architecture model to generate a fault graph and apply transformations into a fault tree representation to support common mode analysis, cut set determination and probabilistic analysis.

Source-Level Performance, Energy, Reliability, Power and Thermal (PERPT) Simulation

With ever increasing design complexities, traditional cycle-accurate or instruction-set simulations are often too slow or too inaccurate for system prototyping in early design stages. As an alternative, host-compiled or source-level software simulation has been proposed, but existing approaches have largely focused on timing simulation only. In this paper, we propose a novel source-level simulation infrastructure that provides a full range of performance, energy, reliability, power and thermal (PERPT) estimation. Using a fully automated, retargetable back-annotation framework, intermediate representation code is statically annotated with timing, energy and resource accesses information obtained from low-level references at basic block granularity. The annotated model is natively compiled and combined with a cache model and occupancy analyzer to provide target performance, energy, soft-error vulnerability and power estimations. Finally, generated power traces are fed into thermal models for further temperature estimation. Comprehensive evaluations of our source-level models for PERPT estimations are performed. We applied our approach to PowerPC targets running various industry benchmark suites. source-level simulations are evaluated for different PERPT metrics and with cache models at various levels of detail to explore the speed and accuracy tradeoffs. More than 90% accuracy can be achieved for timing, energy, reliability and power estimation, and an average error of 0.05 K exists in steady-state thermal estimation. Simulation speeds range from 180 to 5740 MIPS for different types of metrics at different abstraction levels.

A Tool to Build Up-To-Date Gene Annotations for Affymetrix Microarrays

Genome-wide expression profiling and genotyping is widely applied in functional genomics research, ranging from stem cell studies to cancer, in drug response studies, and in clinical diagnostics. The Affymetrix GeneChip microarrays represent the most popular platform for such assays. Nevertheless, due to rapid and continuous improvement of the knowledge about the genome, the definition of many of the genes and transcripts change, and new genes are discovered. Thus the original probe information is out-dated for a number of Affymetrix platforms, and needs to be re-defined. It has been demonstrated, that accurate probe set definition improves both coverage of the gene expression analysis and its statistical power. Therefore we developed a method that incorporates the most recent genome annotations into the annotation of the microarray probe sets, using tools from the next generation sequencing. Additionally our method allows to quickly build project specific gene annotation models, as well as for comparison of microarray to RNAseq data.

Validating Genome-Wide Association Candidates Controlling Quantitative Variation in Nodulation.

Genome-wide association (GWA) studies offer the opportunity to identify genes that contribute to naturally occurring variation in quantitative traits. However, GWA relies exclusively on statistical association, so functional validation is necessary to make strong claims about gene function. We used a combination of gene-disruption platforms (Tnt1 retrotransposons, hairpin RNA-interference constructs, and CRISPR/Cas9 nucleases) together with randomized, well-replicated experiments to evaluate the function of genes that an earlier GWA study in Medicago truncatula had identified as candidates contributing to variation in the symbiosis between legumes and rhizobia. We evaluated ten candidate genes found in six clusters of strongly associated single nucleotide polymorphisms, selected on the basis of their strength of statistical association, proximity to annotated gene models, and root or nodule expression. We found statistically significant effects on nodule production for three candidate genes, each validated in two independent mutants. Annotated functions of these three genes suggest their contributions to quantitative variation in nodule production occur through processes not previously connected to nodulation, including phosphorous supply and salicylic acid-related defense response. These results demonstrate the utility of GWA combined with reverse mutagenesis technologies to discover and validate genes contributing to naturally occurring variation in quantitative traits. The results highlight the potential for GWA to complement forward genetics in identifying the genetic basis of ecologically and economically important traits.

Semantic Interoperability in Heterogeneous IoT Infrastructure for Healthcare

Interoperability remains a significant burden to the developers of Internet of Things’ Systems. This is due to the fact that the IoT devices are highly heterogeneous in terms of underlying communication protocols, data formats, and technologies. Secondly due to lack of worldwide acceptable standards, interoperability tools remain limited. In this paper, we proposed an IoT based Semantic Interoperability Model (IoT-SIM) to provide Semantic Interoperability among heterogeneous IoT devices in healthcare domain. Physicians communicate their patients with heterogeneous IoT devices to monitor their current health status. Information between physician and patient is semantically annotated and communicated in a meaningful way. A lightweight model for semantic annotation of data using heterogeneous devices in IoT is proposed to provide annotations for data. Resource Description Framework (RDF) is a semantic web framework that is used to relate things using triples to make it semantically meaningful. RDF annotated patients’ data has made it semantically interoperable. SPARQL query is used to extract records from RDF graph. For simulation of system, we used Tableau, Gruff-6.2.0, and Mysql tools.

The Acquarossa Memory Project: reconstructing an Etruscan town

Digital techniques and cultural heritage connect, in an innovative way, new and old within the Humanities. In this new project, an Etruscan townscape will be recreated; modelled results created by the 4D Research Lab will be integrated in an Archaeological Park and Museum in such a way that international scholars and visitors can acknowledge and study 3D reconstructions of a series of Etruscan houses within their successive phases of creation, function, reception, destruction, and reconstruction. Acquarossa is an Etruscan town near Viterbo, Italy. Excavations carried out by the Swedish Institute in Rome revealed a series of Etruscan houses, inhabited from the 8th century BC until the middle of the 6th century BC, when the town was suddenly and inexplicably abandoned. The houses were left to crumble and the remains of the foundations, the walls and the decorated roofs, as well as the thousands of household utensils, were all found in situ. It is one of the very few examples of an intact Etruscan townscape, with a unique set of family dwellings from the past. The remnants of the houses were partially reconstructed in the 1980s and covered with soil, but others were left to be destroyed by weather conditions. The site was left to vanish completely. Since 2014, the 4D Research Lab of the Faculty of Humanities of the University of Amsterdam has been involved in an interdisciplinary research project, in collaboration with the private partner Azienda Agrituristica Raffaele Rocchi, the proprietor of the site, which focuses on the reconstruction of a set of houses in annotated 3D models. The 3D models will be used to build ‘actual’ guesthouses at the site itself, for which permission from the Soprintendenza per i Beni archeologici del Lazio e dell’Etruria Meridionale has been granted. The project aims to analyse the house architecture, roof decoration and building processes through 3D modelling and scanning.

Personalized Image Annotation Using Deep Architecture

In image-centric social networks, such as Instagram and Pinterest, users tend to share photos with several tags. These tags describe the content of the image or provide additional contextual information, and therefore may not be necessarily tied to image content and usually carry personal preference. Annotating images in social networks in a personalized manner is in demand. However, the existing image annotation models, which rely only on image content information, cannot capture the user’s tagging preference. In this paper, we propose a deep architecture for personalized image annotation by leveraging the wealth of information in user’s tagging history. The proposed architecture consists of three components: two components for learning features of the image content and user’s history tags and the other one for combining the two learned features to predict the tags. We also explore two ways to model user’s history tags: 1) simply average the embeddings of user’s history tags and 2) model user’s history tags with a sequence model by long short-term memory recurrent neural network. We evaluate our proposed deep architecture on a large-scale and realistic data set, consisting of ~22.8 million public images uploaded by ~4.69 million users. Experimental results show that our proposed deep architecture is effective on a personalized image annotation task.

Advanced Model for Human Action Annotation Based on Background Subtraction Using Learning Vector Quantitation with Co-occurrence Matrix Features

This paper presents an advanced model for human action annotation. The proposed technique is splitting human objects in two parts, upper and under part of human beings. From these two parts, a model to extract the feature vectors by GLCM was proposed as feature for classification. The first method is to extract the Haralick features out of GLCM and the next step is the normalization process for converting co-occurrence feature matrix into various vectors as feature for classification. The research employs learning vector quantification to classify all feature vectors. Finally, the experiment conducted by utilizing Weizmann dataset shows that this approach method achieves an accuracy of 84.7%.

The SWISS-MODEL Repository-new features and functionality.

SWISS-MODEL Repository (SMR) is a database of annotated 3D protein structure models generated by the automated SWISS-MODEL homology modeling pipeline. It currently holds >400 000 high quality models covering almost 20% of Swiss-Prot/UniProtKB entries. In this manuscript, we provide an update of features and functionalities which have been implemented recently. We address improvements in target coverage, model quality estimates, functional annotations and improved in-page visualization. We also introduce a new update concept which includes regular updates of an expanded set of core organism models and UniProtKB-based targets, complemented by user-driven on-demand update of individual models. With the new release of the modeling pipeline, SMR has implemented a REST-API and adopted an open licencing model for accessing model coordinates, thus enabling bulk download for groups of targets fostering re-use of models in other contexts. SMR can be accessed at https://swissmodel.expasy.org/repository.

Automatic image annotation by combining generative and discriminant models

Generative model based image annotation methods have achieved good annotation performance. However, due to the problem of “semantic gap”, these methods always suffer from the images with similar visual features but different semantics. It seems promising to separate these images from semantic relevant ones by using discriminant models, since they have shown excellent generalization performance. Motivated to gain the benefits of both generative and discriminative approaches, in this paper, we propose a novel image annotation approach which combine the generative and discriminative models through local discriminant topics in the neighborhood of the unlabeled image. Singular Value Decomposition(SVD) is applied to group the images of the neighborhood into different topics according to their semantic labels. The semantic relevant images and the irrelevant ones are always assigned into different topics. By exploiting the discriminant information between different topics, Support Vector Machine(SVM) is applied to classify the unlabeled image into the relevant topic, from which the more accurate annotation will be obtained by reducing the bad influence of irrelevant images. The experiments on the ECCV 2002 [3] and NUS-WIDE [34] benchmark show that our method outperforms state-of-the-art annotation models.

A Context-Driven Extractive Framework for Generating Realistic Image Descriptions.

Automatic image annotation methods are extremely beneficial for image search, retrieval, and organization systems. The lack of strict correlation between semantic concepts and visual features, referred to as the semantic gap, is a huge challenge for annotation systems. In this paper, we propose an image annotation model that incorporates contextual cues collected from sources both intrinsic and extrinsic to images, to bridge the semantic gap. The main focus of this paper is a large real-world data set of news images that we collected. Unlike standard image annotation benchmark data sets, our data set does not require human annotators to generate artificial ground truth descriptions after data collection, since our images already include contextually meaningful and real-world captions written by journalists. We thoroughly study the nature of image descriptions in this real-world data set. News image captions describe both visual contents and the contexts of images. Auxiliary information sources are also available with such images in the form of news article and metadata (e.g., keywords and categories). The proposed framework extracts contextual-cues from available sources of different data modalities and transforms them into a common representation space, i.e., the probability space. Predicted annotations are later transformed into sentence-like captions through an extractive framework applied over news articles. Our context-driven framework outperforms the state of the art on the collected data set of approximately 20 000 items, as well as on a previously available smaller news images data set.

Cognitive computing and proposed approaches to conceptual organization of case law knowledge bases: a proposed model for information preparation, indexing, and analysis

Carole Hafner's scholarship on the conceptual organization of case law knowledge bases (COC) was an original approach to distilling a library's worth of cases into a manageable subset that any given legal researcher could review. Her approach applied concept indexation and concept search based on an annotation model of three interacting components combined with a system of expert legal reasoning to aid in the retrieval of pertinent case law. Despite the clear value this tripartite approach would afford to researchers in search of cases with similar fact patterns and desired (or undesired) outcomes, this approach has not been applied consistently in the intervening years since its introduction. Specifically, the conceptual representation of domain concepts and the case frames were not pursued by researchers, and they were not applied by the legal case indexing services that came to dominate the electronic case law market. Advances since Hafner's original scholarship in the form of (1) digitized case law and related materials; (2) computer science analytical protocols; and (3) more advanced forms of artificial intelligence approaches present the question of whether Hafner's COC model could move from the hypothetical to the real.

A Tutorial on Pharmacodynamic Scripting Facility inSimcyp.

The Simcyp Simulator provides a framework for mechanistic Physiologically‐Based Pharmacokinetic/Pharmacodynamic modeling of potentially interacting drugs. It also provides a scripting facility, using the Lua language, for developing customized pharmacodynamic and toxicity models driven by drug concentrations at the site of action. We present an overview of the scripting facility including the scripting language, the editor, and how scripts are embedded within the Simulator. Examples incorporating differential equations and including inter‐individual variability on parameters are presented.

A linguistically-motivated annotation model of modality in English and Spanish: Insights from MULTINOT

In this paper we present current work on the design and validation of a linguistically-motivated annotation model of modality in English and Spanish in the context of the MULTINOT project. Our annotation model captures four basic modal meanings and their subtypes, on the one hand, and provides a fine-grained characterisation of the syntactic realisations of those meanings in English and Spanish, on the other. We validate the modal tagset proposed through an agreement study performed on a bilingual sample of four hundred sentences extracted from original texts of the MULTINOT corpus, and discuss the difficult cases encountered in the annotation experiment. We also describe current steps in the implementation of the proposed scheme for the large-scale annotation of the bilingual corpus using both automatic and manual procedures.

A Spatial-Temporal Topic Model for the Semantic Annotation of POIs in LBSNs

Semantic tags of points of interest (POIs) are a crucial prerequisite for location search, recommendation services, and data cleaning. However, most POIs in location-based social networks (LBSNs) are either tag-missing or tag-incomplete. This article aims to develop semantic annotation techniques to automatically infer tags for POIs. We first analyze two LBSN datasets and observe that there are two types of tags, category-related ones and sentimental ones, which have unique characteristics. Category-related tags are hierarchical, whereas sentimental ones are category-aware. All existing related work has adopted classification methods to predict high-level category-related tags in the hierarchy, but they cannot apply to infer either low-level category tags or sentimental ones. In light of this, we propose a latent-class probabilistic generative model, namely the spatial-temporal topic model (STM), to infer personal interests, the temporal and spatial patterns of topics/semantics embedded in users’ check-in activities, the interdependence between category-topic and sentiment-topic, and the correlation between sentimental tags and rating scores from users’ check-in and rating behaviors. Then, this learned knowledge is utilized to automatically annotate all POIs with both category-related and sentimental tags in a unified way. We conduct extensive experiments to evaluate the performance of the proposed STM on a real large-scale dataset. The experimental results show the superiority of our proposed STM, and we also observe that the real challenge of inferring category-related tags for POIs lies in the low-level ones of the hierarchy and that the challenge of predicting sentimental tags are those with neutral ratings.

Image Annotation by Propagating Labels from Semantic Neighbourhoods

Automatic image annotation aims at predicting a set of semantic labels for an image. Because of large annotation vocabulary, there exist large variations in the number of images corresponding to different labels ("class-imbalance"). Additionally, due to the limitations of human annotation, several images are not annotated with all the relevant labels ("incomplete-labelling"). These two issues affect the performance of most of the existing image annotation models. In this work, we propose 2-pass k-nearest neighbour (2PKNN) algorithm. It is a two-step variant of the classical k-nearest neighbour algorithm, that tries to address these issues in the image annotation task. The first step of 2PKNN uses "image-to-label" similarities, while the second step uses "image-to-image" similarities, thus combining the benefits of both. We also propose a metric learning framework over 2PKNN. This is done in a large margin set-up by generalizing a well-known (single-label) classification metric learning algorithm for multi-label data. In addition to the features provided by Guillaumin et al. (2009) that are used by almost all the recent image annotation methods, we benchmark using new features that include features extracted from a generic convolutional neural network model and those computed using modern encoding techniques. We also learn linear and kernelized cross-modal embeddings over different feature combinations to reduce semantic gap between visual features and textual labels. Extensive evaluations on four image annotation datasets (Corel-5K, ESP-Game, IAPR-TC12 and MIRFlickr-25K) demonstrate that our method achieves promising results, and establishes a new state-of-the-art on the prevailing image annotation datasets.

Patching Holes in the Chlamydomonas Genome.

The Chlamydomonas genome has been sequenced, assembled, and annotated to produce a rich resource for genetics and molecular biology in this well-studied model organism. However, the current reference genome contains ∼1000 blocks of unknown sequence (‘N-islands’), which are frequently placed in introns of annotated gene models. We developed a strategy to search for previously unknown exons hidden within such blocks, and determine the sequence, and exon/intron boundaries, of such exons. These methods are based on assembly and alignment of short cDNA and genomic DNA reads, completely independent of prior reference assembly or annotation. Our evidence indicates that a substantial proportion of the annotated intronic N-islands contain hidden exons. For most of these, our algorithm recovers full exonic sequence with associated splice junctions and exon-adjacent intronic sequence. These new exons represent de novo sequence generally present nowhere in the assembled genome, and the added sequence improves evolutionary conservation of the predicted encoded peptides.

MONO+KM: Knowledge Management in Collaborative Project Development

<p>In the dynamics of collaborative project management, participating organizations make great efforts and provide technical, technological and human resources to achieve a product they can hardly develop individually.</p><p>Although there are tools to integrate, monitor and manage processes for such projects, it is not uncommon to find technological support to manage the knowledge generated during their execution. Usually this knowledge is part of the experience of participants but it is not recovered or documented nor is it used an organizational level, thus losing an important asset.</p><p>In this study, we propose a technique in which a knowledge management approach is applied to the management of collaborative projects and where knowledge is expressed in terms of decisions.</p><p>This is achieved through the analysis of verbal interactions that occur among participants in these projects, the identification and recovery of decisions Grounded Theory Method (GTM) techniques, and the specification of a set of concrete usage scenarios.</p><p>The technique was applied in MONO, a framework for integration, control and optimization of production processes in which digital content companies in the creative industry work collaboratively.</p><p>The study provides an annotation model, which, without being intrusive, allows for the recovery and structuring of knowledge expressed as decisions, thus making possible its replication in other domains.</p>