Image Tagging Research Articles

Learn to Predict Sets Using Feed-Forward Neural Networks.

This paper addresses the task of set prediction using deep feed-forward neural networks. A set is a collection of elements which is invariant under permutation and the size of a set is not fixed in advance. Many real-world problems, such as image tagging and object detection, have outputs that are naturally expressed as sets of entities. This creates a challenge for traditional deep neural networks which naturally deal with structured outputs such as vectors, matrices or tensors. We present a novel approach for learning to predict sets with unknown permutation and cardinality using deep neural networks. In our formulation we define a likelihood for a set distribution represented by a) two discrete distributions defining the set cardinally and permutation variables, and b) a joint distribution over set elements with a fixed cardinality. Depending on the problem under consideration, we define different training models for set prediction using deep neural networks. We demonstrate the validity of our set formulations on relevant vision problems such as: 1) multi-label image classification where we outperform the other competing methods on the PASCAL VOC and MS COCO datasets, 2) object detection, for which our formulation outperforms popular state-of-the-art detectors, and 3) a complex CAPTCHA test, where we observe that, surprisingly, our set-based network acquired the ability of mimicking arithmetics without any rules being coded.

Automatic Image Tagging Based on Text Analysis Using Natural Language Processing Techniques

I have seen that in any newspaper there are many articles related to different things and the article consists of some text and one or more images which are related to the given text. Some of the images given in the article have caption (which depicts the image) and the other images do not have any caption. Hence if I download the image which has no caption then after download, I will not understand what the image specifically describes.
 To address this kind of situation it is necessary to give the image some caption which depicts about the situation of the image and the information about the image. After download the image if I give some relevant tag to the image then it will be helpful for all.
 In modern era images are taken and stored digitally. Also, the prices of the image capturing devices are decreasing. Hence people are taking more and more pictures but handling of pictures are necessary at this situation. Additionally, there is a huge uptick in the use of web-based image-sharing programmes or applications. One of the most widely used of these applications is Flickr, which was acquired by Yahoo and now stores over 4 billion images.
 Image tagging by hand takes time and is subjective. Nobody has time to tag their images. Additionally, because people differ and think in different ways, comparable images will be tagged in various ways by different people. Differences in language, emotion, vocabulary, education, culture, taste, etc. may be the root of this.

Quantification of left ventricular strain and torsion by joint analysis of 3D tagging and cine MR images.

Cardiovascular magnetic resonance (CMR) imaging is the gold standard for the non-invasive assessment of left-ventricular (LV) function. Prognostic value of deformation metrics extracted directly from regular SSFP CMR images has been shown by numerous studies in the clinical setting, but with some limitations to detect torsion of the myocardium. Tagged CMR introduces trackable features in the myocardium that allow for the assessment of local myocardial deformation, including torsion; it is, however, limited in the quantification of radial strain, which is a decisive metric for assessing the contractility of the heart. In order to improve SSFP-only and tagged-only approaches, we propose to combine the advantages of both image types by fusing global shape motion obtained from SSFP images with the local deformation obtained from tagged images. To this end, tracking is first performed on SSFP images, and subsequently, the resulting motion is utilized to mask and track tagged data. Our implementation is based on a recent finite element-based motion tracking tool with mechanical regularization. Joint SSFP and tagged images registration performance is assessed based on deformation metrics including LV strain and twist using human and in-house porcine datasets. Results show that joint analysis of SSFP and 3DTAG images provides better quantification of LV strain and twist as either data source alone.

TelecomNet: Tag-Based Weakly-Supervised Modally Cooperative Hashing Network for Image Retrieval.

We are concerned with using user-tagged images to learn proper hashing functions for image retrieval. The benefits are two-fold: (1) we could obtain abundant training data for deep hashing models; (2) tagging data possesses richer semantic information which could help better characterize similarity relationships between images. However, tagging data suffers from noises, vagueness and incompleteness. Different from previous unsupervised or supervised hashing learning, we propose a novel weakly-supervised deep hashing framework which consists of two stages: weakly-supervised pre-training and supervised fine-tuning. The second stage is as usual. In the first stage, we propose two formulations Tag-basEd weakLy-supErvised Modally COoperative hashing Network (TelecomNet) and Generalized TelecomNet (GTelecomNet). Rather than performing supervision on tags, TelecomNet first learns an observed semantic embedding vector for each image from attached tags and then uses it to guide hashing learning. GTelecomNet introduces a novel semantic network to exploit more precise semantic information. By carefully designing the optimization problem, they can well leverage tagging information and image content for hashing learning. The framework is general and does not depend on specific deep hashing methods. Empirical results on real world datasets show that they significantly increase the performance of state-of-the-art deep hashing methods.

A Vision-Based Cattle Recognition System Using TensorFlow for Livestock Water Intake Monitoring

This letter presents a new method for identifying individual cattle for the purpose of tracking and efficiently measuring their daily water intake using a vision-based machine learning system. In addition to the current solution of using radio frequency identification (RFID), the proposed system uses TensorFlow Object Detection to detect labels on the RFID tag. The proposed system can be integrated into the current water intake monitoring system and alleviates the errors introduced by the unreliable RFID readers. The system allows users to train an object recognition model that can recognize and differentiate the labels on the ear tags of individual farm animals, so the drinking events can be recorded by the water intake monitoring system. The models are trained using custom datasets of manually annotated tag images with pretrained model architectures from TensorFlow 2 Model Zoo. The system is tested using images from event-triggered weather-proof cameras deployed in the grazing site. Experimental results of the system showed an accuracy of around 90%. In comparison to other present methods, this newly proposed system provides scalability and flexibility, making it an attractive vision-based solution for machine learning systems in agriculture.

Do Hexad User Types Matter? Effects of (Non-) Personalized Gamification on Task Performance and User Experience in an Image Tagging Task

The perception of gamification elements differs across users, which is why personalizing gamified systems is important. Past research showed that the Hexad user types model is particularly suitable for this purpose by demonstrating correlations between user types and gamification elements. However, previous studies were mostly survey-based, i.e. relied on participants' rating of gamification elements based on e.g. textual descriptions or storyboards. Thus, the question whether personalization based on Hexad user types provides benefits in implemented gameful systems was neglected. We contribute to this by investigating the effects of (contra-) tailoring the set of gamification elements to a user's Hexad type on performance and user experience, assessed with survey and physiological measures, in an image tagging context. In a lab study (N=29), we found that gamification increases performance and affects psychophysiological measures of arousal. Moreover, we demonstrate that personalization increases enjoyment, positively-valenced affective experiences and participants' absorption in the task at hand.

Flagging the Merited Lesions-Fibroblast 1 and 4 Imaging to Map the Key Avid Domains for Spiked Antigenicity Using SBRT In Situ Vaccination and Metronomic Radiation with Anti PDL-1 Therapy for Augmented Abscopal-Radscopal Responses in Disseminated Cancers

<h3>Purpose/Objective(s)</h3> Immune stimulating effects of SBRT (stereotactic body radiotherapy) for releasing tumor antigens and the immuno-modulatory property of low dose radiation has been under evaluation over years. Conceptual Radscopal effect for lesions receiving a non-tumoricidal scattered low dose radiation showing abscopal responses led to the concept of combining SBRT to highly avid larger metastatic lesions and low dose radiation to smaller lesions is exciting. However, choosing the right lesions for SBRT to release adequate and appropriate tumor antigen spike to cause a systemic response has been a challenge and area of interest. Cancer Associated Fibroblasts (CAF) 1 and 4 can produce growth factors and cytokines such as transforming growth factor-β (TGF-β) and vascular endothelial growth factor (VEGF) to promote angiogenesis and recruit immunosuppressive cells into the tumor stroma leading to immune evasion and resistant immune microenvironment. Fibroblast Activated protein tagged to CAF 1 and 4 whole body Molecular imaging aids us in identifying regions. Based on the avidity values of CAF 1 and 4 tagged imaging, the most immune resistant (CAF Rich) target lesions were chosen for SBRT in our study. Prominent usage of Anti PDL-1 inhibitors has been proven to be beneficial in disseminated cancers, concept of SBRT with metronomic radiation could be used as immune potentiator with accurate mapping of Immune resistant areas. <h3>Materials/Methods</h3> Five patients with disseminated cancers with indications for approved immune check point inhibitors were recruited after institutional ethical committee clearance. Patients received at least 3 cycles of check point inhibitor therapy prior to initiation of immuno-rad therapy. Based on CAF molecular imaging, 1-3 lesions with maximal avidity >10 standard uptake value were chosen for SBRT to a dose of 24 Gy in 3 fractions while all other metastatic less avid smaller lesions received Low dose 1 Gy metronomic radiotherapy using Image guided radiotherapy. Patients were continued on immunotherapy for 3 more cycles before a re-assessment molecular imaging was done. <h3>Results</h3> Nasopharyngeal (NPC), hepatocellular (HCC), 2 lung and renal cell cancer (RCC) patients were recruited. No grade 2 or above normal tissue toxicity was noted. <h3>Conclusion</h3> Effectively tackling the tumor immune evasion mechanisms by choosing the right lesions for SBRT acting as an in situ-vaccination by extensive release of antigens based on individual CAF biomarker profiling would ultimately aid in improving responses to Immunotherapy. The early results of our study are encouraging.

Deepfake attack prevention using steganography GANs.

Deepfakes are fake images or videos generated by deep learning algorithms. Ongoing progress in deep learning techniques like auto-encoders and generative adversarial networks (GANs) is approaching a level that makes deepfake detection ideally impossible. A deepfake is created by swapping videos, images, or audio with the target, consequently raising digital media threats over the internet. Much work has been done to detect deepfake videos through feature detection using a convolutional neural network (CNN), recurrent neural network (RNN), and spatiotemporal CNN. However, these techniques are not effective in the future due to continuous improvements in GANs. Style GANs can create fake videos with high accuracy that cannot be easily detected. Hence, deepfake prevention is the need of the hour rather than just mere detection. Recently, blockchain-based ownership methods, image tags, and watermarks in video frames have been used to prevent deepfake. However, this process is not fully functional. An image frame could be faked by copying watermarks and reusing them to create a deepfake. In this research, an enhanced modified version of the steganography technique RivaGAN is used to address the issue. The proposed approach encodes watermarks into features of the video frames by training an "attention model" with the ReLU activation function to achieve a fast learning rate. The proposed attention-generating approach has been validated with multiple activation functions and learning rates. It achieved 99.7% accuracy in embedding watermarks into the frames of the video. After generating the attention model, the generative adversarial network has trained using DeepFaceLab 2.0 and has tested the prevention of deepfake attacks using watermark embedded videos comprising 8,074 frames from different benchmark datasets. The proposed approach has acquired a 100% success rate in preventing deepfake attacks. Our code is available at https://github.com/shahidmuneer/deepfakes-watermarking-technique.

Enhanced Uptake and Phototoxicity of C60@albumin Hybrids by Folate Bioconjugation.

Fullerenes are considered excellent photosensitizers, being highly suitable for photodynamic therapy (PDT). A lack of water solubility and low biocompatibility are, in many instances, still hampering the full exploitation of their potential in nanomedicine. Here, we used human serum albumin (HSA) to disperse fullerenes by binding up to five fullerene cages inside the hydrophobic cavities. Albumin was bioconjugated with folic acid to specifically address the folate receptors that are usually overexpressed in several solid tumors. Concurrently, tetramethylrhodamine isothiocyanate, TRITC, a tag for imaging, was conjugated to C60@HSA in order to build an effective phototheranostic platform. The in vitro experiments demonstrated that: (i) HSA disperses C60 molecules in a physiological environment, (ii) HSA, upon C60 binding, maintains its biological identity and biocompatibility, (iii) the C60@HSA complex shows a significant visible-light-induced production of reactive oxygen species, and (iv) folate bioconjugation improves both the internalization and the PDT-induced phototoxicity of the C60@HSA complex in HeLa cells.

Mars, a molecule archive suite for reproducible analysis and reporting of single-molecule properties from bioimages.

The rapid development of new imaging approaches is generating larger and more complex datasets, revealing the time evolution of individual cells and biomolecules. Single-molecule techniques, in particular, provide access to rare intermediates in complex, multistage molecular pathways. However, few standards exist for processing these information-rich datasets, posing challenges for wider dissemination. Here, we present Mars, an open-source platform for storing and processing image-derived properties of biomolecules. Mars provides Fiji/ImageJ2 commands written in Java for common single-molecule analysis tasks using a Molecule Archive architecture that is easily adapted to complex, multistep analysis workflows. Three diverse workflows involving molecule tracking, multichannel fluorescence imaging, and force spectroscopy, demonstrate the range of analysis applications. A comprehensive graphical user interface written in JavaFX enhances biomolecule feature exploration by providing charting, tagging, region highlighting, scriptable dashboards, and interactive image views. The interoperability of ImageJ2 ensures Molecule Archives can easily be opened in multiple environments, including those written in Python using PyImageJ, for interactive scripting and visualization. Mars provides a flexible solution for reproducible analysis of image-derived properties, facilitating the discovery and quantitative classification of new biological phenomena with an open data format accessible to everyone.

Design Strategies for Responsive Fluorine‐19 Magnetic Resonance Probes Using Paramagnetic Metal Complexes

AbstractMagnetic resonance imaging (MRI) is a powerful and widely used in vivo imaging technique that enables whole body imaging without ionizing radiation. In clinical practice, 1H MRI is employed for imaging anatomical and physiological states via monitoring of protons in water and lipids. In order to monitor biochemical processes at the molecular level, several research groups are exploring responsive MRI agents that alter their signal upon interaction with an analyte or biological environment of interest. Fluorine (19F) MRI agents are promising due to the 19F nucleus having similar magnetic resonance (MR) properties to proton and the absence of endogenous 19F in living systems, resulting in no background signal. In order to make responsive 19F MR agents for molecular imaging and analysis, fluorinated platforms must be developed in which their 19F MR signal changes after interacting with a target analyte. A promising strategy is to use paramagnetic metals to modulate the 19F MR signal by altering the relaxation rates and/or chemical shift of an appended 19F imaging tag. In this concept, we provide an overview of the theoretical principles and molecular design strategies that have been exploited in the design of responsive 19F MR agents, with a specific focus on agents based on small molecule paramagnetic metal ion chelates.

Detection Method of Athlete Joint Injury Based on Deep Learning Model.

The research on accurate and intelligent segmentation of knee joint MRI images is of great significance to reduce the work intensity of clinical doctors and nurses. In order to solve the problem that knee joint MRI image segmentation model needs a large number of high-quality tagged images and excessive labeling workload, a semisupervised learning segmentation network model based on 3D scSE-UNet is proposed. The model adopts a self-training semisupervised learning framework and adds a cSE-block+ module on the basis of the 3D UNet model. This module can enhance the effective features of the feature image from two aspects of space and channel, while suppressing irrelevant features and preserving image edge information more completely. In order to solve the problem of rough edge of pseudolabel caused by model segmentation, a fully connected conditional random field is added to refine the edge of pseudolabel in the process of model training. The effectiveness of the model is verified by open source MRNet dataset and OAI dataset. The results show that the proposed model can achieve the segmentation effect of fully supervised learning through a small number of labeled images and effectively reduce the dependence of knee joint MRI image segmentation on expert labeling data.

Quantum algorithms for structured prediction

We introduce two quantum algorithms for solving structured prediction problems. We first show that a stochastic gradient descent that uses the quantum minimum finding algorithm and takes its probabilistic failure into account solves the structured prediction problem with a runtime that scales with the square root of the size of the label space, and in $$\tilde{O}\left (1/\epsilon \right )$$ with respect to the precision, 𝜖, of the solution. Motivated by robust inference techniques in machine learning, we then introduce another quantum algorithm that solves a smooth approximation of the structured prediction problem with a similar quantum speedup in the size of the label space and a similar scaling in the precision parameter. In doing so, we analyze a variant of stochastic gradient descent for convex optimization in the presence of an additive error in the calculation of the gradients, and show that its convergence rate does not deteriorate if the additive errors are of the order $$O(\sqrt{\epsilon})$$ . This algorithm uses quantum Gibbs sampling at temperature Ω(𝜖) as a subroutine. Based on these theoretical observations, we propose a method for using quantum Gibbs samplers to combine feedforward neural networks with probabilistic graphical models for quantum machine learning. Our numerical results using Monte Carlo simulations on an image tagging task demonstrate the benefit of the approach.

Multimodal imaging of capsid and cargo reveals differential brain targeting and liver detargeting of systemically-administered AAVs

The development of gene delivery vehicles with high organ specificity when administered systemically is a critical goal for gene therapy. We combine optical and positron emission tomography (PET) imaging of 1) reporter genes and 2) capsid tags to assess the temporal and spatial distribution and transduction of adeno-associated viruses (AAVs). AAV9 and two engineered AAV vectors (PHP.eB and CAP-B10) that are noteworthy for maximizing blood-brain barrier transport were compared. CAP-B10 shares a modification in the 588 loop with PHP.eB, but also has a modification in the 455 loop, added with the goal of reducing off-target transduction. PET and optical imaging revealed that the additional modifications retained brain receptor affinity. In the liver, the accumulation of AAV9 and the engineered AAV capsids was similar (∼15% of the injected dose per cc and not significantly different between capsids at 21 h). However, the engineered capsids were primarily internalized by Kupffer cells rather than hepatocytes, and liver transduction was greatly reduced. PET reporter gene imaging after engineered AAV systemic injection provided a non-invasive method to monitor AAV-mediated protein expression over time. Through comparison with capsid tagging, differences between brain localization and transduction were revealed. In summary, AAV capsids bearing imaging tags and reporter gene payloads create a unique and powerful platform to assay the pharmacokinetics, cellular specificity and protein expression kinetics of AAV vectors in vivo, a key enabler for the field of gene therapy.

Exercise-induced myocardial edema in master triathletes: Insights from cardiovascular magnetic resonance imaging

BackgroundStrenuous exercise has been associated with functional and structural cardiac changes due to local and systemic inflammatory responses, reflecting oxidative, metabolic, hormonal, and thermal stress, even in healthy individuals. We aimed to assess changes in myocardial structure and function using cardiovascular magnetic resonance (CMR) imaging in master triathletes early after a full-distance Ironman Triathlon race.Materials and methodsTen master triathletes (age 45 ± 8 years) underwent CMR within 3 h after a full-distance Ironman Triathlon race (3.8 km swimming, 180 km cycling, and 42.2 km running) completed with a mean time of 12 ± 1 h. All the triathletes had a 30-day follow-up CMR. Cine balanced steady-state free precession, T2-short tau inversion recovery (STIR), tagging, and late gadolinium enhancement (LGE) imaging sequences were performed on a 1.5-T MR scanner. Myocardial edema was defined as a region with increased T2 signal intensity (SI) of at least two SDs above the mean of the normal myocardium. The extent of myocardial edema was expressed as the percentage of left ventricular (LV) mass. Analysis of LV strain and torsion by tissue tagging included the assessment of radial, longitudinal, and circumferential peak systolic strain, rotation, and twist.ResultsCompared with postrace, biventricular volumes, ejection fraction, and LV mass index remained unchanged at 30-day follow-up. Global T2 SI was significantly higher in the postrace CMR (postrace 10.5 ± 6% vs. follow-up 3.9 ± 3.8%, P = 0.004) and presented with a relative apical sparing distribution (P < 0.001) matched by reduction of radial peak systolic strain of basal segments (P = 0.003). Apical rotation and twist were significantly higher immediately after the competition compared with follow-up (P < 0.05).ConclusionStrenuous exercise in master triathletes is associated with a reversible regional increase in myocardial edema and reduction of radial peak systolic strain, both presenting with a relative apical sparing pattern.

Effective Image Tagging: A Survey on Techniques to Improve Image Retrieval Experience

Images are generally searched using textual queries. For relevant images to retrieved effectively, they need to properly tagged. Tagging can be done manually or automatically. This paper is a survey on techniques to improve image retrieval experience by computing better tags for images. It also discusses some problems with the current techniques which opens up opportunities for further research. Key Words: image, tagging, searching

GridScore: a tool for accurate, cross-platform phenotypic data collection and visualization

BackgroundPlant breeding and crop research rely on experimental phenotyping trials. These trials generate data for large numbers of traits and plant varieties that needs to be captured efficiently and accurately to support further research and downstream analysis. Traditionally scored by hand, phenotypic data is nowadays collected using spreadsheets or specialized apps. While many solutions exist, which increase efficiency and reduce errors, none offer the same familiarity as printed field plans which have been used for decades and offer an intuitive overview over the trial setup, previously recorded data and plots still requiring scoring.ResultsWe introduce GridScore which utilizes cutting-edge web technologies to reproduce the familiarity of printed field plans while enhancing the phenotypic data collection process by adding advanced features like georeferencing, image tagging and speech recognition. GridScore is a cross-platform open-source plant phenotyping app that combines barcode-based systems with a guided data collection approach while offering a top-down view onto the data collected in a field layout. GridScore is compared to existing tools across a wide spectrum of criteria including support for barcodes, multiple platforms, and visualizations.ConclusionCompared to its competition, GridScore shows strong performance across the board offering a complete manual phenotyping experience.

A pediatric wrist trauma X-ray dataset (GRAZPEDWRI-DX) for machine learning

Digital radiography is widely available and the standard modality in trauma imaging, often enabling to diagnose pediatric wrist fractures. However, image interpretation requires time-consuming specialized training. Due to astonishing progress in computer vision algorithms, automated fracture detection has become a topic of research interest. This paper presents the GRAZPEDWRI-DX dataset containing annotated pediatric trauma wrist radiographs of 6,091 patients, treated at the Department for Pediatric Surgery of the University Hospital Graz between 2008 and 2018. A total number of 10,643 studies (20,327 images) are made available, typically covering posteroanterior and lateral projections. The dataset is annotated with 74,459 image tags and features 67,771 labeled objects. We de-identified all radiographs and converted the DICOM pixel data to 16-Bit grayscale PNG images. The filenames and the accompanying text files provide basic patient information (age, sex). Several pediatric radiologists annotated dataset images by placing lines, bounding boxes, or polygons to mark pathologies like fractures or periosteal reactions. They also tagged general image characteristics. This dataset is publicly available to encourage computer vision research.

Salient object detection with image-level binary supervision

Recent deep learning based salient object detection (SOD) methods have achieved impressive performance. However, while fully-supervised methods require a large amount of labeled data, weakly-supervised methods still require a considerable human effort. To address this problem, we propose a novel weakly-supervised method for salient object detection based on only binary image tags, which are much cheaper to collect. Our basic idea is to construct a dataset of images that are labeled as either salient (with salient objects) or non-salient (without salient objects), and leverage such binary labels as supervision to learn a salient object detector based on existing unsupervised methods. In particular, we propose a target saliency map hallucinator, which can synthesize pseudo ground truth saliency maps for the salient images in the training data solely from binary labels. We can then use the pseudo ground truth labels to train a salient object detector. Experimental results show that our method performs comparably to the state-of-the-art weakly-supervised methods, but requires considerably less human supervision.

Mm-Wave Chipless RFID Decoding: Introducing Image-Based Deep Learning Techniques

Chipless RFID tag decoding has some inherent degrees of uncertainty because there is no handshake protocol between chipless tags and readers. This article initially compares the outcome of different pattern recognition methods to decode some frequency-based tags in the mm-wave spectrum. It will be shown that these pattern recognition methods suffer from almost 2&#x0025;&#x2013;5&#x0025; false decoding rate. To overcome this misdecoding problem, two novel methods of making images of the chipless tags are presented. The first method is making 2-D images based on side-looking aperture radar concepts, and the second one is making virtual 2-D images from the 1-D backscattering signals. Then, a 2-D decoding algorithm is suggested based on a convolutional neural network to decode those tag images and compare the results. It is shown that this combined decoding method has very high accuracy, and it almost eliminates any ambiguity and false decoding problems. This is the first time a deep learning method is used with image construction methods to decode chipless RFID tags.