Object Labels Research Articles

Pick-Object-Attack: Type-specific adversarial attack for object detection

Many recent studies have shown that deep neural models are vulnerable to adversarial samples: images with imperceptible perturbations, for example, can fool image classifiers. In this paper, we present the first type-specific approach to generating adversarial examples for object detection, which entails detecting bounding boxes around multiple objects present in the image and classifying them at the same time, making it a harder task than against image classification. We specifically aim to attack the widely used Faster R-CNN by changing the predicted label for a particular object in an image: where prior work has targeted one specific object (a stop sign), we generalize to arbitrary objects, with the key challenge being the need to change the labels of all bounding boxes for all instances of that object type. To do so, we propose a novel method, named Pick-Object-Attack. Pick-Object-Attack successfully adds perturbations only to bounding boxes for the targeted object, preserving the labels of other detected objects in the image. In terms of perceptibility, the perturbations induced by the method are very small. Furthermore, for the first time, we examine the effect of adversarial attacks on object detection in terms of a downstream task, image captioning; we show that where a method that can modify all object types leads to very obvious changes in captions, the changes from our constrained attack are much less apparent.

Multifocal imaging for precise, label-free tracking of fast biological processes in 3D

Many biological processes happen on a nano- to millimeter scale and within milliseconds. Established methods such as confocal microscopy are suitable for precise 3D recordings but lack the temporal or spatial resolution to resolve fast 3D processes and require labeled samples. Multifocal imaging (MFI) allows high-speed 3D imaging but is limited by the compromise between high spatial resolution and large field-of-view (FOV), and the requirement for bright fluorescent labels. Here, we provide an open-source 3D reconstruction algorithm for multi-focal images that allows using MFI for fast, precise, label-free tracking spherical and filamentous structures in a large FOV and across a high depth. We characterize fluid flow and flagellar beating of human and sea urchin sperm with a z-precision of 0.15 µm, in a volume of 240 × 260 × 21 µm, and at high speed (500 Hz). The sampling volume allowed to follow sperm trajectories while simultaneously recording their flagellar beat. Our MFI concept is cost-effective, can be easily implemented, and does not rely on object labeling, which renders it broadly applicable.

Beyond the Shape of Things: Infants Can Be Taught to Generalize Nouns by Objects’ Functions

Two-year-olds typically extend labels of novel objects by the objects’ shape (shape bias), whereas adults do so by the objects’ function. Is this because shape is conceptually easier to comprehend than function? To test whether the conceptual complexity of function prevents infants from developing a function bias, we trained twelve 17-month-olds (function-training group) to focus on objects’ functions when labeling the objects over a period of 7 weeks. Our training was similar to previously used methods in which 17-month-olds were successfully taught to focus on the shape of objects, resulting in a precocious shape bias. We exposed another 12 infants (control group) to the same objects over 7 weeks but without labeling the items or demonstrating their functions. Only the infants in the function-training group developed a function bias. Thus, the conceptual complexity of function was not a barrier for developing a function bias, which suggests that the shape bias emerges naturally because shape is perceptually more accessible than function.

Class-Selective Mini-Batching and Multitask Learning for Visual Relationship Recognition

An image can be described by the objects within it, and interactions between those objects. A pair of object labels together with an interaction label is known as a visual relationship, and is represented as a triplet of the form (subject, predicate, object). Recognising visual relationships in images is a challenging task, owing to the combinatorially large number of possible relationship triplets, which leads to an extreme multiclass classification problem. In addition, the distribution of visual relationships in a dataset tends to be long-tailed, i.e. most triplets occur rarely compared to a small number of dominating triplets. Three strategies to address these issues are investigated. Firstly, instead of predicting the full triplet, models can be trained to predict each of the three elements separately. Secondly a multitask learning strategy is investigated, where shared network parameters are used to perform the three separate predictions. Thirdly, a class-selective mini-batch construction strategy is used to expose the network to more of the rare classes during training. Experiments demonstrate that class-selective mini-batch construction can improve performance on classes in the long tail of the data distribution, possibly at the expense of accuracy on the small number of dominating classes. It is also found that a multitask model neither improves nor impedes performance in any significant way, but that its smaller size may be beneficial. In an effort to better understand the behaviour of the various models, a novel evaluation approach for visual relationship recognition is introduced. We conclude that the use of semantics can be helpful in the modelling and evaluation process.

Multi-Task Deep Learning Model for Classification of Dental Implant Brand and Treatment Stage Using Dental Panoramic Radiograph Images.

It is necessary to accurately identify dental implant brands and the stage of treatment to ensure efficient care. Thus, the purpose of this study was to use multi-task deep learning to investigate a classifier that categorizes implant brands and treatment stages from dental panoramic radiographic images. For objective labeling, 9767 dental implant images of 12 implant brands and treatment stages were obtained from the digital panoramic radiographs of patients who underwent procedures at Kagawa Prefectural Central Hospital, Japan, between 2005 and 2020. Five deep convolutional neural network (CNN) models (ResNet18, 34, 50, 101 and 152) were evaluated. The accuracy, precision, recall, specificity, F1 score, and area under the curve score were calculated for each CNN. We also compared the multi-task and single-task accuracies of brand classification and implant treatment stage classification. Our analysis revealed that the larger the number of parameters and the deeper the network, the better the performance for both classifications. Multi-tasking significantly improved brand classification on all performance indicators, except recall, and significantly improved all metrics in treatment phase classification. Using CNNs conferred high validity in the classification of dental implant brands and treatment stages. Furthermore, multi-task learning facilitated analysis accuracy.

Inferring Camouflaged Objects by Texture-Aware Interactive Guidance Network

Camouflaged objects, similar to the background, show indefinable boundaries and deceptive textures, which increases the difficulty of detection task and makes the model rely on features with more information. Herein, we design a texture label to facilitate our network for accurate camouflaged object segmentation. Motivated by the complementary relationship between texture labels and camouflaged object labels, we propose an interactive guidance framework named TINet, which focuses on finding the indefinable boundary and the texture difference by progressive interactive guidance. It maximizes the guidance effect of refined multi-level texture cues on segmentation. Specifically, texture perception decoder (TPD) makes a comprehensive analysis of texture information in multiple scales. Feature interaction guidance decoder (FGD) interactively refines multi-level features of camouflaged object detection and texture detection level by level. Holistic perception decoder (HPD) enhances FGD results by multi-level holistic perception. In addition, we propose a boundary weight map to help the loss function pay more attention to the object boundary. Sufficient experiments conducted on COD and SOD datasets demonstrate that the proposed method performs favorably against 23 state-of-the-art methods.

태권도 사범의 감정노동 및 직무스트레스 원인 탐색

This study was conducted to investigate the causes of emotional labor and job stress among Taekwondo masters. 139 research participants were collected through objective labeling and data was collected using semi-structured open questionnaires. The collected data was coded using Microsoft office 365 Excel and analyzed through a group of experts to conclude that:<BR> First, the causes of emotional labor recognized by Taekwondo masters were analyzed as deep acting(emotional contagion) and surface acting(emotional restraint, emotional dissonance). Second, it was analyzed that the causes of job stress recognized by Taekwondo masters were interpersonal (interpersonal conflict, communication, trainee ecology), and working environment(time control, work burden, communication).

Automatic Extraction and Labelling of Memorial Objects From 3D Point Clouds

This research addresses the problem of automatic extraction of memorial objects from cultural heritage sites represented as scenes of 3D point clouds. Point clouds provide a fine spatial resolution and accurate proxy of the real world. However, how to use them directly is not always obvious. This is especially true for applications where extensive training data or computational resources are not available. In this paper, we present a methodology for automatic segmentation and labelling of cultural heritage objects from 3D point cloud scenes. The proposed methodology is based on machine learning techniques and, in particular, makes use of the concept of transfer learning. Memorial objects are segmented from the scene based on their geometric shape characteristic through a conditional multi-scale partitioning scheme. Then, high-level latent feature descriptors are extracted by a convolutional neural network pre-trained on different 3D object models from a standard dataset (e.g., ModelNet). Based on these descriptors, a classification model (multilayer perceptron) is trained and applied to obtain semantic labels. Experiments demonstrated that the proposed methodology is effective for the extraction and labelling of grave marker objects from cultural heritage sites.

Multi-modal 3D object detection by 2D-guided precision anchor proposal and multi-layer fusion

3D object detection, of which the goal is to obtain the 3D spatial structure information of the object, is a challenging topic in many visual perception systems, e.g., autonomous driving, augmented reality, and robot navigation. Most existing region proposal network (RPN) based 3D object detection methods generate anchors in the whole 3D searching space without using semantic information, which leads to the problem of inappropriate anchor size generation. To tackle the issue, we propose a 2D-guided precision anchor generation network (PAG-Net). Specifically speaking, we utilize a mature 2D detector to get 2D bounding boxes and category labels of objects as prior information. Then the 2D bounding boxes are projected into 3D frustum space for more precise and category-adaptive 3D anchors. Furthermore, current feature combination methods are early fusion, late fusion, and deep fusion, which only fuse features from high convolutional layers and ignore the data missing problem of point clouds. To obtain more efficient fusion of RGB images and point clouds features, we propose a multi-layer fusion model, which conducts nonlinear and iterative combinations of features from multiple convolutional layers and merges the global and local features effectively. We encode point cloud with the bird’s eye view (BEV) representation to solve the irregularity of point cloud. Experimental results show that our proposed approach improves the baseline by a large margin and outperforms most of the state-of-the-art methods on the KITTI object detection benchmark.

A distributional perspective on the gavagai problem in early word learning

Word learning entails the mapping of an auditory word-form to its appropriate grammatical category (e.g., noun, verb, adjective), but before that mapping can occur, the naïve learner must infer which of the myriad of possible referents of that word was intended by the speaker. This creates a computational explosion of referential ambiguity referred to as the gavagai problem. In a set of corpus analyses of parent-directed speech to young infants, we describe the distributional information available to early word learners, with a focus on nouns and adjectives that refer to whole objects and object properties. And in two experiments on word-learning in adults spanning seven different distributional conditions, we document how variations in the ratio of novel labels for objects and properties affect the robustness of word learning. Our results suggest that the language input to 6- to 20-month-olds is robustly populated with high-frequency object words and high-frequency property words, but their co-occurrence is sparse. Although this distributional information slightly favors object words over property words, a more plausible account of the whole-object bias in early word learning is the inability to encode the details of an object/event during rapid naming. Our results from adults, presented with novel labels for multi-referent objects in a cross-situational statistical learning paradigm, also reveal this whole-object bias as well as the absence of property-label generalization to novel objects, even when the distribution of labels is shifted almost exclusively to property words. These results are discussed in terms of the relative ease of mapping auditory word-forms to whole objects vs. object properties, thereby limiting the combinatorics of the gavagai problem, especially in infants with immature encoding and memory representation abilities.

Video object detection from one single image through opto-electronic neural network

An opto-electronic neural network is designed for video object detection from a long-exposure blurred image. This network combines an optical encoder, convolutional neural network decoder, and object detection module, which are jointly optimized end-to-end. The joint loss is adopted for updating the network according to the physical constraints of hardware via back-propagation. A high-speed refreshed spatial light modulator is used as the encoder part of the network to generate coded sub-images, and then, a single blurred image is obtained after a common camera. The rest of the network is used for video object detection. Both simulations and experiments demonstrate that our framework can successfully retrieve object labels and bounding boxes at different moments in the long exposure. To the best of our knowledge, this is the first work investigating video object detection from a single motion-degraded image.

Prediction error boosts retention of novel words in adults but not in children

How do we update our linguistic knowledge? In seven experiments, we asked whether error-driven learning can explain under what circumstances adults and children are more likely to store and retain a new word meaning. Participants were exposed to novel object labels in the context of more or less constraining sentences or visual contexts. Both two-to-four-year-olds (Mage = 38 months) and adults were strongly affected by expectations based on sentence constraint when choosing the referent of a new label. In addition, adults formed stronger memory traces for novel words that violated a stronger prior expectation. However, preschoolers' memory was unaffected by the strength of their prior expectations. We conclude that the encoding of new word-object associations in memory is affected by prediction error in adults, but not in preschoolers.

Relation Regularized Scene Graph Generation

Scene graph generation (SGG) is built on top of detected objects to predict object pairwise visual relations for describing the image content abstraction. Existing works have revealed that if the links between objects are given as prior knowledge, the performance of SGG is significantly improved. Inspired by this observation, in this article, we propose a relation regularized network (R2-Net), which can predict whether there is a relationship between two objects and encode this relation into object feature refinement and better SGG. Specifically, we first construct an affinity matrix among detected objects to represent the probability of a relationship between two objects. Graph convolution networks (GCNs) over this relation affinity matrix are then used as object encoders, producing relation-regularized representations of objects. With these relation-regularized features, our R2-Net can effectively refine object labels and generate scene graphs. Extensive experiments are conducted on the visual genome dataset for three SGG tasks (i.e., predicate classification, scene graph classification, and scene graph detection), demonstrating the effectiveness of our proposed method. Ablation studies also verify the key roles of our proposed components in performance improvement.

Learning Graph Convolutional Networks for Multi-Label Recognition and Applications.

The task of multi-label image recognition is to predict a set of object labels that present in an image. As objects normally co-occur in an image, it is desirable to model the label dependencies to improve the recognition performance. To capture and explore such important information, we propose graph convolutional networks (GCNs) based models for multi-label image recognition, where directed graphs are constructed over classes and information is propagated between classes to learn inter-dependent class-level representations. Following this idea, we design two particular models that approach multi-label classification from different views. In our first model, the prior knowledge about the class dependencies is integrated into classifier learning. Specifically, we propose Classifier Learning GCN (C-GCN) to map class-level semantic representations (e.g., word embeddings) into classifiers that maintain the inter-class topology. In our second model, we decompose the visual representation of an image into a set of label-aware features and propose prediction learning GCN (P-GCN) to encode such features into inter-dependent image-level prediction scores. Furthermore, we also present an effective correlation matrix construction approach to capture inter-class relationships and consequently guide information propagation among classes. Empirical results on generic multi-label image recognition demonstrate that both of the proposed models can obviously outperform other existing state-of-the-arts. Moreover, the proposed methods also show advantages in some other multi-label classification related applications.

An efficient sampling-based visualization technique for big data clustering with crisp partitions

The data cluster tendency is an emerging need for exploring the big data cluster analysis tasks. The data are evaluated based on the number of clusters is known as cluster tendency. Many visualization techniques have been developed for the detection of cluster tendency. Some of the existing techniques include Visual Assessment Tendency (VAT), spectral-based VAT (SpecVAT), and improved VAT (iVAT), are considerably succeeded for an assessment of cluster tendency for small datasets. A bigVAT is another method that was recently developed for the estimation of cluster tendency of big data. It is perfect for deriving the clustering tendency in visual form for big data. However, it is intractable to explore the data clusters for large volumes of data objects. The proposed work addresses the clustering problem of bigVAT with the derivation of sampling-based crisp partitions. The crisp partitions will accurately predict the cluster labels of data objects. This research is based on big synthetic and big real-life datasets for demonstrating the performance efficiency of the proposed work.

Multivariate time-series clustering based on component relationship networks

Clustering is a powerful technique for providing class labels of data objects for learning guidance. Because traditional clustering methods rarely consider the component correlations of a multivariate time series(MTS), an MTS clustering method based on a component relationship network (CRN) is proposed in the present study. An MTS dataset is mapped to a multi-relationship network (MRN), which consists of a set of CRNs. Every CRN reflects the relationship of the MTS data under each component. The proposed method applies two steps. First, the distance function and K-nearest neighbors are combined to transform an MTS dataset into an MRN. Second, a non-negative matrix factorization is designed to identify time-series clusters. Because asynchronous time-series data exist in the form of an approximate case, we recommend an improved penalty-coefficient based dynamic time-warping algorithm to measure the similarity between two-time sequences. The similarity can reflect the correlation between the asynchronous MTS data. Through an experiment, we compared the proposed method to other clustering methods and discussed of the effects of the parameters in detail. The numerical results of the experiment demonstrate that the proposed method can improve the accuracy and quality of MTS data clustering.

An algorithm for labels aggregation in taxonomy-based crowd-labeling

Crowdsourcing provides a convenient solution for many information processing problems that are still hard or even intractable by modern AI techniques, but are relatively simple for many people. However, complete crowdsourcing solution cannot go by without a quality control mechanisms, as the results received from participants are not always reliable. The paper considers taxonomy-based crowd-labeling - a form of crowdsourcing, in which participants label objects with tags, and there exists an explicit taxonomy relation on the set of tags. We propose a method and an algorithm for label aggregation, allowing to estimate the likelihood of the true object label from a set of noisy labels received from the crowd, and to estimate the expected crowd members’ accuracy. The proposed method and algorithm can be used in a wide range of crowd-labeling applications (e.g., classification of scientific literature collections, software repositories, etc.).

Perceptual Connectivity Influences Toddlers’ Attention to Known Objects and Subsequent Label Processing

While recent research suggests that toddlers tend to learn word meanings with many “perceptual” features that are accessible to the toddler’s sensory perception, it is not clear whether and how building a lexicon with perceptual connectivity supports attention to and recognition of word meanings. We explore this question in 24–30-month-olds (N = 60) in relation to other individual differences, including age, vocabulary size, and tendencies to maintain focused attention. Participants’ looking to item pairs with high vs. low perceptual connectivity—defined as the number of words in a child’s lexicon sharing perceptual features with the item—was measured before and after target item labeling. Results revealed pre-labeling attention to known items is biased to both high- and low-connectivity items: first to high, and second, but more robustly, to low-connectivity items. Subsequent object–label processing was also facilitated for high-connectivity items, particularly for children with temperamental tendencies to maintain focused attention. This work provides the first empirical evidence that patterns of shared perceptual features within children’s known vocabularies influence both visual and lexical processing, highlighting the potential for a newfound set of developmental dependencies based on the perceptual/sensory structure of early vocabularies.

3-D Relation Network for visual relation recognition in videos

Video visual relation recognition aims at mining the dynamic relation instances between objects in the form of 〈subject,predicate,object〉, such as “person1-towards-person2” and “person-ride-bicycle”. Existing solutions treat the problem as several independent sub-tasks, i.e., image object detection, video object tracking and trajectory-based relation prediction. We argue that such separation results in the lack of information flow between different sub-models, which creates redundant representation while each sub-task cannot share a common set of task-specific features. Toward this end, we connect these three sub-tasks in an end-to-end manner by proposing the 3-D relation proposal that serves as a bridge for relation feature learning. Specifically, we put forward a novel deep neural network, named 3DRN, to fuse the spatio-temporal visual characteristics, object label features, and spatial interactive features for learning the relation instances with multi-modal cues. In addition, a three-staged training strategy is also provided to facilitate large-scale parameter optimization. We conduct extensive experiments on two public datasets with different emphasis to demonstrate the effectiveness of the proposed end-to-end feature learning method for visual relation recognition in videos. Furthermore, we verify the potential of our approach by tackling the video relation detection task.

Manifold Preserving CNN for Pixel-Based Object Labelling in Images for High Dimensional Feature spaces

Deep CNN’s have achieved an excellent performance in computer vision and image processing methods, designating them as a state-of-art in this domain. CNN based applications have achieved tremendous advancement towards vision computing with high dimensional object labelling in images. The complex nature of High Dimensional (HD) images limits the performance of CNN’s. In high dimensional feature space, the pixel-based image labelling is a complex problem for the parsing of objects in an image. To overcome this issue, we have studied a two-stage end-to-end framework that uses manifold embedding based patch-wise CNN architecture to extract the features and classify the image for labelled classes. We have investigated the deep-features with an information fusion technique for low dimensional feature space compression by using pre-trained CNNs and spatiality preserving manifold embedding in the first stage. The cost of pixel-based labelling in HD feature space is very high, so researchers have tried to encapsulate maximum information within the minimum image size. Therefore, in this stage, we have first increased the valuable information by concatenating the deep spatial features and then embedding the massive dataset by using manifold preservation. In stage-2, the image patches are extracted and passed into three layers of convolution-pooling pair and two layers of fully connected pair using parameter tuning. The training dataset is prepared in the form of pixel-label pairs. Subsequently, the proposed method has been evaluated on publicly available images and compared with the previously proposed schemes. The proposed method has outperformed the previous techniques in accuracy and computation time with a significant margin.