One-shot Learning Research Articles

Interactive Trimap Generation for Digital Matting Based on Single-Sample Learning

Image matting refers to the task of estimating the foreground of images, which is an important problem in image processing. Recently, trimap generation has attracted considerable attention because designing a trimap for every image is labor-intensive. In this paper, a two-step algorithm is proposed to generate trimaps. To use the proposed algorithm, users must only provide some clicks (foreground clicks and background clicks), which are employed as the input to generate a binary mask. One-shot learning technique achieves remarkable progress on semantic segmentation, we extend this technique to perform the binary mask prediction task. The mask is further used to predict the trimap using image dilation. Extensive experiments were performed to evaluate the proposed algorithm. Experimental results show that the trimaps generated using the proposed algorithm are visually similar to the user-annotated ones. Comparing with the interactive matting algorithms, the proposed algoritm is less labor-intensive than trimap-based matting algorithm and achieved more accuate results than scribble-based matting algorithm.

Tracklet Self-Supervised Learning for Unsupervised Person Re-Identification

Existing unsupervised person re-identification (re-id) methods mainly focus on cross-domain adaptation or one-shot learning. Although they are more scalable than the supervised learning counterparts, relying on a relevant labelled source domain or one labelled tracklet per person initialisation still restricts their scalability in real-world deployments. To alleviate these problems, some recent studies develop unsupervised tracklet association and bottom-up image clustering methods, but they still rely on explicit camera annotation or merely utilise suboptimal global clustering. In this work, we formulate a novel tracklet self-supervised learning (TSSL) method, which is capable of capitalising directly from abundant unlabelled tracklet data, to optimise a feature embedding space for both video and image unsupervised re-id. This is achieved by designing a comprehensive unsupervised learning objective that accounts for tracklet frame coherence, tracklet neighbourhood compactness, and tracklet cluster structure in a unified formulation. As a pure unsupervised learning re-id model, TSSL is end-to-end trainable at the absence of source data annotation, person identity labels, and camera prior knowledge. Extensive experiments demonstrate the superiority of TSSL over a wide variety of the state-of-the-art alternative methods on four large-scale person re-id benchmarks, including Market-1501, DukeMTMC-ReID, MARS and DukeMTMC-VideoReID.

Automated Facial Recognition in Older Photographs Using One-Shot Learning in Siamese Networks and Transfer Learning

A lot of historical information comes in various forms, such as old documents, papers, photographs, videos, audio, and even artefacts and sculptures. Photographs, audio, and videos are especially important because they effectively convey information. When we convert these into digital versions, it becomes easy to share, access online or offline, copy, move around, back up, and store in numerous places. However, a challenge with digital content is that it is often difficult to search due to the absence of readable text. Consequently, we cannot effectively analyse and utilise critical information. To make it useful, we manually look at pictures and add tags to create labels. While basic labels suffice for most searches, it becomes more complicated when dealing with a large number of photographs. Enhancements in search capabilities are needed to make the process easier, quicker, and more efficient. Fortunately, recent technological advances, such as artificial intelligence, provide us with facilities to simplify this process. This paper explores how artificial intelligence can streamline this process, enhancing search efficiency and enabling automatic identification and tagging of individuals in photos, thus facilitating easier access and analysis of digital archives.It is anticipated that manual tagging efforts could be reduced by approximately 80%, and the searchability of photographs could be enhanced by about 84%.

Feature Deformation Meta-Networks in Image Captioning of Novel Objects

This paper studies the task of image captioning with novel objects, which only exist in testing images. Intrinsically, this task can reflect the generalization ability of models in understanding and captioning the semantic meanings of visual concepts and objects unseen in training set, sharing the similarity to one/zero-shot learning. The critical difficulty thus comes from that no paired images and sentences of the novel objects can be used to help train the captioning model. Inspired by recent work (Chen et al. 2019b) that boosts one-shot learning by learning to generate various image deformations, we propose learning meta-networks for deforming features for novel object captioning. To this end, we introduce the feature deformation meta-networks (FDM-net), which is trained on source data, and learn to adapt to the novel object features detected by the auxiliary detection model. FDM-net includes two sub-nets: feature deformation, and scene graph sentence reconstruction, which produce the augmented image features and corresponding sentences, respectively. Thus, rather than directly deforming images, FDM-net can efficiently and dynamically enlarge the paired images and texts by learning to deform image features. Extensive experiments are conducted on the widely used novel object captioning dataset, and the results show the effectiveness of our FDM-net. Ablation study and qualitative visualization further give insights of our model.

One-Shot Image Classification by Learning to Restore Prototypes

One-shot image classification aims to train image classifiers over the dataset with only one image per category. It is challenging for modern deep neural networks that typically require hundreds or thousands of images per class. In this paper, we adopt metric learning for this problem, which has been applied for few- and many-shot image classification by comparing the distance between the test image and the center of each class in the feature space. However, for one-shot learning, the existing metric learning approaches would suffer poor performance because the single training image may not be representative of the class. For example, if the image is far away from the class center in the feature space, the metric-learning based algorithms are unlikely to make correct predictions for the test images because the decision boundary is shifted by this noisy image. To address this issue, we propose a simple yet effective regression model, denoted by RestoreNet, which learns a class agnostic transformation on the image feature to move the image closer to the class center in the feature space. Experiments demonstrate that RestoreNet obtains superior performance over the state-of-the-art methods on a broad range of datasets. Moreover, RestoreNet can be easily combined with other methods to achieve further improvement.

One-Shot Learning for Long-Tail Visual Relation Detection

The aim of visual relation detection is to provide a comprehensive understanding of an image by describing all the objects within the scene, and how they relate to each other, in < object-predicate-object > form; for example, < person-lean on-wall > . This ability is vital for image captioning, visual question answering, and many other applications. However, visual relationships have long-tailed distributions and, thus, the limited availability of training samples is hampering the practicability of conventional detection approaches. With this in mind, we designed a novel model for visual relation detection that works in one-shot settings. The embeddings of objects and predicates are extracted through a network that includes a feature-level attention mechanism. Attention alleviates some of the problems with feature sparsity, and the resulting representations capture more discriminative latent features. The core of our model is a dual graph neural network that passes and aggregates the context information of predicates and objects in an episodic training scheme to improve recognition of the one-shot predicates and then generate the triplets. To the best of our knowledge, we are the first to center on the viability of one-shot learning for visual relation detection. Extensive experiments on two newly-constructed datasets show that our model significantly improved the performance of two tasks PredCls and SGCls from 2.8% to 12.2% compared with state-of-the-art baselines.

Few-Shot Knowledge Graph Completion

Knowledge graphs (KGs) serve as useful resources for various natural language processing applications. Previous KG completion approaches require a large number of training instances (i.e., head-tail entity pairs) for every relation. The real case is that for most of the relations, very few entity pairs are available. Existing work of one-shot learning limits method generalizability for few-shot scenarios and does not fully use the supervisory information; however, few-shot KG completion has not been well studied yet. In this work, we propose a novel few-shot relation learning model (FSRL) that aims at discovering facts of new relations with few-shot references. FSRL can effectively capture knowledge from heterogeneous graph structure, aggregate representations of few-shot references, and match similar entity pairs of reference set for every relation. Extensive experiments on two public datasets demonstrate that FSRL outperforms the state-of-the-art.

Learning One-Shot Imitation From Humans Without Humans

Humans can naturally learn to execute a new task by seeing it performed by other individuals once, and then reproduce it in a variety of configurations. Endowing robots with this ability of imitating humans from third person is a very immediate and natural way of teaching new tasks. Only recently, through meta-learning, there have been successful attempts to one-shot imitation learning from humans; however, these approaches require a lot of human resources to collect the data in the real world to train the robot. But is there a way to remove the need for real world human demonstrations during training? We show that with Task-Embedded Control Networks, we can infer control polices by embedding human demonstrations that can condition a control policy and achieve one-shot imitation learning. Importantly, we do not use a real human arm to supply demonstrations during training, but instead leverage domain randomisation in an application that has not been seen before: sim-to-real transfer on humans. Upon evaluating our approach on pushing and placing tasks in both simulation and in the real world, we show that in comparison to a system that was trained on real-world data we are able to achieve similar results by utilising only simulation data. Videos can be found here: https://sites.google.com/view/tecnets-humans.

Method for Whale Re-identification Based on Siamese Nets and Adversarial Training

Training Convolutional Neural Networks that do well in one-shot learning settings can have wide range of impacts on real-world datasets. In this paper, we explore an adversarial training method that learns a Siamese neural network in an end-to-end fashion for two models—ConvNets model that learns image embeddings from input image pair, and head model that further learns the distance between those embeddings. We further present an adversarial mining approach that efficiently identifies and selects harder examples during training, which significantly boosts the model performance over difficult cases. We have done a comprehensive evaluation using a public whale identification dataset hosted on Kaggle platform, and report state-of-the-art performance benchmarking against result of other 2000 participants.

Deep reinforcement one-shot learning for artificially intelligent classification in expert aided systems

In recent years there has been a sharp rise in applications, in which significant events need to be classified but only a few training instances are available. These are known as cases of one-shot learning. To handle this challenging task, organizations often use human analysts to classify events under high uncertainty. Existing algorithms use a threshold-based mechanism to decide whether to classify an object automatically or send it to an analyst for deeper inspection. However, this approach leads to a significant waste of resources since it does not take the practical temporal constraints of system resources into account. By contrast, the focus in this paper is on rigorously optimizing the resource consumption in the system which applies to broad application domains, and is of a significant interest for academic research, industrial developments, as well as society and citizens benefit. The contribution of this paper is threefold. First, a novel Deep Reinforcement One-shot Learning (DeROL) framework is developed to address this challenge. The basic idea of the DeROL algorithm is to train a deep-Q network to obtain a policy which is oblivious to the unseen classes in the testing data. Then, in real-time, DeROL maps the current state of the one-shot learning process to operational actions based on the trained deep-Q network, to maximize the objective function. Second, the first open-source software for practical artificially intelligent one-shot classification systems with limited resources is developed for the benefit of researchers and developers in related fields. Third, an extensive experimental study is presented using the OMNIGLOT dataset for computer vision tasks, the UNSW-NB15 dataset for intrusion detection tasks, and the Cleveland Heart Disease Dataset for medical monitoring tasks that demonstrates the versatility and efficiency of the DeROL framework.

One-shot learning of view-invariant object representations in newborn chicks

Can newborn brains perform one-shot learning? To address this question, we reared newborn chicks in strictly controlled environments containing a single view of a single object, then tested their object recognition performance across 24 uniformly-spaced viewpoints. We found that chicks can build view-invariant object representations from a single view of an object: a case of one-shot learning in newborn brains. Chicks can also build the same view-invariant object representation from different views of an object, showing that newborn brains converge on common object representations from different sets of sensory inputs. Finally, by rearing chicks with larger numbers of object views, we found that chicks develop enhanced recognition for familiar views. These results illuminate the earliest stages of object recognition, revealing (1) powerful one-shot learning that builds invariant object representations from the first views of an object and (2) view-based learning that enriches object representations, producing enhanced recognition for familiar views.

Using a Convolutional Siamese Network for Image-Based Plant Species Identification with Small Datasets.

The application of deep learning techniques may prove difficult when datasets are small. Recently, techniques such as one-shot learning, few-shot learning, and Siamese networks have been proposed to address this problem. In this paper, we propose the use a convolutional Siamese network (CSN) that learns a similarity metric that discriminates between plant species based on images of leaves. Once the CSN has learned the similarity function, its discriminatory power is generalized to classify not just new pictures of the species used during training but also entirely new species for which only a few images are available. This is achieved by exposing the network to pairs of similar and dissimilar observations and minimizing the Euclidean distance between similar pairs while simultaneously maximizing it between dissimilar pairs. We conducted experiments to study two different scenarios. In the first one, the CSN was trained and validated with datasets that comprise 5, 10, 15, 20, 25, and 30 pictures per species, extracted from the well-known Flavia dataset. Then, the trained model was tested with another dataset composed of 320 images (10 images per species) also from Flavia. The obtained accuracy was compared with the results of feeding the same training, validation, and testing datasets to a convolutional neural network (CNN) in order to determine if there is a threshold value t for dataset size that defines the intervals for which either the CSN or the CNN has better accuracy. In the second studied scenario, the accuracy of both the CSN and the CNN—both trained and validated with the same datasets extracted from Flavia—were compared when tested on a set of images of leaves of 20 Costa Rican tree species that are not represented in Flavia.

How stable is perception in #TheDress and #TheShoe?

#TheDress is perceived by some people as black and blue while others perceive it as white and gold. We have previously shown that the first encounter with #TheDress strongly biases its perception. This percept remained stable during the experiment, suggesting a role of one-shot learning. #TheShoe is another image that elicits similar bimodal color percepts. Here, we investigated how percepts change over time in both #TheShoe and #TheDress. First, we show that the important role of one-shot learning, which we found for #TheDress extends to #TheShoe. Similarly to our previous results with the dress, hiding large parts of the image with occluders biased the percept of the shoe. The percept did not change for the majority of observers when the occluders were removed. Second, we investigated if and how percepts switch over a time course of 14 days. We found that although some observers experienced percept switches, the percept was largely stable for most observers.

Memories of control: One-shot episodic learning of item-specific stimulus-control associations

The repeated pairing of a particular stimulus with a specific cognitive control process, such as task switching, can bind the two together in memory, resulting in the formation of stimulus-control associations. These bindings are thought to guide the context-sensitive application of cognitive control, but it is not presently known whether such stimulus-control associations are only acquired through slow, incremental learning or could also be mediated by episodic memories of a single experience, so-called one-shot learning. Here, we tested this episodic control-binding hypothesis by probing whether a single co-occurrence of a stimulus and the control process of task switching would lead to significant performance benefits (reduced task switch cost) when that stimulus later re-occurred under the same as opposed to different control demands. Across three experiments, we demonstrate that item-specific stimulus-control associations can be formed based on a single exposure, providing the first strong evidence for episodic memory guidance of cognitive control.

A Bayesian Approach to the Naming Game Model

We present a novel Bayesian approach to semiotic dynamics, which is a cognitive analog of the naming game model restricted to two conventions. The model introduced in this paper provides a general framework for studying the combined effects of cognitive and social dynamics. The one-shot learning that characterizes the agent dynamics in the basic naming game is replaced by a word-learning process in which agents learn a new word by generalizing from the evidence garnered through pairwise-interactions with other agents. The principle underlying the model is that agents—like humans—can learn from a few positive examples and that such a process is modeled in a Bayesian probabilistic framework. We show that the model presents some analogies with the basic two-convention naming game model but also some relevant differences in the dynamics, which we explain through a geometric analysis of the mean-field equations.

One-Shot Learning for Deformable Medical Image Registration and Periodic Motion Tracking

Deformable image registration is a very important field of research in medical imaging. Recently multiple deep learning approaches were published in this area showing promising results. However, drawbacks of deep learning methods are the need for a large amount of training datasets and their inability to register unseen images different from the training datasets. One shot learning comes without the need of large training datasets and has already been proven to be applicable to 3D data. In this work we present a one shot registration approach for periodic motion tracking in 3D and 4D datasets. When applied to a 3D dataset the algorithm calculates the inverse of the registration vector field simultaneously. For registration we employed a U-Net combined with a coarse to fine approach and a differential spatial transformer module. The algorithm was thoroughly tested with multiple 4D and 3D datasets publicly available. The results show that the presented approach is able to track periodic motion and to yield a competitive registration accuracy. Possible applications are the use as a stand-alone algorithm for 3D and 4D motion tracking or in the beginning of studies until enough datasets for a separate training phase are available.

Author Correction: Ferroelectric ternary content-addressable memory for one-shot learning

Author Correction: Ferroelectric ternary content-addressable memory for one-shot learning

Scale and translation-invariance for novel objects in human vision

Though the range of invariance in recognition of novel objects is a basic aspect of human vision, its characterization has remained surprisingly elusive. Here we report tolerance to scale and position changes in one-shot learning by measuring recognition accuracy of Korean letters presented in a flash to non-Korean subjects who had no previous experience with Korean letters. We found that humans have significant scale-invariance after only a single exposure to a novel object. The range of translation-invariance is limited, depending on the size and position of presented objects. To understand the underlying brain computation associated with the invariance properties, we compared experimental data with computational modeling results. Our results suggest that to explain invariant recognition of objects by humans, neural network models should explicitly incorporate built-in scale-invariance, by encoding different scale channels as well as eccentricity-dependent representations captured by neurons’ receptive field sizes and sampling density that change with eccentricity. Our psychophysical experiments and related simulations strongly suggest that the human visual system uses a computational strategy that differs in some key aspects from current deep learning architectures, being more data efficient and relying more critically on eye-movements.

Object Detection-Based One-Shot Imitation Learning with an RGB-D Camera

End-to-end robot learning has achieved a great success for robots to obtain various manipulation skills. It learns a function which maps visual information to robotic action directly. Because of the diversity of target objects, most end-to-end robot learning approaches have focused on a single object-specific task with a limited capability of generalization. In this work, an object detection-based one-shot learning method is proposed, which separates the semantic understanding from robot control. It enables a robot to acquire similar manipulation skills efficiently and to have the ability to cope with new objects with a single demonstration. This approach mainly has two modules: the object detection network and the motion policy network. With RGB images, the object detection network tries to output the task-related semantic keypoint of the target object, which is the center of the container in this application, and the motion policy network generates the motion action based on the depth map and the detected keypoint. To evaluate this proposed pipeline, a series of experiments are conducted on typical placing tasks in different simulation scenarios and, additionally, the learned policy is transferred from simulation to the real world without any fine-tuning.

One-Shot Recognition of Manufacturing Defects in Steel Surfaces

Quality control is an essential process in manufacturing to make the product defect-free as well as to meet customer needs. The automation of this process is important to maintain high quality along with the high manufacturing throughput. With recent developments in deep learning and computer vision technologies, it has become possible to detect various features from the images with near-human accuracy. However, many of these approaches are data intensive. Training and deployment of such a system on manufacturing floors may become expensive and time-consuming. The need for large amounts of training data is one of the limitations of the applicability of these approaches in real-world manufacturing systems. In this work, we propose the application of a Siamese convolutional neural network to do one-shot recognition for such a task. Our results demonstrate how one-shot learning can be used in quality control of steel by identification of defects on the steel surface. This method can significantly reduce the requirements of training data and can also be run in real-time.