Triplet Learning Research Articles

Ranking-based triplet loss function with intra-class mean and variance for fine-grained classification tasks

This paper proposed a deep ranking model for triplet selection to efficiently learn similarity metric from top ranked images. A modified distance criterion described in the current work leverages the intra-category variance in metric learning of a triplet network by learning a local sample structure. A multicolumn fusion architecture is used to capture different levels of variance, which when incorporated in the loss function strengthens it and optimizes the objective of the triplet networks. This enables a fine-grained classification strategy. State-of-the-art techniques use a group-sensitive triplet sampling to deal with this issue. However, these have the disadvantage of increased group sampling computations. Experiments are conducted over a variety of benchmark datasets including Model40, PatternNet, and In-Shop Clothing. The main purpose of these experiments are to verify whether the triplet learning technique can be applied over different kinds of data. Results demonstrate that the current work provides superior results in most cases. These results can further be improved with specific parameter tunings and ensembling techniques wherever applicable.

Visual statistical learning and orthographic awareness in Chinese children with and without developmental dyslexia

This study examined the role of visual statistical learning in reading and writing and its relationship to orthographic awareness in Hong Kong Chinese children with and without developmental dyslexia. Thirty-five 7- to 8-year-old children with developmental dyslexia and 37 chronologically age-matched controls were tested on visual statistical learning, orthographic awareness, nonverbal cognitive ability, Chinese word reading, and word dictation tasks. Visual statistical learning was assessed using a triplet learning paradigm that required children to detect the temporal order of visual stimuli. Orthographic awareness was measured with a novel character invention task that required children to create pseudocharacters using untaught stroke patterns according to the rules of Chinese character orthography. Children with dyslexia performed significantly worse than their age-matched controls on both the visual statistical learning and orthographic awareness tasks. Furthermore, visual statistical learning was significantly associated with orthographic awareness and word reading. These findings suggest that Chinese children with dyslexia are impaired in visual statistical learning and that such deficits may be related to disrupted orthographic learning abilities, thereby contributing to their reading difficulties.

A Probabilistic Zero-Shot Learning Method via Latent Nonnegative Prototype Synthesis of Unseen Classes.

Zero-shot learning (ZSL), a type of structured multioutput learning, has attracted much attention due to its requirement of no training data for target classes. Conventional ZSL methods usually project visual features into semantic space and assign labels by finding their nearest prototypes. However, this type of nearest neighbor search (NNS)-based method often suffers from great performance degradation because of the nonuniform variances between different categories. In this article, we propose a probabilistic framework by taking covariance into account to deal with the above-mentioned problem. In this framework, we define a new latent space, which has two characteristics. The first is that the features in this space should gather within the classes and scatter between the classes, which is implemented by triplet learning; the second is that the prototypes of unseen classes are synthesized with nonnegative coefficients, which are generated by nonnegative matrix factorization (NMF) of relations between the seen classes and the unseen classes in attribute space. During training, the learned parameters are the projection model for triplet network and the nonnegative coefficients between the unseen classes and the seen classes. In the testing phase, visual features are projected into latent space and assigned with the labels that have the maximum probability among unseen classes for classic ZSL or within all classes for generalized ZSL. Extensive experiments are conducted on four popular data sets, and the results show that the proposed method can outperform the state-of-the-art methods in most circumstances.

Deep learning for patient-specific quality assurance: Identifying errors in radiotherapy delivery by radiomic analysis of gamma images with convolutional neural networks.

Patient-specific quality assurance (QA) for intensity-modulated radiation therapy (IMRT) is a ubiquitous clinical procedure, but conventional methods have often been criticized as being insensitive to errors or less effective than other common physics checks. Recently, there has been interest in the application of radiomics, quantitative extraction of image features, to radiotherapy QA. In this work, we investigate a deep learning approach to classify the presence or absence of introduced radiotherapy treatment delivery errors from patient-specific QA. Planar dose maps from 186 IMRT beams from 23 IMRT plans were evaluated. Each plan was transferred to a cylindrical phantom CT geometry. Three sets of planar doses were exported from each plan corresponding to (a) the error-free case, (b) a random multileaf collimator (MLC) error case, and (c) a systematic MLC error case. Each plan was delivered to the electronic portal imaging device (EPID), and planned and measured doses were used to calculate gamma images in an EPID dosimetry software package (for a total of 558 gamma images). Two radiomic approaches were used. In the first, a convolutional neural network with triplet learning was used to extract image features from the gamma images. In the second, a handcrafted approach using texture features was used. The resulting metrics from both approaches were input into four machine learning classifiers (support vector machines, multilayer perceptrons, decision trees, and k-nearest-neighbors) in order to determine whether images contained the introduced errors. Two experiments were considered: the two-class experiment classified images as error-free or containing any MLC error, and the three-class experiment classified images as error-free, containing a random MLC error, or containing a systematic MLC error. Additionally, threshold-based passing criteria were calculated for comparison. In total, 303 gamma images were used for model training and 255 images were used for model testing. The highest classification accuracy was achieved with the deep learning approach, with a maximum accuracy of 77.3% in the two-class experiment and 64.3% in the three-class experiment. The performance of the handcrafted approach with texture features was lower, with a maximum accuracy of 66.3% in the two-class experiment and 53.7% in the three-class experiment. Variability between the results of the four machine learning classifiers was lower for the deep learning approach vs the texture feature approach. Both radiomic approaches were superior to threshold-based passing criteria. Deep learning with convolutional neural networks can be used to classify the presence or absence of introduced radiotherapy treatment delivery errors from patient-specific gamma images. The performance of the deep learning network was superior to a handcrafted approach with texture features, and both radiomic approaches were better than threshold-based passing criteria. The results suggest that radiomic QA is a promising direction for clinical radiotherapy.

Visual Statistical Learning With Stimuli Presented Sequentially Across Space and Time in Deaf and Hearing Adults.

This study investigated visual statistical learning (VSL) in 24 deaf signers and 24 hearing non-signers. Previous research with hearing individuals suggests that SL mechanisms support literacy. Our first goal was to assess whether VSL was associated with reading ability in deaf individuals, and whether this relation was sustained by a link between VSL and sign language skill. Our second goal was to test the Auditory Scaffolding Hypothesis, which makes the prediction that deaf people should be impaired in sequential processing tasks. For the VSL task, we adopted a modified version of the triplet learning paradigm, with stimuli presented sequentially across space and time. Results revealed that measures of sign language skill (sentence comprehension/repetition) did not correlate with VSL scores, possibly due to the sequential nature of our VSL task. Reading comprehension scores (PIAT-R) were a significant predictor of VSL accuracy in hearing but not deaf people. This finding might be due to the sequential nature of the VSL task and to a less salient role of the sequential orthography-to-phonology mapping in deaf readers compared to hearing readers. The two groups did not differ in VSL scores. However, when reading ability was taken into account, VSL scores were higher for the deaf group than the hearing group. Overall, this evidence is inconsistent with the Auditory Scaffolding Hypothesis, suggesting that humans can develop efficient sequencing abilities even in the absence of sound.

Commentary: Musicians' Online Performance during Auditory and Visual Statistical Learning Tasks.

A commentary on: Musicians' Online Performance during Auditory and Visual Statistical Learning Tasks by Mandikal Vasuki, P. R., Sharma, M., Ibrahim, R. K., and Arciuli, J. (2017). Front. Hum. Neurosci. 11:114. doi: 10.3389/fnhum.2017.00114 Statistical learning (SL) is the extraction of the underlying statistical structure from sensory input (Frost et al., 2015). The extent to which this ability is domain-general (with a single central mechanism underpinning SL in any modality) or domain-specific (where the SL mechanism differs by modality) remains a central question in statistical learning (Frost et al., 2015), and two approaches have been adopted to tackle this. First is to examine the extent to which predominantly domain-specific skills such as language proficiency (Arciuli and von Koss Torkildsen, 2012) and musical expertise (Schon and Francois, 2011), and domain-general skills such as working memory and general IQ (Siegelman and Frost, 2015), correlate with SL ability. Second is to compare SL performance across modalities, or even examine cross-modal transfer (Durrant et al., 2016). Mandikal Vasuki et al. (2017) (and the sister paper: Mandikal Vasuki et al., 2016) make an important contribution by adopting both of these approaches. They compare auditory and visual SL using the Saffran triplet learning paradigm (Saffran et al., 1999) in musicians and non-musicians. The three key findings are that musicians are better than non-musicians at segmentation of auditory stimuli only, there is no correlation between auditory and visual performance, and that auditory performance is better overall. This last result could be due to privileged auditory processing of sequential stimuli (Conway et al., 2009), or it could just reflect differences in perceptual or memory capabilities across modalities. However, the fact that SL performance in one modality does not predict performance in another is hard to explain if a single mechanism underlying both is posited. Combined with the fact that overall better performance was found in musicians only in the auditory modality, a domain-specific SL mechanism seems to offer the most parsimonious explanation of this data.

Large scale product search with spatial quantization and deep ranking

Product image search aims to retrieve similar product images based on a query image. While deep learning based features work well in retrieving images of the same category (e.g. “searching for T-shirts from all the clothing images”), they perform poorly when retrieving variants of images within the same category (e.g. “searching for uniform of Chelsea football club from all T-shirts image”), since it requires fine-grained matching on image details. In this paper, we present a spatial quantization approach that utilizes spatial pyramid pooling (SPP) and vector of locally aggregated descriptors (VLAD) to extract more discriminative features for instance-aware product search. By using the proposed spatial quantization, spatial information is encoded into the image feature to improve the fine grained product image search. We also present an triplet learning to rank method to finetune the deep learning model on product image search task. Finally, the experiments conducted on a large scale real world dataset provided by Alibaba large-scale image search challenge (ALISC) demonstrate the effectiveness of our method.

Event simultaneity does not eliminate age deficits in implicit probabilistic sequence learning.

Recent studies have shown age-related deficits in learning subtle probabilistic sequential relationships. However, virtually all sequence learning studies have displayed successive events one at a time. Here we used a modified Triplets Learning Task to investigate if an age deficit occurs even when sequentially-presented predictive events remain in view simultaneously. Twelve young and 12 old adults observed two cue events and responded to a target event on each of a series of trials. All three events remained in view until the subject responded. Unbeknownst to participants, the first cue predicted one of four targets on 80% of the trials. Learning was indicated by faster and more accurate responding to these high-probability targets than to low-probability targets. Results revealed age deficits in sequence learning even with this simultaneous display, suggesting that age differences are not due solely to general processing declines, but rather reflect an age-related deficit in associative learning.

Implicit sequence learning in people with Parkinson's disease.

Implicit sequence learning involves learning about dependencies in sequences of events without intent to learn or awareness of what has been learned. Sequence learning is related to striatal dopamine levels, striatal activation, and integrity of white matter connections. People with Parkinson’s disease (PD) have degeneration of dopamine-producing neurons, leading to dopamine deficiency and therefore striatal deficits, and they have difficulties with sequencing, including complex language comprehension and postural stability. Most research on implicit sequence learning in PD has used motor-based tasks. However, because PD presents with motor deficits, it is difficult to assess whether learning itself is impaired in these tasks. The present study used an implicit sequence learning task with a reduced motor component, the Triplets Learning Task (TLT). People with PD and age- and education-matched healthy older adults completed three sessions (each consisting of 10 blocks of 50 trials) of the TLT. Results revealed that the PD group was able to learn the sequence, however, when learning was examined using a Half Blocks analysis (Nemeth et al., 2013), which compared learning in the 1st 25/50 trials of all blocks to that in the 2nd 25/50 trials, the PD group showed significantly less learning than Controls in the 2nd Half Blocks, but not in the 1st. Nemeth et al. (2013) hypothesized that the 1st Half Blocks involve recall and reactivation of the sequence learned, thus reflecting hippocampal-dependent learning, while the 2nd Half Blocks involve proceduralized behavior of learned sequences, reflecting striatal-based learning. The present results suggest that the PD group had intact hippocampal-dependent implicit sequence learning, but impaired striatal-dependent learning. Thus, sequencing deficits in PD are likely due to striatal impairments, but other brain systems, such as the hippocampus, may be able to partially compensate for striatal decline to improve performance.

Does a simultaneous memory load affect older and younger adults’ implicit associative learning?

ABSTRACTThis study investigated the effects of a simultaneous memory load on implicit associative sequence learning using the Triplets Learning Task (TLT). Participants in the Simultaneous condition held a secondary task memory load during the TLT, while those in the Sequential condition also performed both tasks, but successively, rather than simultaneously. Thus, the Simultaneous condition had a memory load during the TLT, while the Sequential condition did not. Probe blocks without the secondary task allowed separation of effects on learning from effects on its expression. Results revealed that the simultaneous memory load affected older, but not younger adults, by suppressing the expression of learning, not learning itself. Thus, older and younger adults can learn probabilistic associations while holding a simultaneous memory load, but the load can limit the extent to which older adults adapt their performance to environmental structure. Results are consistent with theories which propose that implicit associative learning does not call on limited capacity resources, and highlight the importance of distinguishing effects of dual tasks on the expression of learning from those on learning itself.

The Effects of Aging on the Neural Basis of Implicit Associative Learning in a Probabilistic Triplets Learning Task

Few studies have investigated how aging influences the neural basis of implicit associative learning, and available evidence is inconclusive. One emerging behavioral pattern is that age differences increase with practice, perhaps reflecting the involvement of different brain regions with training. Many studies report hippocampal involvement early on with learning becoming increasingly dependent on the caudate with practice. We tested the hypothesis that the contribution of these regions to learning changes with age because of differential age-related declines in the striatum and hippocampi. We assessed age-related differences in brain activation during implicit associative learning using the Triplets Learning Task. Over three event-related fMRI runs, 11 younger and 12 healthy older adults responded to only the third (target) stimulus in sequences of three stimuli ("triplets") by corresponding key press. Unbeknown to participants, the first stimulus' location predicted one target location for 80% of trials and another target location for 20% of trials. Both age groups learned associative regularities but differences in favor of the younger adults emerged with practice. The neural basis of learning (response to predictability) was examined by identifying regions that showed a greater response to triplets that occurred more frequently. Both age groups recruited the hippocampus early, but with training, the younger adults recruited their caudate whereas the older adults continued to rely on their hippocampus. This pattern enables older adults to maintain near-young levels of performance early in training, but not later, and adds to evidence that implicit associative learning is supported by different brain networks in younger and older adults.

Statistical Learning Is Related to Reading Ability in Children and Adults

There is little empirical evidence showing a direct link between a capacity for statistical learning (SL) and proficiency with natural language. Moreover, discussion of the role of SL in language acquisition has seldom focused on literacy development. Our study addressed these issues by investigating the relationship between SL and reading ability in typically developing children and healthy adults. We tested SL using visually presented stimuli within a triplet learning paradigm and examined reading ability by administering the Wide Range Achievement Test (WRAT-4; Wilkinson & Robertson, 2006). A total of 38 typically developing children (mean age of 9;5 years, range 6;4-12;5) and 37 healthy adults (mean age of 21 years, range 18-34) were assessed. In children, SL was significantly related to reading ability. Importantly, this relationship was independent of grade and also age. The adult data, too, revealed that SL was significantly related to reading ability. A regression analysis of the combined child and adult data revealed that SL accounted for a unique amount of variance in reading ability, after age and attention had been taken into consideration. For the first time, this study provides empirical evidence that a capacity for more effective SL is related to higher reading ability in the general population.

Age Differences in Implicit Learning of Probabilistic Unstructured Sequences

It is unclear whether implicit probabilistic learning, the acquisition of regularities without intent or explicit knowledge, declines with healthy aging. Because age differences in previous work might reflect motor or rule learning deficits, we used the implicit Triplets Learning Task with reduced motor sequencing and non-rule-based associations. Fifteen young and 15 old adults responded only to the last event in a series of discrete 3-event sequences or triplets. A randomly chosen set of triplets occurred with high frequency, so there was no underlying rule to be learned. Both age groups learned associative regularities, but age differences in favor of the young emerged with practice. Discussion. Age differences may reflect the different neural regions that are involved as training progresses, which differ in the extent to which they are compromised by aging.

Dopamine transporter genotype predicts implicit sequence learning

Implicit learning, the non-conscious acquisition of sequential and spatial environmental regularities, underlies skills such as language, social intuition, or detecting a target in a complex scene. We examined relationships between a variation of the dopamine transporter (DAT1) gene (SLC6A3), which influences dopamine transporter expression in the striatum, and two forms of implicit learning that differ in the regularity to be learned and in striatal involvement. Participants, grouped as 9-repeat carriers or 10/10 homozygotes, completed the triplets learning task (TLT) and the spatial contextual cueing task (SCCT). The TLT assesses sequence learning, recruiting the striatal system, particularly as training continues. In contrast, the SCCT assesses spatial context learning, recruiting medial temporal brain networks. For both tasks, participants demonstrated learning in faster and/or more accurate responses to repeating patterns or spatial arrays. As predicted, TLT learning was greater for the 9-repeat carriers than the 10/10 group (despite equal overall accuracy and response speed) whereas there were no significant group differences in SCCT. Thus, presence of the DAT1 9-repeat allele was beneficial only for implicit sequence learning, indicating the influence of DAT1 genotype on one form of implicit learning and supporting evidence that implicit learning of sequential dependencies and spatial layouts recruit different neural systems.

Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity

Classical experiments on spike timing-dependent plasticity (STDP) use a protocol based on pairs of presynaptic and postsynaptic spikes repeated at a given frequency to induce synaptic potentiation or depression. Therefore, standard STDP models have expressed the weight change as a function of pairs of presynaptic and postsynaptic spike. Unfortunately, those paired-based STDP models cannot account for the dependence on the repetition frequency of the pairs of spike. Moreover, those STDP models cannot reproduce recent triplet and quadruplet experiments. Here, we examine a triplet rule (i.e., a rule which considers sets of three spikes, i.e., two pre and one post or one pre and two post) and compare it to classical pair-based STDP learning rules. With such a triplet rule, it is possible to fit experimental data from visual cortical slices as well as from hippocampal cultures. Moreover, when assuming stochastic spike trains, the triplet learning rule can be mapped to a Bienenstock-Cooper-Munro learning rule.