Multi-representational learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNs

Abstract

Offline Signature Verification (OSV) is a challenging pattern recognition task, especially in the presence of skilled forgeries that are not available during training. This study aims to tackle its challenges and meet the substantial need for generalization for OSV by examining different loss functions for Convolutional Neural Network (CNN). We adopt our new approach to OSV by asking two questions: 1. which classification loss provides more generalization for feature learning in OSV?, and 2. How integration of different losses into a unified multi-loss function lead to an improved learning framework?These questions are studied based on analysis of three loss functions, including cross entropy, Cauchy-Schwarz divergence, and hinge loss. According to complementary features of these losses, we combine them into a dynamic multi-loss function and propose a novel ensemble framework for simultaneous use of them in CNN. Our proposed Multi-Loss Snapshot Ensemble (MLSE) consists of several sequential trials. In each trial, a dominant loss function is selected from the multi-loss set, and the remaining losses act as a regularizer. Different trials learn diverse representations for each input based on signature identification task. This multi-representation set is then employed for the verification task. An ensemble of SVMs is trained on these representations, and their decisions are finally combined according to the selection of most generalizable SVM for each user.We conducted two sets of experiments based on two different protocols of OSV, i.e., writer-dependent and writer-independent on three signature datasets: GPDS-Synthetic, MCYT, and UT-SIG. Based on the writer-dependent OSV protocol, On UT-SIG, we achieved 6.17% Equal Error Rate (EER) which showed substantial improvement over the best EER in the literature, 9.61%. Our method surpassed state-of-the-arts by 2.5% on GPDS-Synthetic, achieving 6.13%. Our result on MCYT was also comparable to the best previous results. The second set of experiments examined the robustness of our proposed method to the arrival of new users enrolled in the OSV system based on the writer-independent protocol. The results also confirmed that our proposed system efficiently performed the verification of new users enrolled in the OSV system.