Cross-Modal Retrieval for CPSS Data

Abstract

Data generated and collected from Cyber-Physical-Social Systems (CPSS) that usually in the forms of image, audio, video, and text, are complex and heterogeneous. How to deal with the cross-modal retrieval problem for heterogeneous CPSS data has drawn considerable interests recently. The hashing based methods have been widely studied in building bilateral semantic associations of binary codes for cross-model retrieval. However, most existing methods discard binary constraints and learn linear projections as hashing functions. Moreover, none of them consider the cross-modal retrieval application in the scenario of Cloud-Fog-Edge computing. Therefore, how to learn more compact and discriminative binary codes with discrete constraints and nonlinear hashing functions for CPSS data in the Cloud-Fog-Edge architecture is still an open problem. In this paper, we propose a nonlinear discrete cross-modal hashing (NDCMH) method based on concise binary classification for CPSS data which fully investigates the nonlinear relationship embedding, discrete optimization as well as the hashing functions learning. Different from previous methods, our work presents a concise but promising cross-modal hashing method that builds a direct connection between original CPSS data and binary codes, which can alleviate the impact of large quantization loss. Furthermore, we execute the cross-modal retrieval service at cloud and fog. Specifically, hashing functions are deployed at the fog plane to reduce the amount data transfer and storage need on the cloud. Extensive experiments carried out on typical CPSS datasets demonstrate that the proposed NDCMH significantly outperforms other state-of-the-art methods.