Discovering patterns in co-occurrences data between objects and groups of concepts is a useful task in many domains, such as healthcare data analysis, information retrieval, and recommender systems. These relational representations come from objects' behaviors in different views, posing a challenging task of integrating information from these views to uncover the shared latent structures. The problem is further complicated by the high dimension of data and the large ratio of missing data. We propose a new paradigm of learning semantic relations using tensor factorization, by jointly factorizing multi-view tensors and searching for a consistent underlying semantic space across each views. We formulate the idea as an optimization problem and propose efficient optimization algorithms, with a special treatment of missing data as well as high-dimensional data. Experiments results show the potential and effectiveness of our algorithms.