Knowledge graphs (KGs) have been widely used to represent relationships among entities, while KGs cannot capture new relationships between entities emerging along time. Since news often provides the latest information regarding the new entities and relationships, there is an opportunity to connect emerging relationships from news timely. However, it is a challenging task due to the source heterogeneity of structured KGs and unstructured news texts. In order to address the issue, we propose a tensor-based framework to capture the complex interactions among multiple types of relations, entities and text descriptions. We further develop an efficient Text-Aware MUlti-RElational learning method (TAMURE) that can learn the embedding representations of entities and relation types from both KGs and news, by jointly factorizing the interaction parameters. Furthermore, the complexity of TAMURE is linear in the number of parameters, which makes it suitable to large-scale KGs and news texts. Extensive experiments via TensorFlow demonstrate the effectiveness of the proposed TAMURE model compared with nine state-of-the-art methods on real-world datasets.