Biomedical knowledge graphs, which can help with the understanding of complex biological systems and pathologies, have begun to play a critical role in medical practice and research. However, challenges remain in their embedding and use due to their complex nature and the specific demands of their construction. Existing studies often suffer from problems such as sparse and noisy datasets, insufficient modeling methods, and non-uniform evaluation metrics. In this work, we established a comprehensive knowledge graph (KG) system for the biomedical field in an attempt to bridge the gap. Here we introduced PharmKG, a multi-relational, attributed biomedical knowledge graph, composed of more than 500,000 individual interconnections between genes, drugs, and diseases, with 29 relation types over a vocabulary of ~8,000 disambiguated entities. Each entity in PharmKG is attached with heterogeneous, domain-specific information obtained from multi-omics data, i.e. gene expression, chemical structure, and disease word embedding while preserving the semantic and biomedical features. For baselines, we offered 9 state-of-the-art knowledge graph embedding (KGE) approaches and a new biological, intuitive, graph neural network-based KGE method that uses a combination of both global network structure and heterogeneous domain features. Based on the proposed benchmark, we conducted extensive experiments to assess these KGE models using multiple evaluation metrics. Finally, we discussed our observations across various downstream biological tasks and provide insights and guidelines for how to use a knowledge graph in biomedicine. We hope that the unprecedented quality and diversity of PharmKG will lead to advances in biomedical knowledge graph construction, embedding, and application.

[Update New Version]: We have updated the PharmKG-8k dataset we used in the paper, including the training/validation/test set. If you found this dataset useful for your research, please cite our paper: PharmKG: A Dedicated Knowledge Graph Benchmark for Biomedical Data Mining.