Advanced persistent threat (APT) attacks present a significant challenge for any organization, as they are difficult to detect due to their elusive nature and characteristics. In this article, we conduct a comprehensive literature review to investigate the various APT attack detection systems and approaches and classify them based on their threat model and detection method. Our findings reveal common obstacles in APT attack detection, such as correctly attributing anomalous behavior to APT attack activities, limited availability of public datasets and inadequate evaluation methods, challenges with detection procedures, and misinterpretation of requirements. Based on our findings, we propose a reference architecture to enhance the comparability of existing systems and provide a framework for classifying detection systems. In addition, we look in detail at the problems encountered in current evaluations and other scientific gaps, such as a neglected consideration of integrating the systems into existing security architectures and their adaptability and durability. While no one-size-fits-all solution exists for APT attack detection, this review shows that graph-based approaches hold promising potential. However, further research is required for real-world usability, considering the systems’ adaptability and explainability.