This dataset contains the training and test set used in the ICDAR 2019 Competition on Image Retrieval for Historical Handwritten Documents. This competition investigates the performance of large-scale retrieval of historical document images based on writing style. Based on large image data sets provided by cultural heritage institutions and digital libraries, providing a total of 20 000 document images representing about 10 000 writers, divided in three types: writers of (i) manuscript books, (ii) letters, (iii) charters and legal documents. We focus on the task of automatic image retrieval to simulate common scenarios of humanities research, such as writer retrieval. The training data set encompasses images from (i) Letters A, where each writer contributed one or three images; (ii) Manuscripts, where each writer was represented by five consecutive images from a single book. In total, it contains 300 writers contributing one page, 100 writers contributing three pages, and 120 writers contributing five pages resulting in 1200 images of 520 writers. The test data set contains 20 000 images: About 7 500 pages stem from isolated documents (partially anonymous writers, contributing one page each), and about 12 500 pages are from writers that contributed three or five pages. If you use this dataset, please cite: V. Christlein, A. Nicolaou, M. Seuret, D. Stutzmann, A. Maier: "ICDAR 2019 Competition on Image Retrieval for Historical Handwritten Documents", in 15th International Conference on Document Analysis and Recognition, 2019, Sydney, Australia