Abstract Background Patients with a rare disease (RD) are often diagnosed too late or not at all. Clinical decision support systems (CDSSs) could support the diagnostic process in rare diseases (RDs). The MIRACUM (Medical Informatics in Research and Medicine) consortium, which is one of four funded consortia in the German Medical Informatics Initiative, will develop a CDSS for RDs based on distributed clinical data from ten university hospitals. This qualitative study aims to investigate (1) the relevant organizational conditions for the operation of a CDSS for RDs, (2) which data is necessary for decision support, and (3) the appropriate user group for such a CDSS. Methods Interviews were carried out with RDs experts. Participants were recruited from staff physicians at the Rare Disease Centers (RDCs) at the MIRACUM locations, which offer diagnosis and treatment of RDs. An interview guide was developed with a category-guided deductive approach. The interviews were recorded on an audio device and then transcribed into written form. We continued data collection until all interviews were completed. Afterwards, data analysis was performed using Mayring’s qualitative content analysis approach. Results A total of seven experts were included in the study, from seven of the eight MIRACUM locations which have established an RDC. The results show that administrative staff and physicians from RDC B-centers, representing different medical specialties, are involved in the diagnostic process. The experts cited various software programs used for diagnostic support and considered both physicians and experts from the B-centers to be potential users of a CDSS. Furthermore, the experts explained that RDs exist which cannot be fully differentiated, but rather described only by their overall symptoms or findings: diagnosis is dependent on the disease or disease group. At the end of the diagnostic process, most centers prepare a summary of the patient case. Conclusions This qualitative study is a first step towards establishing the requirements for the development of a CDSS for RDs. However, further research is necessary to create solutions by also including the experts on RDs.