What is EvidenceGrader? Assessment of the quality of medical evidence is a critical step in the systematic review of clinical evidence. We aim to develop and evaluate a machine-learning system to enable automation of this step. The system should provide an overall quality rating for each outcome as well as finer-grained justification for different quality criteria according to the GRADE formalisation framework. Our goal is to work towards evaluating the quality of bodies of evidence (BoE) for a specific clinical question, rather than assessing the quality of individual primary studies. The dataset We have built a dataset of 13,459 instances built from quality-related information available in Cochrane Database of Systematic Reviews. To obtain the full dataset, a request needs to be made first through: https://community.cochrane.org/form/cochrane-library-data-request, upon which we can share all dataset files. The file fold_ids.json available here contains only the identifiers of summary-of-findings tables from which we extracted the data instances. One instance corresponds to one outcome per summary-of-finding table (there can be more than one outcome). The entire collection is split into 10 folds.