Out-of-hospital cardiac arrest (OHCA) data debriefing and clinical research often require the retrospective analysis of large datasets containing defibrillator files from different vendors and clinical annotations by the emergency medical services.To introduce and evaluate a methodology to automatically extract cardiopulmonary resuscitation (CPR) quality data in a uniform and systematic way from OHCA datasets from multiple heterogeneous sources.A dataset of 2236 OHCA cases from multiple defibrillator models and manufacturers was analyzed. Chest compressions were automatically identified using the thoracic impedance and compression depth signals. Device event time-stamps and clinical annotations were used to set the start and end of the analysis interval, and to identify periods with spontaneous circulation. A manual audit of the automatic annotations was conducted and used as gold standard. Chest compression fraction (CCF), rate (CCR) and interruption ratio were computed as CPR quality variables. The unsigned error between the automated procedure and the gold standard was calculated.Full-episode median errors below 2% in CCF, 1 min-1 in CCR, and 1.5% in interruption ratio, were measured for all signals and devices. The proportion of cases with large errors (>10% in CCF and interruption ratio, and >10 min-1 in CCR) was below 10%. Errors were lower for shorter sub-intervals of interest, like the airway insertion interval.An automated methodology was validated to accurately compute CPR metrics in large and heterogeneous OHCA datasets. Automated processing of defibrillator files and the associated clinical annotations enables the aggregation and analysis of CPR data from multiple sources.