AbstractBackgroundEpidemiologic research on severe maternal morbidity often relies on a screening definition of the outcome because a gold standard approach requires medical record review.ObjectiveTo determine the validity of screening or identification criteria to classify cases of severe maternal morbidity using the definition of severe maternal morbidity proposed by the American College of Obstetricians and Gynecologists (ACOG).MethodsFrom all singleton deliveries at Magee‐Womens Hospital in Pittsburgh, Pennsylvania (2010‐2011; n = 19 307), we selected all deliveries that had at least one screening or identification criteria for severe maternal morbidity (n = 349) and a random sample of deliveries with no case identification criteria (n = 349). Screen‐positive deliveries were a delivery with any of the following: Centers for Disease Control and Prevention International Classification of Diseases 9th Revision diagnosis and procedure codes for the identification of severe maternal morbidity; prolonged post‐partum length of stay; or maternal intensive care unit admission. We identified true cases through detailed chart review using the suggested diagnoses in the 2016 ACOG and SMFM Obstetric Care Consensus on severe maternal morbidity. We calculated the positive and negative predictive values of the screening criteria.ResultsApproximately 1.8% of deliveries screened positive for severe maternal morbidity. After medical record review, 166 screen‐positive deliveries were true cases (48% positive predictive value), and 347 screen‐negative cases were true negatives (99% negative predictive value). Two screen‐negative cases were false negatives. If we applied the negative predictive value to the population, 109 true cases would be missed with these criteria.ConclusionThe criteria we used to identify potential cases of severe acute maternal morbidity had poor performance in our cohort. In the absence of resources to apply the gold standard outcome definition to a large population, validation data and analytic strategies that incorporate measurement error are essential to estimate the direction and magnitude of the resulting bias.