The opportunity to improve patient safety is significant and the pressure to improve it is increasing. An approach to evaluate an organization's progress with patient safety efforts has not been clearly articulated, and existing efforts to monitor safety are likely inadequate. We present a framework to monitor patient safety, combining valid rate-based measures to evaluate outcomes and processes of care, and non-rate-based measures to evaluate structure and context of care. We present an example of how the safety scorecard from this framework is used to monitor patient safety at The Johns Hopkins Hospital and in over 150 ICUs in Michigan, New Jersey, and Rhode Island.