With 87% of providers using electronic health records (EHRs) in the United States, EHRs have the potential to contribute to population health surveillance efforts. However, little is known about using EHR data outside syndromic surveillance and quality improvement. We created an EHR-based population health surveillance system called the New York City (NYC) Macroscope and assessed the validity of diabetes, hyperlipidemia, hypertension, smoking, obesity, depression, and influenza vaccination indicators. The NYC Macroscope uses aggregate data from a network of outpatient practices. We compared 2013 NYC Macroscope prevalence estimates with those from a population-based, in-person examination survey, the 2013–2014 NYC Health and Nutrition Examination Survey. NYC Macroscope diabetes, hypertension, smoking, and obesity prevalence indicators performed well, but depression and influenza vaccination estimates were substantially lower than were survey estimates. Ongoing validation will be important to monitor changes in validity over time as EHR networks mature and to assess new indicators. We discuss NYC’s experience and how this project fits into the national context. Sharing lessons learned can help achieve the full potential of EHRs for population health surveillance.