Peganum harmala L. is a traditional Chinese and Uygur medicine used to treat cancer. Bioactivity‐guided fractionation was applied to determine the cytotoxic constituents from P. harmala. A novel triterpenoid and a phenolic glycoside were isolated and identified, as well as seven known compounds. The novel metabolites were elucidated to be 3α‐acetoxy‐27‐hydroxyolean‐12‐en‐28‐oic acid methyl ester (1, OA) and N‐acetyl‐9‐syringinoside (9). Some compounds exhibited potent cytotoxicity against human tumor cells. Among them, OA showed the highest cytotoxicity against human lung cancer cells A549 with an IC50 value of 8.03 ± 0.81 μm. OA had a potent anti‐NSCLC cell activity by interfering with the epidermal growth factor receptor (EGFR) activation and its downstream signaling, and could exert an antiproliferative effect by inactivation of EGFR‐driven antiapoptotic pathway followed by the release of mitochondrial cytochrome c, which might prove to be a promising leading compound for the development of an anti‐lung cancer drug.