Do opportunities to punish non-punishers help to stabilize cooperation? Or do opportunities to punish punishers harm cooperation and its benefits by deterring first order punishment and wasting resources? We compare treatments of a decision experiment without peer punishment and with one order of punishment to ones in which subjects can be punished for punishing or for failing to punish. Our treatments with higher-order punishment achieve as much improvement in cooperation as those with only one punishment stage. We see evidence of social norms in action, but no evidence of punishing failure to punish. These results suggest that higher-order punishment is neither critical to nor a major deterrent to cooperation.