Third party punishment is crucial for sustaining cooperative behavior. Still, little is known about its determinants. In this paper we use laboratory experiments to investigate a long-conjectured interaction between group identification and bystanders' punishment preferences using a novel measure of these preferences. We induce minimal groups and give a bystander the opportunity to punish the perpetrator of an unfair act against a defenseless victim. We elicit the bystander's valuation for punishment in four cases - when the perpetrator, the victim, both or neither are members of the bystander's group. We generate testable predictions about the rank order of punishment valuations from a simple framework incorporating group-contingent preferences for justice which are largely confirmed. Finally, we conduct control sessions where groups are not induced. Comparing punishment across treatment and control suggests that third-party punishers tend to treat others as in-group members unless otherwise divided.