We pose a simple research question: what do people infer about an action based on the fact that it is punished? Amid all the discussion of what punishment should and can accomplish or communicate, relatively little thought has been given to what punishment actually does signal (but c.f. Bilz, 2016; Murphy & Hampton, 1990). We present empirical evidence that speaks to the most basic way punishment may be understood by the lay public. In Part I, we briefly discuss some possible answers. In Part II, we present four studies. The first set demonstrates that information about punishment can lead to inferences about an action’s morality and harmfulness. We then show that punishment seems to signal harm — but not moral wrongness — uniquely, compared to other messages of social disapproval. Finally, our data show that, at least for some acts, even minimal messages of punishment can significantly change inferences of harm.