The distinction between literal and figurative language (metonymies, metaphors, etc.) is often not made formally explicit, or, if formal criteria exist, insufficient. This poses problems for an adequate computational treatment of these phenomena. The basic criterion for delineating literal from figurative speech we propose is centered around the notion of categorization conflicts that follow from the context of the utterance. In addition, we consider the problem of granularity, which is posed by the dependence of our approach on the underlying ontology.