Picture-word interference studies typically show that semantically related distractor words embedded within a picture slow picture-naming responses, relative to unrelated ones. This semantic interference effect is commonly interpreted as arising from the competition of lexical-semantic (e.g., Schriefers, Meyer, & Levelt, 1990) or lexical-phonological (e.g., Starreveld & La Heij, 1996) codes. The experiment reported here tests a crucial assumption shared by these accounts--namely, that the effect reflects a lexical, rather than a nonverbal, conceptual conflict. Pictures were named while participants attempted to ignore embedded distractors that were in either verbal or pictorial format. The presence of both words and pictures substantially interfered with naming responses, but only words, not pictures, were found to induce semantic interference. These findings support the claim that for semantic interference to arise, both target picture and distractor have to be lexicalized. Consequently, a general conceptual locus of the effect can be excluded, and the claim that semantic interference is based on a lexical conflict is confirmed.