Pica is an extremely troublesome behaviors and causes a particularly high burdens for family caregivers. It can lead to dangerous medical consequences, such as malnutrition and intoxication, which sometimes require emergency medical treatment. Its neural basis and underlying mechanisms, however, have not been clarified. We addressed these issues by investigating patients with acquired brain injury who presented with pica. We found that semantic memory deficits were most often associated with pica rather than frontal release signs or changes in eating behaviors. Individuals with pica displayed lesion in the posterior part of the middle temporal gyrus, although spread across a wide area. Taken together, our findings suggest that semantic memory deficits, because of temporal lobe damage might be associated with pica. Moreover, Alzheimer's disease and semantic dementia are the two main degenerative disease etiologies that cause pica. Our findings may also be applied to degenerative disorders as temporal lobe involvement and semantic memory deficits are common. The most common items ingested in pica are daily necessities that are easy to swallow or drink; as such, these items should be avoided in patients with pica.