Smart devices with the capability to record audio can create a trade-off for users between convenience and privacy. To understand how users experience this trade-off, we report on data from 35 interview, focus group, and design workshop participants. Participants' perspectives on smart-device audio privacy clustered into the pragmatist, guardian, and cynic perspectives that have previously been shown to characterize privacy concerns in other domains. These user groups differed along four axes in their audio-related behaviors (for example, guardians alone say they often move away from a microphone when discussing a sensitive topic). Participants surfaced three usage phases that require design consideration with respect to audio privacy: 1) adoption, 2) in-the-moment recording, and 3) downstream use of audio data. We report common design solutions that participants created for each phase (such as indicators showing when an app is recording audio and annotations making clear when an advertisement was selected based on past audio recording).