For a long time, magnetic tape has been commonly utilized as one of physical media for recording and playing music. In this research, we propose a novel interactive musical instrument called ParaSampling that utilizes the technology of magnetic sound recording, and a improvisational sound playing method based on the instrument. While a conventional cassette tape player has a single tapehead, which rigidly placed, our instrument utilizes multiple handheld tapehead modules as an interface. Players can hold the interfaces and press them against the rotating magnetic tape at an any point to record or reproduce sounds The player can also easily erase and rewrite the sound recorded on the tape. With this instrument, they can achieve improvised and unique musical expressions through tangible and spatial interactions. In this paper, we describe the system design of ParaSampling, the implementation of the prototype system, and discuss music expressions enabled by the system.